We've added support for GitLab.com commit statuses to Buildkite Pipelines ✅

Buildkite can now send status updates to GitLab for each build state (scheduled, started, running, passed, failed, blocked, canceled, skipped). Status updates appear on commits and merge requests in GitLab. These link back to the corresponding Buildkite builds.

Turn this on today by connecting to GitLab.com:

Then update your favorite pipeline's repository settings page to turn on commit status reporting ✨

For more details, see our GitLab documentation.

You can now send Slack notifications from Buildkite Test Engine!

Slack notifications can be triggered when a test state changes, or when a test label is added or removed.

Additionally Slack notifications can be filtered to only tests owned by particular teams.

Read more about Test Engine Slack notifications.

The Flaky Resolution feature is being deprecated and will be removed for all users in the coming week.

The My Assignments feature is being deprecated and will be removed for all users in the coming week.

Users can continue to review tests owned by their teams using the filtering functionality on the Tests and Flaky tests pages.

As part of this change, the Flaky Summary Mailer will also be updated to show the flakiest tests owned by your team, rather than those that were previously assigned.

You can now send webhooks from Buildkite Test Engine!

Webhooks can be triggered by the following events:

• Test state changed
• Test label added or removed

Additionally webhook events can be filtered to only tests owned by particular teams.

Read more about Test Engine webhooks.

You can now save your favorite test filters in Test Engine for quick access.

The new filter bar in the test list lets you add filters using execution tags and test labels. Once you've set up a view you like, click Save to name it. Your saved filters will then appear as a view in your suite navigation for easy access.

Buildkite Portals are user-defined GraphQL operations exposed via authenticated URL endpoints — offering a new, secure way to access the Buildkite API.

Designed with machine-to-machine operations in mind, Portals are not tied to user-owned access tokens. Instead, they use ephemeral, scoped tokens that can only perform the operations defined in an approved GraphQL document.

Key benefits of Portals include:

• Scoped access — Tokens are restricted to the exact operations defined by organization admins. This enables fine-grained, least-privilege access to your Buildkite data.
• Ephemeral tokens — Tokens are short-lived and single-purpose, ideal for secure machine-to-machine communication.
• User-invoked flow — Optional interactive flows allow users to generate tokens on-demand, similar to OAuth.
• Userless authentication — Tokens are not tied to any individual user, avoiding disruptions if team members are removed.

Getting started with Portals

Portals can be created by administrators from organization settings page. Each portal is assigned a unique endpoint with the following URL structure: https://portal.buildkite.com/organizations/{organization.slug}/portals/{portal.slug}

Portals support passing variables to enable dynamic operations and offer multiple authentication mechanisms for flexibility and security.

For more information, see the Portals documentation.

Buildkite Test Engine now supports test splitting (smart bin packing for faster CI) and auto-quarantining (automatic detection and isolation of failing tests) for Python PyTest suites.

• ✅ Cut down your critical path time.
• ✅ Keep flaky tests from blocking your pipeline.
• ✅ Spend less time on manual triage.

To get started, install the Test Engine Client to:

• Automatically split tests evenly across parallel jobs.
• Quarantine unstable tests based on customizable heuristics.

For full setup instructions, see the Test Engine docs.

Within Linux Hosted Agents, the Docker binaries have now been updated. These new versions are as follows:

• Docker Client -> From 25.0.4 to 27.3.1
• Docker Buildx -> From 0.13.0 to 0.22.0
• Docker Compose -> From 2.24.7 to 2.31.0

You can now label your tests with Buildkite Test Engine!

Labels allow you to:

• Organize tests to be more meaningful to your team and organization.
• Categorize tests, and therefore, can be used to filter tests within Test Engine.

Labels may be applied to or removed from tests:

• Manually through the Buildkite interface.
• Automatically through the automatic quarantine or test execution tags features.
• Using the REST API.

Start labelling your tests with Test Engine today.

We're extending the preview period for our new build page beyond the originally announced April 24th date. This extension gives us time to incorporate your valuable feedback and implement several significant enhancements before making it the default experience.

Why we're extending

Your feedback has been instrumental in shaping the new build page. We're working on performance optimizations and feature additions based directly on your input, with some substantial improvements in the pipeline.

Rather than rushing these changes, we want to:

• Complete the implementation of key performance improvements
• Incorporate requested features that enhance your workflow
• Ensure a seamless transition when the new experience becomes the default

What's next

We remain fully committed to the new build page with its enhanced navigation, powerful table views, improved build management, and better annotation support. If you haven't tried it yet, we encourage you to opt in through the button on any build page.

When our next wave of improvements is ready, we'll announce a new deprecation timeline for the classic build page, giving everyone ample time to adjust to the changes before they become standard.

Please continue sharing your feedback at support@buildkite.com - it directly influences our development priorities.

You can now add custom tags to your tests — unlocking a whole new dimension (literally) of insight and control.

Whether you want to tag tests by the infrastructure they run on, the services they touch, or framework versions they depend on, custom tags let you define what matters — and then slice your test data by those attributes.

🔍 Find root causes faster

Spot patterns in failures and slowdowns by breaking down your suite along your real-world dimensions — like container image, compute pool, or dependency version.

📊 Insights tailored to your stack

Move beyond generic test metrics. Aggregations by custom tags give you visibility into the things that actually cause pain in your CI.

💸 Drive cost and performance optimizations

Use tags to identify high-cost, slow, or flaky tests by environment — and target improvements where they’ll have the biggest impact.

Start tagging your tests with Test Engine today.

The Changelog, ChangelogAuthor, ChangelogConnection, and ChangelogEdge types have been removed from the GraphQL API.

Please use the changelog Atom feed (https://buildkite.com/changelog.atom) instead.

Buildkite has joined the the GitHub Secret Scanning Program to enhance security for your API tokens. This program helps detect and alert us when a Buildkite API access token is leaked in a public GitHub repository.

What happens when a token is detected:

• For tokens found in public repositories or npm packages: GitHub immediately notifies Buildkite, and we automatically revoke the affected token to prevent unauthorized access. The token owner and organization admins receive notifications about the incident.
• For tokens found in private repositories with secret scanning enabled: Repository admins and the committer are alerted directly through GitHub's interface, where they can view and manage the detected secrets.

FAQ's

Do I need to enable anything to get this protection?

• For public repositories, protection is automatic with no configuration needed.
• For private repositories, repository administrators need to enable GitHub Secret Scanning.

What types of Buildkite tokens are protected?

• Currently, only Buildkite API access tokens

How will I be notified if my token is revoked?

• The owner of the token and the admins of the associated organization will receive an email from Buildkite.

What should I do if I receive a notification about a leaked token?

• Generate a new access token for your Buildkite user account.

It is now possible to pause individual Buildkite agents. Pausing an agent is similar to stopping an agent, but unlike stopping, a paused agent remains running and can resume work later on.

Pausing an agent is a useful alternative to stopping an agent, especially when resources are tied to the lifetime of the agent, such as a cloud instance configured to terminate when the agent exits. By pausing an agent, you can investigate problems in its environment more easily, without the worry of jobs being dispatched to it.

Some examples of maintenance operations that could benefit from pausing an agent include:

• pruning Docker caches
• emptying temporary directories
• updating code mirrors
• installing software updates
• compacting or vacuuming databases

You can pause agents in either the UI, GraphQL API or Rest API. Paused agent counts will be reflected in the platform.

To fully benefit from pausing, we recommend upgrading your agents to the latest released version.

See Pausing and resuming agents on Buildkite Docs for more information.

Buildkite now supports setting commit statuses on GitLab self-managed instances. This integration enables real-time status visibility in merge requests and can block merging until builds pass.

You can configure commit statuses today from your organization's repository providers page:

https://buildkite.com/organizations/-/repository-providers

Look under API Settings for configuration instructions:

Then go to your favorite pipeline's repository settings page to turn on commit status reporting ✨

You can now see the source IP ranges used by your Buildkite hosted agents in your cluster.

We're introducing expiry timestamps for agent registration tokens in response to customer feedback around security compliance and token lifecycle management.

You can now set an expiry time when creating tokens via both GraphQL (expiresAt) and REST (expires_at) APIs. The timestamp must be in ISO8601 format (2025-01-01T00:00:00Z).

This change enables automated token rotation through API integration, replacing the previous manual rotation process for long-lived tokens.

Important details:

• Existing tokens will continue to work without expiry
• Expired tokens will prevent new agent registrations but won't affect currently connected agents
• The UI will display tokens as "expired" after their expiry date
• Expiry dates cannot be modified after token creation
• There is no maximum expiry duration but a minimum of 10 minutes in the future is required

Below is a GraphQL example showing the creation of a token with an expiry.

mutation createToken {
  clusterAgentTokenCreate(input: {
    organizationId: "",
    description: "a token with an expiration",
    clusterId:"",
    expiresAt: "2026-01-01T00:00:00Z"
  }) {
    tokenValue
  }
}

Since turning on the new build page for everyone, we've been listening to your feedback and working hard to make improvements. Thank you for bearing with us. We're excited to announce a number of new changes that we hope you'll find valuable, with more on the way ahead of deprecating the classic build page on April 24, 2025.

Performance Improvements: faster Canvas and Table

To make the page faster for you, we’ve added virtualisation to the canvas and table views – rendering only what needs to be rendered, making a significant improvement for builds with hundreds or thousands of steps. On our backlog is to give the same treatment to the sidebar.

Persistent sidebar between builds

Keep your sidebar consistent as you navigate through builds now that we persist collapsed states in the build page sidebar (ie. Failed, Running, Passed). With large builds showing hundreds of passed steps you don't need to see, once you collapse Passed steps they'll stay like that on all consecutive builds.

Get the same view by default on your builds

Going between builds is now better as your preferred view persists. Your personal selection of "Canvas", "Table", or "Waterfall" is remembered as you navigate across builds. These views are now consolidated into a single "Steps" tab where you control which visualization you'd like to use.

Many people gave us feedback about situations where the canvas or table weren’t particularly useful and now they’ll be able to stay on their preferred view.

More intuitive filtering

We’ve moved step filtering out of the sidebar and into the “Steps” view to allow users to filter down their pipeline steps whilst continuing to have the overall context of the pipeline. This lets the sidebar remain a navigational aid even when narrowing down on failed steps.

An added benefit: debugging some job slowness using the waterfall view? Waterfall now supports job-level filtering.

Less noise in the sidebar

To make the sidebar more useful, we’ve improved the way we surface the relevant jobs based on the current state of a step. Here’s an example of this in action when viewing a build with a failing step with a high number of parallelism.

You’ll see how we present the failed job, but we initially hide the passed jobs.

We have also changed the way we display failed steps, by giving them a slight red background. Making it easier to see failed steps in your pipeline of the sidebar. Particularly useful when using the Pipeline view.

What's next?

We are still looking at opportunities to improve the page and make migration from the classic build UI easier. In particular we are considering improvements to annotations, layout and information architecture as well as unblocking block steps.

Customers often need to send custom slack or email notifications when steps fail. If an important build or deploy step fails, then someone needs to know about it. But, this hasn't been the simplest process until now.

You can now reference a step's information (id, key, label, type, state and outcome) in conditionals by prefixing with step. (such as step.id). This is particularly useful when you need to notify someone as soon as a step fails. See detailed descriptions of these values in our docs on conditional variables.

Here's how simple it is now:

steps:
  - command: false
    notify:
      - slack:
          channels: ["#engineering"]
          message: "Important step failed, please fix."
        if: step.outcome == "hard_failed"
  - command: sleep 1000

Previously, the recommended solution required conditionally uploading a new step to handle notifications:

steps:
  - label: "Fail"
    command: exit 1
    key: "fail"
  - label: "Run regression only if fail check is failed"
    depends_on: "fail"
    command: |
      if [ $$(buildkite-agent step get "outcome" --step "fail") == "hard_failed" ]; then
         cat <<- YAML | buildkite-agent pipeline upload
         steps:
           - label: notify"
             command: notification.sh
      YAML
      fi

For quick reference, options for step.outcome are neutral, passed, errored, hard_failed or soft_failed. step.state can be one of ignored, waiting_for_dependencies, ready, running, failing or finished. step.type exposes the step's type such as command or trigger. Step key and label are user defined values.

This new syntax works with all notification types (Slack, email, webhooks, etc.) and can be combined with other conditions using standard operators (&&, ||). Each step's notification can only reference its own outcome - you can't reference other steps' outcomes directly in the notification conditional.

Notifications are sent immediately when the step completes with the specified outcome, providing faster feedback when issues occur.