Tests API

List tests

Lists the tests in a suite, along with aggregated duration, reliability, and execution metrics for each test over a time range.

curl -H "Authorization: Bearer $TOKEN" \
  -H "Buildkite-Version: 2026-08-01" \
  -X GET "https://api.buildkite.com/v2/analytics/organizations/{org.slug}/suites/{suite.slug}/tests"

[
  {
    "id": "01867216-8478-7fde-a55a-0300f88bb49b",
    "url": "https://api.buildkite.com/v2/analytics/organizations/my_great_org/suites/my_suite_name/tests/01867216-8478-7fde-a55a-0300f88bb49b",
    "web_url": "https://buildkite.com/organizations/my_great_org/analytics/suites/my_suite_name/tests/01867216-8478-7fde-a55a-0300f88bb49b",
    "scope": "User#email",
    "name": "is correctly formatted",
    "location": "./spec/models/user_spec.rb:42",
    "file_name": "./spec/models/user_spec.rb",
    "labels": ["flaky"],
    "reliability": 0.98,
    "duration_avg": 0.213,
    "duration_sum": 23.856,
    "duration_min": 0.108,
    "duration_max": 1.942,
    "executions_count": 112,
    "executions_count_by_result": {
      "passed": 110,
      "failed": 2
    }
  }
]

The Buildkite-Version request header opts in to the versioned response shown above, which includes the aggregated metrics. Requests made without this header receive a response which contains only the test attributes id through labels.

The aggregated metrics in each test are calculated over the time range set by the period, min_timestamp, and max_timestamp query string parameters. Tests without any executions recorded in Test Engine during the requested time range will not be present in the response.

`reliability`	The reliability of the test, calculated from its passed and failed executions and expressed as a decimal fraction. This is `null` when the test has no passed or failed executions in the time range.
`duration_avg`	The average execution duration, in seconds.
`duration_sum`	The total execution duration, in seconds.
`duration_min`	The shortest execution duration, in seconds.
`duration_max`	The longest execution duration, in seconds.
`executions_count`	The number of executions in the time range.
`executions_count_by_result`	The number of executions in the time range, broken down by result. The `passed` and `failed` counts are always present. The `skipped`, `pending`, and `unknown` counts are only present when they're non-zero.

Query string parameters

Some query string parameters support one or more of the additional operators listed below.

`!`	Not equal Example: Tests labeled with `foo` but not `bar` - `labels=foo,!bar`
`~`	Group contains Example: tests with one or more failed executions within the requested time period `tags=result:~failed`
`^`	Group only Example: tests with only failed executions within the requested time period `tags=result:^failed`
`*`	Starts with Example: tests with executions on branches with the staging- prefix within the requested time period `branch=staging-*`

`labels`	Filters the results by a comma-separated list of test labels. Cannot be combined with `label`. Example: `?labels=flaky,!slow` Supported operators: `!` Available with `Buildkite-Version` header >= `2026-08-01`
`label`	Filters the results by a single test label. This parameter is a legacy alternative to `labels` and cannot be combined with it. Example: `?label=flaky`
`branch`	Only aggregates executions from the branch whose name is specified by the `branch` value. Example: `?branch=main` Supported operators: `! *` Available with `Buildkite-Version` header >= `2026-08-01`
`owners`	Filters the results by a comma-separated list of test owner slugs. Example: `?owners=my-team,another-team` Supported operators: `!` Available with `Buildkite-Version` header >= `2026-08-01`
`state`	Filters the results by test state. Valid values are `enabled`, `muted`, and `skipped`. Example: `?state=muted` Available with `Buildkite-Version` header >= `2026-08-01`
`tags`	Filters the results by a comma-separated list of execution tags, using `key:value` syntax. Example: `?tags=framework:rspec,ci:true` Supported operators: `! *`. The `result` tag also supports `^ ~`. Available with `Buildkite-Version` header >= `2026-08-01`
`period`	Aggregates metrics over the given relative time `period`, for example `7days` or `28days`. The periods available to your organization depend on its maximum time window quota. Cannot be combined with `min_timestamp` or `max_timestamp`. Example: `?period=28days` Available with `Buildkite-Version` header >= `2026-08-01`
`min_timestamp`	The start of the aggregation window, as an ISO 8601 timestamp. Defaults to your organization's default period before the current time. Cannot be combined with `period`. Example: `?min_timestamp=2026-07-01T00:00:00Z` Available with `Buildkite-Version` header >= `2026-08-01`
`max_timestamp`	The end of the aggregation window, as an ISO 8601 timestamp. Defaults to the current time. Cannot be combined with `period`. Example: `?max_timestamp=2026-07-23T00:00:00Z` Available with `Buildkite-Version` header >= `2026-08-01`
`sort_by`	The metric to sort the results by. Valid values are `duration_avg`, `duration_sum`, `duration_min`, `duration_max`, and `reliability`. The default value is `duration_avg`. Example: `?sort_by=reliability` Available with `Buildkite-Version` header >= `2026-08-01`
`order`	The direction to sort the results in. Valid values are `asc` and `desc`. The default value is `desc`. Example: `?order=asc` Available with `Buildkite-Version` header >= `2026-08-01`

Optional request headers:

`Buildkite-Version`	Request an API version using a date in `YYYY-MM-DD` format. Set to `2026-08-01` or a later date to receive test metrics in the response. Without this header, or with a date before `2026-08-01`, the response uses the legacy format without metrics. Example: `Buildkite-Version: 2026-08-01`

Invalid format returns an error

If a non-blank Buildkite-Version header value is not in YYYY-MM-DD format, the API returns a 400 response with {"message": "Buildkite-Version must be in format YYYY-MM-DD"}. A blank value is treated as an omitted header, so the API returns the legacy 200 response instead.

This endpoint is paginated.

Required scope: read_suites

Success response: 200 OK

Get a test

curl -H "Authorization: Bearer $TOKEN" \
  -X GET "https://api.buildkite.com/v2/analytics/organizations/{org.slug}/suites/{suite.slug}/tests/{test.id}"

{
  "id": "01867216-8478-7fde-a55a-0300f88bb49b",
  "url": "https://api.buildkite.com/v2/analytics/organizations/my_great_org/suites/my_suite_name/tests/01867216-8478-7fde-a55a-0300f88bb49b",
  "web_url": "https://buildkite.com/organizations/my_great_org/analytics/suites/my_suite_name/tests/01867216-8478-7fde-a55a-0300f88bb49b",
  "scope": "User#email",
  "name": "is correctly formatted",
  "location": "./spec/models/user_spec.rb:42",
  "file_name": "./spec/models/user_spec.rb",
  "labels": ["flaky"]
}

Required scope: read_suites

Success response: 200 OK

Find a test with scope and name

In some situations, you may not have access to UUID to make a call to Test Engine API. You can locate a test record using its scope and name to retrieve the UUID from the response.

curl -H "Authorization: Bearer $TOKEN" \
  -X POST "https://api.buildkite.com/v2/analytics/organizations/{org.slug}/suites/{suite.slug}/tests/find" \
  -H "Content-Type: application/json" \
  -d '{
    "scope": "User#email",
    "name": "is correctly formatted"
  }'

{
  "id": "01867216-8478-7fde-a55a-0300f88bb49b",
  "url": "https://api.buildkite.com/v2/analytics/organizations/my_great_org/suites/my_suite_name/tests/01867216-8478-7fde-a55a-0300f88bb49b",
  "web_url": "https://buildkite.com/organizations/my_great_org/analytics/suites/my_suite_name/tests/01867216-8478-7fde-a55a-0300f88bb49b",
  "scope": "User#email",
  "name": "is correctly formatted",
  "location": "./spec/models/user_spec.rb:42",
  "file_name": "./spec/models/user_spec.rb",
}

Required scope: read_suites

Success response: 200 OK

Add or remove labels from a test

curl -H "Authorization: Bearer $TOKEN" \
  -X PATCH "https://api.buildkite.com/v2/analytics/organizations/{org.slug}/suites/{suite.slug}/tests/{test.id}/labels" \
  -H "Content-Type: application/json" \
  -d '{
    "operator": "add",
    "labels": ["flaky", "slow"]
  }'

{
    "file_name": "./spec/features/cool_spec.rb",
    "id": "ccd837ee-d484-8864-a6ee-29cfae965bd8",
    "labels": [
        "flaky", "slow"
    ],
    "location": "./spec/features/cool_spec.rb:232",
    "name": "one plus one",
    "scope": "A fancy feature",
    "url": "https://api.buildkite.com/v2/analytics/organizations/acme-inc/suites/acme-suite/tests/ccd837ee-d484-8864-a6ee-29cfae965bd8",
    "web_url": "https://buildkite.com/organizations/acme-inc/analytics/suites/acme-suite/tests/ccd837ee-d484-8864-a6ee-29cfae965bd8"
}

Required request body properties:

`operator`	The operation that will be apply to labels. `"add"` or `"remove"`.
`labels`	The labels that will be added or removed. Example: `["flaky"]`.

Required scope: write_suites

Success response: 200 OK