New `tests` plugin for running your test suites

The new tests Buildkite plugin is the easiest way to run your test suites on Buildkite. Add the plugin to your step and set its command to bktec run, and bktec (Buildkite Test Engine client) splits your tests across parallel jobs out of the box. You also get test analytics, flaky test detection, timing, and history, with nothing extra to wire up.

steps:
  - label: "RSpec"
    command: bktec run
    parallelism: 4
    plugins:
      - tests#v1.0.0:
          test-runner: rspec
          result-path: tmp/rspec-result.json

The plugin works with every runner bktec supports, including RSpec, Jest, Cypress, Playwright, pytest, gotest, Cucumber, and NUnit.

Analytics out of the box

With bktec v2.7.0, the plugin uploads test results by default, so there is no separate test collector to install or configure. You get flaky test detection, per-test timing, and failure history from the first build. This can be disabled if you prefer to use a test collector instead.

Zero setup

The plugin handles everything behind the scenes:

• Downloads bktec and makes it available to the step (can be disabled if you prefer to install it yourself)
• Generates a short-lived OIDC token for authentication, so there are no API tokens to rotate
• Creates the test suite from the pipeline on the first run if it doesn't exist
• Exposes all bktec options as plugin attributes, so you don't need to manage environment variables yourself

Get started

Add the plugin to any step that runs bktec run or bktec plan. See the plugin README for the full list of options.