Test Engine overview

Test Engine is the testing layer of Buildkite Pipelines. It collects test results from the jobs your pipelines run, then provides tools to track, analyze, and act on those results across any testing framework. Test Engine also accepts results from non-Buildkite CI systems, so you can use it alongside an existing CI/CD setup while migrating to Buildkite Pipelines.

Use Test Engine to:

  • Detect and quarantine flaky tests so they stop blocking builds.
  • Split tests across parallel jobs to reduce build duration.
  • Monitor test performance, reliability, and ownership over time.
  • Surface the slowest and least reliable tests in each suite.
Screenshot of test suite trend showing six metrics over the last day

Get started

The recommended way to set up a new test suite is to add the Tests Buildkite plugin to the step that runs your tests. Learn more in the Test collection overview.

If you're looking for deeper framework insights, you can also add the test collector for your testing framework (also known as a test runner):

:rspec: RSpec :ruby: minitest :jest: Jest :mocha: Mocha :cypress: Cypress :jasmine: Jasmine :playwright: Playwright :vitest: Vitest :swift: Swift :android: Android :pytest: pytest :golang: Go :junit: JUnit :dotnet: .NET :elixir: Elixir :rust: Rust

If a Buildkite test collector is not available for one of these test runners, you can use other test collection mechanisms instead.

Manual setup

New to Test Engine? Work through the Add a test suite section of the Pipelines getting started tutorial, which walks you through creating a test suite, configuring a test collector for your project, and automating the test runner with Buildkite Pipelines.

Core features

Data retention

The execution data uploaded to Test Engine is stored in S3 and deleted after 120 days.

API & references

Learn more about: