CI environments

Recommended environment variables

If you're manually providing environment variables, we strongly recommend setting the following variables:

• run_env[key]: A required variable that sends the UUID for the build, letting you group batches of data by the key.
• run_env[branch]: Sends the branch or reference for this build, enabling you to filter data by branch.
• run_env[url]: Provides the URL for the build on your CI provider, giving you a handy link back to the CI build.
• run_env[commit_sha]: Sends the commit hash for the head of the branch, enabling automatic flaky test detection in your builds.
• run_env[message]: Forwards the commit message for the head of the branch, helping you identify different runs more easily.

Containers and test collectors

If you're using containers within your CI system, then the environment variables used by test collectors may not be exposed to those containers by default. Make sure to export your CI environment's variables and your Buildkite API token to your containerized builds and tests.

For example, by default Docker does not receive the host's environment variables. To pass them through to the Docker container, use the --env option:

  docker run \
    --env BUILDKITE_ANALYTICS_TOKEN \
    --env BUILDKITE_BUILD_ID \
    --env BUILDKITE_BUILD_NUMBER \
    --env BUILDKITE_JOB_ID \
    --env BUILDKITE_BRANCH \
    --env BUILDKITE_COMMIT \
    --env BUILDKITE_MESSAGE \
    --env BUILDKITE_BUILD_URL \
    bundle exec rspec

Review the following sections for the environment variables expected by test collectors.

Buildkite

During Buildkite pipeline runs, test collectors upload information from the following environment variables, and test importers use the following field names:

CircleCI

During CircleCI workflow runs, test collectors upload information from the following environment variables, and test importers use the following field names:

For CircleCI runs:

run_env[key]=$CIRCLE_WORKFLOW_ID-$CIRCLE_BUILD_NUM

GitHub Actions

During GitHub Actions workflow runs,test collectors upload information from the following environment variables, and test importers use the following field names:

For GitHub Action runs:

run_env[key]=$GITHUB_ACTION-$GITHUB_RUN_NUMBER-$GITHUB_RUN_ATTEMPT

Other CI providers

If you're using other CI providers (or containers), then set environment variables for test collectors to gather information about your builds and tests. If you don't set these environment variables, then Test Analytics lacks the details needed to produce useful reports.

Each environment variable corresponds to a run_env key in the payload https://analytics-api.buildkite.com/v1/uploads. Read Importing JSON to learn how these keys are used to make API calls.