Test Collector Buildkite Plugin 
A Buildkite plugin for uploading JSON or JUnit files to Buildkite Test Engine ✨
Options
These are all the options available to configure this plugin’s behaviour.
Required
files
(string or array of strings)
One or more patterns of files to upload to Test Engine, relative to the root of the searching path (./
by default). May contain *
to match any number of characters of any type (unlike shell expansions, it will match /
and .
if necessary). Can be either a single pattern in a string or any number of them in an array.
format
(string)
Format of the file.
Only the following values are allowed: junit
, json
Optional
api-token-env-name
(string)
Name of the environment variable that contains the Test Engine API token.
Default value: BUILDKITE_ANALYTICS_TOKEN
api-url
(string)
Full URL for the API to upload to. Defaults to https://analytics-api.buildkite.com/v1/uploads
base-path
(string)
Where to search for files to upload. Defaults to the working directory .
branches
(string)
String containing a regex to only do an upload in branches that match it (using the case-insensitive bash =~
operator against the BUILDKITE_BRANCH
environment variable).
For example:
prod
will match any branch name that contains the substringprod
^stage-
will match all branches that start withstage-
-ISSUE-[0-9]*$
will match branches that end withISSUE-X
(where X is any number)
Important: you may have to be careful to escape special characters like $
during pipeline upload
debug
(boolean)
Print debug information to the build output.
Default value: false
.
Can also be enabled with the environment variable BUILDKITE_ANALYTICS_DEBUG_ENABLED
.
exclude-branches
(string)
String containing a regex avoid doing an upload in branches that match it (using the case-insensitive bash =~
operator against the BUILDKITE_BRANCH
environment variable ).
For example:
prod
will exclude any branch name that contains the substringprod
^stage-
will exclude all branches that start withstage-
-SECURITY-[0-9]*$
will exclude branches that end withSECURITY-X
(where X is any number)
Important:
- you may have to be careful to escape special characters like
$
during pipeline upload - exclusion of branches is done after the inclusion (through the
branches
option)
follow-symlinks
(boolean)
By default the plugin will not follow symlinked folders, set this option to true
to do so. This will add the -L
option to the find
command used to get the files to upload.
missing-error
(integer)
Exit code of the hook when no files are found to upload as long as the step’s command has not failed already.
This should allow you to use a special exit code to soft-fail on when no files to upload. Warning: make sure that the exit code used can not be generated by your command to avoid hiding actual step failures.
Default value: 1
tags
(array of strings)
A list of tags to apply to all test results in the upload, in the format key=value
.
For example:
tags:
- "arch=arm64"
- "language.version=1.2.3"
timeout
(number)
Maximum number of seconds to wait for each file to upload before timing out.
Default value: 30
annotation-link
(boolean)
Adds an annotation to the build run with a link to the uploaded report.
Default value: false
upload-concurrency
(number)
The number of concurrent file uploads to perform to the Buildkite Test Engine API.
Default value: 1
Requirements
This plugin requires jq
for parsing JSON data. If jq
is not found on the agent, sed
will be used as a fallback. Ensure that sed
is also available to handle scenarios where jq
cannot be used.
Fallback Behavior
If jq
is unavailable, the plugin will attempt to parse the results using sed
. This ensures that the plugin remains functional even if the preferred JSON parser is missing.
Examples
Upload a JUnit file
To upload a JUnit file to Test Engine from a build step:
steps:
- label: "🔨 Test"
command: "make test"
plugins:
- test-collector#v1.11.0:
files: "test/junit-*.xml"
format: "junit"
Upload a JSON file
To upload a JSON file to Test Engine from a build step:
steps:
- label: "🔨 Test"
command: "make test"
plugins:
- test-collector#v1.11.0:
files:
- "test-data-*.json"
format: "json"
Using build artifacts
You can also use build artifacts generated in a previous step:
steps:
# Run tests and upload
- label: "🔨 Test"
command: "make test --junit=tests-N.xml"
artifact_paths: "tests-*.xml"
- wait
- label: "🔍 Test Engine"
command: buildkite-agent artifact download "tests-*.xml" .
plugins:
- test-collector#v1.11.0:
files: "tests-*.xml"
format: "junit"
Branch filtering
Only upload on the branches that end with -qa
steps:
- label: "🔨 Test"
command: "make test"
plugins:
- test-collector#v1.11.0:
files: "test-data-*.json"
format: "json"
branches: "-qa$"
Do not upload on the branch that is exactly named legacy
:
steps:
- label: "🔨 Test"
command: "make test"
plugins:
- test-collector#v1.11.0:
files: "test-data-*.json"
format: "json"
exclude-branches: "^legacy$"
Only upload on branches that start with stage-
but do not contain hotfix
steps:
- label: "🔨 Test"
command: "make test"
plugins:
- test-collector#v1.11.0:
files: "test-data-*.json"
format: "json"
branches: "^stage-"
exclude-branches: "hotfix"
⚒ Developing
You can use the bk cli to run the pipeline locally:
bk local run
:warning: bk local run
was removed on v3. You can use v2 and get the binaries from https://github.com/buildkite/cli/releases/tag/v2.0.0
Or if you want to run just the tests, you can use the docker Plugin Tester:
docker run --rm -ti -v "${PWD}":/plugin buildkite/plugin-tester:latest
👩💻 Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/buildkite-plugins/test-collector-buildkite-plugin
📜 License
The package is available as open source under the terms of the MIT License.