Android collectors

To use Buildkite Test Engine with your Android projects, use the test-collector-android package.

The recommended way to set up a new test suite is to add the Tests Buildkite plugin to the step that runs your tests. It works with every runner that bktec supports, and the entire setup is configuration-only — you can get a test suite running through changes to pipeline.yml alone, with no modifications to your application code.

Use the language-specific test collector documented on this page when you want deeper framework integration — such as custom execution tags, span annotations, or richer per-framework data. Language-specific collectors still pair well with the Tests Buildkite plugin, but adding one requires changes to your application code.

You can also upload test results by importing JSON or JUnit XML.

Android

Before you start, make sure your tests run with access to CI environment variables.

Create a test suite and copy the test suite API token.
Securely set the BUILDKITE_ANALYTICS_TOKEN secret on your CI to the API token from the previous step.

This will need to be on your CI server, if running the BuildKite collector using CI, or otherwise on your local machine.

Unit Test Collector. In your top-level build.gradle.kts file, add the following to your classpath:

buildScript {
    ...
    dependencies {
        ...
        classpath("com.buildkite.test-collector-android:unit-test-collector-plugin:0.1.0")
    }
}

Then, in your app-level build.gradle.kts, add the following plugin:

plugins {
    id("com.buildkite.test-collector-android.unit-test-collector-plugin")
}

That's it!

Instrumented Test Collector. In your app-level build.gradle.kts file,

Add the following dependency:

androidTestImplementation("com.buildkite.test-collector-android:instrumented-test-collector:0.1.0")

android {
    ...

    defaultConfig {
        ...

        buildConfigField(
            "String",
            "BUILDKITE_ANALYTICS_TOKEN",
            "\"${System.getenv("BUILDKITE_ANALYTICS_TOKEN")}\""
        )
    }
}

Sync Gradle, and rebuild the project to ensure the BuildConfig is generated.

Create the following class in your androidTest directory, for example, src/androidTest/java/com/myapp/MyTestCollector.kt

class MyTestCollector : InstrumentedTestCollector(
    apiToken = BuildConfig.BUILDKITE_ANALYTICS_TOKEN
)

Again, in your app-level build.gradle.kts file, instruct Gradle to use your test collector:

testInstrumentationRunnerArguments += mapOf(
    "listener" to "com.mycompany.myapp.MyTestCollector" // Make sure to use the correct package name here
)

Note: This test collector uploads test data using the device under test. Make sure your Android device/emulator has network access.

Commit and push your changes:

git checkout -b add-buildkite-test-engine
git commit -am "Add Buildkite Test Engine"
git push origin add-buildkite-test-engine

Once you're done, in your Test Engine dashboard, you'll see analytics of test executions on all branches that include this code.

If you don't see branch names, build numbers, or commit hashes in Test Engine, then read CI Environments to learn more about exporting your environment to the collector.

Debugging

To enable debugging output, create and set BUILDKITE_ANALYTICS_DEBUG_ENABLED environment variable to true on your test environment (CI server or local machine).

For instrumented tests debugging, access the variable using buildConfigField and pass it through your MyTestCollector class. Refer the example project for implementation.