1. Resources
  2. /
  3. Examples
  4. /
  5. RSpec JUnit

Buildkite RSpec JUnit Example

This repository is an example Buildkite pipeline that collects JUnit test failures from RSpec and annotates the build with the buildkite-agent annotate command.

See the full Getting Started Guide for step-by-step instructions on how to get this running.

Screenshot of Buildkite RSpec JUnit example pipeline

How it works

This example demonstrates how to:

  • Run RSpec tests that produce JUnit XML output
  • Parse the JUnit report for failed tests
  • Use the buildkite-agent annotate command to display those failures directly in the Buildkite UI

The annotation helps you quickly identify failing tests without digging through logs.

Test annotations use 2 features within Buildkite:

  1. The buildkite-agent annotate command that allows you add arbiraty HTML and Markdown to the top of a build page
  2. A wait step with continue_on_failure: true which tells the pipeline to continue in the event of a failed step

The pipeline.yml file can be seen here: https://github.com/buildkite/rspec-junit-example/blob/master/.buildkite/pipeline.yml

Here’s the first step:

- label: ":rspec:"
  artifact_paths: "tmp/rspec-*.xml"
  commands:
    - "bundle"
    - "rspec --format progress --format RspecJunitFormatter --out tmp/rspec-$BUILDKITE_JOB_ID.xml"
  plugins:
    - docker#v1.0.0:
        image: "ruby:2.4"
        workdir: /app

This step runs our “rspec” command within a Ruby docker container, and stores the JUnit XML data to an artifact at “tmp/rspec-f59b96d7-5d75-42a6-ab9b-d4cea59db2b2.xml”

The next step:

 - wait: ~
   continue_on_failure: true

Tells Buildkite that if there’s a failure in the previous stage of the pipeline (in this case, the “RSpec” tests) that it should continue the pipeline. Usually, the wait step will fail the pipeline at this point since there was an error - but this option changes that behaviour.

- label: ":junit:"
  commands:
    - .buildkite/junit.sh

The last step is the one that parses our JUnit XML files, and turns them into a Buildkite annotation. Here’s the script that creates the actual annotation:

#!/bin/bash

set -euo pipefail

mkdir -p tmp

echo "--- :junit: Download the junits"
buildkite-agent artifact download tmp/rspec-*.xml tmp

echo "--- :junit::ruby: Processing the junits"
docker run --rm -v "$(pwd):/app" ruby:2.4 bash -c "cd /app && gem install nokogiri --quiet --silent && ruby /app/.buildkite/lib/junit.rb /app/tmp/*.xml" > tmp/annotation.md

echo "--- :buildkite: Creating annotation"
buildkite-agent annotate --context junit --style error < tmp/annotation.md

We wrote our JUnit XML parser in Ruby (you can write it in what ever language you like), but all it does it parse all the rspec-*.xml files, grabs the errors, and turns it some HTML that looks like this:

<p>There were 3 failures:</p>

<details>
<summary><code>Calculator#multiply multiplies numbers together in spec.lib.calculator_spec</code></summary>

<code><pre>expected: 8
     got: 6

(compared using eql?)

./spec/lib/calculator_spec.rb:21:in `block (3 levels) in '</pre></code>

in <a href="https://github.com/buildkite/rspec-junit-example/blob/HEAD/#f59b96d7-5d75-42a6-ab9b-d4cea59db2b2">Job #f59b96d7-5d75-42a6-ab9b-d4cea59db2b2</a>
</details>

...
</div>

We then pipe that HTML into the buildkite-agent annotate command and change the style of annotation to error so it shows up as red and with a cross in the Buildkite UI.

- wait

- command: ".buildkite/deploy.sh"
  label: ":rocket:"

The final 2 steps demonstrate that the pipeline doesn’t continue any further since there was an error in one of the previous stages. If this wait step also had continue_on_failure: true then it would also continue.

License

See LICENSE.md (MIT)

More examples

Start turning complexity into an advantage

Create an account to get started with a 30-day free trial. No credit card required.

Buildkite Pipelines

Platform

  1. Pipelines
  2. Public pipelines
  3. Test Engine
  4. Package Registries
  5. Mobile Delivery Cloud
  6. Pricing

Hosting options

  1. Self-hosted agents
  2. Mac hosted agents
  3. Linux hosted agents

Resources

  1. Docs
  2. Blog
  3. Changelog
  4. Example pipelines
  5. Plugins
  6. Webinars
  7. Case studies
  8. Events
  9. Migration Services
  10. Comparisons

Company

  1. About
  2. Careers
  3. Press
  4. Brand assets
  5. Contact

Solutions

  1. Replace Jenkins
  2. Workflows for AI/ML
  3. Testing at scale
  4. Monorepo mojo
  5. Bazel orchestration

Legal

  1. Terms of Service
  2. Acceptable Use Policy
  3. Privacy Policy
  4. Subprocessors
  5. Service Level Agreement

Support

  1. System status
  2. Forum
© Buildkite Pty Ltd 2025