Back in October, Buildkite Agent v3.5.0 included a bug fix to ensure that plugins are always executed in the correct order. In the process of fixing that bug, we've updated our recommended syntax for using plugins to use an array instead of a map, to make it clearer that the plugins are executed in order.

For example, if your pipeline uses the following syntax, we recommend updating it:

1
2
3
4
5
6
7
steps:
  - label: '<img class="emoji not-prose size-[1em] inline align-[-0.1em]" title="shipit" alt=":shipit:" src="https://buildkiteassets.com/emojis/img-buildkite-64/shipit.png" draggable="false" />'
    plugins:
      docker-login#v2.0.1:
        username: ci
      docker-compose#v2.6.0:
        push: app

The new recommended syntax, which works on all versions of the Buildkite Agent, changes the plugin property to be an array:

1
2
3
4
5
6
7
steps:
  - label: '<img class="emoji not-prose size-[1em] inline align-[-0.1em]" title="shipit" alt=":shipit:" src="https://buildkiteassets.com/emojis/img-buildkite-64/shipit.png" draggable="false" />'
    plugins:
      - docker-login#v2.0.1:
          username: ci
      - docker-compose#v2.6.0:
          push: app

We've been working on a command-line tool for Buildkite for a while. Recently it's been upgraded with a local pipeline runner for testing out pipelines locally on your development machine.

We use this for testing plugins, and quickly iterating on pipelines in development. Secretly, it was just an experiment to use the image support in iTerm 2 for rendering inline custom emojis 😉💥

Download the latest release and let us know what you think.

You can now delete artifacts that you no longer want stored on Buildkite with the new delete button in the Artifacts tab:

When you delete an artifact, it's permanently deleted from our AWS S3 storage and removed from the job’s artifact list. If you’re using your own S3 bucket or Google Cloud Storage bucket for artifact storage, you'll need to manually remove them from your artifact store (don’t worry, we'll remind you to do this in the confirmation message).

If you’re performing deletions over a large number of builds, or wanting to automate deletion in general, we've updated the REST APIs to add support both artifact deletion and job log deletion.

We’ve updated the pipeline view of parallel jobs, so instead of showing every individual job they’re now shown as a single, expandable group 🎲

Each parallel job on the build page now also shows their number, and the parallel group size, alongside their label 🏷

Parallel jobs are a powerful way to reduce your build time by distributing your tests across agents 🌪 You can read about them in the parallel job documentation, or see them in action in the parallel testing screencast.

Happy 2019! To kick off the year, we've got some new agent changes fresh off the press!

Better error messages for cancelled jobs 🚨

When a job is cancelled, the agent receives the instruction and starts to shutdown the running job. The job is given 10 seconds to finish, after which it's terminated. Previously what had happened wasn't clear. We now clearly show that a command was interrupted by a signal:

Configurable grace period for cancellation ⏱

By default we give cancelled jobs 10 seconds to finish before they terminate. For some things like a Terraform tear-down, this can take a lot longer. We added a config of cancel-grace-period to customize that value.

1
2
# Wait 600 seconds (10 minutes) for cancellation
buildkite-agent start --cancel-grace-period 600

Future improvements will allow this to be set at a pipeline or step level.

Checkout directory is removed on error 🧹

Checkouts can fail for a variety of reasons which can sometimes leave the git repository in a corrupted state that prevents future checkouts.

The default behaviour for the agent is now to remove the checkout directory if there is a terminal checkout failure.

Windows agents run hooks after cancel 🎣

Windows unfortunately doesn't handle signals like posix systems do, so on Windows the cancellation behaviour was to simple terminate the job. This meant that artifacts weren't uploaded and hooks like pre-exit didn't run, which left things like docker-compose without a way to cleanup.

We now use Windows Terminal Process Groups and the CTRL_BREAK message to emulate signal handling in windows, which now means it should behave much like it's fairer cousins!

Windows gets terminal colors! 🖍

Ok this is a small improvement, but it represents our ongoing efforts to take advantage of the neat things that the Windows Terminal team is doing building for Terminal apps.

Spawn Multiple Agents 🐙

Currently we recommend that multiple agents on a host are run via your init system (launchd, systemd, etc). The new --spawn feature allows for multiple parallel agents to be spawned from the one installation.

We're still figuring out how this might fit into the bigger picture, but keen to see how folks use this feature!

Datadog metrics (Beta) 🐶

We are working with the Datadog team on becoming an official integration. First steps are getting some simple job metrics into Datadog.

You can use the new config metrics-datadog and metrics-datadog-host.

Let us know what you think at https://forum.buildkite.community/t/about-our-datadog-integration/216

Vendored Plugins 📦

We now support loading plugins from within your repository. This lets you vendor plugins to ensure that a third-party can't change them.

1
2
3
4
5
steps:
  - label: 'Test with vendored docker plugin'
    plugins:
      - './.buildkite/plugins/docker':
          image: 'hello-world'

Check out https://github.com/buildkite/agent/pull/878 for a bit more context.

Resolve $BUILDKITE_COMMIT correctly at pipeline upload ⚙️

We now resolve manually triggered builds with a commit of HEAD to the actual commit value in $BUILDKITE_COMMIT for your pipeline uploads.

See https://github.com/buildkite/agent/issues/456 for more context. I'm sorry it took us so long!

Better handling of submodules 🧤

We discovered a few gnarly bugs where we weren't ssh keyscanning submodules correctly and also a few cases where we weren't cleaning up lingering submodules in the checkout phase. These are now squashed!

Everyone's favourite illustration from the Buildkite website has been repackaged and deployed as a set of transparent stickers, ready for you to launch onto any surface you choose ✨

You can order your own set of 🚀 stickers from the Buildkite Shop.

You can now link directly to a job from a build annotation! 🔗

This should make the annotations generated by neat annotation generators like test-summary-buildkite-plugin and rspec-buildkite all the more handy.

You can now upload a user avatar directly on Buildkite. 🤳🏻 You’ll find the option to either drag-and-drop, or browse for an avatar in your Personal Settings!

If you don’t upload an avatar directly, we’ll continue to use your Gravatar, or if you use Buildkite via Single Sign-On, the avatar from your SSO provider.

Finally, organization admins can find a new option to upload an icon under Organization Settings. 🖼 That icon will be used in the organization switcher, as well as shown alongside Single Sign-On prompts.

We've released a new major version of Elastic Stack for AWS.

The Buildkite Elastic CI Stack gives you a private, autoscaling Buildkite Agent cluster. Use it to parallelize legacy tests across hundreds of nodes, run tests and deployments for all your Linux-based services and apps, or run AWS ops tasks.

The new version is focused on moving the underlying instances to Amazon Linux 2, which brings with it modern versions of systemd amongst others.

Check out the full release notes at https://github.com/buildkite/elastic-ci-stack-for-aws/releases/tag/v4.0.0 and let us know what you think at https://forum.buildkite.community/t/upcoming-elastic-stack-v4/78.

The Buildkite pipeline schema is now available in the JSON Schema Store and on GitHub, allowing you to autocomplete and validate your pipeline YAML files as you write them 📝✨

Any editor extension that supports RedHat YAML Language Server will automatically pick up support, including:

• vscode-yaml for VS Code
• ide-yaml for Atom
• coc-yaml for vim and neovim
• Eclipse Che

If you’ve built your own dynamic pipeline tools or plugins you can use the full JSON schema in your tests, or to validate your pipelines before uploading them to Buildkite.

There's a new screencast available at buildkite.com/screencasts/docker 🎉

The Docker screencast walks through a Docker workflow in Buildkite: building, testing, and deploying an image to production. The pipeline and code you'll see in the screencast are also available on GitHub: https://github.com/buildkite/screencast-examples/

You can check out the Docker screencast, and all the others, at buildkite.com/screencasts

Some of you may not know this, but Buildkite actually used to be called Buildbox! We changed our name back in December 2014.

From today, BUILDBOX_* environment variables won't be generated for new jobs.

We've confirmed there are no agents running this version connected to Buildkite, however we can't tell if you're referencing any BUILDBOX_* environment variables in your build scripts. If you are, please update them to use their BUILDKITE_* equivalent.

You can see our environment variable documentation for a full list of current job environment variables.

We've just shipped a small update to the build page to include the user who canceled the build 🙅‍♀️🙅‍♂️

And if your build was canceled due to Build Skipping (e.g. if you’ve enabled "Cancel Intermediate Builds") we'll show that too:

We've just shipped a new beta feature giving you the ability to define your initial build steps using the same pipeline YAML format that you use in source code.

This means that you can now configure trigger steps, block steps with input fields, and plugins independently of your pipeline’s source repository. YAML steps will become the default for all new pipelines once the beta is complete.

Head on over to your "Pipeline Settings" page to convert your pipeline to YAML Steps.

You can read more about the feature and give us feedback in the Buildkite Community Forum

Two-factor authentication for user accounts has just been released 🚀📟✨

You can find two-factor authentication in the sidebar of your Personal Settings. Set up your One Time Password with your favorite authenticator application, and don't forget to save your recovery codes somewhere secure 🗃💾💿

Buildkite Docker Plugin v2.0.0 has been released, and includes:

• Support for multi-line commands
• A new command option for using single-binary/command line Docker images
• Windows container support

1
2
3
4
5
6
steps:
  - command: 'dotnet publish -c Release -o published'
    plugins:
      - docker#v2.0.0:
          image: 'microsoft/dotnet:latest'
          always-pull: true

See all the details, and upgrade instructions, on the release notes: https://github.com/buildkite-plugins/docker-buildkite-plugin/releases/tag/v2.0.0

Build pipelines that finish with a click to unblock step now have a new state: Blocked! 🚧

You'll see some nice new icons across the Buildkite website and the build will now show as "Blocked" in notification services.

For now, webhooks and the API are unchanged so we don't affect your existing integrations. Keep an eye out for these changes in a future update to the API!

We’ve just released Buildkite Agent v2.2 (Brangelina) 💑💔

So what's changed since Buildkite Agent 2.1.14?

Features

• The buildkite-agent binary is now built with Golang 1.7 giving us support for macOS Sierra
• 💨 The agent now talks HTTP2 making calls to the Agent API that little bit faster
• 📦 The binary is a statically compiled (no longer requiring libc)
• meta-data-ec2 and meta-data-ec2-tags can now be configured using BUILDKITE_AGENT_META_DATA_EC2 and BUILDKITE_AGENT_META_DATA_EC2_TAGS environment variables

(Possible)? Breaking changes

We've removed our dependency of libc for greater compatibly across *nix systems which has had a few side effects:

• We've had to remove support for changing the process title when an agent starts running a job since it required libc to work. This feature has only ever been available to users running 64-bit Ubuntu. We'd like to bring this feature back in the future in a way that doesn't have us relying on libc
• The agent will now use Golangs internal DNS resolver instead of the one on your system. This probably won't effect you in any real way, unless you've setup some funky DNS settings for agent.buildkite.com

The release has been published to the Buildkite package repositories. For instructions on upgrading your agent please see the Upgrading section in the corresponding section of our installation documentation.

Today we’ve shipped some changes to simplify the Buildkite invitation system, including revamped invitation emails, automatic email address verification and a new screen for you to double check invitations before accepting them.

You can get the lowdown on our Building Buildkite blog post 🌟

Update: We accidentally introduced a bug in this release. Please upgrade to the latest agent instead.

It's a cold winter's day here in Melbourne, but never too cold for new Buildkite Agent release: v2.1.14 (Frosty) ⛄️

What’s new since v2.1.13?

• 🔍 SSH key scanning should be more resilient, whether or not you hash your known hosts file
• 🏅 Commands executed by the Bootstrap script correctly preserve positional arguments and handle interpolation better
• 🌈 ANSI color sequences are a little more resilient
• ✨ Git clean and clone flags can now be supplied in the Agent configuration file or on the command line
• 📢 Docker Compose will now be a little more verbose when the Agent is in Debug mode
• 📑 $BUILDKITE_DOCKER_COMPOSE_FILE now accepts multiple files separated by a colon (:), like $PATH

The release has been published to the Buildkite package repositories. For instructions on upgrading your agent please see the Upgrading section in the corresponding section of our installation documentation.

For a full list of what's changed see the GitHub release page