We've had a lot to talk about at Buildkite lately! We relaunched our blog, we've been updating and revamping our documentation, and have a ton of upcoming projects, customer stories, and tutorials we can't wait to share.

If that sounds like a lot to keep track of, we've got you covered! We're launching a 📰 newsletter 📰 to provide curated highlights, so you can stay in the know whatever your schedule ✨

Sign up here and you'll get a handy digest of our favorite features, updates, and blog posts delivered directly to your inbox 🗞📬🚀

For the latest news as it happens, this changelog will remain your one-stop source for Buildkite updates. You can also check out the blog and our docs directly to stay up to date on all things Buildkite 💚

Nobody likes accidentally leaking secrets in a build log. The Buildkite agent can now redact secrets present in your environment variables before they are uploaded to your logs 👷🏻

Turn it on for your agents, and add your custom variable patterns.

Thanks to Buildkite customer David Baggerman for doing the initial work on this feature 💚

With the release of v3.28.1 of the Buildkite Agent, we've introduced a new command to remove existing annotations from a build 📝🚫

buildkite-agent annotation remove removes an annotation entirely, where previously you could only blank their content by passing an empty string to buildkite-agent annotate.

This allows for some creative uses like showing bits of context while builds, tests, deployments or lints run, and removing them from view when they're no longer relevant.

We're looking forward to hearing about what you make with the extra annotation abilities 😄🎉

For teams needing detailed observability in their CI/CD pipelines, we've recently released support for configuring Datadog as a tracing backend on the Buildkite agent to help monitor build and job metrics 🕵🏻

Huge thanks to our open source collaborators 💚

To get started, install the Datadog Agent on your Buildkite Agent machine(s) and configure the following environment variable: BUILDKITE_TRACING_BACKEND="datadog"

For more details, check out the agent configuration docs.

To support customers with inbound network firewalls, Buildkite's Meta API lists the IP addresses that Buildkite uses to send outbound traffic such as webhooks and commit statuses.

To improve network resilience, new IP addresses were added to the Meta API earlier this month. If you use network ACL/firewall rules to allow-list traffic from Buildkite, these should be updated immediately.

During the week of 2021-04-19 traffic will intermittently be switched to the new IPs for short periods as a low-impact method of uncovering issues. During the following week of 2021-04-26 outbound traffic will be permanently switched to the new addresses.

Folks using Terraform to manage their infrastructure, great news! We've published an official Buildkite provider to the Terraform Registry

For details of the configuration syntax and supported resources, check out the official docs on the Terraform Registry.

A big thanks to Buildkite customer Jarryd Tilbrook who developed the initial version of the provider and was happy for us to pick it up 💚

If you only want to build tags, you can now optionally switch off building branches 🚦

We've added a new setting: Build branches that is on by default for new pipelines. To only build on tags, disable Build branches and ensure that your Build tags box is checked:

You can find the Build tags and the new Build branches setting in your pipeline's Repository settings when using GitHub, GitHub Enterprise, Bitbucket, or Bitbucket Server.

We've heard that some Buildkite customers are considering ARM in production, so we've made sure that Elastic Stack for AWS v5.1.0 can help you get there ✨

v5.1.0 adds support for the following AWS instance types with an ARM-powered Graviton CPU:

• m6g.*
• m6gd.*
• t4g.*
• a1.*
• c6g.*
• c6gd.*
• r6g.*
• t6gd.*

To use these instance types, follow the upgrade guide for v5.1.0 and set your InstanceType stack parameter 🚀

Update, June 2021: We are no longer offering Buildkite-hosted macOS agents. For more information on spinning up your own Buildkite agents on macOS, see this guide from Stark and Wayne or check out the docs.

We are excited to announce an exclusive program for our macOS agents managed service, providing the first scalable CI platform for Mac and iOS development 🚀

To learn more and to resister your interest, check out our introduction to Hosted Mac Agents and our blog post about the announcement 📚✨

Those occasional times when you need to grab a copy of all the environment variables on a job, wouldn't it be great if there was a button for that? 🤔

Now there is! We've added a button to the job Environment tab to copy all of your environment variables and their values straight into your clipboard 🚀

Apple machines with the new ARM-based M1 chip are being delivered to Buildkite customers, and starting from agent 3.26.0 we have an experimental binary that will run natively on these systems.

Users with M1 based hardware can install the native binary by:

• Using Homebrew
• Running our curl install script
• Downloading the darwin-arm64 binary directly from GitHub

If you manage a lot of pipelines you will be pleased to hear that we've added the ability to archive pipelines that are no longer in use. Archived pipelines have all their historical data preserved, but won't run new builds, and are not shown on the Pipelines page by default. You'll find a list of archived pipelines in the Team Selector:

Archive pipelines from the Settings page, under Pipeline Management, or using the GraphQL API.

Happy Archiving!

As part of the transition to GraphQL, we have added graphql_id for the following REST API resources. The graphql_id is used to target specific resources when running queries and mutations on GraphQL API.

• Agent
• Build
• Job
• Organization
• Pipeline
• Team
• User

Here is an example response from pipelines which includes graphql_id:

1
2
3
4
5
{
  "id": "849411f9-9e6d-4739-a0d8-e247088e9b52",
  "graphql_id": "UGlwZWxpbmUtLS1lOTM4ZGQxYy03MDgwLTQ4ZmQtOGQyMC0yNmQ4M2E0ZjNkNDg=",
  "url": "https://api.buildkite.com/v2/organizations/acme-inc/pipelines/my-pipeline"
}

If you have any questions or feedback, please let us know in the Forum or send us an email 📨

Establishing a shared timeline when troubleshooting or digging into recent builds can be a challenge, especially for distributed teams. To help make this a little easier, we’ve added the ability to toggle between local time and UTC by clicking on a timestamp ⏱

As an added bonus, UTC times include the full date and are shown in 24-hour time regardless of your browser’s default locale settings ✨

We've updated our GitHub app, adding the option to automatically create webhooks for new pipelines 🔀

The Buildkite GitHub App now additionally requests the webhook read & write permission for Repositories. We'll only use this permission to create new webhooks when you explicitly check the 'Auto-create webhooks' box during pipeline setup 🔏

Teams with large repositories, we'd love for you to try out the git-mirrors experiment flag, available on all agents v3.10.0 and above. This enables you to have a single git clone per host, to help speed up git operations and reduce network and disk usage 👯

You can get started using git mirrors by setting the git-clone-mirror-flags experimental agent config option.

Let us know about any questions or issues on the Shared Git Checkouts topic in the Buildkite Community Forum 📝

With the new block step attribute, blocked_state, you can specify what state the build should have when it’s blocked by each block step — green passed, yellow running, or red failed.

When you’ve got a block step that requires user input, having the build look or behave like it's passed isn’t always what you want. Setting blocked_state will affect the build state within the Buildkite UI, source provider integrations, webhooks, and notifications. This gives you more control over how your builds interact with PRs and custom integrations.

Add the new blocked_state attribute to your block steps to change the blocked status colour from the default green.

You can now mix positive and negative branch filter patterns for more precise control over which branches to build 🔀

We've done our best to reach out via email if your pipeline already had a mix of patterns that will be affected by this change 👋🏻

If you're seeing unexpected builds, or are left waiting for a build that isn't showing up, we'd suggest a quick review of your pipeline's branch filter patterns.

For more information on branch filter patterns and some example code, check out the updated branch configuration docs.

We've released a new major version of the Elastic CI Stack 🌈

The latest version includes:

• Experimental Windows Server 2019 support
• Support for four new regions: ap-east-1 (Hong Kong), me-south-1 (Bahrain), af-south-1 (Cape Town), eu-south-1 (Milan)
• Support for managing Agent tokens with AWS SSM
• A blazing fast Agent Auto Scaler by AWS Lambda - up to 300% faster initial scale-ups compares to previous AutoScaling rules

And much, much more! Check out the full changelog for this release on GitHub.

We'd love for you to share any feedback you have on the v5 announcement post in our forum 🤓

Thank you to the many users who provided bug reports and contributions that made this release possible 💚

Debugging job and agent issues has become a whole lot easier with the shiny updates we've just released to the Timeline tab 📖

The new and improved Timeline tab has:

• Live updating with agent events
• Events from the dispatcher showing when the jobs are scheduled and assigned, and
• All your job info in one place including concurrency groups, step dependencies, and conditionals

You can find all of these updates under the Timeline tab of any job 🕵🏻‍♀️