Upgrading to Buildkite Agent v3

The Buildkite Agent has changed a lot in v3, but we've tried to keep it as backwards compatible as possible. We'll cover what’s changed and how to upgrade to new agents.

Overview of what has changed

Added:

• Plugins for easily sharing functionality between pipelines and customizing how agents behave
• Variable interpolation in pipeline.yml
• Build annotations
• pre-exit hook

Changed:

• Agent meta-data has been renamed to "tags"
• Much better Windows support, including .BAT hooks support
• Checkout clean no longer ignores files in .gitignore
• The bootstrap (run as a sub-process for every job) has moved from a shell script to buildkite-agent bootstrap. This means it's written in golang and cross-platform.

Deprecated:

• Built-in Docker and Docker Compose support has been deprecated. The same functionality is available from the dedicated plugins: docker-compose and docker.

Bootstrap customizations

If you customized your bootstrap.sh file, you will need to move the changes to hooks, or update your bootstrap.sh to call buildkite-agent bootstrap.

Docker and Docker Compose support

In v2 we supported a variety of environment variables like BUILDKITE_DOCKER_COMPOSE_CONTAINER and BUILDKITE_DOCKER. These are deprecated in favour of the docker-compose and docker pipeline plugin.

You can keep using the old environment variables in v3, but they will be removed in v4.

Steps using BUILDKITE_DOCKER_COMPOSE_CONTAINER

This is a step that uses the v2 BUILDKITE_DOCKER_COMPOSE_CONTAINER environment variable to run the command in a docker-compose container:

steps:
  - label: '🔨 Tests'
    command: 'scripts/tests.sh'
    env:
      BUILDKITE_DOCKER_COMPOSE_CONTAINER: app

The same action with the docker-compose plugin looks like this:

steps:
  - label: '🔨 Tests'
    command: 'scripts/tests.sh'
    plugins:
      - docker-compose#v1.8.4:
          run: app

Steps using BUILDKITE_DOCKER

This is a step that uses the v2 BUILDKITE_DOCKER environment variable to run the command in docker container:

steps:
  - label: '🔨 Tests'
    command: 'scripts/tests.sh'
    env:
      BUILDKITE_DOCKER: true

There isn't a direct conversion for this at present. You can either add a docker-compose file and use the docker-compose plugin, or if you want to run your build in a docker container without providing a Dockerfile or a docker-compose file, you can use the docker plugin:

steps:
  - label: '🔨 Tests'
    command: 'scripts/tests.sh'
    plugins:
      - docker#v1.1.1:
          image: "node:7"
          workdir: /app

Environment variables in your pipeline.yml

Previously we didn't support environment variable interpolation, such as ${MY_VARIABLE_NAME} or $MY_VARIABLE_NAME. If you have any of these in your pipeline.yml, they will now be interpolated. To render the literal text, you will need to escape the dollar signs, for example $$MY_VARIABLE_NAME.

See the environment variable substitution for more details.

Checkout clean no longer ignores files in .gitignore

Older agents didn't remove files from your working directory that were ignored by git. The new default values for git clean are -fxdq. If you've previously overridden your git-clean-flags in your config, it might be a good chance to comment them out and use the standard behavior.

Upgrading from a 2.0 agent

To upgrade, install the new 3.0 agent using one of the standard installation methods. To make installation easier, there are packages for each of the major operating systems.

You can test your updated agents in parallel to your existing agents by using agent tags to create a new queue for 3.0 builds.

Upgrading from 3.0-beta to a stable 3.0 agent

To upgrade a Ubuntu / Debian 3.0 beta agent:

• Edit /etc/apt/sources.list.d/buildkite-agent.list and replace the word unstable (or experimental) with stable
• Run sudo apt-get update && sudo apt-get upgrade -y buildkite-agent

To upgrade a Red Hat / CentOS 3.0 beta agent:

• Edit /etc/yum.repos.d/buildkite-agent.repo and replace the word unstable (or experimental) with stable
• Run sudo yum clean expire-cache && sudo yum update buildkite-agent

If you didn't install the agent using the above packages, update the agent like you did originally and you will get the latest stable version.