Buildkite Agent Plugins

Plugins provide a way to extend the functionality that Buildkite natively offers. They act as a external repository of Hooks that are referenced from your pipeline.yml file, downloaded and run as part of your builds.

Plugins can be hosted on GitHub, BitBucket or a private git repository, and are automatically fetched as required.

Buildkite offers a collection of useful plugins to get you started, including a Docker Compose plugin and a Junit Annotate plugin.

Using a plugin

Plugins are only available for pipelines using pipeline.yml files.

Plugins can be used in two ways:

  • as a standalone step, performing an operation
  • as a library, making commands available to that step's script

An example of what a plugin might look like in your code:

steps:
  - plugins:
      detect-clowns#v2.0.0: ~

This example will install the buildkite-plugins/detect-clowns-buildkite-plugin when your pipeline runs this build step.

Plugins are automatically fetched from where they are hosted, based on the name of the project. See Plugin sources for how you can host your own.

A real-life example is the Docker Compose plugin. It provides an integration with docker-compose that assists with running your builds within a docker environment:

steps:
  # Prebuild the app image, upload it to a registry for later steps
  - name: "Docker Build"
    plugins:
      docker-compose#v1.3.2:
        build: app
        image-repository: index.docker.io/org/repo

  - wait

  # Use the app image built above to run concurrent tests
  - name: "Docker Test %n"
    command: test.sh
    parallelism: 25
    plugins:
      docker-compose#v1.3.2:
        run: app

Plugin sources

If you refer to a plugin just by name (say detect-clowns), it defaults to https://github.com/buildkite-plugins/detect-clowns-buildkite-plugin. If you used myorg/detect-clowns it would refer to https://github.com/myorg/detect-clowns-buildkite-plugin.

You can also use fully qualified Git URLs like:

  • https://github.com/my-org/my-plugin.git#v1.0.0
  • ssh://git@github.com/my-org/my-plugin.git#v1.0.0
  • file:///a-local-path/my-plugin.git#v1.0.0

This syntax lets you refer to your own private Git repositories for plugin locations. Branches, tags and commits are all valid after the #.

We recommend that plugins be pinned to a specific version to prevent behaviour changing unexpectedly. This is done with a git tag of vX.X.X.

Developing a plugin

The best place to get started is the Buildkite Plugins repository. You can also find third-party plugins with the buildkite-plugin tag on GitHub.

Anatomy of a plugin

Plugins are essentially a collection of hooks with extra configuration. The agent passes the plugin configuration from the pipeline.yml to the plugin hooks as environment variables.

A good example is the docker plugin, which runs a command within a docker container. The pipeline invocation of this plugin looks like:

steps:
  - command: yarn install && yarn run test
    plugins:
      docker#v1.0.0:
        image: node
        workdir: /app

In its root directory the plugin has a plugin.yml that describes what it does, the executables required on the agent machine, and a schema of the plugin’s configuration options:

name: Docker
description: Runs build steps within a docker container.
author: https://github.com/buildkite
requirements:
  - docker
configuration:
  properties:
    image:
      type: string
    workdir:
      type: string
  required:
    - image
    - workdir

The plugin’s hook scripts are placed in the hooks/ directory. Here’s the simplified logic of the command hook, which would exist at hooks/command:

#!/bin/bash
set -euo pipefail

echo "--- Running ${BUILDKITE_COMMAND} in ${BUILDKITE_PLUGIN_DOCKER_IMAGE}"

docker run -it --rm \
  --volume "${PWD}:${BUILDKITE_PLUGIN_DOCKER_WORKDIR}" \
  --workdir "${BUILDKITE_PLUGIN_DOCKER_WORKDIR}" \
  "${BUILDKITE_PLUGIN_DOCKER_IMAGE}" bash -c "${BUILDKITE_COMMAND}"

The environment variables are named BUILDKITE_PLUGIN_{NAME}_{KEY}, or if a list is provided then BUILDKITE_PLUGIN_{NAME}_{KEY}_{IDX}.

For example image: "node" becomes the environment variable BUILDKITE_PLUGIN_DOCKER_IMAGE="node", and links: [ "db", "redis" ] becomes the enviroment variables BUILDKITE_PLUGIN_DOCKER_LINKS_0="db" and BUILDKITE_PLUGIN_DOCKER_LINKS_1="redis".

Plugin tools

The following are some helpful tools when developing plugins:

  • Plugin Tester - A Docker image containing tools that simplifies writing BATS tests, and getting your plugin tested (so meta).
  • Plugin Linter - A linter that checks your plugin for best practices.
  • Shellcheck Plugin - A plugin for running Shellcheck on your hook scripts.
  • JSON Schema Lint - A tool that allows for validating your JSON schema with YAML.
  • JSON Schema - The JSON Schema spec.
  • Understanding JSON Schema - A tutorial to help understand how to write JSON Schema.
  • bksr - A command line tool for running Buildkite pipeline steps locally.

Security and reliability

Plugins introduce a third-party dependency in your pipeline. They provide a way to introduce executable code into your pipeline beyond scripts in your repository and access to the pipeline.yml.

Recommended security practices:

  • Always pin to a particular version of the plugin, e.g. docker-compose#287293c4. You can refer to any git ref. The agent will check this out the first time it encounters it and then re-use that checked out code.

  • Follow the Unofficial Bash Strict Mode and check your work with shellcheck.

  • Apply special care to avoid executing arbitrary code passed in to plugins by config. As these are just environment variables, anything that can set environment variables can mess with plugin configuration.

For particularly high-security setups, you can disable plugins selectively by removing them from the BUILDKITE_PLUGINS variable in your environment hook, or run agents with the --no-plugins flag.

We love feedback

Agent plugins are still new, and we are evolving them as we learn more. We'd love feedback on how you use them and how we might make them better.