Securing your Buildkite Agent

In cases where a Buildkite Agent is being deployed into a sensitive environment there are a few default settings and techniques which may be adjusted.

Securely storing secrets

For best practices and recommendations about secret storage in the Agent, see the Managing Secrets guide.

Disabling automatic ssh-keyscan

By default the agent will automatically accept the Git SSH host using the ssh-keyscan command when doing the first checkout on a new agent host. The agent runs a similar command to this:

ssh-keyscan "<host>" >> "~/.ssh/known_hosts"

If you choose to disable this functionality, you'll need to manually perform your first checkout, or ensure the SSH fingerprint of your source code host is already present on your build machine.

This operation can be disabled by setting no-ssh-keyscan in one of the following ways:

  • Environment variable: BUILDKITE_NO_SSH_KEYSCAN=true
  • Command line flag: --no-ssh-keyscan
  • Configuration setting: no-ssh-keyscan=true

Disabling command eval

By default the agent allows you to run any command on the build server (e.g. make test). You can disable command evaluation and allow only the execution of scripts (with no ability to pass command line flags). Disabling command evaluation will also disable plugin execution.

Once disabled your build steps will need to be checked into your repository as scripts, and the only way to pass arguments is via environment variables.

To disable command line evaluation use one of the following settings:

  • Environment variable: BUILDKITE_NO_COMMAND_EVAL=1
  • Command line flag: --no-command-eval
  • Configuration setting: no-command-eval=true

Custom hooks

If you have a custom 'command' hook, using no-command-eval will have no effect on your command execution. See Allowing a List of Plugins and Custom bootstrap scripts for examples of how to completely lock down your agent from arbitrary code execution.

Disabling plugins

As plugins execute in the same way as local hooks, they can pose a potential security risk. If you're using third party plugins, you'll be executing the third party's code on your agent.

You can disable plugins with the command line flag: --no-plugins.

Disabling local hooks

If you have configured your agent to do very specific things with global hooks (i.e. run commands within a Docker container or a chroot environment), you probably don't want them overridden with local ones. Disabling local hooks will also disable plugins.

You can disable local hooks with the command line flag: --no-local-hooks.

Strict checks using agent hooks

You can use the agent’s environment hook to add strict checks for what repositories, commands, and plugins are allowed to run. The environment hook will be run before any source code is checked out onto the machine, and before any commands are executed.

For example, the following environment hook allows only a single file from a single repository to be executed on the agent machine:

#!/bin/bash
set -euo pipefail

if [[ "${BUILDKITE_REPO}" != "git@server:repo.git" ]]; then
  echo "Repository not allowed: ${BUILDKITE_REPO}"
  exit 1
fi

if [[ "${BUILDKITE_COMMAND}" != "some-script.sh" ]]; then
  echo "Command not allowed: ${BUILDKITE_COMMAND}"
  exit 1
fi

Allowing a list of plugins

Using the agent’s environment hook, you can create a list of plugins that an agent is allowed to run using the BUILDKITE_PLUGINS environment variable. For example of how to do this, see the buildkite/buildkite-allowed-plugins-hook-example repository on GitHub.

Customizing the bootstrap

The Buildkite Agent comes with a default bootstrap script, but can be configured to run your own. Providing your own bootstrap provides the highest level of security and control of your agent machine. You can use it to customize your agent, sanitize command output, and implement your own security logic.

The Buildkite Agent is separated into a daemon executable and a bootstrap executable. The daemon is responsible for communicating with the Buildkite API and executing the bootstrap. The bootstrap is responsible for checking out source code, calling hooks, running commands, and uploading build artifacts. The bootstrap is passed environment variables by the daemon process, and has its output streams and exit status captured.

For example, the following custom bootstrap will print out the environment variables passed by the main agent process, print a hello world, and exit with a failure status:

#!/bin/bash
set -euo pipefail

env

echo "Hello world"

exit 1

Forcing clean checkouts

By default Buildkite will reuse (after cleaning) a previous checkout. This may not be safe if building commits from untrusted sources (e.g. 3rd party pull requests). To force a clean checkout every time, set BUILDKITE_CLEAN_CHECKOUT=true in the environment.

Disallowing hooks in the repository

The default bootstrap script will execute hooks from the agent configuration (global) and the repository (local). Running local hooks may be undesirable if you are building commits from untrusted sources (e.g. 3rd party pull requests).

Local hooks can be disabled by setting BUILDKITE_NO_LOCAL_HOOKS=true in the environment.

If local hooks are disabled and one is in the checkout, the job will fail.

NOTE: If building untrusted commits, you should also contain the build scripts and anything else that may be influenced by the repository contents within chroots, containers, VMs, etc as is appropriate for your needs.

Running the Agent behind a proxy

To run the agent behind a proxy, you'll need to export the following proxy environment variables for your process manager:

  • http_proxy
  • https_proxy

Both of these variables should be set to the URL for your proxy server.

For example, if using systemd, create a directory named "/etc/systemd/system/buildkite-agent.service.d" that contains a proxy.conf file.

An example systemd proxy.conf file:

[Service]
# Proxy Env Vars
Environment=http_proxy=http://username:password@proxyserver:8080/
Environment=https_proxy=http://username:password@proxyserver:8080/

After creating this file, systemd will require a reload and the buildkite-agent service will require a restart.