Managing pipeline secrets

Using a secrets storage service

If your team uses an external secrets storage service, such as AWS Secrets Manager or HashiCorp Vault, you can keep managing credentials there and retrieve them at runtime.

Buildkite provides various plugins that integrate reading and exposing secrets to your build steps using secrets storage services, such as the following. If a plugin for the service you use is not listed below or in the Buildkite plugins directory, please contact support.

A secrets storage service can be used with either self-hosted agents in your own infrastructure, as part of a hybrid Buildkite architecture, or with Buildkite hosted agents.

Without a secrets storage service

For self-hosted agents without access to a secrets store, the following approaches load credentials from agent infrastructure. Conditional injection limits accidental exposure. Enforce authorization separately using workload and credential boundaries.

Exporting secrets with environment hooks

You can use the Buildkite agent's environment hook to export secrets to a job.

The environment hook is a shell script that is sourced at the beginning of a job. It runs within the job's shell, so you can use it to conditionally run commands and export secrets within the job.

By default, the environment hook file is stored in the agent's hooks directory. The path to this directory varies by platform; read the installation instructions for the path on your platform. The path can also be overridden by the hooks-path setting.

For example, to expose a Test Engine API token to a specific pipeline, create an environment script in your agent's hooks directory that checks for the pipeline slug before exporting the secret:

#!/bin/bash
set -euo pipefail

if [[ "$BUILDKITE_PIPELINE_SLUG" == "pipeline-one" ]]; then
  export BUILDKITE_ANALYTICS_TOKEN="oS3AG0eBuUJMWRgkRvek"
fi

Adding conditional checks, such as the pipeline slug and step identifier, helps to limit accidental disclosure of secrets. For example, suppose you have a step that runs a script expecting a SECRET_DEPLOYMENT_ACCESS_TOKEN environment variable, like this one:

steps:
  - command: scripts/trigger-deploy
    key: trigger-deploy

In your environment hook, you can export the deployment token only when the job is the deployment step in a specific pipeline:

#!/bin/bash
set -euo pipefail

if [[ "$BUILDKITE_PIPELINE_SLUG" == "my-app" && "$BUILDKITE_STEP_KEY" == "trigger-deploy" ]]; then
  export SECRET_DEPLOYMENT_ACCESS_TOKEN="bd0fa963610b..."
fi

The script exports SECRET_DEPLOYMENT_ACCESS_TOKEN only for the named pipeline and step. Since this script runs for every job, you can extend it to selectively export all of the secrets used on that agent.

Storing secrets with the Elastic CI Stack for AWS

When using the Elastic CI Stack for AWS with your own AWS account and environment, you can store your secrets inside your stack's encrypted S3 secrets bucket. An env file at the root of the bucket applies to all pipelines using that bucket. To load secrets for a specific pipeline, put the file under a prefix matching the pipeline slug.

For example, to expose a GITHUB_MY_APP_DEPLOYMENT_ACCESS_TOKEN environment variable to jobs in the my-app pipeline, create the following env file on your local development machine:

GITHUB_MY_APP_DEPLOYMENT_ACCESS_TOKEN="xxx-yyy-zzz"

You then upload the env file, encrypted, into the secrets S3 bucket with the following command. Replace your-secrets-bucket with your bucket name:

# Upload the env
aws s3 cp --acl private --sse aws:kms env "s3://your-secrets-bucket/my-app/env"
# Remove the original file
rm env

This file exposes the token to every job in my-app, not just a deployment step. For per-step secret injection, use an agent environment hook instead of putting conditional shell logic in the S3 environment file.