Managing Pipeline Secrets
When you need to use secret values in your pipelines, there are some best practices you should follow to ensure they stay safely within your infrastructure and are never stored in, or sent to, Buildkite.
On this page:
Using a secrets storage service
There are also various Buildkite Plugins that integrate reading and exposing secrets to your build steps using secrets storage services.
Storing secrets in environment hooks
If you don’t use a secrets storage service, the recommended place to store secrets is in agent environment hooks. Agent hooks are stored on your agent machine, are only accessible by you, and can conditionally expose secrets to your pipeline steps.
For example, say you had the following deploy command step in a pipeline:
This step runs the
scripts/trigger-github-deploy script from the repository, and creates a GitHub Deployment using a GitHub personal access token. The script has the following source code:
To set up the
GITHUB_MY_APP_DEPLOYMENT_ACCESS_TOKEN secret for the step, you would create an
environment agent hook on your agent machine to conditionally export it. For example:
Adding conditional checks, such as the pipeline slug and the step identifier, helps to limit accidental use and disclosure of secrets.
Storing secrets with the Elastic CI Stack for AWS
To store secrets when using the Elastic CI Stack for AWS, place them inside your stack’s encrypted S3 bucket. Unlike regular agent hooks, the Elastic CI Stack’s
env hooks are per-pipeline.
For example, to expose a
GITHUB_MY_APP_DEPLOYMENT_ACCESS_TOKEN environment variable to a step with identifier
trigger-github-deploy, you would create the following
env file on your local development machine:
You then upload the
env file, encrypted, into the secrets S3 bucket with the following command:
# Upload the env aws s3 cp --acl private --sse aws:kms env "s3://elastic-ci-stack-my-stack-secrets-bucket/my-app/env" # Remove the original file rm env
See the Elastic CI Stack for AWS readme for more information and examples.
Anti-pattern: Storing secrets in your pipeline settings
You should never store secrets on your Buildkite Pipeline Settings page. Not only does this expose the secret value to Buildkite, but pipeline settings are often returned in REST and GraphQL API payloads.
Never store secret values in your Buildkite pipeline settings.
Anti-pattern: Storing secrets in your pipeline.yml
You should never store secrets in the
env block at the top of your pipeline steps, whether it's in a
pipeline.yml file or the YAML steps editor.
Never store secrets in the
env section of your pipeline.
Anti-pattern: Referencing secrets in your pipeline YAML
You should never refer to secrets directly in your
pipeline.yml file, as they may be interpolated during the pipeline upload and sent to Buildkite. For example:
Referencing secrets in your steps risks them being interpolated, uploaded to Buildkite, and shown in plain text in the "Uploaded Pipelines" list in the job’s Timeline tab.
The Buildkite agent does redact strings that match the values off of environment variables whose names match common password patterns such as
To prevent the risk of interpolation, it is recommended that you replace the command block with a script in your repository, for example:
Use build scripts instead of
command blocks for steps that use secrets.
If you must define your script in your steps, you can prevent interpolation by using the