Managing pipeline secrets
When you need to use secret values in your pipelines, there are some best practices you should follow to ensure they stay safely within your infrastructure and are never stored in, or sent to, Buildkite.
There are also various Buildkite plugins that integrate reading and exposing secrets to your build steps using secrets storage services.
If you don't use a secrets storage service, then you can use the Buildkite agent's
environment hook to export secrets to a job.
environment hook is a shell script that is sourced at the beginning of a job.
It runs within the job's shell, so you can use it to conditionally run commands and export secrets within the job.
By default, the
environment hook file is stored in the agent's
The path to this directory varies by platform; read the installation instructions for the path on your platform.
The path can also be overridden by the
For example, to expose a Test Analytics API token to a specific pipeline, create an
environment script in your agent's
hooks directory that checks for the pipeline slug before exporting the secret:
Adding conditional checks, such as the pipeline slug and step identifier, helps to limit accidental disclosure of secrets.
For example, suppose you have a step that runs a script expecting a
SECRET_DEPLOYMENT_ACCESS_TOKEN environment variable, like this one:
environment hook, you can export the deployment token only when when the job is the deployment step in a specific pipeline:
The script exports
SECRET_DEPLOYMENT_ACCESS_TOKEN only for the named pipeline and step.
Since this script runs for every job, you can extend it to selectively export all of the secrets used on that agent.
To store secrets when using the Elastic CI Stack for AWS, place them inside your stack's encrypted S3 bucket.
Unlike hooks defined in agent
the Elastic CI Stack for AWS's
env hooks are defined per-pipeline.
For example, to expose a
variable to a step with identifier
trigger-github-deploy, you would create the
env file on your local development machine:
You then upload the
env file, encrypted, into the secrets S3 bucket with the
# Upload the env
aws s3 cp --acl private --sse aws:kms env "s3://elastic-ci-stack-my-stack-secrets-bucket/my-app/env"
# Remove the original file
See the Elastic CI Stack for AWS readme for more information and examples.
You should never store secrets on your Buildkite Pipeline Settings page. Not only does this expose the secret value to Buildkite, but pipeline settings are often returned in REST and GraphQL API payloads.
You should never store secrets in the
env block at the top of your pipeline steps, whether it's in a
pipeline.yml file or the YAML steps editor.
You should never refer to secrets directly in your
pipeline.yml file, as they may be interpolated during the pipeline upload and sent to Buildkite. For example:
Referencing secrets in your steps risks them being interpolated, uploaded to Buildkite, and shown in plain text in the "Uploaded Pipelines" list in the job's Timeline tab.
The Buildkite agent does redact strings that match the values off of environment variables whose names match common password patterns such as
To prevent the risk of interpolation, it is recommended that you replace the command block with a script in your repository, for example:
Use build scripts instead of
command blocks for steps that use secrets.
If you must define your script in your steps, you can prevent interpolation by using the