Secure your first production pipeline

Use these steps as a starting configuration for one production pipeline in Buildkite Pipelines, then add controls for its risk profile. This example isn't a complete security assessment. Each step ends with a check that confirms the control works. The steps apply to every Buildkite plan, unless a step names a plan-specific option and its alternative.

Before you start

Start with an existing pipeline connected to a repository you control. If you haven't created one yet, follow Getting started with Pipelines. You'll need a Buildkite organization administrator for organization security settings and access to configure teams, clusters, and the pipeline.

Decide whether to use Buildkite hosted or self-hosted agents. Review the shared responsibilities for your hosting model before configuring production access.

Configure and verify the pipeline

Require SSO or 2FA: On the Pro and Enterprise plans, configure single sign-on (SSO) with your identity provider. On other plans, or if you don't use SSO, select Settings > Security, select Enforce Two-factor authentication, and then select Update Access Control. Learn more in Enforce two-factor authentication.

To verify, log in as a member of your Buildkite organization in a private browser window, and confirm that you're asked to sign in through your identity provider or enter a 2FA code.
Give the pipeline its own team: Enable teams, create a team for the people who maintain the pipeline, and give that team Full Access to the pipeline. Remove the pipeline from the Everyone team, and give any other team that needs the pipeline Build & Read or Read Only access.

To verify, log in as a user who isn't a member of the new team, and confirm that they can't edit the pipeline's settings.
Run the pipeline in its own cluster: Create a cluster, create a queue with the key production, and move the pipeline to that cluster. Choose the queue type for your hosting model:

For Buildkite hosted agents, create a Buildkite hosted queue with the key production. Buildkite manages the agent credentials; you don't create an agent token.

For self-hosted agents, create a self-hosted queue with the key production, and configure your agents to connect to it. Create the agent token with an expiration date using the REST API, and store it in your secrets manager:
```
curl -H "Authorization: Bearer $TOKEN" \
  -X POST "https://api.buildkite.com/v2/organizations/{org.slug}/clusters/{cluster.id}/tokens" \
  -H "Content-Type: application/json" \
  -d '{
    "description": "Production agents",
    "expires_at": "2027-01-01T00:00:00Z"
  }'
```
A cluster created in the Buildkite interface starts with a self-hosted default queue. The examples below explicitly select production for both the upload and deployment steps, so neither relies on the default queue.

To verify, confirm that the pipeline belongs to the new cluster and the queue key is production. For self-hosted agents, confirm that only your production agents are connected to that queue and that the cluster's Agent Tokens page shows the token's expiration date. For hosted agents, confirm that the queue is a Buildkite hosted queue with the intended image settings.
Keep the pipeline steps in your repository: Replace the steps in the pipeline's YAML steps editor with a single upload step, and define the rest of the steps in a .buildkite/pipeline.yml file, which is protected by branch protection rules in your source control system. Learn more in Defining your pipeline steps.
```
steps:
  - label: ":pipeline: Pipeline upload"
    command: buildkite-agent pipeline upload --reject-secrets
    timeout_in_minutes: 5
    agents:
      queue: production
```
The --reject-secrets option enables the pipeline upload secret check on agent v3.

To verify, open a pull request that changes .buildkite/pipeline.yml, and confirm that it can't be merged without a review. Run a build and confirm that the upload step runs on the production queue rather than waiting on default.
Provide secrets at runtime: Create each secret as a Buildkite secret in the pipeline's cluster, and load it only in the step that needs it. Buildkite secrets are redacted from job logs automatically. Also set a command timeout on the step to limit how long it can run:
```
steps:
  - label: "Deploy"
    command: scripts/deploy.sh
    timeout_in_minutes: 30
    secrets:
      - DEPLOY_API_TOKEN
    agents:
      queue: production
```
Replace scripts/deploy.sh with your deployment script, and DEPLOY_API_TOKEN with the secret it needs.

To verify, search the pipeline's settings and pipeline.yml file for the secret's value, and confirm that it doesn't appear. After the step runs, confirm that the value doesn't appear in the job log, or appears only as [REDACTED].
Block builds from untrusted sources: In the pipeline's GitHub settings, keep Allow builds from third-party forked repositories cleared, so that a fork can't run modified steps with access to your secrets. Learn more in Running builds on pull requests.

To verify, confirm that the option is cleared, and that no build in the pipeline's history was created from a third-party fork.
Write and test a response runbook: Record how to remove a compromised user, rotate DEPLOY_API_TOKEN, and revoke agent tokens if you use self-hosted agents, and who to contact at Buildkite. Learn more in Incident response.

To verify, rehearse credential rotation in a test environment using only the runbook, and confirm that builds still pass and the old credential no longer works. For self-hosted agents, also rehearse replacing an agent token and revoking the old token, and confirm that the agents reconnect.

Review the remaining controls

After you complete these steps, review the remaining baseline controls in each area of the Security checklist and apply those relevant to your workload. Use your risk profile to select additional controls and track outstanding work.