Enforcing security controls

Authentication and session security in the Buildkite interface, APIs and CLI

Risk: Unauthorized access through credential compromise, user impersonation, session hijacking, overprivileged API keys.

Controls:

• Enforce either Single sign-on (SSO) or Two-factor authentication (2FA/MFA) for all access to the Buildkite interface.
• Use time-scoped API tokens with automated rotation.
• Apply least privilege principle when scoping API keys.
• Restrict API tokens to specific IP ranges where possible.

Source code security and version control integrity

Risk: Compromised repository access, unsigned commits, unauthorized branch access.

Controls:

• Use the Buildkite GitHub App integration for secure repository connections.
• Enforce SCM signed commits and branch protection rules on GitHub and GitLab with Buildkite Pipelines conditionals.
• Map Buildkite users to SCM identities with team-based permissions. Use agent hooks to ensure only authorized team members can trigger builds. You can also see a live example to discover how agent hooks operate in builds.
• Utilize programmatic team management alongside pre-merge hooks to verify that commit authors have appropriate permissions before allowing build execution.
• Disable triggering builds on forks for public pipelines and repositories to ensure open source contributors are unable to substantially alter a pipeline to extract secrets.

Dependencies and package management

Risk: Malicious or typosquatted packages that can execute arbitrary code during builds, vulnerable dependencies that persist in packaged images and production deployments.

Controls:

• Integrate with a container scanning tool to keep track of a software bill of materials (SBOM) for your packages. For example, see the following list of community-maintained SBOM generation tools.
• Use Buildkite's official security and compliance plugins (or write your own plugin) to integrate with your existing security scanning infrastructure for source code, container testing, and vulnerability assessment.
• Run automated dependency and malware scanning on every merge using established tools such as GuardDog, Snyk, Aqua Trivy (also available as a Trivy Buildkite plugin), or cloud security services across your software supply chain.
• Use pipeline templates (a Buildkite Enterprise-only feature), dynamic pipelines, and agent hooks to ensure security scans cannot be bypassed by modifying pipeline.yml files. Use pipeline templates to standardize security testing across all the pipelines in a Buildkite organization.
• Track dependencies using Buildkite Annotations to document exact package versions in each build. This creates an auditable record enabling targeted remediation when vulnerabilities are discovered.
• Establish automated response workflows that trigger notifications and remediation processes when critical CVEs are identified.

Secrets management

Risk: Exposed secrets in logs, environment variables, or compromised agents.

Controls:

• Leverage Buildkite's secrets plugins for secrets management and the Buildkite secrets feature to ensure that secrets are only available where explicitly required. Note that Buildkite automatically redacts secrets in logs.
• Integrate external secrets management using dedicated secrets storage services such as AWS Secrets Manager or HashiCorp Vault.
• Export secrets with environment hooks for agent-level secrets, rather than injecting them at build runtime. If you absolutely have to inject your secrets at runtime, avoid storing them as static environment variables.
• Establish environment-specific cluster and queue segmentation of your builds to restrict access so that builds in a queue can only access the secrets they require to run.
• Monitor how secrets are accessed within your CI/CD environment by reviewing the Audit Log.
• Use additional secret scanning tools such as git-secrets to prevent accidental commits of secrets to repositories before they enter the build process.
• Consider using strict pipeline upload guards, such as the reject-secrets option for buildkite-agent pipeline upload commands.
• Have incident response procedures for secret compromise, including automated revocation and rotation processes. Note that cluster maintainers can revoke tokens using the REST API for rapid containment.

Buildkite Agent security

Risk: Buildkite Agent compromise leading to privilege escalation, lateral movement, data access, malicious code execution.

Controls:

• Set granular command authorization controls for what the buildkite-agent user is allowed to run, restricting executable operations to predefined security parameters.
• Configure automated regular credential rotation, such as setting automatic expiration dates on your agent tokens to limit their window of opportunity to be compromised.
• Upgrade your Buildkite Agents on a regular basis.
• Deploy ephemeral build environments using isolated virtual machines or containers. Ensure that your deployment environment is secure by installing minimal operating systems, disabling inbound SSH access, and enforcing strict network egress controls.
• Isolate pipelines with sensitive builds to run on dedicated agent pools within their own cluster. This way, you're ensuring that critical workloads cannot be affected by compromise of less secure environments—for example, open-source repositories with unverified code.
• Enable pipeline signing and verification mechanisms.
• Set appropriate job time limits to limit the potential duration of malicious code execution on compromised agents.
• Utilize OIDC-based authentication to establish secure, short-lived credential exchange between agents and cloud infrastructure, leveraging session tags to add strong unique claims.
• Disable command evaluation where appropriate and enforce script-only execution instead.
• Learn more about making your virtual machine or container running the buildkite-agent process more secure in Securing your Buildkite Agent.

API access token compromise

Risk: Compromised or overprivileged Buildkite API access tokens enabling unauthorized pipeline access, code execution, and data theft.

Controls:

• Create API access tokens with only the minimal required scopes. Use portals to limit GraphQL query scope. Review permissions regularly to match current needs.
• Establish rotation of access tokens with defined expiration periods. Automate rotation where possible to limit exposure windows.
• Bind access tokens to specific IP addresses or network segments. Use network address translation (NAT) with centralized egress routing for enhanced monitoring and rapid compromise detection.
• Deploy access tokens within dedicated virtual private clouds (VPCs) using Buildkite’s Elastic CI Stack for AWS for network isolation.
• Monitor access token usage patterns through the Audit Log. Set up alerts on unusual patterns: unexpected locations, excessive API calls, unauthorized resource access.

Network and transport security

Risk: Interception of traffic between agents, the Buildkite API, and artifact storage, as well as data tampering, exposure, and unauthorized external communications potentially allowing malicious code injection.

Controls:

Buildkite enforces TLS encryption by default for all platform communications, ensuring traffic to and from Buildkite services is encrypted in transit. To further tighten your network security, you can take these additional steps:

• For self-hosted agents, implement a zero trust architecture (ZTA) with least-privilege egress rules.
• Monitor network traffic for anomalies or suspicious connection attempts from build agents.
• Consider taking your infrastructure fully into the cloud with the help of Buildkite hosted agents or by running your agents in AWS or in Google Cloud.
• Harden your cloud infrastructure perimeter using AWS PrivateLink or VPC endpoints for the AWS services, or Private Google Access for Google Cloud.

Artifact storage and integrity

Risk: Artifact tampering, data exfiltration, compromised deployments.

Controls:

• Enforce encryption at rest and in transit when storing and transferring build artifacts.
• Use cloud storage for storing build artifacts. You can use Buildkite Package Registries or other supported private cloud storage options:
  • AWS S3 buckets
  • Google Cloud Storage buckets
  • Azure Blob containers
• Implement artifact signing using Buildkite's SLSA provenance feature, or alternatively using in-toto or cosign, and establish verification processes before deployment to document artifact provenance and detect tampering.
• Enforce KMS signing of your pipelines.

Consistent pipeline-as-code approach

Risk: Inconsistent security implementations across teams and projects within your Buildkite organization, creating undetected security blind spots and gaps.

Controls:

• Adopt an infrastructure-as-code (IaC) approach and mandate the exclusive use of the Buildkite Terraform provider for all pipeline configuration management, implementing a mandatory two-reviewer approval process for infrastructure changes.

• Restrict pipeline configuration access to Buildkite organization administrators only by making your pipelines Read Only to your teams.
• Set zero-tolerance policies for manual pipeline overrides, with any unauthorized modifications triggering immediate alerts within your security information and event management (SIEM) system to ensure rapid incident response and maintain configuration integrity.
• Establish a "break glass" protocol that is tied to SIEM alerts in case someone has to make manual modifications to Buildkite's systems outside of the automated IaC workflow.
• Deploy agent-level lifecycle hooks as these cannot be bypassed or avoided through a pipeline.yml modification or other developer-level code change. You can also customize the hooks to scan your pipeline.yml files to validate their structure and contents, and ensure that those files conform to your Buildkite organization's security requirements.
• Use ephemeral Buildkite Agents (like the Buildkite Agent Stack for Kubernetes) or tools such as Ansible or Puppet to force configuration changes on persistent hosts.
• Mandate comprehensive security scanning (including container vulnerability and static code analysis scanning) and SBOM generation for all builds. For instance, use pipeline templates to ensure every pipeline in your Buildkite organization inherits predetermined configurations and maintains consistent baseline protections.
• Restrict plugin usage to private or version-pinned plugins to prevent supply chain attacks and ensure reproducible builds with known, vetted components.
• Use only verified Docker images.
• Scope pipelines to specific agent queues to maintain separation between environments and prevent unauthorized access across build processes.
• Use permission models to target appropriate agents for builds, ensuring sensitive workloads run only on designated, secured infrastructure.

Monitoring, anomaly detection, logging

Risk: Insufficient monitoring and logging resulting in undetected malicious activity, delayed incident response, and prolonged exposure to security threats within a CI/CD environment.

Controls:

• Export or stream all of your Buildkite Pipelines metrics to your preferred monitoring and observability platform to maintain visibility across CI/CD pipeline activities. Learn more about Buildkite Pipelines' observability platform integrations from the Observability section of the Integrations page (for example, see the OpenTelemetry integration capabilities in Buildkite).
• Set up Amazon EventBridge to consume Buildkite's Audit Log and integrate that information with your SIEM system.
• Monitor logs for anomalies (unusual IPs, secret access patterns, build frequency spikes) and configure automated alerts.

Incident response and recovery

Risk: Security incidents involving leaked secrets, compromised credentials, or unauthorized access to build environments.

Controls:

• Contact support@buildkite.com immediately upon discovering any security incident. Enterprise Premium Support customers can report an incident through their priority support channel. Early notification allows Buildkite to assist with immediate remediation steps.
• Buildkite's incident response team can audit access logs to identify which users and IP addresses accessed builds containing leaked information. For Enterprise plan customers, older logs can be rehydrated for in-depth forensic analysis.

Further questions

If you didn't find coverage of a security-related question that interests you here, feel free to contact support@buildkite.com.