buildkite-agent start

The Buildkite agent's start command is used to manually start an agent and register it with Buildkite.

Starting an agent

Usage

buildkite-agent start [options...]

Description

When a job is ready to run it will call the "bootstrap-script" and pass it all the environment variables required for the job to run. This script is responsible for checking out the code, and running the actual build script defined in the pipeline.

The agent will run any jobs within a PTY (pseudo terminal) if available.

Example

$ buildkite-agent start --token xxx

Options

--no-color #

Don't show colors in logging (default: false)
Environment variable: $BUILDKITE_AGENT_NO_COLOR
--debug #

Enable debug mode. Synonym for --log-level debug. Takes precedence over --log-level (default: false)
Environment variable: $BUILDKITE_AGENT_DEBUG
--log-level string #

Set the log level for the agent, making logging more or less verbose. Defaults to notice. Allowed values are: debug, info, error, warn, fatal (default: "notice")
Environment variable: $BUILDKITE_AGENT_LOG_LEVEL
--experiment string #

Enable experimental features within the buildkite-agent
Environment variable: $BUILDKITE_AGENT_EXPERIMENT
--profile string #

Enable a profiling mode, either cpu, memory, mutex or block
Environment variable: $BUILDKITE_AGENT_PROFILE
--config string #

Path to a configuration file
Environment variable: $BUILDKITE_AGENT_CONFIG
--name string #

The name of the agent
Environment variable: $BUILDKITE_AGENT_NAME
--priority string #

The priority of the agent (higher priorities are assigned work first)
Environment variable: $BUILDKITE_AGENT_PRIORITY
--acquire-job string #

Start this agent and only run the specified job, disconnecting after it's finished
Environment variable: $BUILDKITE_AGENT_ACQUIRE_JOB
--reflect-exit-status #

When used with --acquire-job, causes the agent to exit with the same exit status as the job (default: false)
Environment variable: $BUILDKITE_AGENT_REFLECT_EXIT_STATUS
--disconnect-after-job #

Disconnect the agent after running exactly one job. When used in conjunction with the --spawn flag, each worker booted will run exactly one job (default: false)
Environment variable: $BUILDKITE_AGENT_DISCONNECT_AFTER_JOB
--disconnect-after-idle-timeout int #

The maximum idle time in seconds to wait for a job before disconnecting. The default of 0 means no timeout (default: 0)
Environment variable: $BUILDKITE_AGENT_DISCONNECT_AFTER_IDLE_TIMEOUT
--disconnect-after-uptime int #

The maximum uptime in seconds before the agent stops accepting new jobs and shuts down after any running jobs complete. The default of 0 means no timeout (default: 0)
Environment variable: $BUILDKITE_AGENT_DISCONNECT_AFTER_UPTIME
--cancel-signal-timeout duration #

The amount of time given to a subprocess to handle the cancel signal before SIGKILL is sent (default: 10s)
Environment variable: $BUILDKITE_CANCEL_SIGNAL_TIMEOUT
--enable-job-log-tmpfile #

Store the job logs in a temporary file BUILDKITE_JOB_LOG_TMPFILE that is accessible during the job and removed at the end of the job (default: false)
Environment variable: $BUILDKITE_ENABLE_JOB_LOG_TMPFILE
--job-log-path string #

Location to store job logs created by configuring enable-job-log-tmpfile, by default job log will be stored in TempDir
Environment variable: $BUILDKITE_JOB_LOG_PATH
--write-job-logs-to-stdout #

Writes job logs to the agent process' stdout. This simplifies log collection if running agents in Docker (default: false)
Environment variable: $BUILDKITE_WRITE_JOB_LOGS_TO_STDOUT
--job-logs-otlp #

Export job logs directly as OpenTelemetry log records using the OTEL_EXPORTER_OTLP_LOGS_* / OTEL_EXPORTER_OTLP_* environment configuration (default: false)
Environment variable: $BUILDKITE_JOB_LOGS_OTLP
--shell string #

The shell command used to interpret build commands, e.g /bin/bash -e -c (default: "/bin/bash -e -c")
Environment variable: $BUILDKITE_SHELL
--hooks-shell string #

The shell command used to interpret hooks commands, e.g pwsh -Command
Environment variable: $BUILDKITE_HOOKS_SHELL
--queue string #

The queue the agent will listen to for jobs. If not set, the agent will use the default queue. Overwrites the queue tag in the agent's tags
Environment variable: $BUILDKITE_AGENT_QUEUE
--tags string #

A comma-separated list of tags for the agent (for example, "linux" or "mac,xcode=8")
Environment variable: $BUILDKITE_AGENT_TAGS
--tags-from-host #

Include tags from the host (hostname, machine-id, os) (default: false)
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_HOST
--tags-from-ec2-meta-data #

Include the default set of host EC2 meta-data as tags (instance-id, instance-type, ami-id, and instance-life-cycle)
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_EC2_META_DATA
--tags-from-ec2-meta-data-paths string #

Include additional tags fetched from EC2 meta-data using tag & path suffix pairs, e.g "tag_name=path/to/value"
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_EC2_META_DATA_PATHS
--tags-from-ec2-tags #

Include the host's EC2 tags as tags (default: false)
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_EC2_TAGS
--tags-from-ecs-meta-data #

Include the host's ECS meta-data as tags (container-name, image, and task-arn) (default: false)
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_ECS_META_DATA
--tags-from-gcp-meta-data #

Include the default set of host Google Cloud instance meta-data as tags (instance-id, machine-type, preemptible, project-id, region, and zone)
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_GCP_META_DATA
--tags-from-gcp-meta-data-paths string #

Include additional tags fetched from Google Cloud instance meta-data using tag & path suffix pairs, e.g "tag_name=path/to/value"
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_GCP_META_DATA_PATHS
--tags-from-gcp-labels #

Include the host's Google Cloud instance labels as tags (default: false)
Environment variable: $BUILDKITE_AGENT_TAGS_FROM_GCP_LABELS
--wait-for-ec2-tags-timeout duration #

The amount of time to wait for tags from EC2 before proceeding (default: 10s)
Environment variable: $BUILDKITE_AGENT_WAIT_FOR_EC2_TAGS_TIMEOUT
--wait-for-ec2-meta-data-timeout duration #

The amount of time to wait for meta-data from EC2 before proceeding (default: 10s)
Environment variable: $BUILDKITE_AGENT_WAIT_FOR_EC2_META_DATA_TIMEOUT
--wait-for-ecs-meta-data-timeout duration #

The amount of time to wait for meta-data from ECS before proceeding (default: 10s)
Environment variable: $BUILDKITE_AGENT_WAIT_FOR_ECS_META_DATA_TIMEOUT
--wait-for-gcp-labels-timeout duration #

The amount of time to wait for labels from GCP before proceeding (default: 10s)
Environment variable: $BUILDKITE_AGENT_WAIT_FOR_GCP_LABELS_TIMEOUT
--fail-on-missing-tags #

Exit the agent with an error if any enabled cloud tag source (EC2, ECS, GCP) fails to return tags (default: false)
Environment variable: $BUILDKITE_AGENT_FAIL_ON_MISSING_TAGS
--skip-checkout #

Skip the git checkout phase entirely
Environment variable: $BUILDKITE_SKIP_CHECKOUT
--checkout-override-mode string #

Controls which sources may override the agent's checkout settings; one of [strict from-job none]. strict makes the agent authoritative against pipeline/step env, secrets, hooks, plugins, and the Job API. from-job (default) lets hooks, plugins, and the Job API set checkout vars, blocks secrets, and keeps the agent's checkout flags authoritative over pipeline/step env; pipeline/step env may still set the checkout timeout, submodules, skip-checkout, and skip-fetch-existing-commits toggles that the agent leaves unset, matching earlier agent behavior, and may set the sparse-checkout paths and mode outright. none additionally lets pipeline/step env and secrets set them. All mirror configuration and submodule clone config stay agent-authoritative in every mode. The checkout SSH key and Git LFS toggle are not governed by this flag and stay job-settable in every mode. Disabling command-eval forces this to strict. (default: "from-job")
Environment variable: $BUILDKITE_CHECKOUT_OVERRIDE_MODE
--git-checkout-flags string #

Flags to pass to "git checkout" command (default: "-f")
Environment variable: $BUILDKITE_GIT_CHECKOUT_FLAGS
--git-clone-flags string #

Flags to pass to "git clone" command (default: "-v")
Environment variable: $BUILDKITE_GIT_CLONE_FLAGS
--git-clean-flags string #

Flags to pass to "git clean" command (default: "-ffxdq")
Environment variable: $BUILDKITE_GIT_CLEAN_FLAGS
--git-commit-verification string #

Verify that the commit being built exists on the specified branch; one of strict or off (default: "strict")
Environment variable: $BUILDKITE_GIT_COMMIT_VERIFICATION
--git-fetch-flags string #

Flags to pass to "git fetch" command (default: "-v --prune")
Environment variable: $BUILDKITE_GIT_FETCH_FLAGS
--git-sparse-checkout-paths string #

Comma-separated list of paths for git sparse checkout. When set, only the listed paths are materialized in the working tree. Paths are interpreted according to --git-sparse-checkout-mode.
Environment variable: $BUILDKITE_GIT_SPARSE_CHECKOUT_PATHS
--git-sparse-checkout-mode string #

Changes how the sparse checkout paths are interpreted; available modes are [cone no-cone]. cone treats each path as a directory to include, which is git's default and enables faster pattern matching, but it accepts only directory names. no-cone treats each path as a gitignore-style pattern, which allows globs and exclusions such as !/docs/, and requires git >= 2.35. Git LFS objects are not scoped to the sparse paths in no-cone mode. (default: "cone")
Environment variable: $BUILDKITE_GIT_SPARSE_CHECKOUT_MODE
--git-clone-mirror-flags string #

Flags to pass to "git clone" command when mirroring (default: "-v")
Environment variable: $BUILDKITE_GIT_CLONE_MIRROR_FLAGS
--git-mirrors-path string #

Path to where mirrors of git repositories are stored
Environment variable: $BUILDKITE_GIT_MIRRORS_PATH
--git-mirror-checkout-mode string #

Changes how clones of a mirror are made; available modes are [dissociate reference]. In dissociate mode, clones from a mirror uses the git clone --dissociate flag, which copies underlying objects from the mirror, making the clone robust to changes in the mirror such as garbage collection, at the expense of additional disk usage and setup time. reference mode does not pass --dissociate, which causes the clone to directly use objects from the mirror, which is more fragile and can cause the clone to break under entirely normal operation of the mirror, but is slightly faster to clone and uses less disk space. (default: "reference")
Environment variable: $BUILDKITE_GIT_MIRROR_CHECKOUT_MODE
--git-mirrors-lock-timeout int #

Seconds to lock a git mirror during clone, should exceed your longest checkout (default: 300)
Environment variable: $BUILDKITE_GIT_MIRRORS_LOCK_TIMEOUT
--git-mirrors-skip-update #

Skip updating the Git mirror (default: false)
Environment variable: $BUILDKITE_GIT_MIRRORS_SKIP_UPDATE
--git-checkout-timeout int #

Seconds to allow for each git checkout attempt before it is killed and retried (0 means no timeout) (default: 0)
Environment variable: $BUILDKITE_GIT_CHECKOUT_TIMEOUT
--git-submodule-clone-config string #

Comma separated key=value git config pairs applied before git submodule clone commands such as update --init. If the config is needed to be applied to all git commands, supply it in a global git config file for the system that the agent runs in instead
Environment variable: $BUILDKITE_GIT_SUBMODULE_CLONE_CONFIG
--git-skip-fetch-existing-commits #

Skip git fetch if the commit already exists in the local git directory (default: false)
Environment variable: $BUILDKITE_GIT_SKIP_FETCH_EXISTING_COMMITS
--checkout-attempts int #

Number of checkout attempts (including the initial attempt). Failed attempts are retried with exponential backoff (factor of 2, starting at 1s: 1s, 2s, 4s, ...) (default: 6)
Environment variable: $BUILDKITE_CHECKOUT_ATTEMPTS
--bootstrap-script string #

The command that is executed for bootstrapping a job, defaults to the bootstrap sub-command of this binary
Environment variable: $BUILDKITE_BOOTSTRAP_SCRIPT_PATH
--build-path string #

Path to where the builds will run from
Environment variable: $BUILDKITE_BUILD_PATH
--hooks-path string #

Directory where the hook scripts are found
Environment variable: $BUILDKITE_HOOKS_PATH
--additional-hooks-paths string #

Additional directories to look for agent hooks
Environment variable: $BUILDKITE_ADDITIONAL_HOOKS_PATHS
--sockets-path string #

Directory where the agent will place sockets (default: "$HOME/.buildkite-agent/sockets")
Environment variable: $BUILDKITE_SOCKETS_PATH
--plugins-path string #

Directory where the plugins are saved to
Environment variable: $BUILDKITE_PLUGINS_PATH
--health-check-addr string #

Start an HTTP server on this addr:port that returns whether the agent is healthy, disabled by default
Environment variable: $BUILDKITE_AGENT_HEALTH_CHECK_ADDR
--no-pty #

Do not run jobs within a pseudo terminal (default: false)
Environment variable: $BUILDKITE_NO_PTY
--no-ssh-keyscan #

Require known SSH host keys for the default checkout instead of accepting new keys (default: false)
Environment variable: $BUILDKITE_NO_SSH_KEYSCAN
--no-command-eval #

Don't allow this agent to run arbitrary console commands, including plugins; also forces checkout-override-mode to 'strict' (default: false)
Environment variable: $BUILDKITE_NO_COMMAND_EVAL
--no-plugins #

Don't allow this agent to load plugins (default: false)
Environment variable: $BUILDKITE_NO_PLUGINS
--no-plugin-validation #

Don't validate plugin configuration and requirements (default: true)
Environment variable: $BUILDKITE_NO_PLUGIN_VALIDATION
--plugins-always-clone-fresh #

Always make a new clone of plugin source, even if already present (default: false)
Environment variable: $BUILDKITE_PLUGINS_ALWAYS_CLONE_FRESH
--no-local-hooks #

Don't allow local hooks to be run from checked out repositories (default: false)
Environment variable: $BUILDKITE_NO_LOCAL_HOOKS
--no-git-submodules #

Don't automatically checkout git submodules (default: false)
Environment variable: $BUILDKITE_NO_GIT_SUBMODULES, $BUILDKITE_DISABLE_GIT_SUBMODULES
--no-feature-reporting #

Disables sending a list of enabled features back to the Buildkite mothership. We use this information to measure feature usage, but if you're not comfortable sharing that information then that's totally okay :) (default: false)
Environment variable: $BUILDKITE_AGENT_NO_FEATURE_REPORTING
--allowed-repositories string #

A comma-separated list of regular expressions representing repositories the agent is allowed to clone (for example, "^git@github.com:buildkite/.*" or "^https://github.com/buildkite/.*")
Environment variable: $BUILDKITE_ALLOWED_REPOSITORIES
--enable-environment-variable-allowlist #

Only run jobs where all environment variables are allowed by the allowed-environment-variables option, or have been set by Buildkite (default: false)
Environment variable: $BUILDKITE_ENABLE_ENVIRONMENT_VARIABLE_ALLOWLIST
--allowed-environment-variables string #

A comma-separated list of regular expressions representing environment variables the agent will pass to jobs (for example, "^MYAPP_.*$"). Environment variables set by Buildkite will always be allowed. Requires --enable-environment-variable-allowlist to be set
Environment variable: $BUILDKITE_ALLOWED_ENVIRONMENT_VARIABLES
--allowed-plugins string #

A comma-separated list of regular expressions representing plugins the agent is allowed to use (for example, "^buildkite-plugins/.*$" or "^/var/lib/buildkite-plugins/.*")
Environment variable: $BUILDKITE_ALLOWED_PLUGINS
--opentelemetry-metrics #

Enable agent metrics export over OpenTelemetry OTLP. Configure OTLP with standard OTEL_EXPORTER_OTLP_* env vars (default: false)
Environment variable: $BUILDKITE_OPENTELEMETRY_METRICS
--log-format string #

The format to use for the logger output (default: "text")
Environment variable: $BUILDKITE_LOG_FORMAT
--spawn int #

The number of agents to spawn in parallel (mutually exclusive with --spawn-per-cpu) (default: 1)
Environment variable: $BUILDKITE_AGENT_SPAWN
--spawn-per-cpu int #

The number of agents to spawn per cpu in parallel (mutually exclusive with --spawn) (default: 0)
Environment variable: $BUILDKITE_AGENT_SPAWN_PER_CPU
--spawn-with-priority string #

Assign priorities to every spawned agent (when using --spawn or --spawn-per-cpu). Pass "static" (the configured --priority for every agent, or no priority if unset), "ascending" (1, 2, 3, ...), or "descending" (-1, -2, -3, ...). Descending helps jobs be assigned across all hosts when the value of --spawn varies between hosts (default: "static")
Environment variable: $BUILDKITE_AGENT_SPAWN_WITH_PRIORITY
--cancel-signal string #

The signal to use for cancellation (default: "SIGTERM")
Environment variable: $BUILDKITE_CANCEL_SIGNAL
--cancel-cleanup-timeout duration #

Extra time for a stopping agent to upload logs and artifacts after a job process exits or is killed, before the agent forcefully exits (default: 5s)
Environment variable: $BUILDKITE_CANCEL_CLEANUP_TIMEOUT
--opentelemetry-tracing #

Enable tracing for build jobs with OpenTelemetry OTLP. Configure OTLP with standard OTEL_EXPORTER_OTLP_* env vars (default: false)
Environment variable: $BUILDKITE_OPENTELEMETRY_TRACING
--telemetry-service-name string #

Service name to use when reporting telemetry. (default: "buildkite-agent")
Environment variable: $BUILDKITE_TELEMETRY_SERVICE_NAME
--verification-jwks-file string #

Path to a file containing a JSON Web Key Set (JWKS), used to verify job signatures.
Environment variable: $BUILDKITE_AGENT_VERIFICATION_JWKS_FILE
--signing-jwks-file string #

Path to a file containing a signing key. Passing this flag enables pipeline signing for all pipelines uploaded by this agent. For hmac-sha256, the raw file content is used as the shared key. When using Docker containers to upload pipeline steps dynamically, use environment variable propagation (for example, "docker run -e BUILDKITE_AGENT_JWKS_FILE") to allow all steps within the pipeline to be signed.
Environment variable: $BUILDKITE_AGENT_SIGNING_JWKS_FILE
--signing-jwks-key-id string #

The JWKS key ID to use when signing the pipeline. If omitted, and the signing JWKS contains only one key, that key will be used.
Environment variable: $BUILDKITE_AGENT_SIGNING_JWKS_KEY_ID
--signing-aws-kms-key string #

The KMS KMS key ID, or key alias used when signing and verifying the pipeline.
Environment variable: $BUILDKITE_AGENT_SIGNING_AWS_KMS_KEY
--signing-gcp-kms-key string #

The GCP KMS key resource name used when signing and verifying the pipeline. Format: projects/*/locations/*/keyRings/*/cryptoKeys/*/cryptoKeyVersions/*
Environment variable: $BUILDKITE_AGENT_SIGNING_GCP_KMS_KEY
--debug-signing #

Enable debug logging for pipeline signing. This can potentially leak secrets to the logs as it prints each step in full before signing. Requires debug logging to be enabled (default: false)
Environment variable: $BUILDKITE_AGENT_DEBUG_SIGNING
--verification-failure-behavior string #

The behavior when a job is received without a valid verifiable signature (without a signature, with an invalid signature, or with a signature that fails verification). One of: [block warn]. Defaults to block (default: "block")
Environment variable: $BUILDKITE_AGENT_JOB_VERIFICATION_NO_SIGNATURE_BEHAVIOR
--disable-warnings-for string #

A list of warning IDs to disable
Environment variable: $BUILDKITE_AGENT_DISABLE_WARNINGS_FOR
--ping-mode string #

Selects available protocols for dispatching work to this agent. One of auto (default, prefer streaming, but fall back to polling when necessary), poll-only, or stream-only. (default: "auto")
Environment variable: $BUILDKITE_AGENT_PING_MODE
--token string #

Your agent token. Prefix with file:// to read the token from a file, or fd:// to read it from an inherited file descriptor
Environment variable: $BUILDKITE_AGENT_TOKEN
--endpoint string #

The Agent API endpoint (default: "https://agent-edge.buildkite.com/v3")
Environment variable: $BUILDKITE_AGENT_ENDPOINT
--no-http2 #

Disable HTTP2 when communicating with the Agent API (default: false)
Environment variable: $BUILDKITE_NO_HTTP2
--debug-http #

Enable HTTP debug mode, which dumps all request and response bodies to the log (default: false)
Environment variable: $BUILDKITE_AGENT_DEBUG_HTTP
--trace-http #

Enable HTTP trace mode, which logs timings for each HTTP request. Timings are logged at the debug level unless a request fails at the network level in which case they are logged at the error level (default: false)
Environment variable: $BUILDKITE_AGENT_TRACE_HTTP
--kubernetes-exec #

This is intended to be used only by the Buildkite k8s stack (github.com/buildkite/agent-stack-k8s); it enables a Unix socket for transporting logs and exit statuses between containers in a pod (default: false)
Environment variable: $BUILDKITE_KUBERNETES_EXEC
--kubernetes-container-start-timeout duration #

Timeout for waiting for all containers to start in a Kubernetes pod (default: 5m) (default: 5m0s)
Environment variable: $BUILDKITE_KUBERNETES_CONTAINER_START_TIMEOUT
--job-context-dir string #

The directory for files the agent uses to coordinate with the processes running the job: the job env files, the job timeout marker file, and, in Kubernetes mode, the coordination socket (buildkite.sock). With agent-stack-k8s, leave this unset: the stack manages the shared /workspace volume that Kubernetes mode defaults to. When running --kubernetes-exec under your own orchestration, set this on every container to the path where the shared volume is mounted in that container. Outside Kubernetes mode it defaults to the system temporary directory and rarely needs changing
Environment variable: $BUILDKITE_JOB_CONTEXT_DIR
--redacted-vars string #

Pattern of environment variable names containing sensitive values (default: "*_PASSWORD", "*_SECRET", "*_TOKEN", "*_PRIVATE_KEY", "*_SSH_KEY", "*_ACCESS_KEY", "*_SECRET_KEY", "*_CONNECTION_STRING", "*_API_KEY")
Environment variable: $BUILDKITE_REDACTED_VARS
--strict-single-hooks #

Enforces that only one checkout hook, and only one command hook, can be run (default: false)
Environment variable: $BUILDKITE_STRICT_SINGLE_HOOKS
--no-multipart-artifact-upload #

For Buildkite-hosted artifacts, disables the use of multipart uploads. Has no effect on uploads to other destinations such as custom cloud buckets (default: false)
Environment variable: $BUILDKITE_NO_MULTIPART_ARTIFACT_UPLOAD
--artifact-upload-concurrency int #

Maximum number of concurrent artifact upload operations used by jobs started by this agent. When unset, artifact uploads use their default (default: 0)
Environment variable: $BUILDKITE_ARTIFACT_UPLOAD_CONCURRENCY

Setting tags

Each agent has tags (in 2.x we called this metadata) which can be used to group and target the agents in your build pipelines. This way you're free to dynamically scale your agents and target them based on their capabilities rather than maintaining a static list.

To set an agent's tags you can set it in the configuration file (buildkite-agent.cfg):

tags="docker=true,ruby2=true"

or with the --tags command line flag:

buildkite-agent start --tags "docker=true" --tags "ruby2=true"

or with the BUILDKITE_AGENT_TAGS an environment variable:

env BUILDKITE_AGENT_TAGS="docker=true,ruby2=true" buildkite-agent start

Agent targeting

Once you've started agents with tags you can target them in the build pipeline using agent query rules.

Here's an example of targeting agents that are running with the tag postgres and value of 1.9.4:

pipeline.yml
steps:
  - command: "script.sh"
    agents:
      postgres: "1.9.4"

You can also match for any agent with a postgres tag by omitting the value after the = sign, or by using *, for example:

pipeline.yml
steps:
  - command: "script.sh"
    agents:
      postgres: '*'

Partial wildcard matching (for example, postgres=1.9* or postgres=*1.9) is not yet supported.

Setting agent defaults

Use a top-level agents block to set defaults for all steps in a pipeline.

If you specify multiple tags, your build will only run on agents that have all the specified tags.

The queue tag

The queue tag works differently from other tags, and can be used for isolating jobs and agents. See the Queues overview page for more information about using queues.

If you specify a queue and agent tags, your build will only run on agents that match all of the specified criteria.

For example, if a job has the following agent targeting rules, an agent with both queue=test and postgres=1.9.4 should be present. Otherwise, the job will not dispatch to an agent.

steps:
  - command: "script.sh"
    agents:
      postgres: '1.9.4'
      queue: test

Sourcing tags from Amazon Web Services

You can load an agent's tags from the underlying Amazon EC2 instance using --tags-from-ec2-tags for the instance tags and --tags-from-ec2-meta-data to load the EC2 metadata (for example, instance ID and instance type).

Sourcing tags from Google Cloud

You can load an agent's tags from the underlying Google Cloud metadata using --tags-from-gcp-meta-data.

Run a job on the agent that uploaded it (also known as node affinity)

You can configure your agent and your pipeline steps so that the steps run on the same agent that performed pipeline upload. This is sometimes referred to as "node affinity", but note that what we describe here does not involve Kubernetes (where the term is more widely used).

Normally, we recommend against doing this. The usual practice is to allow jobs to run on whichever agent is available, or to target according to specific criteria (for example, you might want certain jobs to run on a particular operating system). Targeting a specific agent can cause reliability issues (the job can't run if the agent is offline), and can result in work being unevenly distributed between agents (which is inefficient).

First, set the agent hostname tag.

You can do this when starting the agent. This uses the system hostname:

buildkite-agent start --tags "hostname=`hostname`"

Or you can add it to the agent config file, along with any other tags:

tags="hostname=`hostname`"

Then, make sure you are using pipeline upload to upload a pipeline.yml. In Buildkite's YAML steps editor:

steps:
  - command: "buildkite-agent pipeline upload"

Finally, in your pipeline.yml, set hostname: "$BUILDKITE_AGENT_META_DATA_HOSTNAME" on any commands that you want to stick to the agent that uploaded the pipeline.yml. For example:

steps:
  - command: echo "I will stick!"
    agents:
      hostname: "$BUILDKITE_AGENT_META_DATA_HOSTNAME"
  - command: echo "I might not"

When Buildkite uploads the pipeline, $BUILDKITE_AGENT_META_DATA_HOSTNAME is replaced with the agent's hostname tag value. In effect, the previous example becomes:

steps:
  - command: echo "I will stick!"
    agents:
      hostname: "agents-computer-hostname"
  - command: echo "I might not"

This means the first step in the example can only run on an agent with the hostname "agents-computer-hostname". This is the hostname of the agent that uploaded the job. The second step may run on the same agent, or a different one.

Run a single job

--acquire-job value allows you to start an agent and only run the specified job, stopping the agent after it's finished. This means that when you start the agent, instead of it waiting for work, it sends a request to Buildkite to check if it can acquire (self-assign and accept) the job. Once the agent acquires the job, it will run the job, then the agent will be stopped when the job is complete. Jobs acquired using this method will ignore agent tags configured on a job.

Getting the job ID for a single job

value is the job ID. There are several ways to find it:

  • Using the Build API's Get a build endpoint. This returns build information, including all jobs in the build.
  • Through the GraphQL API.
  • The BUILDKITE_JOB_ID build environment variable.
  • In outbound job event webhooks.
  • Using the GUI: select a job, and the job ID is the final value in the URL.

When to use

Normally, you don't set up an agent to run a specific job. Instead, you'll have a pool of agents running, waiting for Buildkite to send jobs to them.

--acquire-job is useful if you want to create your own scheduler to run a specific job.