buildkite-agent start
The Buildkite Agent's start command is used to manually start an agent and register it with Buildkite.
Starting an agent
Usage
buildkite-agent start [options...]
Description
When a job is ready to run it will call the "bootstrap-script" and pass it all the environment variables required for the job to run. This script is responsible for checking out the code, and running the actual build script defined in the pipeline.
The agent will run any jobs within a PTY (pseudo terminal) if available.
Example
$ buildkite-agent start --token xxx
Options
--config value #
|
Path to a configuration file |
|---|---|
--name value #
|
The name of the agent |
--priority value #
|
The priority of the agent (higher priorities are assigned work first) |
--acquire-job value #
|
Start this agent and only run the specified job, disconnecting after it's finished |
--disconnect-after-job #
|
Disconnect the agent after running exactly one job. When used in conjunction with the `--spawn` flag, each worker booted will run exactly one job |
--disconnect-after-idle-timeout value #
|
The maximum idle time in seconds to wait for a job before disconnecting. The default of 0 means no timeout (default: 0) |
--cancel-grace-period value #
|
The number of seconds a canceled or timed out job is given to gracefully terminate and upload its artifacts (default: 10) |
--enable-job-log-tmpfile #
|
Store the job logs in a temporary file `BUILDKITE_JOB_LOG_TMPFILE` that is accessible during the job and removed at the end of the job |
--job-log-path value #
|
Location to store job logs created by configuring `enable-job-log-tmpfile`, by default job log will be stored in TempDir |
--write-job-logs-to-stdout #
|
Writes job logs to the agent process' stdout. This simplifies log collection if running agents in Docker. |
--shell value #
|
The shell command used to interpret build commands, e.g /bin/bash -e -c (default: "/bin/bash -e -c") |
--queue value #
|
The queue the agent will listen to for jobs. If not set, the agent will use the default queue. Overwrites the queue tag in the agent's tags |
--tags value #
|
A comma-separated list of tags for the agent (for example, "linux" or "mac,xcode=8") |
--tags-from-host #
|
Include tags from the host (hostname, machine-id, os) |
--tags-from-ec2-meta-data value #
|
Include the default set of host EC2 meta-data as tags (instance-id, instance-type, ami-id, and instance-life-cycle) |
--tags-from-ec2-meta-data-paths value #
|
Include additional tags fetched from EC2 meta-data using tag & path suffix pairs, e.g "tag_name=path/to/value" |
--tags-from-ec2-tags #
|
Include the host's EC2 tags as tags |
--tags-from-ecs-meta-data #
|
Include the host's ECS meta-data as tags (container-name, image, and task-arn) |
--tags-from-gcp-meta-data value #
|
Include the default set of host Google Cloud instance meta-data as tags (instance-id, machine-type, preemptible, project-id, region, and zone) |
--tags-from-gcp-meta-data-paths value #
|
Include additional tags fetched from Google Cloud instance meta-data using tag & path suffix pairs, e.g "tag_name=path/to/value" |
--tags-from-gcp-labels #
|
Include the host's Google Cloud instance labels as tags |
--wait-for-ec2-tags-timeout value #
|
The amount of time to wait for tags from EC2 before proceeding (default: 10s) |
--wait-for-ec2-meta-data-timeout value #
|
The amount of time to wait for meta-data from EC2 before proceeding (default: 10s) |
--wait-for-ecs-meta-data-timeout value #
|
The amount of time to wait for meta-data from ECS before proceeding (default: 10s) |
--wait-for-gcp-labels-timeout value #
|
The amount of time to wait for labels from GCP before proceeding (default: 10s) |
--git-checkout-flags value #
|
Flags to pass to "git checkout" command (default: "-f") |
--git-clone-flags value #
|
Flags to pass to the "git clone" command (default: "-v") |
--git-clean-flags value #
|
Flags to pass to "git clean" command (default: "-ffxdq") |
--git-fetch-flags value #
|
Flags to pass to "git fetch" command (default: "-v --prune") |
--git-clone-mirror-flags value #
|
Flags to pass to the "git clone" command when used for mirroring (default: "-v") |
--git-mirrors-path value #
|
Path to where mirrors of git repositories are stored |
--git-mirrors-lock-timeout value #
|
Seconds to lock a git mirror during clone, should exceed your longest checkout (default: 300) |
--git-mirrors-skip-update #
|
Skip updating the Git mirror |
--bootstrap-script value #
|
The command that is executed for bootstrapping a job, defaults to the bootstrap sub-command of this binary |
--build-path value #
|
Path to where the builds will run from |
--hooks-path value #
|
Directory where the hook scripts are found |
--sockets-path value #
|
Directory where the agent will place sockets (default: "$HOME/.buildkite-agent/sockets") |
--plugins-path value #
|
Directory where the plugins are saved to |
--no-ansi-timestamps #
|
Do not insert ANSI timestamp codes at the start of each line of job output |
--timestamp-lines #
|
Prepend timestamps on each line of job output. Has no effect unless --no-ansi-timestamps is also used |
--health-check-addr value #
|
Start an HTTP server on this addr:port that returns whether the agent is healthy, disabled by default |
--no-pty #
|
Do not run jobs within a pseudo terminal |
--no-ssh-keyscan #
|
Don't automatically run ssh-keyscan before checkout |
--no-command-eval #
|
Don't allow this agent to run arbitrary console commands, including plugins |
--no-plugins #
|
Don't allow this agent to load plugins |
--no-plugin-validation #
|
Don't validate plugin configuration and requirements |
--no-local-hooks #
|
Don't allow local hooks to be run from checked out repositories |
--no-git-submodules #
|
Don't automatically checkout git submodules [$BUILDKITE_NO_GIT_SUBMODULES, $BUILDKITE_DISABLE_GIT_SUBMODULES] |
--no-feature-reporting #
|
Disables sending a list of enabled features back to the Buildkite mothership. We use this information to measure feature usage, but if you're not comfortable sharing that information then that's totally okay :) |
--allowed-repositories value #
|
A comma-separated list of regular expressions representing repositories the agent is allowed to clone (for example, "^git@github.com:buildkite/.*" or "^https://github.com/buildkite/.*") |
--enable-environment-variable-allowlist #
|
Only run jobs where all environment variables are allowed by the allowed-environment-variables option, or have been set by Buildkite |
--allowed-environment-variables value #
|
A comma-separated list of regular expressions representing environment variables the agent will pass to jobs (for example, "^MYAPP_.*$"). Environment variables set by Buildkite will always be allowed. Requires --enable-environment-variable-allowlist to be set |
--allowed-plugins value #
|
A comma-separated list of regular expressions representing plugins the agent is allowed to use (for example, "^buildkite-plugins/.*$" or "^/var/lib/buildkite-plugins/.*") |
--metrics-datadog #
|
Send metrics to DogStatsD for Datadog |
--metrics-datadog-host value #
|
The dogstatsd instance to send metrics to using udp (default: "127.0.0.1:8125") |
--metrics-datadog-distributions #
|
Use Datadog Distributions for Timing metrics |
--log-format value #
|
The format to use for the logger output (default: "text") |
--spawn value #
|
The number of agents to spawn in parallel (mutually exclusive with --spawn-per-cpu) (default: 1) |
--spawn-per-cpu value #
|
The number of agents to spawn per cpu in parallel (mutually exclusive with --spawn) (default: 0) |
--spawn-with-priority #
|
Assign priorities to every spawned agent (when using --spawn or --spawn-per-cpu) equal to the agent's index |
--cancel-signal value #
|
The signal to use for cancellation (default: "SIGTERM") |
--signal-grace-period-seconds value #
|
The number of seconds given to a subprocess to handle being sent `cancel-signal`. After this period has elapsed, SIGKILL will be sent. Negative values are taken relative to `cancel-grace-period`. The default is `cancel-grace-period` - 1. (default: -1) |
--tracing-backend value #
|
Enable tracing for build jobs by specifying a backend, "datadog" or "opentelemetry" |
--tracing-service-name value #
|
Service name to use when reporting traces. (default: "buildkite-agent") |
--verification-jwks-file value #
|
Path to a file containing a JSON Web Key Set (JWKS), used to verify job signatures. |
--signing-jwks-file value #
|
Path to a file containing a signing key. Passing this flag enables pipeline signing for all pipelines uploaded by this agent. For hmac-sha256, the raw file content is used as the shared key. When using Docker containers to upload pipeline steps dynamically, use environment variable propagation (for example, "docker run -e BUILDKITE_AGENT_JWKS_FILE") to allow all steps within the pipeline to be signed. |
--signing-jwks-key-id value #
|
The JWKS key ID to use when signing the pipeline. If omitted, and the signing JWKS contains only one key, that key will be used. |
--signing-aws-kms-key value #
|
The KMS KMS key ID, or key alias used when signing and verifying the pipeline. |
--debug-signing #
|
Enable debug logging for pipeline signing. This can potentially leak secrets to the logs as it prints each step in full before signing. Requires debug logging to be enabled |
--verification-failure-behavior value #
|
The behavior when a job is received without a valid verifiable signature (without a signature, with an invalid signature, or with a signature that fails verification). One of: [block warn]. Defaults to block (default: "block") |
--disable-warnings-for value #
|
A list of warning IDs to disable |
--token value #
|
Your account agent token |
--endpoint value #
|
The Agent API endpoint (default: " |
--no-http2 #
|
Disable HTTP2 when communicating with the Agent API. |
--debug-http #
|
Enable HTTP debug mode, which dumps all request and response bodies to the log |
--trace-http #
|
Enable HTTP trace mode, which logs timings for each HTTP request. Timings are logged at the debug level unless a request fails at the network level in which case they are logged at the error level |
--no-color #
|
Don't show colors in logging |
--debug #
|
Enable debug mode. Synonym for `--log-level debug`. Takes precedence over `--log-level` |
--log-level value #
|
Set the log level for the agent, making logging more or less verbose. Defaults to notice. Allowed values are: debug, info, error, warn, fatal (default: "notice") |
--experiment value #
|
Enable experimental features within the buildkite-agent |
--profile value #
|
Enable a profiling mode, either cpu, memory, mutex or block |
--redacted-vars value #
|
Pattern of environment variable names containing sensitive values (default: "*_PASSWORD", "*_SECRET", "*_TOKEN", "*_PRIVATE_KEY", "*_ACCESS_KEY", "*_SECRET_KEY", "*_CONNECTION_STRING") |
--strict-single-hooks #
|
Enforces that only one checkout hook, and only one command hook, can be run |
--kubernetes-exec #
|
This is intended to be used only by the Buildkite k8s stack (github.com/buildkite/agent-stack-k8s); it enables a Unix socket for transporting logs and exit statuses between containers in a pod |
--trace-context-encoding value #
|
Sets the inner encoding for BUILDKITE_TRACE_CONTEXT. Must be either json or gob (default: "gob") |
--no-multipart-artifact-upload #
|
For Buildkite-hosted artifacts, disables the use of multipart uploads. Has no effect on uploads to other destinations such as custom cloud buckets |
--tags-from-ec2 #
|
Include the host's EC2 meta-data as tags (instance-id, instance-type, and ami-id) |
--tags-from-gcp #
|
Include the host's Google Cloud instance meta-data as tags (instance-id, machine-type, preemptible, project-id, region, and zone) |
Agent targeting
Once you've started agents with tags you can target them in the build pipeline using agent query rules.
Here's an example of targeting agents that are running with the tag postgres and value of 1.9.4:
steps:
- command: "script.sh"
agents:
postgres: "1.9.4"
You can also match for any agent with a postgres tag by omitting the value after the = sign, or by using *, for example:
steps:
- command: "script.sh"
agents:
postgres: '*'
Partial wildcard matching (for example, postgres=1.9* or postgres=*1.9) is not yet supported.
Use a top-level agents block to set defaults for all steps in a pipeline.
If you specify multiple tags, your build will only run on agents that have all the specified tags.
The queue tag
The queue tag works differently from other tags, and can be used for isolating jobs and agents. See the agent queues documentation for more information about using queues.
If you specify a queue and agent tags, your build will only run on agents that match all of the specified criteria.
For example, if a job has the following agent targeting rules, an agent with both queue=test and postgres=1.9.4 should be present. Otherwise, the job will not dispatch to an agent.
steps:
- command: "script.sh"
agents:
postgres: '1.9.4'
queue: test
Run a job on the agent that uploaded it (also known as node affinity)
You can configure your agent and your pipeline steps so that the steps run on the same agent that performed pipeline upload. This is sometimes referred to as "node affinity", but note that what we describe here does not involve Kubernetes (where the term is more widely used).
First, set the agent hostname tag.
You can do this when starting the agent. This uses the system hostname:
buildkite-agent start --tags "hostname=`hostname`"
Or you can add it to the agent config file, along with any other tags:
tags="hostname=`hostname`"
Then, make sure you are using pipeline upload to upload a pipeline.yml. In Buildkite's YAML steps editor:
steps:
- command: "buildkite-agent pipeline upload"
Finally, in your pipeline.yml, set hostname: "$BUILDKITE_AGENT_META_DATA_HOSTNAME" on any commands that you want to stick to the agent that uploaded the pipeline.yml. For example:
steps:
- command: echo "I will stick!"
agents:
hostname: "$BUILDKITE_AGENT_META_DATA_HOSTNAME"
- command: echo "I might not"
When Buildkite uploads the pipeline, $BUILDKITE_AGENT_META_DATA_HOSTNAME is replaced with the agent's hostname tag value. In effect, the previous example becomes:
steps:
- command: echo "I will stick!"
agents:
hostname: "agents-computer-hostname"
- command: echo "I might not"
This means the first step in the example can only run on an agent with the hostname "agents-computer-hostname". This is the hostname of the agent that uploaded the job. The second step may run on the same agent, or a different one.
Run a single job
--acquire-job value allows you to start an agent and only run the specified job, stopping the agent after it's finished. This means that when you start the agent, instead of it waiting for work, it sends a request to Buildkite to check if it can acquire (self-assign and accept) the job. Once the agent acquires the job, it runs it, then stops the agent when the job is complete.
Getting the job ID for a single job
value is the job ID. There are several ways to find it:
- Using the Build API's Get a build endpoint. This returns build information, including all jobs in the build.
- Through the GraphQL API.
- The
BUILDKITE_JOB_IDbuild environment variable. - In outbound job event webhooks.
- Using the GUI: select a job, and the job ID is the final value in the URL.
When to use
Normally, you don't set up an agent to run a specific job. Instead, you'll have a pool of agents running, waiting for Buildkite to send jobs to them.
--acquire-job is useful if you want to create your own scheduler to run a specific job.