buildkite-agent cache
The Buildkite agent's cache subcommands provide the ability to manage build caches.
Save
Usage
buildkite-agent cache save [options]
Description
Saves files to the cache for the current build based on the cache configuration defined in your cache config file (defaults to .buildkite/cache.yml or .buildkite/cache.yaml).
The cache configuration file defines which files or directories should be cached and their associated cache key.
Note: This feature is currently in development and subject to change. It is not yet available to all customers.
Example
$ buildkite-agent cache save
This will save all caches defined in the cache configuration file. You can also save specific caches by providing their names:
$ buildkite-agent cache save --name "node"
The cache is stored at BUILDKITE_AGENT_CACHE_STORE_URL (or --cache-store-url). The registry is selected by BUILDKITE_AGENT_CACHE_REGISTRY (or --registry); '~' selects the cluster's default registry. If an entry already exists at the same address it is not overwritten unless --force is specified:
$ buildkite-agent cache save --name "node" --force
Force replaces the entire cache at the same save address, without merging files or bypassing registry access policies. Concurrent saves are last-write-wins.
Archives are stored under a digest of their contents. If the store already holds an archive with the same digest and size, the entry is saved without uploading it again. --force always uploads.
Configuration File Format:
The cache configuration file should be in YAML format. cache_key is an ordered list of parts; each part is a literal string or one of { agent: os }, { agent: arch }, { checksum: <file> }, or { env: <VAR> }. A checksum part also accepts an array of file paths and glob patterns (, *, ?), expanded relative to the working directory and hashed together into one digest:
caches:
- name: node
cache_key:
- node
- { agent: os }
- { agent: arch }
- { checksum: [package-lock.json, "patches/*.patch"] }
target_paths:
- node_modules
Options
--no-color #
|
Don't show colors in logging (default: false) |
|---|---|
--debug #
|
Enable debug mode. Synonym for |
--log-level string #
|
Set the log level for the agent, making logging more or less verbose. Defaults to notice. Allowed values are: debug, info, error, warn, fatal (default: "notice") |
--experiment string #
|
Enable experimental features within the buildkite-agent |
--profile string #
|
Enable a profiling mode, either cpu, memory, mutex or block |
--agent-access-token string #
|
The access token used to identify the agent |
--endpoint string #
|
The Agent API endpoint (default: "https://agent-edge.buildkite.com/v3") |
--no-http2 #
|
Disable HTTP2 when communicating with the Agent API (default: false) |
--debug-http #
|
Enable HTTP debug mode, which dumps all request and response bodies to the log (default: false) |
--trace-http #
|
Enable HTTP trace mode, which logs timings for each HTTP request. Timings are logged at the debug level unless a request fails at the network level in which case they are logged at the error level (default: false) |
--name string #
|
Cache name to process (can be specified multiple times; if empty, processes all caches) |
--registry string #
|
The slug of the cache registry to use; '~' selects the cluster's default registry (default: "~") |
--cache-store-url string #
|
The URL of the cache store (e.g., s3://bucket-name); uploads/downloads use ambient credentials |
--cache-config-file string #
|
Path to the cache configuration YAML file (defaults to .buildkite/cache.yml or .buildkite/cache.yaml) |
--concurrency int #
|
Number of concurrent cache operations (default: 2) |
--cache-fail-on-error #
|
Fail the command (non-zero exit) when a cache save or restore fails. By default a cache is best-effort: failures are logged and skipped so they never fail the build |
--force #
|
Save even when an entry already exists at the same address |
Restore
Usage
buildkite-agent cache restore [options]
Description
Restores files from the cache for the current job based on the cache configuration defined in your cache config file (defaults to .buildkite/cache.yml or .buildkite/cache.yaml).
The cache configuration file defines which files or directories should be restored and their associated cache key. An entry is restored when its target_paths (an order-insensitive set) and the mandatory parts of its cache_key match. By default every key part is mandatory; mark a part with fallback_limit: true to make every part after it optional.
Restore replaces the contents of target_paths; it does not merge with existing files. Use job-private target directories. Sharing a remote cache entry between jobs is fine, but restoring into a local directory that another job or process is using can delete its files. If a local target must be shared, coordinate exclusive access for the entire time the cache is in use, not only during restore.
Cleanup is not transactional. If cleanup fails, extraction for that cache does not start, but target paths may already have been modified. These failures remain fatal even when --cache-fail-on-error is false.
Note: This feature is currently in development and subject to change. It is not yet available to all customers.
Example
$ buildkite-agent cache restore
This will restore all caches defined in the cache configuration file. You can also restore specific caches by providing their names:
$ buildkite-agent cache restore --name "node"
The cache is retrieved from BUILDKITE_AGENT_CACHE_STORE_URL (or --cache-store-url), downloaded directly using the agent's ambient credentials. The registry is selected by BUILDKITE_AGENT_CACHE_REGISTRY (or --registry); '~' selects the cluster default.
Configuration File Format:
The cache configuration file should be in YAML format. cache_key is an ordered list of parts; each part is a literal string or one of { agent: os }, { agent: arch }, { checksum: <file> }, or { env: <VAR> }. A checksum part also accepts an array of file paths and glob patterns (, *, ?), expanded relative to the working directory and hashed together into one digest. Any one part may also set fallback_limit: true to make every part after it optional for fallback matching (the marked part itself stays mandatory). In the example below an exact match is preferred, but if the lockfile changed, an entry matching node + os + arch is still restored, as the fallback is specified on arch, making node + os + arch mandatory, but making checksum optional:
caches:
- name: node
cache_key:
- node
- { agent: os }
- { agent: arch, fallback_limit: true }
- { checksum: package-lock.json }
target_paths:
- node_modules
Cache Restoration Results:
The command groups restore results by cache, showing each attempted key and scope in registry search order, including misses, policy denials and the selected hit. Denied attempts include the matched rule name when available. Successful restores identify the selected key and scope and whether the match was exact or a fallback. An incomplete search is distinguished from a miss. Older registries may not provide search diagnostics.
Options
--no-color #
|
Don't show colors in logging (default: false) |
|---|---|
--debug #
|
Enable debug mode. Synonym for |
--log-level string #
|
Set the log level for the agent, making logging more or less verbose. Defaults to notice. Allowed values are: debug, info, error, warn, fatal (default: "notice") |
--experiment string #
|
Enable experimental features within the buildkite-agent |
--profile string #
|
Enable a profiling mode, either cpu, memory, mutex or block |
--agent-access-token string #
|
The access token used to identify the agent |
--endpoint string #
|
The Agent API endpoint (default: "https://agent-edge.buildkite.com/v3") |
--no-http2 #
|
Disable HTTP2 when communicating with the Agent API (default: false) |
--debug-http #
|
Enable HTTP debug mode, which dumps all request and response bodies to the log (default: false) |
--trace-http #
|
Enable HTTP trace mode, which logs timings for each HTTP request. Timings are logged at the debug level unless a request fails at the network level in which case they are logged at the error level (default: false) |
--name string #
|
Cache name to process (can be specified multiple times; if empty, processes all caches) |
--registry string #
|
The slug of the cache registry to use; '~' selects the cluster's default registry (default: "~") |
--cache-store-url string #
|
The URL of the cache store (e.g., s3://bucket-name); uploads/downloads use ambient credentials |
--cache-config-file string #
|
Path to the cache configuration YAML file (defaults to .buildkite/cache.yml or .buildkite/cache.yaml) |
--concurrency int #
|
Number of concurrent cache operations (default: 2) |
--cache-fail-on-error #
|
Fail the command (non-zero exit) when a cache save or restore fails. By default a cache is best-effort: failures are logged and skipped so they never fail the build |
Exec
Usage
buildkite-agent cache exec --name <name> [options] -- <command> [args...]
Description
Runs the given command if the named cache has no result for the current cache key, then saves the cache's target_paths together with the command's output. When the cache has a result, the command doesn't run: target_paths are restored and the saved output is replayed in the build log.
The replayed output is the command's stdout and stderr combined, all written to stdout, so cache exec is for commands whose output goes to the build log. Don't capture or pipe its output, for example with $(...), since a cache hit would include the command's stderr.
Example
$ buildkite-agent cache exec --name frontend_build -- vite build
With this cache configuration, vite build is skipped whenever the sources, lockfile and vite config are unchanged:
caches:
- name: frontend_build
cache_key:
- frontend_build
- { checksum: ["src/**", package-lock.json, vite.config.ts] }
target_paths:
- dist
Options
--no-color #
|
Don't show colors in logging (default: false) |
|---|---|
--debug #
|
Enable debug mode. Synonym for |
--log-level string #
|
Set the log level for the agent, making logging more or less verbose. Defaults to notice. Allowed values are: debug, info, error, warn, fatal (default: "notice") |
--experiment string #
|
Enable experimental features within the buildkite-agent |
--profile string #
|
Enable a profiling mode, either cpu, memory, mutex or block |
--agent-access-token string #
|
The access token used to identify the agent |
--endpoint string #
|
The Agent API endpoint (default: "https://agent-edge.buildkite.com/v3") |
--no-http2 #
|
Disable HTTP2 when communicating with the Agent API (default: false) |
--debug-http #
|
Enable HTTP debug mode, which dumps all request and response bodies to the log (default: false) |
--trace-http #
|
Enable HTTP trace mode, which logs timings for each HTTP request. Timings are logged at the debug level unless a request fails at the network level in which case they are logged at the error level (default: false) |
--name string #
|
Cache name to process (can be specified multiple times; if empty, processes all caches) |
--registry string #
|
The slug of the cache registry to use; '~' selects the cluster's default registry (default: "~") |
--cache-store-url string #
|
The URL of the cache store (e.g., s3://bucket-name); uploads/downloads use ambient credentials |
--cache-config-file string #
|
Path to the cache configuration YAML file (defaults to .buildkite/cache.yml or .buildkite/cache.yaml) |
--concurrency int #
|
Number of concurrent cache operations (default: 2) |
--cache-fail-on-error #
|
Fail the command (non-zero exit) when a cache save or restore fails. By default a cache is best-effort: failures are logged and skipped so they never fail the build |
--redacted-vars string #
|
Pattern of environment variable names containing sensitive values (default: "*_PASSWORD", "*_SECRET", "*_TOKEN", "*_PRIVATE_KEY", "*_SSH_KEY", "*_ACCESS_KEY", "*_SECRET_KEY", "*_CONNECTION_STRING", "*_API_KEY") |