1. Resources
  2. /
  3. Plugins
  4. /
  5. s3-cache-buildkite-plugin

S3 Cache Buildkite Plugin

CI

Save and restore cache to and from AWS S3.

Example

Add the following to your pipeline.yml:

steps:
  - command: npm install && npm test
    plugins:
      - peakon/s3-cache#v2.3.1:
          id: CACHE_IDENTIFIER # optional, default: none
          aws_profile: aws-profile-name # optional, default: none
          restore_dry_run: false # set it to "true" to only check if cacheKey is present on S3 (no download / restoring)
          save:
            - key: 'v1-node-modules-{{ checksum("package-lock.json") }}' # required
              paths: ["node_modules"] # required, array of strings
              when: on_success # optional, one of {always, on_success, on_failure}, default: on_success
              overwrite: false # optional, set true to overwrite cache on S3 even if object already exists
          restore:
            - keys:
                - 'v1-node-modules-{{ checksum "package-lock.json" }}'
                - "v1-node-modules-" # will load latest cache starting with v1-node-modules- (not yet implemented)

Configuration

Prerequisites

Make sure to set BUILDKITE_PLUGIN_S3_CACHE_BUCKET_NAME=your-cache-bucket-name before using this plugin.

Plugin

You can specify either save or restore or both of them for a single pipeline step.

Checking if cache was successfully restored

In some cases you may need to build a conditional logic in the build command based on the results of cache restore operation (for example, to avoid re-generating the cache which already exists and was restored successfully).

To support this use-case, this plugin exports environment variables that can be used during a command step. The feature is opt-in and requires id to be specified in plugin configuration.

For example, this step generates a cache of node_modules (which is then used by all jobs that need it):

steps:
  - command: '[ ! "${BUILDKITE_PLUGIN_S3_CACHE_npm_0_KEY_0_HIT}" =~ ^(true)$ ] && npm install'
    plugins:
      - peakon/s3-cache#2.2.1:
          id: npm
          restore_dry_run: true # This saves runtime, but doesn't check for integrity
          restore:
            - keys: ['v1-node-modules-{{ checksum "package-lock.json" }}']
          save:
            - key: 'v1-node-modules-{{ checksum "package-lock.json" }}'
              paths: ["node_modules"]

Supported functions

  • checksum 'filename(s)' - sha256 hash of filename(s) support glob expansion

  • epoch - time in seconds since Unix epoch (in UTC)

  • .Environment.SOME_VAR - a value of environment variable SOME_VAR

AWS profiles

You can specify a custom AWS profile to be used by AWS CLI

  • in pipeline YAML (aws_profile: profile_name)
  • via BUILDKITE_PLUGIN_S3_CACHE_AWS_PROFILE environment variable (e.g. inside agent environment hook).

Developing

To run the tests:

docker compose run --rm tests

Contributing

  1. Fork the repo
  2. Make the changes
  3. Run the tests
  4. Commit and push your changes
  5. Send a pull request

The plugins listed on this webpage are provided for informational purposes only. They have not undergone any formal security review or assessment. While we strive to provide useful resources, we cannot guarantee the safety, reliability, or integrity of these plugins. Users are strongly advised to conduct their own security evaluations before downloading, installing, or using any plugin. By using these plugins, you acknowledge and accept any risks associated with their use. We disclaim any liability for any harm or damages arising from the use of the plugins listed.

Start turning complexity into an advantage

Create an account to get started with a 30-day free trial. No credit card required.

Buildkite Pipelines

Platform

  1. Pipelines
  2. Pipeline templates
  3. Public pipelines
  4. Test Engine
  5. Package Registries
  6. Mobile Delivery Cloud
  7. Pricing

Hosting options

  1. Self-hosted agents
  2. Mac hosted agents
  3. Linux hosted agents

Resources

  1. Docs
  2. Blog
  3. Changelog
  4. Webinars
  5. Plugins
  6. Case studies
  7. Events
  8. Migration Services
  9. Comparisons

Company

  1. About
  2. Careers
  3. Press
  4. Brand assets
  5. Contact

Solutions

  1. Replace Jenkins
  2. Workflows for AI/ML
  3. Testing at scale
  4. Monorepo mojo
  5. Bazel orchestration

Legal

  1. Terms of Service
  2. Acceptable Use Policy
  3. Privacy Policy
  4. Subprocessors
  5. Service Level Agreement

Support

  1. System status
  2. Forum
© Buildkite Pty Ltd 2025