build-and-push-buildkite-plugin

A concise plugin to assist with build multi-arch container images and pushing them to an ECR repository.

Subsequent pipeline steps can depend_on the step key: build-and-push to ensure that image building and pushing is complete before continuing (see group-key for configuring this value).

Usage

steps:
  - plugins:
    - CatchoftheDay/build-and-push#v1.6.2:
        push-branches: testing,main,master

Configuration

All configuration is optional.

steps:
  - plugins:
    - CatchoftheDay/build-and-push#v1.6.2:
        dockerfile-path: app/Dockerfile
        context-path: "."
        image-name: my-super-special-application
        image-tag: latest
        mutate-image-tag: true
        repository-namespace: wes-digital
        additional-tag: $BUILDKITE_BUILD_NUMBER
        build-args: "secret-from-vault,not-a-secret=42"
        push-branches: testing,main,master
        build-arm: true
        build-x86: true
        scan-image: true
        group-key: build-and-push-0.0.1
        always-pull: false
        additional-plugins: []

dockerfile-path [string]

The relative path of the Dockerfile from the project root. Default: Dockerfile

context-path [string]

The path used as the root of the docker build environment. This effects the source location and availablility of files referenced with ADD in the Dockerfile. Default: . (the project root)

repository-namespace [string]

The namespace of the ECR repository to push to. Default: catch

image-name [string]

The name of your application image. Will be prefixed with catch/ before being pushed to the registry. Default: Buildkite pipeline name ($BUILDKITE_PIPELINE_NAME).

image-tag [string]

A container image tag. Default: First 10 characters of git SHA.

additional-tag [string]

An additional tag for the image, useful if the main tag is dynamic or vis-versa. Default: None

mutate-image-tag [boolean]

Enable an existing tag to be over-written when pushing this image to ECR. WARNING: This will overwrite a tag even if the repository has immutable tags enabled. Default: false

build-args [comma-delimited list]

Additional build-arguments (--build-arg) to pass to docker build. These can be single values (ideally used for secrets that are available to every pipeline step as env vars) or key=value pairs which can be used to pass in non-secret values that aren’t known to every step of the pipeline. Default: ""; GITHUB_TOKEN, BUILDKITE_COMMIT, BUILDKITE_JOB_ID and BUILD_DATE (unit timestamp) are always provided.

push-branches [comma-delimited list]

A list of branch names for which to push a built image to ECR. This can serve as a toggle to be able to test container builds in feature branches but only push those images to ECR for deployable branches. If the build is triggered from a non-branch event (such as a git tag) it will always be pushed to ECR. Default: ""

build-arm [boolean]

Should we build an ARM image? Default: true

build-x86 [boolean]

Should we build an x86 image? Default: true

scan-image [boolean]

Should the container image be scanned the security scanner? This step is non-blocking (but this may change in the future). Default: true

group-key [string]

This is the key assigned to the job group that encapsulates the build tasks. This key is used by subsequent jobs that depend this build completing. Default: build-and-push

always-pull [boolean]

Should the builder always attempt to pull fresh source images. This will ensure it always uses the latest available version of an image tag. Can be disabled to potentially improve build times slightly if there is low risk of the upstream tagged image being updated. Default: true

additional-plugins [array]

Pass through additional plugins to this plugin’s steps, useful in cases where you may need to login to an image registry.

  - label: ":docker: Build and upload container to ECR"
    plugins:
      - CatchoftheDay/build-and-push#v1.6.2:
          additional-plugins:
            - docker-login#v3.0.0:
              server: your-image-registry.io
              username: your_username
              password-env: your_password-env

composer-cache [boolean]

Attempt to utilize a buildkite-cached composer package cache (not a cache of vendor) when building the image. The cache must be available at .composer-cache. The cache will be made available as a build context called composer-cache (see utilising-package-caches for how to take advantag of this in your builds). If the image builds successfully the cache will be resaved at pipeline level so it can be reused as a base even if the manifest changes. See the buildkite cache plugin for further details of how this works. Default: false

example

  - label: cache-composer-deps
    command: |
      composer config -g github-oauth.github.com $${GITHUB_TOKEN};
      composer config -g cache-dir ./.composer-cache;
      composer install --download-only;
    plugins:
      - docker#v5.6.0:
          image: "362995399210.dkr.ecr.ap-southeast-2.amazonaws.com/catch/php-base:8.1-fpm-buster"
          always-pull: true
          mount-ssh-agent: true
          propagate-environment: true
          environment:
            - GITHUB_TOKEN
      - cache#v0.6.0:
          backend: s3
          manifest: composer.lock
          path: .composer-cache
          save: file
          restore: pipeline
  - label: ":docker: Build and upload container to ECR"
    plugins:
      - CatchoftheDay/build-and-push#v1.6.2:
          composer-cache: true
          push-branches: testing,main,master

npm-cache [boolean]

Attempt to utilize a buildkite-cached npm package cache (not a cache of node_modules) when building the image. The cache must be available at .npm-cache. The cache will be made available as a build-context called npm-cache (see utilising-package-caches for how to take advantag of this in your builds). If the image builds successfully the cache will be resaved at pipeline level so it can be reused as a base even if the manifest changes. See the buildkite cache plugin for further details of how this works. Default: false

example

  - label: cache-node-deps
    command: |
      echo "//npm.pkg.github.com/:_authToken=$${GITHUB_TOKEN}" > ~/.npmrc;
      npm config set -g cache ./.npm-cache;
      npm ci;
    plugins:
      - docker#v5.6.0:
          image: "362995399210.dkr.ecr.ap-southeast-2.amazonaws.com/catch/node-base:18-buster-slim"
          always-pull: true
          mount-ssh-agent: true
          propagate-environment: true
          environment:
            - GITHUB_TOKEN
      - cache#v0.6.0:
          backend: s3
          manifest: package-lock.json
          path: .npm-cache
          save: file
          restore: pipeline
  - label: ":docker: Build and upload container to ECR"
    plugins:
      - CatchoftheDay/build-and-push#v1.6.2:
          npm-cache: true
          push-branches: testing,main,master

yarn-cache [boolean]

Attempt to utilize a buildkite-cached npm package cache (not a cache of node_modules) when building the image. The cache must be available at .yarn-cache. The cache will be made available as a build-context called yarn-cache (see utilising-package-caches for how to take advantag of this in your builds). If the image builds successfully the cache will be resaved at pipeline level so it can be reused as a base even if the manifest changes. See the buildkite cache plugin for further details of how this works. Default: false

example

  - label: cache-node-deps
    command: |
      echo "//npm.pkg.github.com/:_authToken=$${GITHUB_TOKEN}" > ~/.npmrc;
      yarn config set cache-folder ./.yarn-cache;
      yarn install --frozen-lockfile;
    plugins:
      - docker#v5.6.0:
          image: "362995399210.dkr.ecr.ap-southeast-2.amazonaws.com/catch/node-base:18-buster-slim"
          always-pull: true
          mount-ssh-agent: true
          propagate-environment: true
          environment:
            - GITHUB_TOKEN
      - cache#v0.6.0:
          backend: s3
          manifest: yarn.lock
          path: .yarn-cache
          save: file
          restore: pipeline
  - label: ":docker: Build and upload container to ECR"
    plugins:
      - CatchoftheDay/build-and-push#v1.6.2:
          yarn-cache: true
          push-branches: testing,main,master

Utilising package caches

Only including the composer-cache: true or npm-cache: true or yarn-cache: true flags isn’t sufficient to take advantage of your package cache. The projects Dockerfile will also need to contain something like the following when performing the install step with the package manager.

composer

RUN --mount=type=cache,from=composer-cache,target=/root/.cache/composer \
    set -ex && \
    composer install \
        --no-scripts \
        --no-progress \
        --no-suggest \
        --prefer-dist \
        --no-dev \
        --no-autoloader \
        --no-interaction

npm

RUN --mount=type=cache,from=npm-cache,target=/root/.npm \
    set -ex && \
    npm ci

yarn

RUN --mount=type=cache,from=yarn-cache,target=/root/.npm \
    set -ex && \
    yarn config set cache-folder /root/.npm && \
    yarn install --frozen-lockfile

This will ensure that package files are picked up from the cache rather than being redownloaded from the internet.