Pre-installed packages

Package comparison

The Elastic CI Stack for AWS AMI includes the following packages:

Handling missing packages

When a package your builds require is not available in the default agent image, you have three options:

• Use a Buildkite plugin that provides the functionality.
• Create a custom container image with the required packages.
• Install packages at runtime using an agent hook.

Using plugins

Plugins can provide tool functionality without modifying your container image. This approach works well for tools with existing plugin support.

For AWS CLI operations, use the aws-assume-role-with-web-identity plugin with OIDC, or provide AWS credentials to a container that includes the AWS CLI.

Browse the plugins directory for plugins that may provide the functionality you need.

Using custom container images

For packages used frequently across many pipelines, create a custom container image based on the Buildkite agent image or another base image.

Create a Dockerfile with the additional packages:

FROM buildkite/agent:latest

USER root

RUN apt-get update && apt-get install -y \
    git-lfs \
    zip \
    pigz \
    python3-pip \
    dnsutils \
    gnupg2 \
    && rm -rf /var/lib/apt/lists/*

USER buildkite-agent

For AWS CLI, install using pip or download the official installer:

FROM buildkite/agent:latest

USER root

RUN apt-get update && apt-get install -y \
    curl \
    unzip \
    && curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
    && unzip awscliv2.zip \
    && ./aws/install \
    && rm -rf awscliv2.zip aws \
    && rm -rf /var/lib/apt/lists/*

USER buildkite-agent

Build and push the image to your container registry:

docker build -t my-registry/buildkite-agent-custom:latest .
docker push my-registry/buildkite-agent-custom:latest

Use the custom image in your pipeline:

steps:
  - label: "Build"
    command: "make build"
    agents:
      queue: kubernetes
    image: my-registry/buildkite-agent-custom:latest

Using agent hooks

For packages needed occasionally or for testing, install them at runtime using an agent hook. This approach adds latency to job startup but avoids maintaining custom images.

Create a pre-command hook that installs the required packages:

#!/bin/bash
set -euo pipefail

if ! command -v zip &> /dev/null; then
  apt-get update && apt-get install -y zip
fi

Create a ConfigMap with the hook:

kubectl create configmap buildkite-hooks \
  --from-file=pre-command=pre-command \
  --namespace buildkite

Configure the controller to use the hook:

config:
  agent-config:
    hooks-path: /buildkite/hooks
    hooksVolume:
      name: buildkite-hooks
      configMap:
        name: buildkite-hooks
        defaultMode: 493

AWS-specific packages

Several packages in the Elastic CI Stack for AWS are AWS-specific and may not be needed when running on Kubernetes:

• amazon-ssm-agent: Provides AWS Systems Manager access. Not applicable in Kubernetes.
• aws-cfn-bootstrap: Used for CloudFormation stack signaling. Not applicable in Kubernetes.
• ec2-instance-connect: Provides SSH access to EC2 instances. Use kubectl exec for pod access instead.
• amazon-cloudwatch-agent: For CloudWatch metrics and logs. Use Kubernetes-native observability tools or configure container logging to forward to CloudWatch if required.
• mdadm and nvme-cli: Low-level disk management tools. Kubernetes manages storage through PersistentVolumes.

If your builds use the AWS CLI for operations like S3 uploads or ECR authentication, include it in a custom container image or use the appropriate Buildkite plugins. See Amazon ECR authentication for ECR-specific guidance.