Linux hosted agents

Sizes

Buildkite offers a selection of Linux instance types (each based on a different combination of size and architecture, known as an instance shape), allowing you to tailor your hosted agent resources to the demands of your jobs. The architectures supported include AMD64 (x64_86) and ARM64 (AArch64).

Note the following about Linux hosted agent instances.

• Extra large instances are available on request.

• To accommodate different workloads, instances are capable of running up to 8 hours.

If you need extra large instances, or longer running hosted agents (over 8 hours), please contact Support at support@buildkite.com.

Concurrency

Linux hosted agents can operate concurrently when running your Buildkite pipeline jobs.

This concurrency is measured by the number of hosted agent machines that are allocated for your jobs at any given time. A machine is counted from the moment it starts booting to acquire a job, since that's when the machine starts consuming capacity. As a result, concurrency may not always match the exact number of running jobs, particularly with short-lived workloads.

The number of Linux hosted agents (of a Buildkite hosted queue) that can process your pipeline jobs concurrently is calculated by your Buildkite plan's maximum combined vCPU value divided by your instance shape's vCPU value. See the Buildkite pricing page for details on the Linux Concurrency that applies to your plan.

For example, if your Buildkite plan provides you with a maximum combined vCPU value of up to 48, and you've configured a Buildkite hosted queue with the LINUX_AMD64_4X16 (Medium AMD64) instance shape, whose vCPU value is 4, then the number of concurrent hosted agents that can run jobs on this queue is 12 (that is, 48 / 4 = 12).

When concurrency limits are exceeded, additional jobs will be queued until sufficient capacity becomes available.

Security

Customer security is paramount to Buildkite, where our source code, build artifacts and deployment processes represent some of our most valuable and sensitive assets.

The shift from a self-hosted to a Buildkite hosted architecture for Buildkite Agents, introduces the potential for new attack vectors and shared responsibility models, and hence, additional security considerations.

The security model for Buildkite hosted agents has the following characteristics to address these security considerations and to mitigate attack risks.

• Infrastructure and isolation security: Buildkite employs a multi-tenant architecture, where each job runs in a completely isolated virtualized environment. Once a job is complete, regardless of its exit status, the virtualized environment is destroyed, along with all its data (except for cache volumes that persist across jobs). This ephemeral approach ensures that customer workloads remain isolated from each other, even though the underlying hardware is shared across multiple customers.

• Physical and operational security: The Buildkite hosted agent fleet operates from multiple Tier 3+ data centers with restricted physical access controls and regular security monitoring. The platform maintains SOC 2 compliance through regular audits of both hardware and software security controls.

Agent images

Buildkite provides a Linux agent image pre-configured with common tools and utilities to help you get started quickly. This image also provides tools required for running jobs on hosted agents.

The image is based on Ubuntu 22.04 and includes the following tools:

• docker
• docker-compose
• docker-buildx
• git-lfs
• node
• aws-cli

You can customize the image that your hosted agents use by creating an agent image.

Create an agent image

Creating an agent image requires you to define a Dockerfile that installs the tools and utilities you require. This Dockerfile should be based on the Buildkite hosted agent base image.

An example Dockerfile that installs the awscli and kubectl:

# Set the environment variable to avoid interactive prompts during awscli installation
ENV DEBIAN_FRONTEND=noninteractive

# Install AWS CLI
RUN apt-get update && apt-get install -y awscli

# Install kubectl using pkgs.k8s.io
RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl \
    && chmod +x kubectl \
    && mv kubectl /usr/local/bin/

You can create an agent image:

1. Select Agents in the global navigation to access the Clusters page.

2. Select the cluster in which to create the new agent image.

   Note: Before continuing, ensure you have created a Buildkite hosted queue (based on Linux architecture) within this cluster. Learn more about how to do this in Create a Buildkite hosted queue.

3. Select Agent Images to open the Agent Images page.

4. Select New Image to open the New Agent Image dialog.

5. Enter the Name for your agent image.

6. In the Dockerfile field, enter the contents of your Dockerfile.

   Notes:

   • The top of the Dockerfile contains the required FROM instruction, which cannot be changed. This instruction obtains the required Buildkite hosted agent base image.
   • Ensure any modifications you make to the existing Dockerfile content are correct before creating the agent image, since mistakes cannot be edited or corrected once the agent image is created.

7. Select Create Agent Image to create your new agent image.

Use an agent image

Once you have created an agent image, you can set it as the default image for any Buildkite hosted queues based on Linux architecture within this cluster. Once you do this for such a Buildkite hosted queue, any agents in the queue will use this agent image in new jobs.

To set a Buildkite hosted queue to use a custom Linux agent image:

1. Select Agents in the global navigation to access the Clusters page.
2. Select the cluster with the Linux architecture-based Buildkite hosted queue whose agent image requires configuring.
3. On the Queues page, select the Buildkite hosted queue based on Linux architecture.
4. Select the Base Image tab to open its settings.
5. In the Agent image dropdown, select your agent image.
6. Select Save settings to save this update.

Delete an agent image

To delete a previously created agent image, it must not be used by any Buildkite hosted queues.

To delete an agent image:

1. Select Agents in the global navigation to access the Clusters page.
2. Select the cluster in which to delete the agent image.
3. Select Agent Images to open the Agent Images page.

4. Select the agent image to delete > Delete.

   Note: If you are prompted that the agent image is currently in use, follow the link/s to each Buildkite hosted queue on the Delete Image message to change the queue's Agent image (from the Base Image tab) to another agent image.

5. On the Delete Image message, select Delete Image and the agent image is deleted.

Using agent hooks

You can create a custom agent image and modify its Dockerfile to embed the following types of job lifecycle hooks as agent hooks:

environment, pre-checkout, checkout, post-checkout, pre-command, command, post-command, pre-artifact, post-artifact, and pre-exit.

Be aware that the pre-bootstrap job lifecycle hook and agent lifecycle hooks operate outside of a job's execution itself, and are therefore not supported within a Buildkite hosted agent context.

To embed hooks in your agent image's Dockerfile:

1. Follow the Create an agent image instructions to begin creating your hosted agent within its Linux architecture-based Buildkite hosted queue.

   As part of this process, modify the agent image's Dockerfile to:

   1. Add the BUILDKITE_ADDITIONAL_HOOKS_PATHS environment variable whose value is the path to where the hooks will be located.
   2. Add any specific hooks to the path defined by this variable.

   An example excerpt from a Dockerfile that would include your own hooks:

   ENV BUILDKITE_ADDITIONAL_HOOKS_PATHS=/custom/hooks
   COPY ./hooks/*.sh /custom/hooks/
   

   This results in an agent image with the directory /custom/hooks that includes any .sh files located at ./hooks/ from where the image is created.

2. Follow the Use an agent image to apply this new agent image to your Buildkite hosted queue.