Clusters

Clusters are a new way of managing your Buildkite agents. They allow teams to self-manage their agent pools, let admins create isolated sets of agents and pipelines within the one Buildkite organization, and help to make agents and queues more discoverable across your organization.

The following diagram shows the architectural change when enabling clusters.

Diagram showing existing architecture and architecture with clusters

Clusters encapsulate groups of agents and pipelines, enabling the following:

  • Clusters are viewable to your entire organization, allowing engineers to better understand the agents and queues available for their pipelines.
  • Individual users or teams can maintain their own clusters. Cluster maintainers can manage queues and agent tokens and add and remove pipelines.
  • Pipelines can be assigned to a cluster, ensuring their builds run only on the agents connected to this cluster. These pipelines can also trigger builds only on other pipelines in the same cluster.

Enable clusters

Any Buildkite administrator can enable clusters for an organization. Once you enable clusters, you can only disable them by contacting support.

To enable clusters:

  1. Navigate to your organization’s pipeline settings.
  2. In Clusters, select Enable Clusters.

Clusters will now appear in the global navigation.

Use clusters alongside unclustered agents and pipelines

Enabling clusters will not impact any of your existing agents or pipelines, nor will you require any workflow-breaking changes for you to try clusters.

Once you’ve enabled clusters, all members of your organization will see Clusters in the global navigation. This will show all your clusters as well as Unclustered agents and pipelines.

Any agents or pipelines not associated with a cluster are called unclustered. To view and manage your unclustered agents, agent tokens, and pipelines, select Unclustered.

To view all running agents in your organization (in a cluster or not), click on All agents in the sidebar.

Set up a cluster

To set up a new cluster:

  1. Navigate to Clusters.
  2. Select Create a Cluster.
  3. Enter a name, description, and emoji.
  4. Select Create Cluster.

Set up queues

When you create your first cluster, it will have an initial default queue.

To create additional queues:

  1. Navigate to the cluster’s Queues.
  2. Select Create a Queue.
  3. Enter a key and description.
  4. Select Create Queue.

Connect agents to a cluster

Agents are associated with a cluster through the cluster’s agent tokens.

To connect an agent:

  1. Navigate to the cluster's Agent tokens.
  2. Copy the auto-generated token.
  3. Use the token with the relevant agents, along with the key from the relevant cluster queue.

You can also create, edit, and revoke other agent tokens from the cluster’s Agent tokens.

Add pipelines to a cluster

Add a pipeline to a cluster to ensure the pipeline’s builds run only on agents connected to that cluster.

To add a pipeline to a cluster:

  1. Navigate to the Pipeline Settings for the pipeline.
  2. Under Cluster Settings, select the relevant cluster.

Add maintainers to a cluster

Only Buildkite administrators or users with the change organization permission can create clusters.

You can assign other users or teams as a cluster’s maintainers to permit them to manage the cluster. Cluster maintainers can:

  • Update or delete the cluster.
  • Manage cluster agent tokens.
  • Add or remove pipelines to the cluster.

To add a maintainer to a cluster:

  1. Navigate to the cluster’s Maintainers.
  2. Select a user or team.
  3. Click Add Maintainer.

Migrate to clusters

If you migrate all your existing agents over to clusters, make sure to add all your pipelines to the relevant clusters. Otherwise, any builds for those pipelines will never find agents to run them.

Pause a queue

You can pause a queue to prevent jobs from being dispatched to agents associated with that queue.

To pause a queue:

  1. Navigate to your cluster’s Queues.
  2. Select the queue you wish to pause.
  3. Select Edit.
  4. Under Queue Management, select Pause Queue.

  5. Enter an optional note in the dialog, and confirm that you wish to pause the queue.

    You can use the note to explain why you're pausing the queue. The note will display on the queue page and any affected builds.

Jobs already dispatched to agents in the queue before pausing will continue to run. New jobs that target the paused queue will wait until the queue is resumed.

Trigger steps do not rely on agents, so they will run unless they have dependencies waiting on the paused queue. The behavior of the jobs they trigger depends on their configuration. If a triggered job targets the paused queue, it will wait until the queue is resumed. If a triggered job does not target the paused queue, it will run as usual.

Select Resume Queue to resume a paused queue. Jobs will resume dispatching to the queue as usual, including any jobs waiting to run.