Resources
/
Examples
/
Concurrency Group Pipeline

steps:
  - command: echo "Running concurrency gate example"
    key: deploy-example

  - label: ":large_green_square: OPEN GATE: Tests"
    key: "start-tests"
    concurrency_group: "tests"
    command: echo "Opening Test gate"

  - wait

  - command: echo "Waiting for Tests to start..."
    key: "wait-start-tests"

  - label: "Tests"
    key: "tests"
    concurrency_group: "tests"
    parallelism: 5
    depends_on: ["wait-start-tests"]
    command: echo "run-tests.sh"

  - label: ":large_red_square: CLOSE GATE: Tests"
    key: "end-tests"
    concurrency_group: "tests"
    command: echo "Closed the Test gate"

  - wait

  - command: echo "Waiting for Tests to end..."
    key: "wait-end-tests"

  - label: ":large_green_square: OPEN GATE: Deploy to Staging"
    key: "start-deploy-staging"
    concurrency_group: "deploy-staging"
    depends_on: ["wait-end-tests"]
    command: echo "Opening Deploy to Staging gate"

  - wait

  - command: echo "Waiting for Deploy to Staging to start..."
    key: "wait-start-deploy-staging"

  - label: "Deploy to Staging"
    key: "deploy-staging"
    concurrency_group: "deploy-staging"
    depends_on: ["wait-start-deploy-staging"]
    command: echo "deploy.sh staging"

  - label: ":large_red_square: CLOSE GATE: Deploy to Staging"
    key: "end-deploy-staging"
    concurrency_group: "deploy-staging"
    command: echo "Closed Deploy to Staging gate"

  - wait

  - command: echo "Waiting for Deploy to Staging to end..."
    key: "wait-end-deploy-staging"

  - label: ":large_green_square: OPEN GATE: Deploy to Production"
    key: "start-deploy-production"
    concurrency_group: "deploy-production"
    depends_on: ["wait-end-tests"]
    command: echo "Closed Deploy to Production gate"

  - wait

  - command: echo "Waiting for Deploy to Production to start..."
    key: "wait-start-deploy-production"

  - label: "Deploy to Production"
    key: "deploy-production"
    concurrency_group: "deploy-production"
    depends_on: ["wait-start-deploy-production"]
    command: echo "deploy.sh production"

  - label: ":large_red_square: CLOSE GATE: Deploy to Production"
    key: "end-deploy-production"
    concurrency_group: "deploy-production"
    command: echo "Closed Deploy to Production gate"

  - wait

  - command: echo "Waiting for Deploy to Production to end..."
    key: "wait-end-deploy-production"

  - label: ":large_green_square: OPEN GATE: Integration Tests"
    key: "start-integration-tests"
    concurrency_group: "integration-tests"
    depends_on: ["wait-end-deploy-staging", "wait-end-deploy-production"]
    command: echo "Closed Integration Tests gate"

  - wait

  - command: echo "Waiting for Integration Tests to start..."
    key: "wait-start-integration-tests"

  - label: "Integration Tests"
    key: "integration-tests"
    concurrency_group: "integration-tests"
    parallelism: 3
    depends_on: ["wait-start-integration-tests"]
    command: echo "integration-tests.sh"

  - label: ":large_red_square: CLOSE GATE: Integration Tests"
    key: "end-integration-tests"
    concurrency_group: "integration-tests"
    command: echo "Closed Integration Tests gate"

  - wait

  - command: echo "Waiting for Integration Tests to end..."
    key: "wait-end-integration-tests"
 
  - label: ":large_green_square: OPEN GATE: Approve Deployment"
    key: "start-approval"
    concurrency_group: "approve-deployment"
    depends_on: ["wait-end-integration-tests"]
    command: echo "Opening Approve Deployment gate"

  - wait

  - command: echo "Waiting for Approve Deployment to start..."
    key: "wait-start-approval"

  - label: "Approve Deployment"
    key: "approval"
    concurrency_group: "approve-deployment"
    depends_on: ["wait-start-approval"]
    command: echo "approve-deployment.sh"

  - label: ":large_red_square: CLOSE GATE: Approve Deployment"
    key: "end-approval"
    concurrency_group: "approve-deployment"
    command: echo "Closed Approve Deployment gate"

  - wait

  - command: echo "Waiting for Approve Deployment to end..."
    key: "wait-end-approval"

Buildkite Concurrency Gates with Parallelism Pipeline Example

This repository demonstrates how to use Buildkite concurrency groups and gates to control the flow of a pipeline. It shows how to combine parallelism, step dependencies, and concurrency limits to model gated workflows.

👉 See this pipeline in action: buildkite.com/buildkite/concurrency-group-pipeline-example

See the full Getting Started Guide for step-by-step instructions on how to get this running.

How it works

This example models a simple gated workflow with concurrency limits and parallelism:

Visual Description

The Running concurrency gate example step prints a quick message to start the pipeline.
The Tests step runs in parallel (up to 5 at once), controlled by a concurrency gate.
When tests finish, the gates for Deploy to Staging and Deploy to Production open. These run independently.
Once both deploy steps complete, Integration Tests run in parallel (up to 3 at once).
After integration tests finish, the Approve Deployment step is triggered.

🔄 Parallelism runs multiple jobs at once. ⛔ Concurrency groups limit how many jobs run at once — even across separate builds.

Think of concurrency like a traffic light: it controls flow, even when you’ve got lots of lanes.

More on this in the Buildkite docs on controlling concurrency.

Pipeline Steps

Running concurrency gate example: Just prints a message to kick things off.
Tests: Runs up to 5 jobs in parallel, using the tests concurrency group.
Deploy to Staging: Waits for tests to finish. Uses the deploy-staging concurrency group.
Deploy to Production: Also waits for tests. Uses the deploy-production concurrency group.
Integration Tests: Waits for both deploy steps. Runs up to 3 jobs in parallel in the integration-tests group.
Approve Deployment: Waits for integration tests, and runs in the approval concurrency group.

Each step uses depends_on and concurrency_group to manage gates and limits.

🧠 Advanced Usage Notes

This setup mixes depends_on with concurrency groups to create gates between phases.

You can:

Gate steps across branches or fan-in/fan-out setups
Limit job execution globally (not just per pipeline)
Create human approval gates at the end of automated flows

Want help modeling a complex pipeline? Reach out to support — we love this stuff.

License

See LICENSE (MIT)

More examples

Node.js Docker

Runs tests inside a Docker container using the official Node.js image.

Updated 7/17/2025

Python Docker

Runs Python tests inside a Docker container using uv for dependency management.

Updated 7/17/2025

Bash

Runs a simple Bash script with artifacts and inline output.

Updated 7/9/2025

Rails Parallel Docker

Runs parallel tests for a Rails app using Docker and Knapsack.

Updated 7/17/2025

🔀

Dynamic Pipeline

Dynamically generates steps using a shell script.

Updated 7/18/2025

Python (pipenv)

Tests a Python project using pipenv for dependency management.

Updated 7/17/2025

Start turning complexity into an advantage

Create an account to get started with a 30-day free trial. No credit card required.

Get started View pricing

Scale-Out Delivery Platform→

Capabilities

Pipelines→

Test Engine→

Package Registries→

Mobile Delivery Cloud→

Bring your own compute

Hosted compute

Replace Jenkins

Workflows for AI/ML

Testing at scale

Monorepo mojo

Bazel orchestration

Example pipelines

Webinars

Blog

Public pipelines

Case studies

Events

Follow Buildkite

About

Careers

Follow Buildkite

Buildkite Concurrency Gates with Parallelism Pipeline Example

How it works

Pipeline Steps

🧠 Advanced Usage Notes

License

More examples

Node.js Docker

Python Docker

Bash

Rails Parallel Docker

Dynamic Pipeline

Python (pipenv)

Start turning complexity into an advantage

Platform

Hosting options

Resources

Company

Solutions

Legal

Support