Buildkite SDK

Share step definitions across pipelines

Use the SDK with ordinary language modules to share individual steps without sharing an entire pipeline. For example, a shared library can define how to build and push a Docker image. Each pipeline then chooses its image name, build context, and other steps. Unlike pipeline templates, these functions can accept parameters and compose steps at runtime.

The following Python example defines a reusable command step. Use Python 3.10 or later and install the SDK as described in Python, then save this module alongside your pipeline generator:

from shlex import quote

def docker_image_step(*, key, image, context):
    return {
        "label": f"Build and push {image}",
        "key": key,
        "command": (
            f"docker build --tag {quote(image)} {quote(context)}"
            f" && docker push {quote(image)}"
        ),
    }

Import the function into a pipeline generator and combine the shared step with pipeline-specific steps:

from buildkite_sdk import Pipeline
from shared_steps import docker_image_step

pipeline = Pipeline()
pipeline.add_step(docker_image_step(
    key="api-image",
    image="registry.example.com/api:example-tag",
    context="services/api",
))
pipeline.add_step({"label": "Test API", "command": "./scripts/test-api"})
print(pipeline.to_yaml())

A second pipeline can import the same function with different parameters:

from buildkite_sdk import Pipeline
from shared_steps import docker_image_step

pipeline = Pipeline()
pipeline.add_step(docker_image_step(
    key="worker-image",
    image="registry.example.com/worker:example-tag",
    context="services/worker",
))
print(pipeline.to_yaml())

Upload each generator's output from its respective pipeline. For example, the API pipeline's upload step is:

steps:
  - label: "Upload API pipeline"
    command: "python3 .buildkite/api_pipeline.py | buildkite-agent pipeline upload"

Replace the example registry, image tags, paths, and test command with values for your project. The agents running the generated image steps need Docker and permission to push to the registry. For Amazon ECR, configure ECR authentication on those agents before the step runs; do not put registry credentials in the generated YAML. The image and test steps run independently. Add step dependencies if a step needs the image first, and use a unique step key for each call within a build.

For pipelines in the same repository, keep the shared module in that repository. To share definitions across repositories, distribute the module as a versioned internal Python package and install a pinned version in each pipeline's generation environment. The same pattern works with Ruby gems, JavaScript packages, or modules in the other supported SDK languages. Test the generated YAML when updating the shared library so consuming pipelines can adopt changes deliberately.

JavaScript and TypeScript (Node.js)

This section explains how to install and use the Buildkite SDK for JavaScript and TypeScript (Node.js-based) projects.

Installing

To install the Buildkite SDK for Node.js to your local development environment, run this command:

npm install @buildkite/buildkite-sdk

Using

The following code example demonstrates how to import the Buildkite SDK into a simple TypeScript script, which then generates a Buildkite Pipelines step for a simple command step that runs echo 'Hello, world!', and then outputs this step to either JSON or YAML format:

const { Pipeline } = require("@buildkite/buildkite-sdk");

const pipeline = new Pipeline();

pipeline.addStep({
    command: "echo 'Hello, world!'",
});

// JSON output
// console.log(pipeline.toJSON());
// YAML output
console.log(pipeline.toYAML());

When you're ready to upload your output JSON or YAML steps to Buildkite Pipelines, you can do so from a currently running pipeline step:

# For example, in your pipeline's Settings > Steps, and with ts-node installed to your agent:
steps:
  - label: ":pipeline: Run dynamic pipeline steps"
    command: ts-node .buildkite/dynamicPipeline.ts | buildkite-agent pipeline upload

API documentation

For more detailed API documentation on the Buildkite SDK for TypeScript, consult the Buildkite SDK's TypeScript API documentation.

Python

This section explains how to install and use the Buildkite SDK for Python projects.

Installing

To install the Buildkite SDK for Python (with uv) to your local development environment, run this command:

uv add buildkite-sdk

Using

The following code example demonstrates how to import the Buildkite SDK into a simple Python script, which then generates a Buildkite Pipelines step for a simple simple command step that runs echo 'Hello, world!', and then outputs this step to either JSON or YAML format:

from buildkite_sdk import Pipeline

pipeline = Pipeline()
pipeline.add_step({"command": "echo 'Hello, world!'"})

# JSON output
# print(pipeline.to_json())
# YAML output
print(pipeline.to_yaml())

When you're ready to upload your output JSON or YAML steps to Buildkite Pipelines, you can do so from a currently running pipeline step:

# For example, in your pipeline's Settings > Steps:
steps:
  - label: ":pipeline: Run dynamic pipeline steps"
    command: python3 .buildkite/dynamic_pipeline.py | buildkite-agent pipeline upload

API documentation

For more detailed API documentation on the Buildkite SDK for Python, consult the Buildkite SDK's Python API documentation.

Go

This section explains how to install and use the Buildkite SDK for Go projects.

Installing

To install the Buildkite SDK for Go to your local development environment, run this command:

go get github.com/buildkite/buildkite-sdk/sdk/go

Using

The following code example demonstrates how to import the Buildkite SDK into a simple Go script, which then generates a Buildkite Pipelines step for a simple command step that runs echo 'Hello, world!', and then outputs this step to either JSON or YAML format:

package main

import (
  "fmt"
  "github.com/buildkite/buildkite-sdk/sdk/go/sdk/buildkite"
)

func main() {
    pipeline := buildkite.Pipeline{}

    pipeline.AddStep(buildkite.CommandStep{
        Command: &buildkite.CommandStepCommand{
            String: buildkite.Value("echo 'Hello, world!"),
        },
    })

    // JSON output
    // json, err := pipeline.ToJSON()
    // if err != nil {
    //     log.Fatalf("Failed to serialize JSON: %v", err)
    // }

    // fmt.Println(json)

    // YAML output
    yaml, err := pipeline.ToYAML()
    if err != nil {
        log.Fatalf("Failed to serialize YAML: %v", err)
    }

    fmt.Println(yaml)
}

When you're ready to upload your output JSON or YAML steps to Buildkite Pipelines, you can do so from a currently running pipeline step:

# For example, in your pipeline's Settings > Steps:
steps:
  - label: ":pipeline: Run dynamic pipeline steps"
    command: go run .buildkite/dynamic_pipeline.go | buildkite-agent pipeline upload

API documentation

For more detailed API documentation on the Buildkite SDK for Go, consult the Buildkite SDK's Go API documentation.

Ruby

This section explains how to install and use the Buildkite SDK for Ruby projects.

Installing

To install the Buildkite SDK for Ruby to your local development environment, run this command:

gem install buildkite-sdk

Using

The following code example demonstrates how to import the Buildkite SDK into a simple Ruby script, which then generates a Buildkite Pipelines step for a simple command step that runs echo 'Hello, world!', along with a label attribute, and then outputs this step to either JSON or YAML format:

require "buildkite"

pipeline = Buildkite::Pipeline.new

pipeline.add_step(
  label: "some-label",
  command: "echo 'Hello, World!'"
)

# JSON output
# puts pipeline.to_json
# YAML output
puts pipeline.to_yaml

When you're ready to upload your output JSON or YAML steps to Buildkite Pipelines, you can do so from a currently running pipeline step:

# For example, in your pipeline's Settings > Steps:
steps:
  - label: ":pipeline: Run dynamic pipeline steps"
    command: ruby .buildkite/dynamic_pipeline.rb | buildkite-agent pipeline upload

API documentation

For more detailed API documentation on the Buildkite SDK for Ruby, consult the Buildkite SDK's Ruby API documentation.

C Sharp

This section explains how to install and use the Buildkite SDK for C# (.NET) projects.

Installing

To install the Buildkite SDK for .NET to your local development environment, run this command:

dotnet add package Buildkite.Sdk

Using

The following code example demonstrates how to import the Buildkite SDK into a simple C# script, which then generates a Buildkite pipeline with a build command step, a wait step, and a test command step, and then outputs these steps to either JSON or YAML format:

using Buildkite.Sdk;
using Buildkite.Sdk.Schema;

var pipeline = new Pipeline();

pipeline.AddStep(new CommandStep
{
    Label = ":hammer: Build",
    Command = "dotnet build"
});

pipeline.AddStep(new WaitStep());

pipeline.AddStep(new CommandStep
{
    Label = ":test_tube: Test",
    Command = "dotnet test"
});

// JSON output for `buildkite-agent pipeline upload`
// Console.WriteLine(pipeline.ToJson());
// YAML output for `buildkite-agent pipeline upload`
Console.WriteLine(pipeline.ToYaml());

When you're ready to upload your output JSON or YAML steps to Buildkite Pipelines, you can do so from a currently running pipeline step:

# For example, in your pipeline's Settings > Steps:
steps:
  - label: ":pipeline: Run dynamic pipeline steps"
    command: dotnet run --project .buildkite/DynamicPipeline.csproj | buildkite-agent pipeline upload

Also included in this section are examples of how to use the Buildkite SDK for C# with other step types, including a more complex command step, a block step, wait step, trigger step, and group step, as well as environment variables.

Command steps

This code example demonstrates a more complex command step with additional options:

pipeline.AddStep(new CommandStep
{
    Label = ":dotnet: Build",
    Key = "build",
    Command = "dotnet build --configuration Release",
    Agents = new AgentsObject { ["queue"] = "linux" },
    TimeoutInMinutes = 30
});

Block steps

This code example demonstrates how to implement a block step:

pipeline.AddStep(new BlockStep
{
    Block = ":rocket: Deploy to Production?",
    Prompt = "Are you sure?"
});

Wait steps

This code example demonstrates how to implement a wait step:

pipeline.AddStep(new WaitStep());
pipeline.AddStep(new WaitStep { ContinueOnFailure = true });

Trigger steps

This code example demonstrates how to implement a trigger step:

pipeline.AddStep(new TriggerStep
{
    Trigger = "deploy-pipeline",
    Build = new TriggerBuild { Branch = "main" }
});

Group steps

This code example demonstrates how to implement a group step:

pipeline.AddStep(new GroupStep
{
    Group = ":test_tube: Tests",
    Steps = new List<IGroupStep>
    {
        new CommandStep { Label = "Unit", Command = "dotnet test" },
        new CommandStep { Label = "Integration", Command = "dotnet test --filter Integration" }
    }
});

Environment variables

This code example demonstrates how to access environment variables:

using Buildkite.Sdk;

var branch = EnvironmentVariable.Branch;
var commit = EnvironmentVariable.Commit;
var buildNumber = EnvironmentVariable.BuildNumber;

Developing the Buildkite SDK

Since the Buildkite SDK is open source, you can make your own contributions to this SDK. Learn more about how to do this from the Buildkite SDK's README.