Group step

Group step attributes

Required attributes:

Optional attributes:

Agent-applied attributes

These attributes are only applied by the Buildkite Agent when uploading a pipeline (buildkite-agent pipeline upload), since they require direct access to your code or repository to process correctly.

Example pipeline, demonstrating various forms of if_changed:

steps:
  # if_changed can specify a single glob pattern.
  # Note that YAML requires some strings to be quoted.
  - label: "Only run if a .go file anywhere in the repo is changed"
    if_changed: "**.go"

  # Braces {,} let you combine patterns and subpatterns.
  # Note that this syntax is whitespace-sensitive: a space within a
  # pattern is treated as part of the file path to be matched.
  - label: "Only run if go.mod or go.sum are changed"
    if_changed: go.{mod,sum}
    # Wrong: go.{mod, sum}

  # Combining the two previous examples:
  - label: "Run if any Go-related file is changed"
    if_changed: "{**.go,go.{mod,sum}}"

  # A less Go-centric example:
  - label: "Run for any changes within app/ or spec/"
    if_changed: "{app/**,spec/**}"

  # From version 3.109 of the Buildkite Agent, lists of patterns
  # are supported. If any changed file matches any of the patterns,
  # the step is run. This can be a more ergonomic alternative to
  # using braces.
  - label: "Run if any Go-related file is changed"
    if_changed:
      - "**.go"
      - go.{mod,sum}

  - label: "Run for any changes in app/ or spec/"
    if_changed:
      - app/**
      - spec/**

  # From version 3.109 of the Buildkite Agent, `include` and
  # `exclude` are supported attributes. As for `if_changed`, these
  # attributes may contain single patterns or lists of patterns.
  # `exclude` eliminates changed files from causing a step to run.
  # If the `exclude` attribute is present, then the `include` attribute,
  # along with its pattern or list of patterns, is required too.
  - label: "Run for changes in spec/, but not in spec/integration/"
    if_changed:
      include: spec/**
      exclude: spec/integration/**

  - label: "Run for api and internal, but not api/docs or internal .py files"
    if_changed:
      include:
        - api/**
        - internal/**
      exclude:
        - api/docs/**
        - internal/**.py

Parallel groups

If you put two or more group steps in a YAML config file consecutively, they will run in parallel. For example:

# 1.sh and 3.sh will start at the same time.
# 2.sh will start when 1.sh finishes, and 4.sh will start
# when 3.sh finishes.
steps:
  - group: "first"
    steps:
      - command: "1.sh"
      - wait
      - command: "2.sh"
  - group: "second"
    steps:
      - command: "3.sh"
      - wait
      - command: "4.sh"

Running jobs in parallel has some limitations:

• Parallel groups will be displayed ungrouped if the build's jobs are truncated because Buildkite doesn't currently store or calculate any information about the number of jobs in a non-parallel group.

• If a parallel step exists within a group, parallel jobs are treated as regular jobs within a step group - so you can't have parallel groups within step groups. So, for example, a group that contains two steps each with parallel: 4 will display eight jobs in it, with no visual indication that those eight jobs are two parallel steps.

• If a parallel job group is within a named group, the groups are handled as though the parallel group isn't there.

• It's impossible to have a parallel job with only some of the jobs within a group, as they're all created on the same YAML step entry.

Using wait steps in job groups

You can have wait steps in a group. Such steps operate independently of other groups. For example, both groups will operate independently here, meaning d.sh won't wait on a.sh to finish. Note also that wait steps are counted in the group step total, so both Group01 and Group02 contain 3 steps.

steps:
  - group: "Group01"
    depends_on: "tests"
    steps:
      - command: "a.sh"
      - wait
      - command: "b.sh"

  - group: "Group02"
    depends_on: "tests"
    id: "toast"
    steps:
      - command: "c.sh"
      - wait
      - command: "d.sh"

  - command: "yay.sh"
    depends_on: "toast"

Group merging

If you upload a pipeline that has a group or label that matches the group of the step that uploaded it, those groups will be merged together in the Buildkite UI.

This merging behavior only applies if the group step with the matching group or label is the first step within the uploaded pipeline.

Note that inside a single pipeline, groups with the same group or label will not be merged in the Buildkite UI.

For example, you have a YAML file:

steps:
  - group: "Setup"
    steps:
      - commands:
        - "buildkite-agent pipeline upload"
        - echo "start"

And this YAML file uploads a pipeline that has a group with the same name:

steps:
  - group: "Setup"
    steps:
      - command: "docker build"

These groups will be merged into one in the UI, and the docker build step will be added to the existing group.

Similarly, if you have a YAML file:

steps:
  - group: ~
    label: "Setup"
    steps:
      - commands:
        - "buildkite-agent pipeline upload"
        - echo "start"

And this YAML file uploads a pipeline that has a group with the same label:

steps:
  - group: ~
    label: "Setup"
    steps:
      - command: echo "proceed"

These groups will be merged into one in the UI, and the echo "proceed" step will be added to the existing group.

Example

Group steps An example of how to group steps in a pipeline github.com/buildkite/group-step-example