Group step

A group step can contain various sub-steps, and display them in a single logical group on the Build page.

For example, you can group all of your linting steps or all of your UI test steps to keep the Build page less messy. Sub-groups and nested groups are not supported.

The group step also helps manage dependencies between a collection of steps, for example, "step X" depends_on everything in "group Y".

A group step can be defined in your pipeline settings or your pipeline.yml file.

Here is an example of using the group step:

pipeline.yml
steps:
  - group: ":lock_with_ink_pen: Security Audits"
    key: "audits"
    steps:
      - label: ":brakeman: Brakeman"
        command: ".buildkite/steps/brakeman"
      - label: ":bundleaudit: Bundle Audit"
        command: ".buildkite/steps/bundleaudit"
      - label: ":yarn: Yarn Audit"
        command: ".buildkite/steps/yarn"
      - label: ":yarn: Outdated Check"
        command: ".buildkite/steps/outdated"

This is how it's displayed in the UI:

Job groups displayed in the Buildkite UI

Only the first 50 jobs within a build header can ever be displayed in the UI, so you might not see all of your groups at all times. However, the jobs are fine and will still show up on the build page.

Group step attributes

Required attributes:

group Name of the group in the UI. In YAML, if you don't want a label, pass a `~`. Can also be provided in the `label` attribute if `null` is provided to the `group` attribute.
Type: string or null
steps A list of steps in the group; at least 1 step is required. Allowed step types: wait, trigger, command/commands, block, input.
Type: array

Optional attributes:

allow_dependency_failure Whether to continue to run this step if any of the steps named in the depends_on attribute fail.
Default: false
depends_on A list of step or group keys that this step depends on. This step or group will only run after the named steps have completed. See managing step dependencies for more information.
Example: "test-suite"
if A boolean expression that omits the step when false. See Using conditionals for supported expressions.
Example: build.message != "skip me"
key A unique string to identify the step, block, or group.
Keys can not have the same pattern as a UUID (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).
Example: "test-suite"
Alias: identifier
label The label that will be displayed in the pipeline visualisation in Buildkite (name of the group in the UI). Supports emoji.
Example: ":hammer: Tests" will be rendered as "🔨 Tests"
notify Allows you to trigger build notifications to different services. You can also choose to conditionally send notifications based on pipeline events.
Example: "github_commit_status:"
skip Whether to skip this step or not. Passing a string provides a reason for skipping this command. Passing an empty string is equivalent to false. Note: Skipped steps will be hidden in the pipeline view by default, but can be made visible by toggling the 'Skipped jobs' icon.
Example: true
Example: false
Example: "My reason"

Parallel groups

If you put two or more group steps in a YAML config file consecutively, they will run in parallel. For example:

pipeline.yml
# 1.sh and 3.sh will start at the same time.
# 2.sh will start when 1.sh finishes, and 4.sh will start
# when 3.sh finishes.
steps:
  - group: "first"
    steps:
      - command: "1.sh"
      - wait
      - command: "2.sh"
  - group: "second"
    steps:
      - command: "3.sh"
      - wait
      - command: "4.sh"

Running jobs in parallel has some limitations:

  • Parallel groups will be displayed ungrouped if the build's jobs are truncated because Buildkite doesn't currently store or calculate any information about the number of jobs in a non-parallel group.
  • If a parallel step exists within a group, parallel jobs are treated as regular jobs within a step group - so you can't have parallel groups within step groups. So, for example, a group that contains two steps each with parallel: 4 will display eight jobs in it, with no visual indication that those eight jobs are two parallel steps.
  • If a parallel job group is within a named group, the groups are handled as though the parallel group isn't there.
  • It's impossible to have a parallel job with only some of the jobs within a group, as they're all created on the same YAML step entry.

Using wait steps in job groups

You can have wait steps in a group. Such steps operate independently of other groups. For example, both groups will operate independently here, meaning d.sh won't wait on a.sh to finish. Note also that wait steps are counted in the group step total, so both Group01 and Group02 contain 3 steps.

pipeline.yml
steps:
  - group: "Group01"
    depends_on: "tests"
    steps:
      - command: "a.sh"
      - wait
      - command: "b.sh"

  - group: "Group02"
    depends_on: "tests"
    id: "toast"
    steps:
      - command: "c.sh"
      - wait
      - command: "d.sh"

  - command: "yay.sh"
    depends_on: "toast"

Group merging

If you upload a pipeline that has a group or label that matches the group of the step that uploaded it, those groups will be merged together in the Buildkite UI.

This merging behavior only applies if the group step with the matching group or label is the first step within the uploaded pipeline.

Note that inside a single pipeline, groups with the same group or label will not be merged in the Buildkite UI.

You can't define the same key twice

Trying to create different groups or steps with the same key attribute will result in an error.

For example, you have a YAML file:

pipeline.yml
steps:
  - group: "Setup"
    steps:
      - commands:
        - "buildkite-agent pipeline upload"
        - echo "start"

And this YAML file uploads a pipeline that has a group with the same name:

pipeline.yml
steps:
  - group: "Setup"
    steps:
      - command: "docker build"

These groups will be merged into one in the UI, and the docker build step will be added to the existing group.

Similarly, if you have a YAML file:

pipeline.yml
steps:
  - group: ~
    label: "Setup"
    steps:
      - commands:
        - "buildkite-agent pipeline upload"
        - echo "start"

And this YAML file uploads a pipeline that has a group with the same label:

pipeline.yml
steps:
  - group: ~
    label: "Setup"
    steps:
      - command: echo "proceed"

These groups will be merged into one in the UI, and the echo "proceed" step will be added to the existing group.

Example

:buildkite: Group steps An example of how to group steps in a pipeline github.com/buildkite/group-step-example