Job Prioritization

By default, jobs are dispatched (taken from the queue and assigned to an agent) on a first-in-first-out basis. However, job priority and pipeline upload time can affect that order.

Prioritizing specific jobs

Job priority is 0 by default, you can prioritize or deprioritize jobs by assigning them a higher or lower integer value. For example:

pipeline.yml
steps:
  - command: "will-run-last.sh"
    priority: -1
  - command: "will-run-first.sh"
    priority: 1

Job priority is considered before jobs are dispatched to agent queues, so jobs with higher priority are assigned before jobs with lower priority, regardless of which has been longest in the queue.

Prioritization can only be set on jobs, and only applies to command jobs, including plugin commands. Setting a default command job priority across a whole build is coming at the start of 2022.

Job dispatch precedence

Jobs are dispatched in the following order:

  1. Job priority in descending order, highest number to lowest (priority)
  2. Date and time scheduled in ascending order, oldest to most recent (scheduled_at). Note that jobs inherit scheduled_at from pipeline upload jobs, meaning jobs that are uploaded by a pipeline in an older build will be dispatched before builds created after that, and the value of scheduled_at cannot be modified.
  3. Upload order in pipeline, first to last.
  4. Internal id in ascending order, used as a tie breaker if all other value are the same, meaning older jobs will be dispatched first.

Example

Here's an example of prioritizing jobs running on a default branch before pull request jobs:

pipeline.yml
steps:
- label: ":pipeline:"
  agents: {queue: uploaders}
  command: |
    if [[ "$${BUILDKITE_BRANCH}" == "$${BUILDKITE_PIPELINE_DEFAULT_BRANCH}" ]]; then
      export PRIORITY=1
    else
      export PRIORITY=0
    fi
    buildkite-agent pipeline upload <<YAML
    steps:
    - label: priority $${PRIORITY}
      command: sleep 3
      priority: $${PRIORITY}
    YAML