🗣️ Update: GitHub has announced that Merge Queue is now generally available.
There's been an audible buzz of excitement about pull request merge queues lately. The merge queue pattern increases the efficiency and stability of software delivery workflows by automatically handling pull request merges into your busiest target branches. Merge queues ensure that pull requests get merged in the correct order, and significantly reduce CI workloads, and potential delivery blockers.
The endless cycle of main diverging from the version in your PR
Prior to merge queue, merging pull requests into main involved:
- Merging
maininto the pull request branch, so changes wouldn’t break main when merged - Kicking off a build in CI (for these newly pulled and merged changes)
- Discovering another PR was merged before the build finished
- Rinse and repeat 🌀😵💫
- Merging the PR into
main
I’d often be rushing to get my PR merged before someone else snuck theirs through, or I’d forget to pull the latest changes from main into my feature branch before pushing to main. Often, my PR merge would fail, because it wasn’t in sync with the main branch. I saw red–quite literally.
Merge queue automatically builds PRs with the previous commit in the queue.
Merge queues automatically build pull request branches with the most recent change (or PR) to be merged ahead of it in the queue. 🙌
The merge queue isn’t new. Uber and Shopify have been using them for close to five years in their pipelines in order maintain high throughput and low commit turnaround time on their busy main branches (think thousands of merges to main per day!). If you have lots of contributors, and high volume of pull requests going to main and if you have frequent flaky merges blocking subsequent commits and creating a backlog of builds, you might find pull request merge queue to be a useful tool,as it minimizes merges tripping over each other on the way to main. GitHub’s pull request merge queues are available in any public repository owned by an organization, or in private repositories owned by organizations using GitHub Enterprise Cloud.)
GitHub merge queue step-by-step:
- The merge queue creates temporary branches to validate pull request changes prefixed with
main/pr-1,main/pr-2etc. - Once required checks specified in the branch protection rules are satisfied, a pull request is added to the merge queue.
- Changes are grouped into a
merge_groupwith the latestbase_branchversion, as well as changes from any other pull requests ahead of it in the queue. - Once the
base_branchrequired checks pass, themerge_groupchanges are merged into thebase_branch
Getting started with GitHub merge queue in your pipelines
Include the gh-readonly-queue/{base_branch}/* filter pattern in your pipeline's Pipeline Settings → GitHub → Branch Limiting. This means builds will automatically run for merge queues.
Include this Branch filter pattern in pipelines with branch limiting.
In your GitHub repository’s Settings → Webhooks configuration check the merge_group checkbox.
Check the merge groups option in your GitHub repo's webhooks settings
You can optionally "skip builds with existing commits" by selecting this in your pipeline’s Pipeline Settings → GitHub → GitHub Settings. It's worth noting that scheduled builds with the same commit message will no longer build with this option selected.
Check the new "Skip builds with existing commits" checkbox.
The efficiency merge queue offers is exciting, though you should keep in mind that configuring branch protection and required checks don’t currently work in Buildkite and this work is on our radar.
Check out the merge queue docs, and if you run into issues let us know at support@buildkite.com or post in the Buildkite community forum.
