Omni Comment Buildkite Plugin

Combine outputs from many jobs into a single comment on pull requests.

Usage

Create a file named omni-comment.yml to define the section IDs that can appear in the comment:

sections:
  - test_results
  - deploy_preview
  - perf_stats

Add the plugin to your Buildkite pipeline:

steps:
  - label: 'Test'
    command: 'npm test'
    plugins:
      - mskelton/omni-comment#v1:
          section: test_results
          message: 'All tests passed! ✅'
          token: '${GITHUB_TOKEN}'

Comment from a file

To comment from a file on disk, you can use the file-path option instead of message:

steps:
  - label: 'Test'
    command: 'npm test > test-results.txt'
    plugins:
      - mskelton/omni-comment#v1:
          section: test_results
          file-path: test-results.txt
          token: '${GITHUB_TOKEN}'

Configuration Options

Option	Description	Required	Default
section	The section ID that matches with the value in omni-comment.yml	✅
token	GitHub auth token for commenting on pull requests	✅
message	Comment body content
file-path	File path containing the comment body
title	A title for the section (creates expandable/collapsible section)
collapsed	Collapse the section by default (only used if title is set)		false
repo	The repository where to create the comment		$BUILDKITE_REPO
pr-number	The pull request number where to create the comment		$BUILDKITE_PULL_REQUEST
config	Path to the config file		omni-comment.yml

How does it work?

I built this action to solve a problem we have at Ramp of lots of CI outputs that each need to post back to the PR via comments. The problem is, the more comments you have, the more noisy it gets.

The idea was, what if you had a single comment that contained everything? Test results, deploy preview URLs, warnings, etc. When you try to build that though, there are some challenges.

First, it should support workflows running in parallel, so the order in which the comments are posted is non deterministic. However, we want the order of sections in the comment to be consistent between runs. Additionally, we need to support updating the comment if you push a new commit, and the test results are now passing instead of failed.

The GitHub issue comments API only supports sending a complete comment body when making updates, so if we just get the current value and send it back with our updates, its possible that two separate jobs update the comment at the same time and one of the updates will be lost. To workaround this, we need a way for jobs to acquire a “lock” on the comment, so that they can safely get the current comment value, make edits, and push the updated value back to GitHub.

What better locking mechanism than reactions! Turns out, the create reaction API will return a 201 Created status when a reaction is newly created and a 200 OK when the reaction already exists. Using this subtle API detail, this action will attempt to acquire a lock by creating the reaction and waiting for a 201 Created status code. If it receives a 200 OK status code, it will sleep and retry until it is able to acquire a lock (it will fail after 30 seconds if it fails to acquire a lock). Once the lock is acquired, the existing comment will be downloaded, edited, and pushed back to GitHub. After updating the comment, the lock is released by deleting the reaction.

Simple right? 😉