---
name: "Monorepo State"
description: "Support plugin for the official Buildkite monorepo-diff plugin — tracks each service's last successful build commit in S3-compatible storage so only genuinely-changed sub-pipelines run."
author: "hasithaishere"
repo: "monorepo-state-buildkite-plugin"
stars: 0
official: false
version: "v0.2.0"
lastUpdated: "2026-07-12T16:15:37.000Z"
---

# Monorepo State Buildkite Plugin

A **support plugin for the official
[`monorepo-diff`](https://github.com/buildkite-plugins/monorepo-diff-buildkite-plugin)
Buildkite plugin** — keeps monorepo sub-pipelines from re-running when nothing
they own has changed.

It tracks each service's **last successful build commit** in an
**S3-compatible bucket**, then computes the changed files since that commit so
monorepo-diff triggers only the sub-pipelines that actually need to run. If a
service's last build **failed**, it is diffed from its previous *success* — so it
re-runs until it passes, even without new changes. (Both the official
`monorepo-diff#…` and the `chronotc/monorepo-diff#…` plugin refs are recognised.)

The heavy lifting is a small Go binary (`bms`, "Buildkite Monorepo State")
delivered by a downloader-shim hook — no runtime on the agent beyond `git`.

> 📖 **Full step-by-step guide:** [docs/USAGE.md](https://github.com/hasithaishere/monorepo-state-buildkite-plugin/blob/HEAD/docs/USAGE.md)

## Why S3

S3 is the universal object-storage contract. The same backend reaches **AWS S3,
MinIO, Cloudflare R2, and Google Cloud Storage (S3 interop)** by changing only an
endpoint URL. Credentials come from the AWS SDK default chain, so the
[`aws-assume-role-with-web-identity`](https://github.com/buildkite-plugins/aws-assume-role-with-web-identity-buildkite-plugin)
plugin, OIDC, instance profiles, and env credentials all work with **no access
keys in your pipeline YAML**.

## How it works

| Phase | Where | What happens |
|-------|-------|--------------|
| **diff** | upload step `pre-command` | The plugin runs first: `bms diff` reads each watched service's last-success hash from S3 and computes the changed files (first run / missing history → all files). The result is **published for the whole build** (see below) and can also be fed to monorepo-diff via `diff: "bms diff"`. |
| **set** | sub-pipeline `post-command` | On success only, `bms set` records the current commit as that service's last-success hash. |

The `environment` hook downloads `bms` and puts it on `PATH` before any command
runs, so monorepo-diff can call `bms diff` regardless of plugin order.

### Published diff output (monorepo-level reuse)

In diff mode the `pre-command` hook computes the changed files once and exposes
them so any step can reuse the result without recomputing:

- **Environment variables** for the current step's command and later hooks:
  - `MONOREPO_STATE_CHANGED_FILES` — newline-separated list.
  - `MONOREPO_STATE_CHANGED_COUNT` — number of changed files.
- **Build meta-data**, readable from *any* step in the pipeline:
  `buildkite-agent meta-data get monorepo-state:changed-files`
  (key configurable via `metadata-key`).

So monorepo-diff's `diff:` can reuse the published value instead of recomputing:

```yaml
      - chronotc/monorepo-diff#v1.2.0:
          diff: "buildkite-agent meta-data get monorepo-state:changed-files"
```

## Usage

**Upload step** — compute changes and trigger sub-pipelines:

```yaml
steps:
  - plugins:
      - hasithaishere/monorepo-state#v0.1.0:
          bucket: my-monorepo-state
          region: ap-southeast-1
          config-path: ".buildkite/pipeline.yml"   # or watch-paths: [...]
      - chronotc/monorepo-diff#v1.2.0:
          diff: "bms diff"
          watch:
            - path: service/app-server
              config: { trigger: app-server-pipeline }
```

**Sub-pipeline** — record success:

```yaml
steps:
  - command: make deploy
    plugins:
      - hasithaishere/monorepo-state#v0.1.0:
          bucket: my-monorepo-state
          region: ap-southeast-1
          service-path: service/app-server
```

After all sub-pipelines pass, record the pipeline-wide baseline once with
`main-pipeline: true` (see [examples/save-step.yml](https://github.com/hasithaishere/monorepo-state-buildkite-plugin/blob/HEAD/examples/save-step.yml)).

## Configuration

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `bucket` | string | **required** | S3-compatible bucket holding state objects. |
| `region` | string | `us-east-1` | Bucket region. R2 wants `auto`. |
| `endpoint-url` | string | — | Custom endpoint for MinIO / R2 / GCS interop. |
| `prefix` | string | `buildkite/store` | Key prefix for all state objects. |
| `force-path-style` | boolean | `false` | Use path-style addressing (required for MinIO). |
| `config-path` | string | — | Pipeline YAML to extract monorepo-diff watch paths from. |
| `watch-paths` | string \| list | — | Service paths to diff (alternative to `config-path`). |
| `forced-deploy-paths` | string \| list | — | Bypass state/git and emit these paths verbatim. |
| `metadata-key` | string | `monorepo-state:changed-files` | Build meta-data key the pre-command hook publishes changed files under. |
| `service-path` | string | — | Service to save state for (save mode). |
| `main-pipeline` | boolean | `false` | Save the pipeline-wide baseline (save mode). |
| `version` | string | the plugin's released version | Override the `bms` binary release to download. Defaults to the version baked into the checked-out plugin (kept in sync by release-please), so pinning `#vX.Y.Z` fetches the matching binary. If the version can't be resolved or downloaded, the build fails. |
| `download` | boolean | `true` | `false` → use a `bms` already on `PATH`. |

Provide exactly one of `watch-paths` / `config-path` for diff, and exactly one of
`service-path` / `main-pipeline` for save.

### State layout

```
<prefix>/<org>/<pipeline>/last-success-hash/<service-path>
<prefix>/<org>/<pipeline>/last-success-hash/main-pipeline
```

## Backend examples

- **AWS S3** — just `bucket` + `region`; credentials from the SDK chain / OIDC.
- **MinIO** — `endpoint-url: https://minio.example.com`, `force-path-style: true`,
  `region: us-east-1`; creds via `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`.
- **Cloudflare R2** — `endpoint-url: https://<account>.r2.cloudflarestorage.com`,
  `region: auto`; R2 API token as AWS creds.
- **Google Cloud Storage (interop)** — `endpoint-url: https://storage.googleapis.com`
  with an HMAC key as AWS creds.

## Development

```bash
go test ./...                       # unit tests
shellcheck hooks/* lib/*.bash       # lint hooks
goreleaser build --snapshot --clean # cross-compile locally
```

Releases are cut by GoReleaser on a tag, publishing `bms_<os>_<arch>` binaries
and `checksums.txt` that the shim verifies.

## Contributing

Contributions are welcome! Please read [CONTRIBUTING.md](https://github.com/hasithaishere/monorepo-state-buildkite-plugin/blob/HEAD/CONTRIBUTING.md) for the
development setup, coding guidelines, and PR checklist. In short:

- Keep Go logic in `internal/*` and the binary thin; hooks stay in Bash.
- `bms diff` must write **only** the changed-file list to stdout (logs to stderr).
- Read config through `internal/config` (env-driven, hermetically testable) —
  never parse plugin YAML by hand.
- Add tests, then make sure these pass before opening a PR:

  ```bash
  go vet ./...
  go test ./...
  shellcheck -x hooks/* lib/*.bash
  bats tests/
  ```

GitHub Actions runs the same checks on every pull request. Use the issue and pull
request templates when reporting bugs, requesting features, or submitting changes.
Found a security issue? Please follow [SECURITY.md](https://github.com/hasithaishere/monorepo-state-buildkite-plugin/blob/HEAD/SECURITY.md) rather than
opening a public issue.

## License

MIT — see [LICENSE](https://github.com/hasithaishere/monorepo-state-buildkite-plugin/blob/HEAD/LICENSE).