The Docker Compose plugin is one of our most used plugins, and we received a request to support Compose v2, so we did it. We also took the opportunity to simplify the cache configuration in pipeline definitions.
Caching before the change:
steps: - label: ":docker: Build Intermediate Image" plugins: - docker-compose#v4.15.0: build: myservice_intermediate # docker-compose.yml is the same as myservice but has `target: intermediate` image-name: buildkite-build-${BUILDKITE_BUILD_NUMBER} image-repository: index.docker.io/myorg/myrepo/myservice_intermediate cache-from: - myservice_intermediate:index.docker.io/myorg/myrepo/myservice_intermediate:${BUILDKITE_BRANCH} - myservice_intermediate:index.docker.io/myorg/myrepo/myservice_intermediate:latest - wait - label: ":docker: Build Final Image" plugins: - docker-compose#v4.15.0: build: myservice image-name: buildkite-build-${BUILDKITE_BUILD_NUMBER} image-repository: index.docker.io/myorg/myrepo cache-from: - myservice:index.docker.io/myorg/myrepo/myservice_intermediate:buildkite-build-${BUILDKITE_BUILD_NUMBER}:intermediate # built in step above - myservice:index.docker.io/myorg/myrepo/myservice:${BUILDKITE_BRANCH} - myservice:index.docker.io/myorg/myrepo/myservice:latest
In that example, there are two groups:
myservice_intermediate:buildkite-build-${BUILDKITE_BUILD_NUMBER}
, named "intermediate."myservice:${BUILDKITE_BRANCH}
and myservice:latest
, with a default name.Now, caching matches Docker's underlying cache-from implementation, so you get more flexibility and resiliency. cache-from
has the form SERVICE:CACHE_FROM_SPEC
and supports any cache type and specification supported by your underlying Docker engine.
😍 Caching now:
steps: - label: ":docker Build an image" plugins: - docker-compose#v5.2.0: build: app push: app:index.docker.io/myorg/myrepo:my-branch cache-from: - "app:type=registry,ref=myregistry:port/myrepo/myapp:my-branch" - "app:myregistry:port/myrepo/myapp:latest" - wait - label: ":docker: Push to final repository" plugins: - docker-compose#v5.2.0: push: - app:myregistry:port/myrepo/myapp:latest
A glimpse into some of the smaller features, fixes, and enhancements our engineering and product teams are constantly shipping.
Fix | Interpolate environment variables as part of group configuration. | Pipelines |
Fix | Fix the 404 issue when clicking rebuild. | Pipelines |
Fix | Fix the UI bug with parallelism by including the number in the build header. | Pipelines |
Fix | Correctly terminate single-job instances in elastic-ci-stack. | Pipelines |
Fix | Ensure audit logs include all pipeline update steps. | Pipelines |
Feature | Allow for ordering annotations by priority using `--priority`. | Pipelines |
Fix | Prevent cluster queues from becoming the default when changing their descriptions. | Pipelines |
Fix | Transition builds to `started` when no longer failing. | Pipelines |
Fix | Fix slow response times when getting `repositories.json` file. | Pipelines |
Fix | Fix title element for default cluster queue page. | Pipelines |
Fix | Fix `Unknown` Slack notifications on skipped builds. | Pipelines |
Enhancement | Add `soft_failed` attribute to trigger jobs API presenter. | Pipelines |
Fix | Fix signed pipelines with wait steps. | Pipelines |
Fix | Resolve `github_commit_status` from `Pending` when using groups and wait steps. | Pipelines |
Fix | Ensure agent registration happens when using clusters. | Pipelines |
Fix | Display correct UI status for group step pills that contain skipped and running jobs. | Pipelines |
Fix | Move jobs in `WAITING_FAILED` and to `CANCELED` when a build is canceled. | Pipelines |
Fix | On Windows, allow case-insensitive interpolation of runtime environment variables. | Pipelines |
Enhancement | Add `ClusterId` to Buildkite environment variables. | Pipelines |
Enhancement | Add support for pipeline templates in the Terraform provider. | Support |
Enhancement | Release v0.2.0 of the Migration CLI. | Support |
Fix | Fix `--target` option implementation in the docker-compose plugin. | Support |
Feature | Add `--buildkite-inline-cache` property to the docker-compose plugin. | Support |
Fix | Fix an issue with the usage of `list-object-v2`. | Support |
Enhancement | Add device-level IO limit options to the Docker plugin. | Support |
Enhancement | Container-definitions configuration is now optional on the ECS plugin. | Support |
Enhancement | Improve SSO IP pinning for users on IPv6 addresses. | Platform |
Enhancement | Expand Buildkite emoji library. | Platform |
Enhancement | Notify organizations when reaching either 50% or 80% of inclusion usage. | Platform |
Enhancement | Allow CSV export of organization members. | Platform |
Enhancement | Improve display of hidden API access token values. | Platform |
Enhancement | Identify inactive notification services via `Last sent` timestamps. | Platform |
Enhancement | Include more information in the empty state when no data is available due to data retention. | Test Analytics |
Fix | Fix missing suite reliability chart hover tooltip. | Test Analytics |
Fix | Adjust suite title and public badge positioning to account for long titles. | Test Analytics |
Enhancement | Update docs to reflect currently available test frameworks. | Test Analytics |
Enhancement | Remove `beta` reference in docs for languages out of beta. | Test Analytics |
Fix | Remove double footer border on Test Analytics views. | Test Analytics |
Fix | Ensure test runner exits with correct exit code in the JavaScript Test Collector. | Test Analytics |
Enhancement | Refresh run issues documentation. | Test Analytics |
Fix | Remove `Continue setup` link when user doesn't have full access to the suite. | Test Analytics |
Enhancement | Remove ActiveSupport dependency from test collector Ruby gem. | Test Analytics |
Enhancement | Update JUnit XML setup instructions to guide users to the Buildkite plugin. | Test Analytics |
Enhancement | Clearly indicate which API calls require access token authentication. | Documentation |
Enhancement | Promote organizations and add new teams endpoints to API documentation. | Documentation |
Enhancement | Provide detail on how to manage clusters, queues, and agent tokens via UI or API. | Documentation |
Enhancement | Explain how to migrate unclustered agents and pipelines to a cluster via UI or API. | Documentation |
Enhancement | Create new documentation section for Buildkite Packages, including a step-by-step tutorial. | Documentation |
Feature | Add `--allowed-environment-variables` option to agent start. | Agent |
Enhancement | Cap agent retries to 24 hours for cases that previously tried forever. | Agent |
Fix | Prevent agent from retrying more requests that will never succeed. | Agent |
Enhancement | Upgrade agent to run on Go 1.22. | Agent |
Enhancement | Log enabled experiments on agent start. | Agent |
Enhancement | Extend graceful cancellation from checkout phase to all job phases. | Agent |
Enhancement | Allow silencing of specific warning logs using `--disable-warnings-for`. | Agent |
Fix | Resolve Git commit information automatically when using a custom checkout hook. | Agent |
Fix | Fix PowerShell hooks on Windows. | Agent |