--- title: "Buildkite plugins, Docker, and shared environment variables" date: "2023-11-23" author: "Lizette Rabuya, Mel Kaulfuss" description: "This blog covers some of the things you'll need to know to have your plugins share environment variables in Docker." tags: "Plugin, CI/CD, DevOps 101" readingTime: "5 minute read" --- # Buildkite plugins, Docker, and shared environment variables Plugins are modular by design, which means you can combine multiple plugins to perform certain actions in your pipeline workflows. There are some common gotchas to be aware of if you're using Docker, and want to expose environment variables created in one pipeline, with another pipeline. This blog helps understand how to solve these issues.

Plugins make life easier—they’re reusable, composable pieces of functionality that extend and adapt your CI/CD workflows. You can easily integrate additional features, tools, and custom scripts to really improve productivity and efficiency. There are a wide array of plugins available in our [plugins directory](https://buildkite.com/docs/plugins/directory), or you can even [create your own](https://buildkite.com/docs/plugins/writinghttps://buildkite.com/docs/plugins/writing). Being modular by design means you can “combine” multiple plugins to carry out a series of actions in a single command step—which is awesome! But…it’s an age-old fact that as soon as dependencies are introduced, things can get complex. Often you’ll have one plugin depending on the results from another plugin, which means plugins have to be defined together and in a [specific order](https://buildkite.com/docs/agent/v3/hooks#job-lifecycle-hooks) to achieve the desired result.  This blog seeks to answer one of the more common questions we hear in support requests. > **Question:** *Why aren’t environment variables set in X plugin available to a Docker container?* > > **Answer:** *Because environment variables aren’t automatically available to Docker containers.* ## Sharing environment variables between plugins Multiple plugins that need to share environment variables should be defined within a single `plugins` step. Each individual step will run on a different agent, meaning environment variables set for a plugin in one step won’t be accessible to a plugin defined in a different `plugins` step. This pipeline uses the [aws-assume-role-with-web-identity](https://github.com/buildkite-plugins/aws-assume-role-with-web-identity-buildkite-plugin) plugin. It exports the AWS credentials so the assumed role can be used in the step.
steps: 
  - label: ":rocket: list AWS env vars"
    command: env | grep AWS
    plugins:
      - aws-assume-role-with-web-identity:
          role-arn: arn:aws:iam::AWS-ACCOUNT-ID:role/SOME-ROLE
The logs show the AWS credentials have been exported.
Image of The logs show that the AWS role has been assumed.
If we use another plugin in the same command step as this plugin, we would assume the exported AWS role credentials would be readily available. However, even if plugins are defined and run together in the same step, there are some sneaky gotchas that snag a lot of users when first using multiple plugins with Docker.

Exposing environment variables to plugins in Docker

The following example uses the `aws-assume-role-with-web-identity` plugin, along with the `docker` plugin.
steps:
  - label: ":face_with_monocle: list all env vars"
    command: env
    plugins:
      - aws-assume-role-with-web-identity:
          role-arn:arn:aws:iam::AWS-ACCOUNT-ID:role/SOME-ROLE
      - docker#v5.7.0:
          image: bash
When listing the environment variables that are available to the `docker` plugin by default, we’d expect to see *something*, because we’ve already validated that we’ve correctly assumed the AWS role necessary to do this. The logs however, show the environment variables were not accessible from the Docker container’s execution environment.
Image of We don't see any environment variables in the logs.
A Docker container does not have access to the environment on its host, as it runs as its own instance. This includes any custom plugins that run in Docker and require access to environment variables. In the following example, our `do-something-with-aws` plugin will need access to the AWS auth credentials to authenticate the `auth-role` configuration.
steps:
  - label: ":face_with_monocle: show user identity"
    command: bash -c 'aws sts get-caller-identity'
    plugins:   
      - aws-assume-role-with-web-identity:        
          role-arn: arn:aws:iam::AWS-ACCOUNT-ID:role/SOME-ROLE    
      - gh-user/do-something-with-aws:
          auth-role:SOME-ROLE
The AWS auth credentials environment variables need to be explicitly defined in the `do-something-with-aws` plugin. You can do this by passing them in using the `-e` or `--env` flag, and the variable name when running the Docker container in the custom plugin’s script.
docker run -e AWS_SESSION_TOKEN -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY .... /
....
The plugin will then have access to the environment variables it needs to do its thing in AWS.
## Exposing environment variables as argument variables for Docker Compose Another common question that crops up is: > How do I expose an environment variable’s value to the `docker-compose` plugin when building a Docker image? The [Docker Compose plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin) is one of our most popular plugins, so if you’re using Docker and Buildkite you’ll be reaching for this plugin—it lets you build, run, and push build steps using Docker Compose. Before getting started, it’s important to be aware that a build’s environment variables won’t automatically propagate as environment variables to the Docker image built using the `docker-compose` plugin. By default, Docker Compose creates whatever environment variables are available for interpolation from the `docker-compose.yml` file, but doesn't automatically pass them into containers. Docker uses argument variables—also known as *build-time variables*. When building an image, these are “announced” in the Dockerfile with the `ARG` instruction. To pass environment variables in as the image is built, the `docker-compose` plugin uses the [`args`](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin#build-arguments) configuration. In the following `.buildkite/pipeline.yml`, the `${BUILDKITE_BUILD_AUTHOR}` environment variable is set as the `IMAGE_AUTHOR` argument variable in the `args` configuration.
- name: ":docker:  Build CI Container"
  key: build-ci
  plugins:
    - docker-compose#v4.15.0: null
        build: app
        args:
          - IMAGE_AUTHOR=${BUILDKITE_BUILD_AUTHOR}
The issue in this scenario is that the `IMAGE_AUTHOR` variable will only be available in the build image, but not on the future containers built from this image. To pass environment variable values from the build image, use the `ARG` variable to set the `ENV` variable, as shown in the following snippet from a Dockerfile:
# expect a build-time variable
ARG IMAGE_AUTHOR

# set the default value
ENV IMAGE_AUTHOR=$IMAGE_AUTHOR
This way, unless overridden, the default value is available to any container started from the image. ## Conclusion These are some of the most common complexities to be aware of when sharing environment variables between plugins and Docker in Buildkite. Plugins become incredibly useful once you've mastered composing them with the right environment variables exposed and shared. You're able to: * Abstract core pieces of functionality from your users. * Centrally manage configurations that can be included in as many pipelines as you need. * Do those things in AWS. Or, do almost anything you can imagine. Plugins unlock a lot of power in customizing your Buildkite pipelines.