Linux and Windows setup for the Elastic CI Stack for AWS
This guide leads you through getting started with the Elastic CI Stack for AWS for Linux and Windows. With the help of the Elastic CI Stack for AWS, you are able to launch a private, autoscaling Buildkite agent cluster in your own AWS account.
Read on for detailed instructions, or jump straight in:
Before you start
Most Elastic CI Stack for AWS features are supported on both Linux and Windows. The following Amazon Machine Images (AMIs) are available by default in all supported regions. The operating system and architecture will be selected based on the values provided for the InstanceOperatingSystem
and InstanceTypes
parameters:
- Amazon Linux 2023 (64-bit x86)
- Amazon Linux 2023 (64-bit ARM, Graviton)
- Windows Server 2019 (64-bit x86)
If you want to use the AWS CLI instead, download config.json.example
, rename it to config.json
, add your Buildkite Agent token (and any other config values), and then run the below command:
aws cloudformation create-stack \
--output text \
--stack-name buildkite \
--template-url "https://s3.amazonaws.com/buildkite-aws-stack/latest/aws-stack.yml" \
--capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM CAPABILITY_AUTO_EXPAND \
--parameters "$(cat config.json)"
Required and recommended skills
The Elastic CI Stack for AWS does not require familiarity with the underlying AWS services to deploy it. However, to run builds, some familiarity with the following AWS services is required:
- AWS CloudFormation
-
Amazon EC2 (to select an EC2
InstanceTypes
stack parameter appropriate for your workload) - Amazon S3 (to copy your git clone secret for cloning and building private repositories)
Elastic CI Stack for AWS provides defaults and pre-configurations suited for most use cases without the need for additional customization. Still, you'll benefit from familiarity with VPCs, availability zones, subnets, and security groups for custom instance networking.
For post-deployment diagnostic purposes, deeper familiarity with EC2 is recommended to be able to access the instances launched to execute Buildkite jobs over SSH or AWS Systems Manager Sessions.
Billable services
Elastic CI Stack for AWS creates its own VPC (virtual private cloud) by default. Best practice is to set up a separate development AWS account and use role switching and consolidated billing. You can check out this external tutorial for more information on how to "Delegate Access Across AWS Accounts".
The Elastic CI Stack for AWS template deploys several billable Amazon services that do not require upfront payment and operate on a pay-as-you-go principle, with the bill proportional to usage.
Service name | Purpose | Required |
---|---|---|
EC2 |
Deployment of instances |
☑️ |
EBS |
Root disk storage of EC2 instances |
☑️ |
Lambda |
Scaling of Auto Scaling group and modifying Auto Scaling group's properties |
☑️ |
Systems Manager Parameter Store |
Storing the Buildkite agent token |
☑️ |
CloudWatch Logs |
Logs for instances and Lambda scaler |
☑️ |
CloudWatch Metrics |
Metrics recorded by Lambda scaler |
☑️ |
S3 |
Charging based on storage and transfers in/and out of the secrets bucket (on by default) |
❌ |
Buildkite services are billed according to your plan.
What's on each machine?
- Amazon Linux 2023
- The Buildkite Agent
- Git and Git LFS
- Docker
- Docker Compose
- AWS CLI - useful for performing any ops-related tasks
- jq - useful for manipulating JSON responses from CLI tools such as AWS CLI or the Buildkite API
For more details on what versions are installed on a given Elastic CI Stack, see the corresponding release announcement.
On both Linux and Windows, the Buildkite agent runs as user buildkite-agent
.
Supported builds
This stack is designed to run your builds in a share-nothing pattern similar to the 12 factor application principals:
- Each project should encapsulate its dependencies through Docker and Docker Compose.
- Build pipeline steps should assume no state on the machine (and instead rely on build meta-data, build artifacts or S3).
- Secrets are configured using environment variables exposed using the S3 secrets bucket.
By following these conventions you get a scalable, repeatable, and source-controlled CI environment that any team within your organization can use.
Custom images
Custom images help teams ensure that their agents have all required tools and configurations before instance launch. This prevents instances from reverting to the base image state when agents restart, which would lose any manual changes made during run time.
Custom AMIs can be used with the Elastic CI Stack for AWS by specifying the ImageId
parameter. You can use any AMI available to your AWS account, but it is recommend starting with Buildkite's base Packer templates as a starting point. The Packer templates used to create the default stack images are available in the packer directory of the Elastic CI Stack for AWS repository.
Requirements
To use the Packer templates provided, you will need the following installed on your system:
- Docker
- Make
- AWS CLI
The following AWS IAM permissions are required to build custom AMIs using the provided packer templates:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CopyImage",
"ec2:CreateImage",
"ec2:CreateKeyPair",
"ec2:CreateSecurityGroup",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteKeyPair",
"ec2:DeleteSecurityGroup",
"ec2:DeleteSnapshot",
"ec2:DeleteVolume",
"ec2:DeregisterImage",
"ec2:DescribeImageAttribute",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus",
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSnapshots",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DetachVolume",
"ec2:GetPasswordData",
"ec2:ModifyImageAttribute",
"ec2:ModifyInstanceAttribute",
"ec2:ModifySnapshotAttribute",
"ec2:RegisterImage",
"ec2:RunInstances",
"ec2:StopInstances",
"ec2:TerminateInstances"
],
"Resource": "*"
}
]
}
It is also recommended that you have a base knowledge of:
- Packer
- HashiCorp configuration language (HCL)
- Bash or PowerShell (depending on the operating system of choice)
Creating an image
To create a custom AMI, use the provided Packer templates to build new images with your modifications. First, make your desired changes to the Packer templates, then run the Makefile
in the root directory to begin the build process.
This Makefile
provides several build targets, each running Packer in a Docker container:
Command | Description |
---|---|
|
Build all AMI variants |
|
Build Amazon Linux 2023 (64-bit x86) AMI only |
|
Build Amazon Linux 2023 (64-bit ARM, Graviton) AMI only |
|
Build Windows Server 2019 (64-bit x86) AMI only |
By default, all builds target the us-east-1
region and use your default AWS profile. The make
command can be prefixed with environment variables to change the behavior of the build.
Variable | Default | Description |
---|---|---|
|
|
Target AWS region for AMI creation |
|
(system default) |
Specific AWS profile to use |
|
(unset) |
Enable Packer debug logging ( |
|
|
Build identifier passed to Packer |
|
|
Whether this is a release build |
|
|
Instance type for ARM64 builds |
|
|
Instance type for AMD64 builds |
|
|
Instance type for Windows builds |
For example, you could build an AMD64 Linux image in the eu-west-1
region using a smaller instance type and a specific AWS profile by running:
AMD64_INSTANCE_TYPE="t3.medium" \
AWS_REGION="eu-west-1" \
AWS_PROFILE="assets-profile" \
make packer-linux-amd64.output
Once your image build is completed, the AMI will be stored in your AWS account and the AMI ID is displayed in your terminal output. You can also find the AMI ID in the corresponding output file (such as packer-linux-amd64.output
).
Launching the stack
Go to the Agents page on Buildkite and select the AWS tab:

Click Launch Stack


After clicking Next, configure the stack using your Buildkite agent token:

If you don't know your agent token, there is a Reveal Agent Token button available on the right-hand side of the Agents page:

By default the stack uses a job queue of default
, but you can specify any other queue name you like.
A common example of setting a queue for a dedicated Windows agent can be achieved with the following in your pipeline.yml
after you've set up your Windows stack:
steps:
- command: echo "hello from windows"
agents:
queue: "windows"
For more information, see Buildkite Agent job queues, specifically Targeting a queue.
Review the parameters, see Elastic CI Stack for AWS parameters for more details.
Once you're ready, check these three checkboxes:
- I acknowledge that AWS CloudFormation might create IAM resources.
- I acknowledge that AWS CloudFormation might create IAM resources with custom names.
- I acknowledge that AWS CloudFormation might require the following capability:
CAPABILITY_AUTO_EXPAND
Then click Create stack:

After creating the stack, Buildkite takes you to the CloudFormation console. Click the Refresh icon in the top right hand corner of the screen until you see the stack status is CREATE_COMPLETE
.

You now have a working Elastic CI Stack for AWS ready to run builds! 🎉
Running your first build
We've created a sample bash-parallel-example sample pipeline for you to test with your new autoscaling stack. Click the Add to Buildkite button below (or on the GitHub README):
Click Create Pipeline. Depending on your organization's settings, the next step will vary slightly:
- If your organization uses the web-based steps editor (default), your pipeline is now ready for its first build. You can skip to the next step.
- If your organization has been upgraded to the YAML steps editor, you should see a Choose a Starting Point wizard. Select Pipeline Upload from the list:
Click New Build in the top right and choose a build message (perhaps a little party :partyparrot:
?):

Once your build is created, head back to AWS EC2 Auto Scaling Groups to watch the Elastic CI Stack for AWS creating new EC2 instances:

Select the buildkite-AgentAutoScaleGroup-xxxxxxxxxxxx group and then the Instances tab. You'll see instances starting up to run your new build and after a few minutes they'll transition from Pending to InService:

Once the instances are ready they will appear on your Buildkite Agents page:

And then your build will start running on your new agents:

Congratulations on running your first Elastic CI Stack for AWS build on Buildkite! 🎉