Elastic CI Stack for AWS
The Elastic CI Stack for AWS gives you a private, autoscaling Buildkite Agent cluster in your own AWS account. This guide will lead you through getting started with the stack 🎉
On this page:
Creating an AWS EC2 Key Pair
The first thing you’ll need is an EC2 Key Pair to allow you to SSH into EC2 instances created by the Elastic CI stack. If you already have a Key Pair that you'd like to use, you can head straight to Launching the Stack.
To create an EC2 Key Pair, sign into the AWS Console, and go to the EC2 service:
Then go to “Key Pairs” under “Network & Security”:
Click the “Create Key Pair” button in the top left hand corner and give it a name.
Launching the Stack
To launch the stack, you’ll first need to create a Buildkite API Access Token for the stack to use. The stack includes a scheduled Lamda function which connects to the Buildkite API to autoscale EC2 instances.
Go to the “Agents” page on Buildkite and select the “AWS” tab:
Click the “Create an API Access Token” button:
The form is pre-filled with everything you’ll need:
Click “Create New API Access Token” and you'll be taken back to the AWS tab on the agents page with the token ready to use.
Now you’re ready to launch the stack! Click “Launch Stack”
After clicking “Next” you’ll configure the stack using the details from Buildkite:
By default the stack uses a job queue of
default but you can specify any other queue name you’d like. See the Buildkite Agent Job Queue docs for more info.
Once you're ready, click the “I acknowledge that this template might cause AWS CloudFormation to create IAM resources.” checkbox then click “Create”:
After creating the stack, you'll be taken to the CloudFormation console. Click the “Refresh” button in the top right hand corner of the screen until you see the stack being created.
You've now got a working Elastic CI Stack ready to run builds! 🎉
Running your first build
Your pipeline is now ready for its first build. Click "New Build" in the top right and choose a build message (perhaps a little party
Once your build is created head back to AWS EC2 Autoscaling console to monitor the Elastic CI stack creating new EC2 instances:
Select the “buildkite-AgentAutoScaleGroup-xxx” group and then the the “Instances” tab. You'll see instances starting up to run your new build and after a few minutes they'll transition from
Once the instances are ready you'll see them appear on your Buildkite Agents page:
And then your build will start running on your new agents:
Congratulations on running your first Elastic CI Stack build on Buildkite! 👏🎉
You can read more about the Elastic CI Stack over on the GitHub repository: