Elastic CI Stack for AWS

The Elastic CI Stack for AWS gives you a private, autoscaling Buildkite Agent cluster in your own AWS account. This guide will lead you through getting started with the stack :cloudformation:🎉

Creating an AWS EC2 Key Pair

The first thing you’ll need is an EC2 Key Pair to allow you to SSH into EC2 instances created by the Elastic CI stack. If you already have a Key Pair that you'd like to use, you can head straight to Launching the Stack.

To create an EC2 Key Pair, sign into the AWS Management Console, click “Services ▼” on top left, then go to the EC2 service:


Then go to “Key Pairs” under “Network & Security”:

AWS EC2 Key Pair

Click the “Create key pair” button in the top left hand corner and give it a name.

AWS EC2 Create Key Pair

Launching the Stack

Go to the Agents page on Buildkite and select the “AWS” tab:

Buildkite AWS Agents

Click “Launch Stack” :red_button:

Launch Buildkite Elastic CI Stack
AWS Select Template Screen

After clicking “Next” you’ll configure the stack using the details from Buildkite:

AWS Parameters

By default the stack uses a job queue of default but you can specify any other queue name you’d like. See the Buildkite Agent Job Queue docs for more info.

Review all configured options. Once you're ready, check both checkboxes:

  • I acknowledge that AWS CloudFormation might create IAM resources with custom names.”_
  • I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND

Then click “Create stack”:

AWS Create Stack Button

After creating the stack, you'll be taken to the CloudFormation console. Click the “Refresh” icon in the top right hand corner of the screen until you see the stack status is CREATE_COMPLETE.

AWS Elastic CI Stack Create Complete

You've now got a working Elastic CI Stack ready to run builds! 🎉

Running your first build

We’ve created a sample bash-parallel-example sample pipeline for you to test with your new autoscaling stack. Click the “Add to Buildkite” button below (or on the GitHub Readme):

Add Bash Example to Buildkite

Click “Create Pipeline”. Your pipeline is now ready for its first build. Click "New Build" in the top right and choose a build message (perhaps a little party :partyparrot:?):

Triggering Buildkite Build

Once your build is created head back to AWS EC2 Auto Scaling Groups to monitor the Elastic CI Stack creating new EC2 instances:

AWS EC2 Auto Scaling Group Menu

Select the “buildkite-AgentAutoScaleGroup-xxxxxxxxxxxx” group and then the “Instances” tab. You'll see instances starting up to run your new build and after a few minutes they'll transition from Pending to InService:

AWS Auto Scaling Group Launching

Once the instances are ready you'll see them appear on your Buildkite Agents page:

Buildkite Connected Agents

And then your build will start running on your new agents:

Your First Build

Congratulations on running your first Elastic CI Stack build on Buildkite! 👏🎉

Further documentation

You can read more about the Elastic CI Stack over on the GitHub repository: