👋 Welcome to Buildkite Pipelines! You can use Pipelines to build your dream CI/CD workflows on a secure, scalable, and flexible platform. This tutorial takes you through creating a basic pipeline from an example.
To complete this tutorial, you'll need:
A Buildkite account. If you don't have one already, create a free account.
When you create a new organization as part of sign-up, you'll be guided through a flow to create and run a starter pipeline. Complete that before continuing, and keep your agent running to continue using it in this tutorial.
To enable the YAML steps editor in Buildkite:
- Select Settings > YAML Migration to open the YAML migration settings.
- Select Use YAML Steps for New Pipelines, then confirm the action in the modal.
Git. This tutorial uses GitHub, but Buildkite can work with any version control system.
Before creating a pipeline, take a moment to understand Buildkite's architecture and the advantages it provides. Buildkite uses a hybrid architecture to separate:
- The control panel
- The build environment
Buildkite runs the control panel as a SaaS product, and you run the build environment on your own infrastructure. In other words, Buildkite handles the orchestration, and you bring the compute. That means you can fine-tune and secure the build environment to suit your particular use case and workflow.
The hybrid architecture reduces the maintenance burden on your team compared to self-hosted solutions and gives you the security and control missing from fully SaaS build tools. See Hybrid architecture to learn more.
The program that executes work is called an agent in Buildkite. An agent is a small, reliable, and cross-platform build runner that connects your infrastructure to Buildkite. It polls Buildkite for work, runs jobs, and reports results. You can install agents on local machines, cloud servers, or other remote machines. You need at least one agent to run builds.
If you're already running an agent, skip to the next step.
To install and run an agent:
Decide where you want to run the agent.
Most engineers start by running an agent on their local machine while playing around with pipeline definitions before setting up a long-term solution.
Follow the instructions for where you want to install the agent.
To install locally, see:
Or see all installation options.
Ensure you configure the agent token, which connects the agent to your Buildkite account.
To confirm that your agent is running, and configured correctly with your credentials, go to Agents. You should see a list of all agents linked to the account and their status.
Pipelines are how Buildkite represents a CI/CD workflow. You define each pipeline with a series of steps to run. When you trigger a pipeline, you create a build, and steps are dispatched as jobs to run on agents. Jobs are independent of each other and can run on different agents.
Next, you'll create a new pipeline based on one of the following example pipelines:
Both result in the same behavior: the pipeline definition is uploaded from the repository (
.buildkite/pipeline.yml), then a script runs that prints output to the logs.
To create a pipeline:
Select Add to Buildkite for the appropriate example based on where your agent is running.
On the New Pipeline page, leave all fields with their pre-filled default values and select Create Pipeline. This associates the example repository with your new pipeline, and adds a step to upload the full pipeline definition from the repository.
On the next page showing your pipeline name, click New Build. In the modal that opens, create a build using the pre-filled details.
- In the Message field, enter a short description for the build. For example, My first build.
- Select Create Build.
The page for the build then opens and begins running:
After triggering the build, you can view the output as it runs and the full results when complete. The output for each step shows in the job list.
Expand the row in the job list to view the output for a step. For example, selecting Example Script shows the following:
In the output, you'll see:
- A pre-command hook ran and printed some text in the logs.
- The agent checked out the repository.
- The agent accessed different environment variables shown in the job environment.
- The script ran and printed text to the logs and uploaded an image as an artifact of the build.
Beyond the log, select one of the other tabs to see the artifacts, a timeline breakdown, and the environment variables.
That's it! You've installed an agent, run a build, and checked the output. 🎉
Now try creating your own pipeline.