So, you need to choose a CI/CD tool for your project, and you're starting to realize just how many options there are. It can be a daunting task. But don't worry, we've got you. While each situation is different, there are some key factors you can use to inform your decision.
When it comes to CI/CD, where your pipelines run can significantly impact your development workflow. So that's a good place to start. Three hosting options dominate the industry:
- Managed: A SaaS model where a company hosts the control plane and compute.
- Self-hosted: An on-premise model where you manage the control plane and compute.
- Hybrid: A mixed approach where a company runs the control plane as a SaaS offering, and you manage the compute.
Each approach has advantages and challenges, so finding the right balance for your project is key. In this blog, we'll consider each option in terms of:
- Security
- Speed
- Convenience
- Scalability
- Cost
We'll explain each approach to hosting in more detail, score them, and then give you a simple decision tree to help you assess your requirements.
Managed CI/CD
Managed CI/CD tools are cloud-based, fully managed platforms provided as a SaaS product. You connect your source code, configure pipelines through their interface, and let them handle the compute infrastructure.
They're generally the simplest CI/CD to set up and use because they abstract away most of the infrastructure details. The tool decides how to run your pipelines and manage scaling. For these reasons, managed CI/CD is typically where most people start when beginning their CI/CD journey.
Managed CI/CD architecture
How does managed CI/CD score?
| Category | Score |
|---|---|
| 🔒 Security | ⭐️☆☆☆☆ |
| 🏃 Speed | ⭐️⭐️⭐️☆☆ |
| 🛠 Convenience | ⭐️⭐️⭐️⭐️⭐️ |
| 📈 Scalability | ⭐️⭐️⭐️☆☆ |
| 💸 Cost | ⭐️⭐️☆☆☆ |
Our reasoning:
- Security: These tools need full access to your source code and secrets. They also require you to open your firewall for internal integrations.
- Speed: Limited transparency and configuration of the build environment means you don't have deep control to optimize the compute performance.
- Convenience: These are simple to set up and get going with—grant access to your code, define the pipeline, and start running builds.
- Scalability: They scale with your organization but are typically not transparent with limited configuration.
- Cost: Since the company does everything for you, they charge for it. These can get expensive as your usage increases.
With managed CI/CD, you're paying for convenience but can lose out on performance and flexibility. This makes it a great fit for simple projects with common workflow requirements.
Popular managed CI/CD tools include:
- CircleCI
- GitHub Actions
- Buildkite Pipelines (hosted agents)
- Harness
Self-hosted CI/CD
Self-hosted CI/CD is the polar opposite of managed CI/CD. In this approach, you are entirely responsible for managing your CI/CD infrastructure, including the build environments, keeping the control plane available, and maintaining and upgrading the platform. You host the control plane and build environment on your own servers, on-premises or in a private cloud. That means you control and run the infrastructure from the job execution to the orchestration.
Setting up self-hosted CI/CD requires skill, expertise, and effort, but you have control over every aspect. This is where many companies end up when security is vital, they have strong compliance needs, and have teams of engineers to run the tool.
Self-hosted CI/CD architecture
How does self-hosted CI/CD score?
| Category | Score |
|---|---|
| 🔒 Security | ⭐️⭐️⭐️⭐️☆ |
| 🏃 Speed | ⭐️⭐️⭐️⭐️⭐️ |
| 🛠 Convenience | ⭐️☆☆☆☆ |
| 📈 Scalability | ⭐️☆☆☆☆ |
| 💸 Cost | ⭐️⭐️⭐️☆☆ |
Our reasoning:
- Security: Everything stays within your security perimeter, so you can lock it down as much as you need. However, popular tools like Jenkins and TeamCity have faced big security incidents from running community plugins and closed-source code on your infrastructure.
- Speed: With full control over the setup, you can tailor it to suit your needs.
- Convenience: Doing everything yourself means exactly that. You’ll need to manage the availability and performance of the control plane and compute.
- Scalability: Adapting the scale requires real engineering effort to keep the control plane available and performant.
- Cost: With popular open-source tools available, the cost can be low or free. However, the hidden cost comes with the maintenance requirements. You’ll typically need multiple infrastructure experts to manage your implementation.
In self-hosted CI/CD products, you get a strong security posture and control over performance. But you pay for it with challenging and ongoing work to maintain and optimize the implementation. Operating the control plane often proves complex and messy over time. This makes it a great fit for large companies with teams of platform engineers to manage the infrastructure.
Popular self-hosted CI/CD tools include:
- Jenkins
- TeamCity
- Bamboo
Hybrid CI/CD
The hybrid approach combines elements of both managed and self-hosted CI/CD. You run the compute resources (on your servers or cloud infrastructure), but use a control plane hosted as a SaaS product to simplify CI/CD management.
The company manages the control plane, which communicates with the compute running on your infrastructure. The control plane includes a UI for handling user authentication, controlling the orchestration, and configuring your account. While the experience may differ between tools, the best also provide dashboards, metrics, and visibility into logs.
You control the build environment and can customize it to your needs. That keeps you in control of compute costs, working directly with cloud providers. You have the freedom to set your own balance between cost and speed.
Hybrid CI/CD architecture
How does hybrid CI/CD score?
| Category | Score |
|---|---|
| 🔒 Security | ⭐️⭐️⭐️⭐️☆ |
| 🏃 Speed | ⭐️⭐️⭐️⭐️⭐️ |
| 🛠 Convenience | ⭐️⭐️⭐️☆☆ |
| 📈 Scalability | ⭐️⭐️⭐️⭐️⭐️ |
| 💸 Cost | ⭐️⭐️⭐️☆☆ |
Our reasoning:
- Security: All the sensitive data remains in your control—source code, secrets, and integration with internal systems. This lets you adopt a zero-trust security posture.
- Speed: You get full control over the build environment, meaning you can optimize it for your use case.
- Convenience: You'll need some infrastructure expertise to manage the build environment, but the effort can also help improve how you operate your product.
- Scalability: There's transparent and configurable scaling across the system.
- Cost: You pay for the SaaS component that controls the orchestration of your builds, but you then have full control over the cost of compute. There's no middleman upselling the compute.
Hybrid CI/CD is the solution companies turn to when they have complex use cases and some infrastructure expertise in the team. They use that expertise to focus on managing the build environment specific to their product, not running, scaling, and basically developing their own CI/CD product.
Popular hybrid CI/CD tools include:
- Buildkite Pipelines
- GitHub self-hosted runners
- CircleCI self-hosted runners
Buildkite pioneered this approach to CI/CD, with other tools later starting to add limited support.
Results and decision tree
| Managed | Hybrid | Self-hosted | |
|---|---|---|---|
| Security | ⭐️☆☆☆☆ | ⭐️⭐️⭐️⭐️☆ | ⭐️⭐️⭐️⭐️☆ |
| Speed | ⭐️⭐️⭐️☆☆ | ⭐️⭐️⭐️⭐️⭐️ | ⭐️⭐️⭐️⭐️⭐️ |
| Convenience | ⭐️⭐️⭐️⭐️⭐️ | ⭐️⭐️⭐️☆☆ | ⭐️☆☆☆☆ |
| Scalability | ⭐️⭐️⭐️☆☆ | ⭐️⭐️⭐️⭐️⭐️ | ⭐️☆☆☆☆ |
| Cost | ⭐️⭐️☆☆☆ | ⭐️⭐️⭐️☆☆ | ⭐️⭐️⭐️☆☆ |
| Average | 2.8 ⭐️ | 4.0 ⭐️ | 2.8 ⭐️ |
As you expect with this kind of thing, each approach has tradeoffs. It’s best to consider what's most important to your project and organization. More complex projects with unique requirements will get the most out of the flexibility and control of self-hosted or hybrid CI/CD. Simple projects you want to move quickly with are best suited to managed CI/CD.
While there are many ways to decide which approach to use, we recommend you start by considering your security requirements. Security is job zero for everyone, so this is what our decision tree looks like for new projects:
Conclusion
While there's no hard and fast rule for deciding the right approach to CI/CD for new projects, hopefully this blog gives you a helpful way to frame the decision. Projects with basic requirements or limited resources may find managed solutions appealing for their convenience. Infrastructure teams with CI/CD domain knowledge may run a self-hosted option. Complex projects with specific requirements might lean towards hybrid solutions to retain more flexibility and control. Many teams start with a managed CI/CD setup before graduating to a hybrid tool as their team and project grows—it's not a one-way door.
Whichever path you choose, we recommend you evaluate your project's requirements for security, speed, convenience, scalability, and cost to make an informed decision about the right CI/CD tool for you.
Buildkite Pipelines is a CI/CD tool designed for developer happiness. Easily follow and decipher logs, get observability into key build metrics, and tune for enterprise-grade speed, scale, and security. Every new signup gets a free 30-day trial to test out the key features. See Buildkite Pipelines to learn more.
