It seems as if you can’t go a few weeks without hearing about a security leak or data breach affecting an organization. Why do they keep happening? And how can you prevent them?
Let’s take a dive into three of the most common avenues that lead to security breaches and discuss ways to keep sensitive information safe.
Leaky S3 Buckets
Let’s start with everyone’s favorite vulnerability—leaky Amazon S3 buckets. A quick web search will lead you to dozens of stories about data breaches that were the result of an unsecured S3 bucket. This has happened to organizations of all sizes. Some recent major breaches have involved Twilio and US telecommunications company Verizon. As recently as March 2021 this continues to be a problem, as an unsecured S3 bucket led to the exposure of more than 50,000 patient records from a medical testing company.
This happens with such frequency that you’d think that many companies would have better control over such an issue. However, according to security firm Skyhigh Networks, 7 percent of all S3 buckets have unrestricted public access. While the percentage is small, in sheer numbers it’s huge considering that thousands of companies use S3 servers for a multitude of purposes ranging from pure storage, to static website hosting, to processing big data.
Professional and amateur security researchers alike have even made businesses out of uncovering unprotected S3 buckets.
S3 buckets are set to private by default. AWS stresses this in its documentation. So what exactly is going on? The reason often comes down to user error and misconfiguration.
Some common mistakes include:
- Granting access to “Any Authenticated AWS User” or “Any AWS User.” This allows anyone with an AWS account (not just within your organization) to access the bucket.
- JSON bucket policies that use opposite or negative syntax to inadvertently block access to approved IPs, while granting access to the broader web.
- Object ACLs that do not inherit bucket ACLs, meaning objects within a bucket could be accessible even if the bucket itself is private.
How to prevent it
Amazon does have some tools to prevent misconfiguration and to ensure that S3 buckets remain private. Most notable is the Block Public Access feature, which allows administrators to set up more centralized controls.
Setting up more controls now however, doesn’t necessarily help you protect the buckets that are already out in the open. Preventing leaks will require continuous monitoring and scans.
Third-party exposure (supply chain attacks)
Also known as third-party or value chain attacks, supply chain attacks involve hacking into an organization by infiltrating less-secure systems provided by third-party vendors and partners. According to Accenture, around 40 percent of cyberattacks of organizations originate from their extended supply chain.
These secondary targets are often less secure than primary targets which is why supply chain attacks are growing.
Well-known breaches
US-based retailer Target was attacked in 2013, when a nefarious group used a third-party supplier’s credentials to access sensitive information. Malware inserted into POS systems in 1800 stores resulted in the theft of credit card information from 40 million customers.
The most recent high-profile supply chain attack was made public in late 2020. Eighteen-thousand customers of the IT networking firm Solarwinds were vulnerable including many US government and national security agencies. A root cause analysis conducted by Crowdstrike found that actors, possibly backed by a foreign government, used a tool called SUNSPOT to insert malware called SUNBURST into a Solarwinds software update.
How to prevent it
Any company that receives software or services from a third-party vendor is at risk for a supply chain attack. Protecting yourself starts with evaluating your relationships with these vendors. Depending on what industry you are in (e.g., financial services or healthcare) there may already be regulatory frameworks in place regarding security issues with vendors.
When working with third-party vendors, demand more auditable proof of security standards. Also consider tighter restrictions on privilege access for external parties.
Misconfigured CI/CD
The Solarwinds hack mentioned previously can also be used to demonstrate a possible avenue for a supply chain attack. Although the details are still unclear, reporting suggests that Solarwinds was compromised by unsecured build infrastructure. Its system was used as a weak link to gain access to private data or insert malicious code. Of particular note is that an insecure password linked to the company’s development system was stored in a public GitHub repository.
We’ll go into more details about CI-specific security issues in an upcoming post, but we want to mention two recent cases that highlight the potential for CI infrastructure to be misused.
Denial of service (DoS) attacks
While DoS is often associated with hackers taking over unsecured IoT and mobile devices, CI’s highly parallelized, ephemeral workloads can present ideal opportunities for these types of attacks.
In July of 2020, the Federal Bureau of Investigation (FBI) overseen by the United States Department of Justice released an advisory warning that malicious actors have exploited built-in network protocols to conduct larger distributed denial of service (DDoS) attacks.
As an example, the law enforcement agency cited a February 2020 vulnerability assessment of Jenkins servers conducted by UK-based security researchers. The group found vulnerabilities in the built-in network discovery protocols of Jenkins servers, which could be used to “amplify DDoS attack traffic 100 times against the online infrastructure of targeted victims across sectors.”
While this Jenkins vulnerability was not exploited and has since been patched, this suggests that development teams should be on high alert for this avenue of intrusion.
Cryptocurrency mining attacks
The CI industry has seen an increase in certain users abusing free tiers in order to mine cryptocurrency. According to TechBeacon, owners of GitHub projects have noticed:
“Random users are forking repos, then pull-requesting a change that includes an obfuscated GitHub Action. Attackers force the free service to run a cryptocurrency miner by pretending it’s part of the project’s CI/CD pipeline. Crucially, the malicious GitHub Action runs before the project owner decides whether to approve the PR.”
The issue is becoming so severe that many CI providers including GitLab and CircleCI are severely restricting or eliminating their free tiers.
Keeping your infrastructure safe
We live in a world where security breaches and exploits are becoming everyday occurrences. While this may seem like an insurmountable challenge, developer teams do have the resources and power to keep their organizations’ infrastructure and data secure.
It requires teams to be diligent in keeping their tools up to date, actively monitoring systems for vulnerabilities, and being aware of the security practices of their external vendors.
Helpful Resources
- Blog - How to Protect Your CI/CD Pipelines from... Yourself: Grant Colgate, Buildkite Engineering Lead, takes us through some useful best practices for setting user and team access control within Buildkite.
- Docs - Securing your Buildkite Agent: A handy guide detailing useful configuration settings for use within sensitive environments.
