Private package management with Packagecloud

Packagecloud is a package management solution recently acquired by Buildkite. It's a cloud-based service that empowers users to distribute software packages in a simple, reliable, and scalable way—without the need to own or maintain any infrastructure. In the future, package management will be natively available on Buildkite.

In Packagecloud, users can create and publish to both public and private repositories (repos), plus invite collaborators to read, push, yank, and promote packages in their repositories. Users can also easily generate, distribute, and revoke master/read tokens for fine-grained access control to their repositories.

As long as Packagecloud supports the packages’ operating system or distribution, any package can be pushed to the same repository—providing a central location for securely and efficiently distributing your packages.

Public vs. private repositories

Public repositories are best suited for users intending to distribute open source software as they are read-accessible by anyone. 

Whereas private repositories are:

• Best suited for users who have proprietary software.
• Read-accessible only with a read token.

All subscribed users can create an unlimited number of public and private repositories. 

Both public and private repositories can be used to store packages for NPM, Debian, Maven, RPM, RubyGem, Python, Alpine, and Helm—or even a combination of all package types within the same repository.

How to create a private repository on Packagecloud

You'll need a Packagecloud account to create a private repository. If you don't already have an account, sign up to create one. You'll be guided through creating a repo during the signup process.

To create a private repository:

1. From your dashboard, select Create a Repository:

2. On the create screen, enter your new repository’s name and select the Private setting:

3. Select Create It!

How to upload a package to a private repository

What you'll need:

• A Packagecloud account with a private repository.
• A package to upload.
• (Optional) The Packagecloud CLI tool installed.

When uploading a package to a private repository, you can choose one of the following methods:

• Using the UI
• Using the CLI
• Using CI integrations

Uploading a package to a private repo using the UI

If you've just created your Packagecloud account, the onboarding flow guides you to try uploading a package in the UI:

Otherwise, to upload a package to a private repo using the UI:

1. From the dashboard, open your private repo.
2. Select Upload a package.
3. Choose a package and an OS/distribution from the dropdown menu.
4. Select Upload:

The package details information screen is only visible to the repo owner and any invited collaborators.

Uploading a package to a private repository using the CLI

After installing and setting up the Packagecloud CLI, you can upload a package with the package_cloud push command—specifying the desired operating system, distribution, and package path.

For example:

• Pushing an Ubuntu/Jammy Debian package:

  1
    package_cloud push username/reponame/ubuntu/jammy packagecloud-test_1.1-2_amd64.deb

• Pushing a RHEL/9 RPM package:

  1
    package_cloud push username/reponame/el/9 packagecloud-test-1.1-1.x86_64.rpm

• Pushing a Node.js package:

  1
    package_cloud push username/reponame/node example-1.0.tgz

Although the commands for pushing different package types may differ slightly, any supported package (Debian, RPM, RubyGem, Python, Node.js, Alpine packages, Java JAR/WAR/AAR, Helm charts and arbitrary files, etc.) can be pushed to the same private repository.

Uploading a package using CI integrations

Teams using a continuous integration strategy to build and deploy their packages can also publish packages to a private Packagecloud repository. 

Packagecloud has step-by-step instructions and integrations with many popular CI platforms such as Buildkite, CircleCI, Jenkins, Travis CI, and GitHub Actions.

Installing and securely distributing your privately hosted packages

All private Packagecloud repositories need to be configured to make the packages inside available for installation. Packagecloud generates an installation script command containing an automatically generated default master token for every private repository:

Running this command provides a fast and easy way to complete important steps for installing a Packagecloud private repository—such as generating a read token, importing the GPG key, and creating the repository config file in the correct directory. Alternatively, users can manually install the repository by following the instructions on the package information screen.

Packagecloud also displays a command to download and install the package once the repository is properly configured on the machine:

Users can also skip installing the repository and directly download the package as a file in the UI using the Download button on the package’s details page.

Managing master tokens and read tokens of private repositories

Packagecloud private repositories enable users to distribute their packages easily and safely. The repository installation script command displayed on the package details page includes the default master token. However, this can be replaced with another user-generated and revocable master token for fine-grained access control to your repositories.

About master tokens

Each repository is created with a default master token, used only for generating descendent read tokens. It cannot be used for reading from private repositories, pushing packages, modifying configurations, or anything else. 

This will allow users to safely create and distribute master tokens for use cases such as sharing with customers/untrusted parties or embedding them in configuration management manifests. Users can just as easily revoke those master tokens and their generated descendant read tokens from the UI or CLI.

To generate a master token from the UI:

1. From your dashboard, open any private repository.
2. In the side navigation, select Tokens.
3. Select Create Master Token.
4. Input a name to identify the token.

To generate a master token from the CLI, run:

1
package_cloud master_token create <username>/<repo_name> <master_token_name>

Replacing:

• <username> with your username.
• <repo_name> with the name of the private repo.
• <master_token_name> with a name to identify the token by.

For more information about the CLI and master tokens, see the documentation.

About read tokens

Read tokens are generated by master tokens. They are used to authenticate against the Packagecloud private repository by sending it along with any requests for repo metadata and packages.

They are read-only and cannot be used to modify the repository or authenticate against any Packagecloud endpoints apart from the servers.

Read tokens are generated under the specific master token used when running the private repository installation script. They can also be generated manually, either upon running the associated command listed in the script or using the UI.

We strongly recommend using one read token per node to maximize isolation.

You can create and revoke read tokens through the UI or API.

To create a read token in the UI:

1. From your dashboard, open any private repository.
2. In the side navigation, select Tokens.
3. Beside the custom master token used to generate the read token, select Show.
4. Select + Create Read Token.
5. Input a name.

To revoke a read token in the UI, open the token and select revoke. All read tokens generated by running the installation script will automatically be associated with the node’s hostname.

To create a read token through the REST API run the following call:

1
UNIQUE_ID=`hostname -f` && curl -XPOST --data "name=${UNIQUE_ID}" https://${master_token}:@packagecloud.io/install/repositories/${username}/${repository}/tokens.text

Replacing:

• UNIQUE_ID with a unique ID for the token.
• hostname with the node's hostname.
• master_token with the value of the master token.
• username with your username.
• repository with the name of the private repository.

To learn more about read tokens, see the documentation.

Adding collaborators to a private repository

Users can invite others to read, push, yank, and promote packages in any of their repositories. This type of invited user is called a collaborator, and their permissions are restricted only to the repositories they are invited to.

Both public and private repositories can have unlimited collaborators.

To add a collaborator to a repository:

1. From your dashboard, open any private repository.
2. In the side navigation, select Settings.
3. Go to the Manage Collaborators section.
4. Add a collaborator.
   1. For existing Packagecloud users, enter their email address or username, then select Add.
   2. For new Packagecloud users, enter their email address, then select Send invite. They'll receive an invitation to sign up to Packagecloud.

Buildkite acquired Packagecloud to add first-class package repository support to its CI/CD products. It's a cloud-based service that empowers users to distribute software packages in a simple, reliable, and scalable way—without the need to own or maintain any infrastructure. Sign up to manage your packages and deploy to any environment from one beautiful interface, on-premise or in the cloud.

In the future, package management will be natively available on Buildkite. Join the waitlist for early access.