Container resources (requests and limits)
Default resources for requests and limits can be allocated to Pods and their containers using the PodSpec patch in the Buildkite Agent Stack for Kubernetes controller's values YAML configuration file, which applies across the board, or within a pipeline's YAML file, which can override those defined in the values YAML configuration file.
Alternatively, Resource Classes can be configured, allowing workload to decide resources by specifying resource_class
Agent tags.
Using resource class
Resource Classes allow you to define reusable resource configurations that can be applied to CI workloads based on agent tags.
To implement the agent configuration options described on this section, version 0.31.0 or later of the Agent Stack for Kubernetes controller is required.
Configuration
Resource classes are defined in the controller configuration under the resource-classes
key:
# values.yaml
config:
resource-classes:
class-name:
resource: # Optional: Kubernetes resource requirements
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
nodeSelector: # Optional: Kubernetes node selector
instance-type: "small"
zone: "us-west-2a"
-
resource (optional): Kubernetes ResourceRequirements object that will be applied to the command container
- requests: Resource requests (CPU, memory, etc.)
- limits: Resource limits (CPU, memory, etc.)
- nodeSelector (optional): Key-value pairs for Kubernetes node selection
Usage
To use a resource class in your CI pipeline, specify the resource_class
agent tag:
# pipeline.yaml
steps:
- label: "Build"
command: "make build"
agents:
resource_class: "medium" # <-- New!
Example configurations
# values.yaml
config:
resource-classes:
xs:
resource:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
gpu:
resource:
requests:
nvidia.com/gpu: "1"
cpu: "1000m"
memory: "2Gi"
limits:
nvidia.com/gpu: "1"
cpu: "2000m"
memory: "4Gi"
nodeSelector:
accelerator: "nvidia-tesla-k80"
spot:
nodeSelector:
node-type: "spot"
instance-type: "large"
Using the PodSpec patch in the controller values YAML configuration file
In the Buildkite Agent Stack for Kubernetes controller's values YAML configuration file, you can specify the default resources (requests and limits) to apply to the Pods and containers:
# values.yaml
agentStackSecret: <name-of-predefined-secrets-for-kubernetes>
config:
pod-spec-patch:
initContainers:
- name: copy-agent
resources:
requests:
cpu: 100m
memory: 50Mi
limits:
memory: 100Mi
containers:
- name: agent # this container acquires the job
resources:
requests:
cpu: 100m
memory: 50Mi
limits:
memory: 1Gi
- name: checkout # this container clones the repository
resources:
requests:
cpu: 100m
memory: 50Mi
limits:
memory: 1Gi
- name: container-0 # the job runs in a container with this name by default
resources:
requests:
cpu: 100m
memory: 50Mi
limits:
memory: 1Gi
Overriding the PodSpec patch for a single job
Following on from the Agent Stack for Kubernetes controller's YAML configuration values file above, all the Kubernetes Jobs created by the controller will have the resources (defined in this file) applied to them. To override these resources for a single job, use the kubernetes
plugin with podSpecPatch
to define container resources. For example:
# pipelines.yaml
agents:
queue: kubernetes
steps:
- name: Hello from a container with more resources
command: echo Hello World!
plugins:
- kubernetes:
podSpecPatch:
containers:
- name: container-0 # <-- Specify this exactly as `container-0`.
resources: # Currently under experimentation to make this more ergonomic.
requests:
cpu: 1000m
memory: 50Mi
limits:
memory: 1Gi
- name: Hello from a container with default resources
command: echo "Hello World!"
Configuring imagecheck-* containers
To define CPU and memory resource limits for your containers, use the image-check-container-cpu-limit
and image-check-container-memory-limit
configuration values:
# values.yaml
config:
image-check-container-cpu-limit: 100m
image-check-container-memory-limit: 128Mi