Pod templates
From v0.32.2 of agent-stack-k8s, the kubernetes plugin allows you to specify a podTemplate. The podTemplate attribute specifies the name of a PodTemplate resource in the same namespace as the stack controller. Pod templates function similarly to podSpec, but hide the details of a podSpec from the pipeline definition.
Stack operators (who can create Kubernetes resources) can set up a shared library of templates. This allows updating pod specs separately from both the stack controller (as with pod-spec-patch) and all the pipelines using them, and avoids storing unnecessary platform details within each pipeline definition.
How to use
- Ensure you are using agent-stack-k8s v0.32.2 or later.
- Create
PodTemplateresources in the same namespace as the stack controller. - Refer to
PodTemplateresources by name using thepodTemplatekey of thekubernetesplugin.
Notes
podTemplate operates similarly to podSpec. It provides the initial spec of a pod that is then adjusted by the stack controller. If a podSpec is provided, podTemplate is ignored. To adjust a podTemplate within a step, use podSpecPatch.
Other options that change the initial spec include:
- Step attributes such as
imageandcommand - The resource class tag
- Checkout parameters, such as
skip -
pod-spec-patchcontroller configuration andpodSpecPatchplugin attribute
In addition to spec (a PodSpec), a PodTemplate can also specify metadata within the template. This metadata is ignored by the stack controller - only spec is used.
Example
This example manifest defines a PodTemplate called go-with-cache in the buildkite namespace, that
- sets the container image to
golang:latest, - configures Go to use a caching tool,
- attaches a persistent volume claim,
- sets a security context to change the user and group.
apiVersion: v1
kind: PodTemplate
metadata:
name: go-with-cache
namespace: buildkite # Note: must be the same namespace as agent-stack-k8s
template:
spec:
containers:
- name: container-0
image: golang:latest
env:
- name: GOCACHEPROG
value: "/tools/go-cacher -cache-server=http://gocached.default.svc.cluster.local:31364"
volumeMounts:
- name: tools
mountPath: /tools
readOnly: true
volumes:
- name: tools
persistentVolumeClaim:
claimName: tools-shared
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1001
Once a PodTemplate has been created in the cluster, it can be referred to from the kubernetes plugin. Here is a pipeline definition that uses the go-with-cache template (defined above) multiple times:
steps:
- label: "Go Build"
command: go build -o /tmp/out .
plugins:
- kubernetes:
podTemplate: go-with-cache
- label: "Go Test"
command: go test ./...
plugins:
- kubernetes:
podTemplate: go-with-cache
- label: "Go Vet"
command: go vet
plugins:
- kubernetes:
podTemplate: go-with-cache
The equivalent pipeline using only podSpec is quite lengthy, with repetitive, deeply-nested configurations containing platform details that are largely irrelevant to the pipeline steps (such as volume configuration):
steps:
- label: "Go Build"
command: go build -o /tmp/out .
plugins:
- kubernetes:
podSpec:
containers:
- name: container-0
image: golang:latest
env:
- name: GOCACHEPROG
value: "/tools/go-cacher -cache-server=http://gocached.default.svc.cluster.local:31364"
volumeMounts:
- name: tools
mountPath: /tools
readOnly: true
volumes:
- name: tools
persistentVolumeClaim:
claimName: tools-shared
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1001
- label: "Go Test"
command: go test ./...
plugins:
- kubernetes:
podSpec:
containers:
- name: container-0
image: golang:latest
env:
- name: GOCACHEPROG
value: "/tools/go-cacher -cache-server=http://gocached.default.svc.cluster.local:31364"
volumeMounts:
- name: tools
mountPath: /tools
readOnly: true
volumes:
- name: tools
persistentVolumeClaim:
claimName: tools-shared
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1001
- label: "Go Vet"
command: go vet
plugins:
- kubernetes:
podSpec:
containers:
- name: container-0
image: golang:latest
env:
- name: GOCACHEPROG
value: "/tools/go-cacher -cache-server=http://gocached.default.svc.cluster.local:31364"
volumeMounts:
- name: tools
mountPath: /tools
readOnly: true
volumes:
- name: tools
persistentVolumeClaim:
claimName: tools-shared
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1001
While other techniques can be used to shorten this example (such as YAML anchors/aliases or the controller pod-spec-patch configuration), they are less flexible than using podTemplate, and changes to the pod spec would require either updating the pipeline or the controller configuration.