Pod mutations is a Cast AI feature that automatically modifies pod specifications when pods are created. By intercepting pod creation at the Kubernetes admission stage, pod mutations apply configuration changes—such as labels, tolerations, node selectors, and Spot Instance settings—without requiring changes to your deployment manifests.

Pod mutations are implemented as Kubernetes Custom Resources (PodMutation), making them fully compatible with GitOps workflows and declarative cluster management. The cluster serves as the single source of truth: mutations created in the Cast AI console sync to your cluster as custom resources, and mutations applied directly to the cluster (via kubectl, Terraform, ArgoCD, etc.) sync back to Cast AI.

Why use pod mutations?

Managing Kubernetes workloads at scale presents several challenges that pod mutations address.

Complex configuration requirements. As clusters grow, manually configuring pod specifications becomes time-consuming and error-prone. Each workload may need specific labels, tolerations, and node selectors to ensure proper scheduling. Pod mutations let you define these configurations once and apply them automatically to matching pods.

Legacy system integration. When onboarding existing clusters to Cast AI, workloads may need to be reconfigured to take advantage of cost optimization features, such as Spot Instances or Node Template consolidation. Traditionally, this requires updating deployment manifests across many repositories. Pod mutations automate this.

Resource fragmentation. Without standardized pod configurations, clusters can become fragmented with too many node groups, leading to inefficient resource utilization. Pod mutations can consolidate workloads onto fewer Node Templates, improving bin-packing and reducing costs.

GitOps and infrastructure-as-code workflows. Teams using declarative tools like ArgoCD or Terraform need mutations to be manageable as code. Because pod mutations are Kubernetes custom resources, they integrate naturally with your existing GitOps workflows, enabling version control, code review, and approval.

Key concepts

PodMutation custom resource

Each pod mutation is represented as a Kubernetes custom resource of kind PodMutation. This resource defines two things: which pods the mutation applies to (via filters), and what changes to make (via configuration options).

apiVersion: pod-mutations.cast.ai/v1
kind: PodMutation
metadata:
  annotations:
    pod-mutations.cast.ai/pod-mutation-id: 6de5fd3f-500d-4b6c-b0b2-adf10f6a009b
    pod-mutations.cast.ai/pod-mutation-name: demo-label-mutation
    pod-mutations.cast.ai/pod-mutation-source: api
  creationTimestamp: "2025-12-17T15:06:05Z"
  generation: 1
  name: api-mutation-6de5fd3f-500d-4b6c-b0b2-adf10f6a009b
  resourceVersion: "74105"
  uid: fb84c26f-1bca-4d73-b243-3659b0f9a987
spec:
  filter:
    pod: {}
    workload: {}
  filterV2:
    pod:
      excludeLabels: {}
      labels: {}
    workload:
      namespaces:
      - type: exact
        value: pod-mutation-demo
  patchesV2:
  - operations:
    - op: add
      path: /metadata/labels/mutated-by
      value: castai
  restartPolicy: deferred
  spotConfig: {}

This example creates a mutation that applies to pods in the pod-mutation-demo namespace and applies the label mutated-by: castaito them.

Two-way synchronization

Pod mutations support two-way sync between the Cast AI console and your cluster:

• Console to cluster: Mutations you create or edit in the Cast AI console are automatically synced to your cluster as PodMutation custom resources.
• Cluster to console: Mutations you apply directly to your cluster (using kubectl, Helm, Terraform, or other tools) are discovered by Cast AI and appear in the console. These mutations display a "(Custom Resource in cluster)" suffix and cannot be edited through the UI—all changes must be made via your cluster management tools.

This architecture means the cluster is always the source of truth. The pod mutator component reads mutation definitions only from the cluster, not from Cast AI APIs.

Single mutation per pod

Only one mutation can apply to any given pod. When multiple mutations have filters that match the same pod, Cast AI uses a specificity scoring system to select the most targeted mutation:

The mutation with the highest total score wins. For example, a mutation targeting "deployment frontend with label tier=web in namespace production" (score: 7) takes precedence over a mutation targeting "all pods in namespace production" (score: 1).

If two mutations have the same score, Cast AI applies tie-breaking rules in this order: fewer workload names wins, then fewer namespaces, then more label conditions, and finally alphabetical order by mutation name.

To avoid relying on this precedence system, design your mutations with mutually exclusive filters whenever possible.

Filter options

Pod mutations provide flexible filtering to target specific workloads:

• Namespace selector: Match pods by namespace name, with support for regex patterns.
• Workload selector: Match by workload name and kind (Deployment, StatefulSet, DaemonSet, etc.), with regex support.
• Pod label selector: Match by pod labels using AND/OR logic to combine multiple conditions.

The AND/OR label filtering lets you create precise targeting rules. For example, you can target pods that have (environment=production AND tier=frontend) OR (app=critical).

Configuration options

Pod mutations can apply various configuration changes:

Multiple configuration types can be combined in a single mutation. They are applied together when a matching pod is created.

How it works

The pod mutator runs as a Kubernetes Mutating Admission Webhook in your cluster. When a pod is created, the following sequence occurs:

1. Interception: The Kubernetes API server sends the pod specification to the pod mutator webhook before the pod is scheduled.

2. Matching: The mutator evaluates all PodMutation custom resources in the cluster to find those with filters matching the new pod.

3. Selection: If multiple mutations match, the mutator selects the most specific one using the scoring system described above.

4. Mutation: The mutator applies the selected mutation's configuration changes to the pod specification.

5. Admission: The modified pod specification is returned to the API server, and the pod proceeds to scheduling with the new configuration.

---
title: Pod Mutations
---
flowchart TD
    subgraph workload[" "]
        W["**Workload**"]
    end
    
    subgraph original["Pod spec (original)"]
        O1["• Labels x"]
        O2["• Toleration x"]
        O3["• NodeSelectors x"]
    end
    
    subgraph mutator["Pod Mutator webhook"]
        M["Modifies the pod spec<br/>(e.g. Modified labels,<br/>Added tolerations,<br/>Removed NodeSelectors, etc)"]
    end
    
    subgraph newspec["New pod spec"]
        N1["• Labels y"]
        N2["• Toleration y"]
        N3["• Affinities y"]
    end
    
    subgraph scheduled[" "]
        S["pod is<br/>scheduled"]
    end
    
    W -->|"defines"| original
    original -->|"admission<br/>request"| mutator
    mutator -->|"returns<br/>mutated<br/>pod spec"| newspec
    newspec --> S
    
    style mutator fill:#c5d9f1,stroke:#666
    style original fill:#e8e8e8,stroke:#666
    style newspec fill:#e8e8e8,stroke:#666
    style workload fill:none,stroke:none
    style scheduled fill:none,stroke:none

Because mutations are applied at creation time, they only affect newly created pods. Existing pods are not modified. Changes to a mutation definition take effect the next time matching pods are created—for example, during a deployment rollout or when pods are rescheduled.

Limitations

• New pods only: Mutations apply only when pods are created. Existing pods are not affected until they are recreated (for example, during a rollout or rescheduling).
• One mutation per pod: When multiple mutations match a pod, only the most specific one is applied. Design filters to be mutually exclusive to avoid unexpected behavior.
• Immutable fields: Some Kubernetes pod fields cannot be modified after creation. JSON Patch mutations that attempt to set these fields will be rejected.
• Distribution drift: With rapid scaling (for example, 0 to 10 replicas instantly), the actual Spot/On-Demand distribution may not match the configured percentage immediately. The system self-corrects as pods are recreated over time.

Related resources

To get started with pod mutations, see the Pod mutations quickstart.

For detailed information on all configuration options and the CRD specification, see the Pod mutations reference.