How it works

🚧

Notice

Spot-webhook is being phased out. Funcionality is moving to nodeTemplates.

CAST AI gives customers the flexibility to run all or a portion of workloads on spot instances without having to modify manifest files. To achieve this, users must install and configure a Mutating Admission Webhook.

When there's a request to schedule a pod, the CAST AI Mutating Admission Webhook (in short, mutating webhook) will mutate the workload manifest - for example, add spot toleration to influence the desired pod placement by the Kubernetes Scheduler.

Mutating Admission Webhook

CAST AI Mutating Admission Webhook presets:

• Spot-only
• Spot-only except kube-system
• Partial Spot
• Custom

📘

Note

Pods that are already running will not be affected.

The Webhook only mutates pods during scheduling. Over time, all pods should eventually be rescheduled and, in turn, mutated. The application owners will release a new version of the workload that will trigger all the replicas to be rescheduled. Evictor or Rebalancing will remove older nodes, put pods for rescheduling, etc.

If you'd like to initiate mutation for the whole namespace immediately, run this command which will recreate all pods:

kubectl -n {NAMESPACE} rollout restart deploy

Spot-only

Please use Node template configuration.

Spot-only except kube-system

Preset allSpotExceptKubeSystem.

This mode works the same as the Spot-only mode, but it forces all pods in the kube-system namespace to be placed on on-demand nodes. This mode is recommended for clusters where the high-availability aspect of the control plane is vitally important, while other pods can tolerate spot interruptions.

Install Spot-only except kube-system

To run all pods, excluding kube-system on spot instances, use:

helm repo add castai-helm https://castai.github.io/helm-charts
helm upgrade -i --create-namespace -n castai-pod-node-lifecycle castai-pod-node-lifecycle \
    castai-helm/castai-pod-node-lifecycle \
    --set staticConfig.preset=allSpotExceptKubeSystem

Partial Spot

Preset partialSpot.

When 100% of pods on spot instances is not desirable, you can use a ratio such as 60% on stable on-demand instances and the remaining 40% of pods in the same ReplicaSet (Deployment / StatefulSet) running on spot instances.

This conservative configuration ensures that there are enough pods on stable computing for the base load. Still, it allows achieving significant savings for pods above the base load by putting them on spot instances. This setup is recommended for all types of environments, from production to development.

Install Partial Spot

For running 40% workload pods on spot instances and keeping the remaining pods of the same ReplicaSet on on-demand instances, use:

helm repo add castai-helm https://castai.github.io/helm-charts
helm upgrade -i --create-namespace -n castai-pod-node-lifecycle castai-pod-node-lifecycle \
    castai-helm/castai-pod-node-lifecycle \
    --set staticConfig.preset=partialSpot

To set a custom ratio for partial Spot, replace 70 with [1-99] as a percentage value:

helm repo add castai-helm https://castai.github.io/helm-charts
helm upgrade -i --create-namespace -n castai-pod-node-lifecycle castai-pod-node-lifecycle \
    castai-helm/castai-pod-node-lifecycle \
    --set staticConfig.defaultToSpot=false --set staticConfig.spotPercentageOfReplicaSet=70

Custom

No preset.

This mode can be adjusted to match your cluster's needs and requirements. Instead of choosing a specific preset, you can configure the behavior yourself.

Schema description of the PodAffinityTerm object can be found in the official kubernetes-api documentation. The property topologyKey is ignored and the property namespaceSelector is not yet supported.

Install Custom

Here is an example of a values.yaml with custom rules defined:

staticConfig:
  defaultToSpot: false
  spotPercentageOfReplicaSet: 30
  IgnorePodsWithNodeSelectorsAffinities: false
  ignorePods:
    - labelSelector:
        matchLabels:
          app.kubernetes.io/name: ignored-pod
  forcePodsToSpot:
    - labelSelector:
        matchExpressions:
          - key: app.kubernetes.io/name
            operator: In
            values:
              - spot-pod-1
              - spot-pod-2
  forcePodsToOnDemand:
    - namespaces:
        - kube-system
        - default

To install the webhook with these custom rules, execute this command:

helm repo add castai-helm https://castai.github.io/helm-charts
helm upgrade -i --create-namespace -n castai-pod-node-lifecycle castai-pod-node-lifecycle \
    castai-helm/castai-pod-node-lifecycle \
    --values values.yaml

Workload level override

The Mutating Webhook is a cluster-level configuration, but it can have exceptions that could be enforced per Deployment or StatefulSet.

kubectl patch deployment resilient-app -p '{"spec": {"template":{"metadata":{"annotations":{"scheduling.cast.ai/lifecycle":"spot"}}}}}'
kubectl patch deployment sensitive-app -p '{"spec": {"template":{"metadata":{"annotations":{"scheduling.cast.ai/lifecycle":"on-demand"}}}}}'
kubectl patch deployment conservative-app -p '{"spec": {"template":{"metadata":{"annotations":{"scheduling.cast.ai/spot-percentage":"50"}}}}}'

Troubleshooting

The mutating webhook will ignore these types of pods:

• Bare pods without ReplicaSet Controller
• Pods in castai-pod-node-lifecycle namespace
• Pods with TopologySpreadConstraints with TopologyKey=Lifecycle
• DaemonSets will get Spot Toleration by default, ensuring DaemonSet Pods can run on spot and on-demand nodes

The CAST AI Mutating webhook pods write logs to thestdOut.

If the cluster has Deployments with 1000+ replicas set higher Memory Requests and Limits, by appending these parameters to Helm command:

--set resources.requests.memory=1G --set resources.limits.memory=1G