Advanced Config

Evictor is a major component responsible for actively bin-packing customer clusters. It is one of the pillars that provides active cost-saving value. Evictor as a component, has multiple modes that it can operate on:

• Default - targets every eviction friendly pod and node in the clusters.
• Aggressive Mode - targets every pod (single replica, bare-pods, jobs, but not Pods with PVC) and node in the cluster, disregarding the number of replicas pods have, which might cause some small interrupts from applications at times.
• Scoped mode - targets only CAST AI created nodes in the cluster

There are several use cases to run Evictor in aggressive mode on specific resources.

For this Advanced Configuration can be used to configure Evictor with additional rules.

F.A.Q

What does Evictor Advanced Configuration do?

It provides additional rules for Evictor to target specific nodes and/or pods to override default Evictor behaviour.

When should Evictor Advanced Configuration be used?

It should be used to achieve granular optimisation to evict specific resources.

What are the current eviction modes available with Advanced Configuration?

The available modes are: removalDisabled, aggressive, and disposable.

Advanced configuration

Advanced Configuration serves as an additional set of rules to work along side the default CAST AI Evictor rules. You can start configuring Evictor with Advanced Configuration, by creating YAML file and passing it to Evictor.

Use case examples

Evictor Advanced Configuration use case for marking Job kind pods as removalDisabled if matchers are fulfilled:

evictionConfig:
  - podSelector:
      namespace: "namespace"
      kind: Job
      labelSelector:
      	matchLabels:
          pod.cast.ai/name: "cron-job"
          app.kubernetes.io/active: "true"
    settings:
      removalDisabled:
        enabled: true

Marking node as disposable:

evictionConfig:
  - nodeSelector:
      labelSelector:
        matchLabels:
          app.kubernetes.io/name: castai-node
          app.kubernetes.io/instance: instance
    settings:
      disposable:
        enabled: true

Disposing pod(s) with attached PVC(s) using disposable flag:

evictionConfig:
  - podSelector:
      namespace: "statefulSet-ns"
      kind: StatefulSet
      labelSelector:
        matchExpressions:
          - key: pod.cast.ai/flag
            operator: In
            values:
              - "true"
    settings:
      disposable:
        enabled: true

Applying aggressive mode even if Evictor is running without aggressive mode turned on:

evictionConfig:
  - podSelector:
      namespace: "namespace"
      kind: ReplicaSet
      labelSelector:
        matchExpressions:
          - key: pod.cast.ai/flag
            operator: In
            values:
              - "true"
    settings:
      aggressive:
        enabled: true

Selectors

Selectors specify criteria for matching specific resources, e.g.: pods or nodes.

If selector satisfies rules, it applies eviction mode specified in settings. If no matches are found or if matches are found with the eviction mode enabled flag set to false, default Evictor rules will be applied for given pod or node selector.

Selector Keys

Selectors have specific matchers for deciding if targeting mode should be applied for resource. matchLabels and matchExpressions follow same structure as in Kubernetes, documentation

Settings

Settings hold one of supported targeting modes and if that mode is enabled.

Targeting Modes

Targeting mode - specifies the eviction type for the matched resource. The specified mode takes precedence and overrides default Evictor behaviour.

NOTE aggressive targeting works same as turning it from console, but only for a certain pod or node. If aggressive mode is turned on in console and specified in advanced configuration it won't have any additional behaviour.

Targeting Mode Keys

How to pass Advanced Configuration?

CAST AI flow

During Evictor initialisation CM (ConfigMap) is created under castai-agent namespace with castai-evictor-config name.

Edit castai-evictor-config CM data, under config.yaml: | part with wanted evictionConfig YAML format contents, e.g.:

• YAML

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: "castai-evictor-config"
    labels:
      
      helm.sh/chart: castai-evictor-1
      app.kubernetes.io/name: castai-evictor
      app.kubernetes.io/instance: release-name
      app.kubernetes.io/version: "version"
      app.kubernetes.io/managed-by: Helm
  data:
    config.yaml: |
  
      evictionConfig:
        - nodeSelector:
            labelSelector:
              matchLabels:
                app.kubernetes.io/name: castai-node
                app.kubernetes.io/instance: instance
          settings:
            removalDisabled:
              enabled: true
        - podSelector:
            namespace: "namespace"
            kind: ReplicaSet
            labelSelector:
              matchLabels:
                pod.cast.ai/name: castai-pod
          settings:
            disposable:
              enabled: true
        - podSelector:
            namespace: "namespace"
            kind: ReplicaSet
            labelSelector:
              matchExpressions:
                - key: pod.cast.ai/flag
                  operator: In
                  values:
                    - "true"
                - key: pod.cast.ai/name
                  operator: Exists
              matchLabels:
                pod-label: "pod-label-value"
          settings:
            disposable:
              enabled: true
  

Manual installation

To pass Evictor Advanced Configuration with manual Evictor installation, you can provide path to created YAML file through helm, by setting file location flag or with set flag. You can find guide how to install Evictor manually here.

• Shell

  --set-file customConfig=<path_to_file>
  

• --set customConfig="<config_string>"