Node configuration

What is node configuration?

The CAST AI provisioner allows you to set node configuration parameters that the platform will apply to provisioned nodes.

A cluster can have multiple Node Configurations. However, you can select only one as the default the Autoscaler will use. You can link the default and other node configurations to Node Templates.

📘
You can link node configuration to multiple node templates, but one node template can have just a single node configuration link.

Node configuration on its own does not influence workload placement. Its sole purpose is to apply user-provided configuration settings on the node during the provisioning process.

You can manage node configurations via UI, API or Terraform.

Feature availability

EKS	AKS	GKE	KOPS
+	+	+	-

Shared configuration options

The following table provides a list of supported cloud-agnostic configuration parameters:

Configuration	Description	Default value
Root volume ratio	CPU to storage (GiB) ratio	1 CPU: 0 GiB
Image version	Image to be used when building CAST AI provisioned node	The latest available for the Kubernetes release
SSH key	Base64-encoded public key or AWS key ID	""
Subnets	Subnet IDs for CAST AI provisioned nodes	All subnets pointing to NAT/Internet Gateways inside the cluster's VPC
Instance tags	Tags/VM labels to be applied on CAST AI provisioned nodes	[]

📘
Check the list of images available for EKS in the AWS documentation

Some configuration options are cloud provider specific. See the table below:

EKS-specific configuration options:

Configuration	Description	Default value
Security groups	Security group IDs for nodes provisioned in CAST AI	Tagged and CAST AI SG
Instance profile ARN	Instance profile ARN for CAST AI provisioned nodes	cast-<cluster-name>-eks-<cluster-id> (only the last 8 digits of the cluster ID)
Dns-cluster-ip	Override the IP address to be used for DNS queries within the cluster	""
Container runtime	Container runtime engine selection: docker or containerd	Unspecified
Init script	A script to be run when building the node	bash ""
Docker configuration	A set of values that will be overwritten in the Docker daemon configuration	JSON {}
Kubelet configuration	A set of values that will be added or overwritten in the kubelet configuration	JSON {}
Volume type	EBS volume type to be used for provisioned nodes	gp3
Volume IOPS	EBS volume IOPS value to be used for provisioned nodes	3000
Volume throughput	EBS volume throughput in MiB/s to be used for provisioned nodes	125
Use IMDS v1	IMDSv1 and v2 are enabled by default, else only IMDSv2 will be allowed	True

AKS-specific configuration options

Configuration	Description	Default value
Max pods per node	Maximum number of pods to be hosted on a node	30

How to create a node configuration

A default node configuration is created during cluster onboarding in the CAST AI-managed mode.

You can choose to modify this configuration or create a new one. If you add a new node configuration that will be applied to all newly provisioned nodes, you will have to mark it as default.

Node configurations are versioned, and when the CAST AI provisioner adds a new node, the latest version of the node configuration is applied.

A new configuration can't be applied to an existing node. If you want to upgrade node configuration on a node or a set of nodes, you need to delete an existing node and wait until Autoscaler replaces it with a new one or rebalance the cluster (fully or partially).

Kubelet configuration examples

You can find all available kubelet settings in the Kubernetes documentation – Kubelet Configuration. Please refer to the version of your cluster.

For example, if you want to add some specific custom taints during node startup, you could do it with the following snippet:

{
    "registerWithTaints": [
        {
            "effect": "NoSchedule",
            "key": "nodes-service-critical",
            "value": "true"
        }
    ]
}

The second example involves configuring kubelet image pulling and setting kube API limits like following:

{
    "eventBurst": 20,
    "eventRecordQPS": 10,
    "kubeAPIBurst": 20,
    "kubeAPIQPS": 10,
    "registryBurst": 20,
    "registryPullQPS": 10
}

Create node configuration with the CAST AI Terraform provider

Use the resource castai_node_configuration from CAST AI terraform provider.

Reference example:

resource "castai_node_configuration" "test" {
  name           = local.name
  cluster_id     = castai_eks_cluster.test.id
  disk_cpu_ratio = 5
  subnets        = aws_subnet.test[*].id
  tags           = {
    env = "development"
  }
  eks {
    instance_profile_arn = aws_iam_instance_profile.test.arn
    dns_cluster_ip       = "10.100.0.10"
    security_groups      = [aws_security_group.test.id]
  }
}