Installation

Prerequisites

Before you install Storage Autoscaling, confirm the following:

• A GKE or EKS cluster running Kubernetes 1.29 or later
• Helm 3 installed on your local machine
• kubectl configured to target your cluster
• A StorageClass with allowVolumeExpansion: true
• Kvisor with storage metrics enabled
• Cloud credentials for capacity expansion and IOPS/throughput management

Check volume expansion support

Run the following command to verify that your StorageClasses allow volume expansion:

kubectl get storageclass -o yaml | grep allowVolumeExpansion

Kvisor with storage metrics

Kvisor populates the Storage autoscaler dashboard in the Cast AI console with savings estimates, storage type breakdowns, resize event logs, and per-PV usage data. Without it, the dashboard will be empty. Storage Autoscaling still functions without Kvisor, but you lose console visibility into what it is doing.

See Configuring Kvisor: Kubernetes storage metrics for installation instructions.

📘

Note

Kvisor also requires cloud volume metadata configuration to collect disk type, size, and provisioned IOPS data per volume.

GKE identity and permissions

IOPS and throughput rightsizing on GKE requires a GCP service account with specific IAM roles, bound to the Kubernetes service account via Workload Identity Federation. The Helm chart also includes nodes/proxy RBAC for reading disk usage metrics.

Required IAM roles

Create a GCP service account with the following IAM roles:

Create the GCP service account

Create the service account and add the IAM bindings:

PROJECT_ID=YOUR_PROJECT_ID # update with your project ID

gcloud iam service-accounts create castai-pv-monitor \
  --project="${PROJECT_ID}" \
  --display-name="castai-pv-monitor controller"

gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
  --member="serviceAccount:castai-pv-monitor@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/compute.storageAdmin"

gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
  --member="serviceAccount:castai-pv-monitor@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/monitoring.viewer"

Configure Workload Identity Federation

Bind the Kubernetes service account to the GCP service account using Workload Identity Federation:

NAMESPACE=castai-agent
KSA_NAME=castai-pv-monitor

gcloud iam service-accounts add-iam-policy-binding \
  "castai-pv-monitor@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:${PROJECT_ID}.svc.id.goog[${NAMESPACE}/${KSA_NAME}]"

EKS identity and permissions

IOPS and throughput rightsizing on EKS requires an IAM policy with specific EC2 and CloudWatch permissions, attached to the Kubernetes service account via IAM Roles for Service Accounts (IRSA). The Helm chart also includes nodes/proxy RBAC for reading disk usage metrics.

Required IAM permissions

Create an IAM policy with the following permissions:

Create the IAM policy

Create the policy and role using IAM Roles for Service Accounts (IRSA):

export AWS_REGION="YOUR_AWS_REGION" # update with your region
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export POLICY_NAME="CastAIPvMonitorPolicy"

aws iam create-policy \
  --policy-name ${POLICY_NAME} \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "EBS",
        "Effect": "Allow",
        "Action": [
          "ec2:DescribeVolumes",
          "ec2:DescribeVolumesModifications",
          "ec2:ModifyVolume",
          "ec2:DescribeInstanceTypes"
        ],
        "Resource": "*"
      },
      {
        "Sid": "CloudWatch",
        "Effect": "Allow",
        "Action": [
          "cloudwatch:GetMetricData"
        ],
        "Resource": "*"
      }
    ]
  }'

Create the IRSA role

Create an IAM role with a trust policy that allows the castai-pv-monitor service account to assume it via OIDC federation:

export CLUSTER_NAME="<YOUR_CLUSTER_NAME>" # update with your cluster name
export NAMESPACE="castai-agent"
export SERVICE_ACCOUNT_NAME="castai-pv-monitor"

export OIDC_PROVIDER=$(aws eks describe-cluster \
  --name "${CLUSTER_NAME}" \
  --query "cluster.identity.oidc.issuer" \
  --output text | sed 's|https://||')

aws iam create-role \
  --role-name CastAIPvMonitorRole \
  --assume-role-policy-document "{
    \"Version\": \"2012-10-17\",
    \"Statement\": [{
      \"Effect\": \"Allow\",
      \"Principal\": {
        \"Federated\": \"arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}\"
      },
      \"Action\": \"sts:AssumeRoleWithWebIdentity\",
      \"Condition\": {
        \"StringEquals\": {
          \"${OIDC_PROVIDER}:sub\": \"system:serviceaccount:${NAMESPACE}:${SERVICE_ACCOUNT_NAME}\"
        }
      }
    }]
  }"

aws iam attach-role-policy \
  --role-name CastAIPvMonitorRole \
  --policy-arn "arn:aws:iam::${AWS_ACCOUNT_ID}:policy/${POLICY_NAME}"

Install the pv-monitor Helm chart

Start with a dry-run deployment so that planned actions are logged without modifying your PVCs.

Add the Cast AI Helm repository:

helm repo add castai-helm https://castai.github.io/helm-charts
helm repo update

helm upgrade --install castai-pv-monitor castai-helm/pv-monitor \
  --namespace castai-agent --create-namespace \
  --set castai.clusterID="<YOUR_CAST_AI_CLUSTER_ID>" \
  --set castai.apiKeySecretRef.name="<YOUR_API_KEY_SECRET_NAME>" \
  --set serviceAccount.annotations."iam\.gke\.io/gcp-service-account"="castai-pv-monitor@${PROJECT_ID}.iam.gserviceaccount.com" \
  --set serviceAccount.name=castai-pv-monitor \
  --set config.dryRun=true

--set serviceAccount.create=false --set serviceAccount.name=castai-pv-monitor

helm upgrade --install castai-pv-monitor castai-helm/pv-monitor \
  --namespace castai-agent --create-namespace \
  --set castai.clusterID="<YOUR_CAST_AI_CLUSTER_ID>" \
  --set castai.apiKeySecretRef.name="<YOUR_API_KEY_SECRET_NAME>" \
  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"="arn:aws:iam::${AWS_ACCOUNT_ID}:role/CastAIPvMonitorRole" \
  --set serviceAccount.name=castai-pv-monitor \
  --set config.dryRun=true

📘

Note

Cast AI API key can also be provided directly, via --set castai.apiKey=YOUR_API_KEY. It will result in an automatic creation of a dedicated secret with the API key, tied to the lifecycle of the chart.

📘

Note

The installation creates a default PVPolicy in dry-run mode that matches all PVCs. No changes are made until you explicitly enable live mode. See How it works for details.

Verify installation

Check pods

Confirm that the deployment is running. You should see two pods: one active leader and one standby replica.

kubectl get pods -n castai-agent

Check the default PVPolicy

Verify that the Helm chart created the default PVPolicy. You should see a policy with priority 0 and dry-run enabled.

kubectl get pvpolicy -n castai-agent

Check the matched policy on a PVC

Once the pods are running, Storage Autoscaling begins evaluating PVCs. Verify that a PVC has been matched to a policy:

kubectl get pvc <pvc-name> -o jsonpath='{.metadata.annotations.storage\.cast\.ai/matched-pvpolicy}'

If the output is empty, no policy matched. See No PVPolicy matching a PVC for troubleshooting steps.

Review dry-run logs

Follow the logs to see what actions would be taken. Confirm that PVCs are being evaluated and that planned actions appear:

kubectl logs -n castai-agent \
  -l app.kubernetes.io/name=castai-pv-monitor \
  --follow

Scrape Prometheus metrics (optional)

If you use Prometheus, forward the metrics port and confirm that PVCs are being tracked:

kubectl port-forward -n castai-agent svc/castai-pv-monitor-metrics 8080:8080
curl http://localhost:8080/metrics | grep pv_monitor_pvcs_monitored

Key metrics to watch:

• pv_monitor_pvcs_monitored
• pv_monitor_pvc_usage_percent
• pv_monitor_pvc_growth_total

Inspect annotations on a PVC

Storage Autoscaling writes status annotations to each managed PVC. Inspect them to confirm that the PVC is being tracked:

kubectl get pvc <pvc-name> -o jsonpath='{.metadata.annotations}' | jq

Look for the following annotations:

• storage.cast.ai/disk-type confirms the cloud disk type was detected.
• storage.cast.ai/current-iops shows the current provisioned IOPS.
• storage.cast.ai/operation-status shows whether a resize is in progress.

On AWS, also check storage.cast.ai/modifications-history to see how many of the 4 ModifyVolume slots have been consumed in the rolling 24-hour window.

Enable live mode

Once you have verified the installation and reviewed the dry-run logs, enable live mode.

Option A: Patch the default PVPolicy

The quickest way to go live is to patch the default policy. This enables all three dimensions for every matched PVC at once:

kubectl patch pvpolicy default -n castai-agent --type=merge -p '{"spec":{"dryRun":false}}'

Option B: Create a scoped PVPolicy (recommended for production)

For more control, create a new PVPolicy with dryRun: false and a higher priority targeting only the PVCs you want to manage live. This leaves the default policy in dry-run mode for everything else, so you can roll out incrementally. See Configuration for examples.

⚠️

Warning

The default PVPolicy enables all three dimensions: storageEnabled, iopsEnabled, and throughputEnabled. Every eligible PVC is managed once you disable dry-run.

Before enabling live mode, review the dry-run logs. You can also create a higher-priority PVPolicy with dry-run enabled to keep specific PVCs in observation mode, or use per-PVC storage.cast.ai/*-enabled: "false" annotations.

Trigger a test resize (optional)

To verify that the full resize pipeline works end to end, set a desired capacity annotation on a PVC. Storage Autoscaling will expand the PVC to at least the specified size through the cloud API.

kubectl annotate pvc <pvc-name> \
  storage.cast.ai/desired-capacity="200Gi" \
  --overwrite

Watch the operation status. It should transition from pending to running to done:

kubectl get pvc <pvc-name> -w -o \
  jsonpath='{.metadata.annotations.storage\.cast\.ai/operation-status}'

If the status shows failed, check the storage.cast.ai/last-error annotation for details. See Troubleshooting for common issues.

Next steps