Private image scanning

By default, the Kvisor agent has the ability to scan both private and public images that are running on nodes managed by the CAST AI cluster controller. To enable scanning on other nodes and have a complete overview of vulnerabilities, you must configure Kvisor to scan images stored in your private registry.

Kvisor supports scanning private images from any private registry using image pull secrets. It also integrates with managed registries provided by cloud providers such as Amazon Elastic Container Registry (ECR) for EKS, Google Container Registry (GCR) for GKE, and Azure Container Registry (ACR) for AKS.

Private registries with image pull secret

To enable the scanning of private images using an image pull secret, follow these steps:

Create an image pull secret in the castai-agent namespace:

kubectl -n castai-agent create secret docker-registry [secret-name] \
  --docker-server=[registry-server] \
  --docker-username=[registry-username] \
  --docker-password=[registry-password]

The [registry-server] can be in one of the following formats:

{registry}, for example docker.io
{registry}/{namespace}, for example docker.io/castai
{registry}/{namespace}/{repository}, for example docker.io/castai/agent

Here's an example for Gitlab private images:

kubectl -n castai-agent create secret docker-registry [secret-name] \
  --docker-server=https://registry.gitlab.com \
  --docker-username=registry-user \
  --docker-password=registry-password

Configure Kvisor to scan private images using the image pull secret:

helm upgrade castai-kvisor castai-helm/castai-kvisor -n castai-agent \
  --reuse-values --set controller.extraArgs.image-private-registry-pull-secret=[secret-name]

Multiple registries

You can configure a single image pull secret to scan images from multiple registries and repositories. This is useful when your applications use images from different sources.

Create a secret with multiple registry configurations:

cat <<EOF | kubectl apply -n castai-agent -f -
apiVersion: v1
kind: Secret
metadata:
  name: <secret-name>
type: kubernetes.io/dockerconfigjson
stringData:
  .dockerconfigjson: |
    {
      "auths":
        {
          "docker.io": {
            "username":"user1",
            "password":"password1",
            "auth":"auth1=="
          },
          "docker.io/my-repository": {
            "username":"user2",
            "password":"password2",
            "auth":"auth2=="
          },
          "registry.gitlab.com": {
            "username":"user3",
            "password":"password3",
            "auth":"auth3=="
          }
        }
    }
EOF

To generate the configuration for individual registries or repositories, use this command:

kubectl -n castai-agent create secret docker-registry [secret-name] \
  --docker-server=[registry-server] \
  --docker-username=[registry-username] \
  --docker-password=[registry-password] \
  --dry-run=client \
  --output go-template='{{ index .data ".dockerconfigjson" | base64decode }}' | jq '.auths'

Amazon Elastic Container Registry (Amazon ECR)

Amazon ECR integrates with EKS Kubernetes clusters and provides secure access to images without the need to configure pull secrets. To enable Kvisor to scan images from ECR, follow these steps:

Enable OIDC provider for your EKS cluster:

eksctl utils associate-iam-oidc-provider \
  --cluster <cluster_name> \
  --approve

Create a service account:

kubectl create serviceaccount castai-kvisor-ecr -n castai-agent

Attach an IAM policy that allows Kvisor image scan jobs to access private images in read-only mode:

eksctl create iamserviceaccount \
  --name castai-kvisor-ecr \
  --namespace castai-agent \
  --cluster <cluster_name> \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly \
  --approve \
  --override-existing-serviceaccounts

Configure Kvisor to use the service account for image scanning:

helm upgrade castai-kvisor castai-helm/castai-kvisor -n castai-agent \
  --reuse-values --set controller.extraArgs.image-scan-service-account=castai-kvisor-ecr

Microsoft Azure Container Registry (ACR)

Microsoft ACR integrates with AKS Kubernetes clusters and provides secure access to images without the need to configure pull secrets. To enable Kvisor to scan images from ACR, follow these steps:

Enable OIDC provider and Workload Identity for your AKS cluster:

az aks update -g <resource_group> -n <cluster_name> --enable-oidc-issuer --enable-workload-identity

Create a Managed Identity and get the ID:

az identity create --name <identity_name> --resource-group <resource_group> --location <location> --subscription <subscription>
export IDENTITY_CLIENT_ID="$(az identity show --resource-group <resource_group> --name <identity_name> --query 'clientId' -o tsv)"

Assign ACR Permissions to the newly-created Managed Identity:

ACR_ID=$(az acr show --name <acr_name> --resource-group <resource_group> --query "id" --output tsv)
az role assignment create --assignee $IDENTITY_CLIENT_ID --role "AcrPull" --scope $ACR_ID

Create a Kubernetes Service Account that is linked with this Identity:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: "${IDENTITY_CLIENT_ID}"
  name: castai-kvisor-aks
  namespace: castai-agent
EOF

Retrieve the OIDC Issuer URL and use it to create a federated identity credential between the managed identity, the service account issuer, and the subject:

export AKS_OIDC_ISSUER="$(az aks show -n <cluster_name> -g <resource_group> --query "oidcIssuerProfile.issuerUrl" -o tsv)"

az identity federated-credential create --name <federated_identity_name> \
  --identity-name <identity_name> \
  --resource-group <resource_group> \
  --issuer "${AKS_OIDC_ISSUER}" \
  --subject system:serviceaccount:castai-agent:castai-kvisor-aks \
  --audience api://AzureADTokenExchange

Configure Kvisor to use the service account for image scanning:

helm upgrade castai-kvisor castai-helm/castai-kvisor -n castai-agent \
  --reuse-values --set controller.extraArgs.image-scan-service-account=castai-kvisor-aks

Google Container Registry/Google Artifact Registry

No additional setup on kvisor side is required.