Create hosted model.

post

https://api.cast.ai/ai-optimizer/v1beta/organizations/{organizationId}/clusters/{hosted_model.cluster_id}/hosted-models

Deploys a model on a CAST AI managed cluster.

Recent Requests

Time	Status	User Agent
Retrieving recent requests…

Loading…

Path Params

organizationId

string

required

The ID of the Organization.

hosted_model.cluster_id

string

required

Body Params

A model that is deployed on a CAST AI managed cluster.

clusterId

string

required

The ID of the Cluster.

service

string

required

Kubernetes service name for the deployed hosted model.

port

int32

required

Port on which the model will be exposed.

nodeTemplateName

string

The name of the node template that will be used to deploy the model. If not provided, a new node template will be created.

vllmConfig

object

Configuration for the vLLM provider.

region

string

The region the model is deployed in.

horizontalAutoscaling

object

The settings for horizontal autoscaling

fallback

object

The fallback model to use when the primary model is not available.

hibernation

object

The settings for hibernation

modelSpecsId

string

required

Model specs ID

edgeLocations

object

List of edge locations where the model can be deployed.

includeInCli

boolean

Whether the auto-created provider for this hosted model should appear in the CLI provider list.

Responses

Language

Credentials

URL

Loading…

Response

Click Try It! to start a request and see the response here! Or choose an example:

application/json

200OK

defaultDefault error response