API Reference

Create hosted model.

post
https://api.cast.ai/ai-optimizer/v1beta/organizations//clusters//hosted-models

Deploys a model on a CAST AI managed cluster.

Path Params
string
required

The ID of the Organization.

string
required
Body Params
string
required

The ID of the Cluster.

string
required

Kubernetes service name for the deployed hosted model.

int32
required

Port on which the model will be exposed.

string

The name of the node template that will be used to deploy the model. If not provided, a new node template will be created.

vllmConfig
object

Configuration specific to the vLLM provider.

string

The region the model is deployed in.

horizontalAutoscaling
object

The settings for horizontal autoscaling.

fallback
object

The fallback to use when the primary model is not available.

hibernation
object

The settings for automatic hibernation and resumption of the hosted model.

string
required

Model specs ID

edgeLocations
object

Edge locations where the model can be deployed

boolean

Whether the auto-created provider for this hosted model should appear in the CLI provider list.

Responses

Updated 5 months ago

Language
Credentials
URL
Response 
Click Try It! to start a request and see the response here! Or choose an example:
application/json
Updated 5 months ago