The AI Enabler Settings page provides a centralized interface for configuring how the Cast AI Enabler Proxy handles your LLM requests. These settings are accessible through the API, but this guide will focus on the intuitive Cast AI console UI that makes managing your AI Enabler settings more straightforward.

Accessing the Settings Page

To access the AI Enabler Settings page:

1. Log in to the Cast AI console
2. Navigate to AI Enabler in the sidebar menu
3. Select Settings from the submenu

The settings page has two tabs, Organization and API Keys, which allow you to configure settings at different levels.

Organization Settings

The Organization tab contains global settings that apply to all LLM requests sent to AI Enabler, unless overridden at the API key level or through request headers.

Routing

Routing is the core function of the AI Enabler that allows Cast AI to direct your requests to the most suitable LLM model for your application.

• Enable Routing: Toggle this setting on to allow AI Enabler to route your requests to the optimal model based on complexity, performance, and cost considerations. When disabled, requests will only be proxied to the model specified in the request.

• How it works: When enabled, Cast AI analyzes each request and routes it to the most suitable model that balances quality, performance, and cost efficiency. For example, simple requests might be sent to more affordable models, while complex requests are routed to more capable models with more advanced reasoning capabilities.

• API Header Override: This setting can be overridden on a per-request basis using the API request header X-Routing-Enabled: true/false

Prompt Sharing

Prompt sharing allows Cast AI to store your prompts to improve routing accuracy and model recommendations.

• Enable Prompt Sharing: This setting is turned on by default to allow Cast AI to store your prompts to help improve routing decisions and provide more accurate model recommendations.

• Privacy Considerations: When enabled, Cast AI will store your prompts to improve the routing algorithm. No personally identifiable information is extracted or stored separately. When disabled, Cast AI will only store the request metadata.

• API Header Override: This setting can be overridden on a per-request basis using the API request header X-Prompt-Sharing-Enabled: true/false

Quality Weight

Quality weight helps you balance cost efficiency and response quality by adjusting the routing algorithm's priorities.

Adjust the slider between Cost and Quality to set your preferred balance. Moving toward Cost prioritizes less expensive models, while moving toward Quality prioritizes models with higher capabilities.

The default setting is 0.5, representing an equal balance between cost and quality.

Organizations focused on minimizing costs might prefer settings closer to the Cost end, while those prioritizing response quality for complex tasks might prefer settings closer to the Quality end.

• API Header Override: This setting can be overridden on a per-request basis using the API request header X-Router-Quality-Weight: 0.0-1.0 (where 0.0 represents maximum cost efficiency, and 1.0 represents maximum quality)

Settings Hierarchy and Override Priority

Cast AI implements a hierarchical structure for AI Enabler settings, where more specific configurations override broader ones. Understanding this priority order is crucial for effectively managing your AI Enabler behavior:

1. Request Headers (Highest Priority): Settings specified in request headers override everything else for that specific request
2. API Key Settings (Medium Priority): Settings configured for a specific API key override organization settings for all requests using that key
3. Organization Settings (Lowest Priority): These global settings serve as the default when no overrides are present

This hierarchy ensures that you can configure appropriate defaults at the organization level while maintaining the flexibility to customize behavior for specific API keys or individual requests as needed.

API Key Settings

The API Keys tab allows you to configure settings specific to individual API keys, which will override the organization-level settings for requests made with those keys.

The table displays all API keys in your organization and their current settings. Per-key settings always override organization settings regardless of the key's scope, but can still be overridden by request headers.

For more information on setting up users, access keys, and service accounts, see Role-Based Access Control.

Managing API Key Settings

To adjust settings for a specific API key:

1. Click on the API key name in the table
2. A settings drawer will open, allowing you to modify settings for that particular key
3. Make your desired changes to routing, prompt sharing, or quality weight
4. Save your changes

These per-key settings will override the organization-level settings for any requests made using this specific API key, giving you fine-grained control over how different keys behave.

API Reference

For developers looking to override settings programmatically, here are the available request headers:

Remember that header overrides have the highest priority and will take precedence over both API key settings and organization settings for the specific request.

For more details on API integration, see the AI Enabler Settings API Reference.

Troubleshooting

Settings changes not taking effect

Check the following:

• Look for request headers that might be overriding your settings
• Verify the API key you're using doesn't have overriding settings that conflict with your expectations
• Verify the API key you're using is scoped to the correct resource (cluster, organization)
• Allow a few minutes for changes to propagate

Remember the priority order: Headers → API Key Settings → Organization Settings. A higher-priority setting will always override a lower-priority one.

For additional assistance with AI Enabler, contact Cast AI support or visit our community Slack channel.