AI Model Providers

The AI Model Provider entity lets you securely store and manage credentials for connecting to upstream LLM services. Use AI Model Providers to:

• Store API keys for OpenAI, Azure, Bedrock, or any other LLM provider
• Centrally manage and rotate credentials across multiple AI Models
• Enforce consistent authentication across your deployments

An AI Model Provider manages outbound credentials, which is distinct from the inbound authentication managed by an AI Auth Strategy. When an AI Consumer calls an AI Model, the AI Auth Strategy checks who they are. The AI Model then uses the AI Model Provider’s credentials to forward the request upstream.

Each AI Model Provider has a type that selects the upstream LLM service and configures provider-specific options. See the schema for supported types, and the per-provider pages under AI Gateway providers for provider-specific configuration and limitations.

AI Model Providers can be created and managed through:

• Konnect UI
• AI Gateway API: /v1/ai-gateways/{aiGatewayId}/model-providers
• kongctl

For configuration examples and step-by-step setup instructions, see Set up an AI Model Provider.

Relationship to AI Models

AI Model Providers and AI Models have a many-to-many relationship: one AI Model Provider can back many AI Models, and one AI Model can route to multiple AI Model Providers. For example, a single openai AI Model Provider might be used by both a chat AI Model and an embeddings AI Model, while a single AI Model might route to OpenAI and Anthropic targets for failover.

When configuring an AI Model, you reference an AI Model Provider by setting the provider field in each item of the targets array. You can reference by name or id. Use id if you plan to rename the AI Model Provider later.

AI Gateway supports the following upstream LLM providers. The AI Model Provider’s type field selects one of these targets. The following provider-specific pages document supported capabilities, configuration requirements, and limitations.

OpenAI

Azure OpenAI

Amazon Bedrock

Anthropic

Gemini

Cohere

Mistral

Hugging Face

Kimi

Llama

xAI

Alibaba Cloud DashScope

Cerebras

DeepSeek

Ollama

Databricks

Vercel

vLLM

AWS SageMaker

The config.auth object declares how AI Gateway authenticates to the upstream AI provider. The shape of auth depends on the AI Model Provider’s type:

• basic: Header- or parameter-based auth. Supports up to one auth header (config.auth.headers) and one auth parameter (config.auth.params). Parameters can be sent as a query string or in the request body (config.auth.params[].location). Used by most AI Model Provider types.
• aws: IAM access-key and assume-role auth. Used by Bedrock.
• azure: Microsoft Entra ID or managed-identity auth. Used by Azure OpenAI.
• gcp: Google service-account auth. Used by Gemini.

Bedrock, Azure OpenAI, and Gemini can also fall back to basic auth.

An AI Model Provider stores the credentials, but doesn’t generate any runtime primitives.

AI Model Provider credentials are passed to the runtime only when an AI Model references the AI Model Provider. At that point, the credentials are then passed to the AI Model.

When you update the credentials of an AI Model Provider, the new credentials are passed to every AI Model that references it the next time a request is made through the AI Model.

You can’t attach AI Policies directly to an AI Model Provider entity instance. AI Policies attach to AI Models, AI Agents, AI Consumers, or AI Consumer Groups to control security, rate limiting, guardrails, and observability.

To apply an AI Policy across requests using a particular AI Model Provider, you can:

1. Set the policy to global: true to apply it to all resources in the gateway.
2. Attach the same policy to each AI Model that references the AI Model Provider.
3. Create an AI Consumer Group with the policy and control access to AI Models via ACLs.

The following example creates an OpenAI AI Model Provider that authenticates with a single bearer-token header. An AI Model can then route to this AI Model Provider by setting the provider field in a targets array item to my-openai-account (or the AI Model Provider id).

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/model-providers \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "OpenAI Production",
      "name": "my-openai-account",
      "type": "openai",
      "config": {
        "auth": {
          "type": "basic",
          "headers": [
            {
              "name": "Authorization",
              "value": "'$OPENAI_AUTH_HEADER'"
            }
          ]
        }
      }
    }
    '

ai_gateway_model_providers:
  - ref: my-openai-account
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: OpenAI Production
    name: my-openai-account
    type: openai
    config:
      auth:
        type: basic
        headers:
        - name: Authorization
          value: !secret {source: !env OPENAI_AUTH_HEADER}