--- title: "AI Gateway providers" description: This page is an introduction to the AI providers available in AI Gateway. url: "/ai-gateway/ai-providers/" canonical_url: "/ai-gateway/ai-providers/" content_type: landing_page products: - AI Gateway tags: - ai canonical: true works_on: - konnect --- # AI Gateway providers The core of [AI Gateway](/ai-gateway/) is the ability to serve [AI Models](/ai-gateway/entities/ai-model/) from various [AI Model Providers](/ai-gateway/entities/ai-model-provider/) via a provider-agnostic API. This normalized API layer affords developers and organizations multiple benefits: * Client applications are shielded from AI Provider API specifics, promoting code reusability * Centralized AI Provider credential management * Developers and organizations have a central point of governance and observability over AI data and usage * Request routing can be dynamic, allowing AI usage to be optimized based on various metrics * AI services can be used by Kong Gateway to augment non-AI API traffic [OpenAI](/ai-gateway/ai-providers/openai/) [Azure AI](/ai-gateway/ai-providers/azure/) [Amazon Bedrock](/ai-gateway/ai-providers/bedrock/) [Amazon SageMaker](/ai-gateway/ai-providers/sagemaker/) [Gemini](/ai-gateway/ai-providers/gemini/) [Vercel](/ai-gateway/ai-providers/vercel/) [Anthropic](/ai-gateway/ai-providers/anthropic/) [Cohere](/ai-gateway/ai-providers/cohere/) [Hugging Face](/ai-gateway/ai-providers/huggingface/) [Llama](/ai-gateway/ai-providers/llama/) [Mistral](/ai-gateway/ai-providers/mistral/) [xAI](/ai-gateway/ai-providers/xai/) [DashScope](/ai-gateway/ai-providers/dashscope/) [Kimi](/ai-gateway/ai-providers/kimi/) [Cerebras](/ai-gateway/ai-providers/cerebras/) [Ollama](/ai-gateway/ai-providers/ollama/) [Databricks](/ai-gateway/ai-providers/databricks/) [DeepSeek](/ai-gateway/ai-providers/deepseek/) [vLLM](/ai-gateway/ai-providers/vllm/) > Note that some providers may not be available or require different configuration steps depending on your AI Gateway version, and some providers don't support all route types. > See the specific provider documentation for more details. ## References * [Load balancing with Kong AI Gateway](/ai-gateway/load-balancing/) * [Kong AI Gateway resource sizing guidelines](/ai-gateway/resource-sizing-guidelines-ai/) ## Frequently Asked Questions - Can I authenticate to Azure AI with Azure Identity? Yes, if AI Gateway is running on Azure, you can configure an [AI Model Provider](/ai-gateway/entities/ai-model-provider/) to detect the designated Managed Identity or User-Assigned Identity of that Azure Compute resource and use it for authentication. In your [AI Model Provider](/ai-gateway/entities/ai-model-provider/) configuration, set `auth.type` to `azure`, then: * Set `auth.use_managed_identity` to `true` to use a system-assigned Managed Identity. * Set `auth.use_managed_identity` to `true` and `auth.client_id` to the client ID to use a user-assigned identity. Then reference this [AI Model Provider](/ai-gateway/entities/ai-model-provider/) in your [AI Model](/ai-gateway/entities/ai-model/) to proxy requests with the appropriate Azure credentials. - How can I set model generation parameters when calling Gemini? You can configure model generation parameters when calling Gemini through AI Gateway: - **Using the Gemini SDK**: 1. Create an [AI Model Provider](/ai-gateway/entities/ai-model-provider/) for Gemini and an [AI Model](/ai-gateway/entities/ai-model/) that references it. 1. Configure parameters like `temperature`, `top_p`, and `top_k` on the client side: ```python model = genai.GenerativeModel( 'gemini-2.5-flash', generation_config=genai.types.GenerationConfig( temperature=0.7, top_p=0.9, top_k=40, max_output_tokens=1024 ) ) ``` - **Using the OpenAI SDK** with AI Gateway: 1. Create an [AI Model](/ai-gateway/entities/ai-model/) with an `openai` entry in its `formats` array, and a target that references your Gemini [AI Model Provider](/ai-gateway/entities/ai-model-provider/). 1. You can configure parameters in one of three ways: - Configure them in the [AI Model](/ai-gateway/entities/ai-model/) only. - Configure them in the client only. - Configure them in both. The client-side values override the model config. - How do I use Gemini's `googleSearch` tool for real-time web searches? Configure an [AI Model](/ai-gateway/entities/ai-model/) that uses a Gemini [AI Model Provider](/ai-gateway/entities/ai-model-provider/), then declare the `googleSearch` tool in your requests. - How do I control aspect ratio and resolution for Gemini image generation? Pass `imageConfig` parameters via `generationConfig` in your image generation requests. - How do I get reasoning traces from Gemini models? Pass `thinkingConfig` parameters via `extra_body` in your requests to enable detailed reasoning traces. - How do I specify model IDs for Amazon Bedrock cross-region inference profiles? For cross-region inference with Amazon Bedrock, prefix the model ID with a geographic identifier in your [AI Model](/ai-gateway/entities/ai-model/) configuration: ``` {geography-prefix}.{provider}.{model-name}... ``` For example: `us.anthropic.claude-sonnet-4-5-20250929-v1:0` ### `us.` Geography: United States ### `eu.` Geography: European Union ### `apac.` Geography: Asia-Pacific ### `global.` Geography: All commercial regions For a full list of supported cross-region inference profiles, see [Supported Regions and models for inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html) in the AWS documentation. - How do I set the FPS parameter for video generation for Amazon Bedrock? Use the `extra_body` feature when sending requests to an [AI Model](/ai-gateway/entities/ai-model/) that proxies Amazon Bedrock video generation in OpenAI format: ```sh curl http://localhost:8000 \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "amazon.nova-reel-v1:0", "prompt": "A large red square that is rotating", "extra_body": { "fps": 24 } }' ``` - How do I use Amazon Bedrock's Rerank API to improve RAG retrieval quality? Configure an [AI Model](/ai-gateway/entities/ai-model/) with a Bedrock [AI Model Provider](/ai-gateway/entities/ai-model-provider/) and set up AWS authentication using IAM credentials or assumed roles. - How do I include guardrail configuration with Amazon Bedrock requests? Add a `guardrailConfig` object to your request body when calling an [AI Model](/ai-gateway/entities/ai-model/) that proxies Amazon Bedrock: ```json { "messages": [ { "role": "system", "content": "You are a scientist." }, { "role": "user", "content": "What is the Boltzmann equation?" } ], "guardrailConfig": { "guardrailIdentifier": "$GUARDRAIL-IDENTIFIER", "guardrailVersion": "1", "trace": "enabled" } } ``` For more details, see [Guardrails and content safety](/ai-gateway/#guardrails-and-content-safety) and the [AWS Bedrock guardrails documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html). - How do I use Cohere's document-grounded chat for RAG pipelines? Configure an [AI Model](/ai-gateway/entities/ai-model/) with a Cohere [AI Model Provider](/ai-gateway/entities/ai-model-provider/) and send queries with candidate documents. The model filters for relevance and returns answers with citations.