Kong AI Gateway cookbooks

A collection of production-ready recipes to build with Kong AI Gateway: LLM routing, guardrails, semantic caching, cost optimization, governance, and MCP integration.

Featured

Model-Based Routing

Dynamically route requests across OpenAI and AWS Bedrock based on prompt complexity analysis.

LLM
View recipe →

Prompt Compression with Headroom

Compress tool results in agent requests with the AI Prompt Compressor Policy and Headroom, and compare a compressed endpoint with an uncompressed one on Kong AI Gateway.

Cost optimization
View recipe →

Voice AI Pipeline Observability

Observe and govern a cascading voice AI pipeline (STT, LLM, TTS) with per-hop telemetry and conversation-level tracing through Langfuse.

Observability
View recipe →

All recipes

AI Rate Limit Alerts

Notify Slack, PagerDuty, or a webhook before a Consumer hits its AI token quota, instead of waiting for HTTP 429.
Cost optimization View recipe →

Basic LLM Routing

Route requests to any supported LLM provider through Kong AI Gateway with AI Consumer authentication and per-request model selection.
LLM View recipe →

Claude Code SSO

Authenticate Claude Code requests via Okta SSO with consumer-group-based model routing and per-tier rate limiting.
Access control View recipe →

Custom LLM Routing

Route chat requests to a small or large model by applying a Pre-function Policy that estimates input token size and rewrites the request model alias.
LLM View recipe →

GitHub Copilot BYOK Custom Endpoint

Route GitHub Copilot Chat and agent traffic through Kong Gateway via Copilot's Bring Your Own Key Custom Endpoint, with per-consumer attribution, request normalization, and token rate limiting.
LLM View recipe →

Guardrail Integrations

Integrate external guardrail services through Kong AI Gateway using dedicated AI Policies and the universal custom guardrail adapter.
Guardrails View recipe →

LLM Cost Optimization

Reduce LLM infrastructure costs using semantic routing, caching, prompt compression, and tiered cost-based rate limiting.
Cost optimization View recipe →

LLM tracing with Arize Phoenix

Export Kong AI Gateway chat completions to Arize Phoenix over OTLP and fill Sessions, user, Input/Output, and Messages with AI Policies.
Observability View recipe →

Mistral AI with Kong AI Gateway

Put Mistral chat and embedding models behind one Kong AI Gateway endpoint, with per-application API keys, so no client ever holds your Mistral key.
LLM View recipe →

Model-Based Routing

Dynamically route requests across OpenAI and AWS Bedrock based on prompt complexity analysis.
LLM View recipe →

Multi-Layer AI Guardrails

Defense-in-depth content safety using regex filtering, semantic analysis, and PII protection in a single gateway route.
Guardrails View recipe →

Prompt Compression with Headroom

Compress tool results in agent requests with the AI Prompt Compressor Policy and Headroom, and compare a compressed endpoint with an uncompressed one on Kong AI Gateway.
Cost optimization View recipe →
ModelContextProtocol

Secure External MCP Gateway

Proxy third-party MCP servers (GitHub, Konnect) through Kong AI Gateway with AI Consumer identification, IdP-based identity mapping, and tool-level ACLs.
MCP View recipe →
ModelContextProtocol

Secure Internal MCP Gateway

Aggregate multiple internal APIs into a single MCP server with OAuth 2.1 authentication and tool-level access control via consumer-group ACLs.
MCP View recipe →
ModelContextProtocol

Secure MCP with Microsoft Entra ID

Protect Kong-converted MCP servers with Microsoft Entra ID using JWKS validation. Interactive clients request the delegated mcp.tools scope. Client credentials with /.default are for automated tests only.
MCP View recipe →

Voice AI Pipeline Observability

Observe and govern a cascading voice AI pipeline (STT, LLM, TTS) with per-hop telemetry and conversation-level tracing through Langfuse.
Observability View recipe →
Something wrong?
Report an Issue | Edit this Page
View all Kong documentation

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help?

Ask in our Slack Community
Contact Support