In AI Gateway running on Kong Gateway, AI functionality is delivered by three proxy plugins that extend Kong Gateway’s core proxying. You build Services and Routes, then attach a plugin to add AI behavior:
- AI Proxy Advanced provides model proxying, transformation, and load balancing across providers and models.
- AI MCP Proxy bridges Kong-managed Services to the Model Context Protocol, converting REST APIs into MCP tools or fronting upstream MCP servers.
- AI A2A Proxy adds observability and gateway control for Agent-to-Agent protocol traffic.
This model works, but it couples every AI concept to Kong Gateway primitives. A single logical model can require a Service, a Route, an AI Proxy Advanced plugin, and several supporting plugins, with the AI intent spread across all of them.
AI Gateway 2.x abstracts those plugins into a purpose-built entity model on its own control plane. You no longer need to configure Services, Routes, and plugins manually. Instead, you declare the AI resource you want, and the control plane provisions the underlying primitives for you.
The following table describes how the two models relate at a high level:
|
V1 (Kong Gateway model)
|
V2 (Native AI Gateway model)
|
Description
|
|
AI Proxy Advanced on a Service or Route
|
AI Model
|
One model entry per virtual model, with one or more targets.
|
Set config.targets[].model.provider on AI Proxy Advanced with inline auth
|
AI Model Provider
|
Provider credentials are now declared once and reused across AI Model entities.
|
Set config.targets[].route_type on AI Proxy Advanced
|
Set capabilities and formats.type on an AI Model
|
The route_type is decomposed into a capabilities array and a format type.
|
Set config.balancer on AI Proxy Advanced
|
Set config.balancer on an AI Model
|
The same load balancing algorithms are available.
|
Set config.vectordb and config.embeddings on AI Proxy Advanced
|
Set config.balancer.AIGatewayModelBalancerSemanticConfig.vectordb and config.balancer.AIGatewayModelBalancerSemanticConfig.embeddings on an AI Model
|
Carried over with the same Redis and pgvector strategies.
|
|
AI MCP Proxy on a Service or Route
|
AI MCP Server
|
Each plugin mode maps directly to an AI MCP Server type value in version 2.x. Additionally, a new upstream-server type is available.
|
Set config.default_acl and config.tools.acl on AI MCP Proxy
|
Set access or tools.access on an AI MCP Server. Configure an AI Consumer or AI Consumer Group
|
ACLs become first-class fields.
|
|
AI A2A Proxy on a Service or Route
|
AI Agent
|
First class A2A support with URL rewriting and A2A analytics built in.
|
|
Plugins
|
Policies
|
AI Policies replace plugins, and can be attached to other entities. The type field on a Policy corresponds to the plugin.
|
|
Consumers and Consumer Groups
|
AI Consumer and AI Consumer Group
|
Managed from the control plane.
|
|
Vault
|
AI Vault and AI Data Plane Certificates
|
Referenceable fields keep the same {vault://…} syntax.
|
Note the following terminology changes:
- AI Policies replace Kong Gateway plugins. All AI Policies have some common parameter. Each AI Policy has a
type which corresponds to a plugin from AI Gateway running on Kong Gateway, such as ai-sanitizer or openid-connect, and their config is the same as the plugin.
- AI Model Providers are now separate reusable entities. This decouples config and credentials of upstream providers from specific models, which allows you to declare an AI Model Provider once and reference it by name from multiple AI Models.
- A Route from AI Gateway running on Kong Gateway is split into two AI Gateway 2.x concepts: a
capabilities list and a formats entry.