AI Request Transformer

Release date 2025/07/03

Bugfix

• Fixed an issue where some of ai metrics was missed in analytics

Known Issues

• If any AI Gateway plugin has been enabled in a self-managed Kong Gateway deployment for more than a week, upgrades from 3.10 versions to 3.11.0.0 will fail due to a license migration issue. This does not affect Konnect deployments.

  A fix will be provided in 3.11.0.1.

  See breaking changes in 3.11 for a temporary workaround.

Release date 2025/03/27

Deprecation

• Deprecated preserve mode in config.route_type. Use config.llm_format instead. The preserve mode setting will be removed in a future release.

Feature

• Allow authentication to Bedrock services with assume roles in AWS.

Bugfix

• Fixed an issue where AI upstream URL trailing would be empty.

• • Fixed an issue where templates weren’t being resolved correctly.
  • The plugins now support nested fields.

Release date 2025/03/20

Bugfix

• Fixed issue of template not being resolved correctly and supported nested fields.

Release date 2025/01/28

Bugfix

• Reverted the analytics container key from “proxy” to “ai-proxy” to align with previous versions.

Release date 2024/12/12

Bugfix

• Fixed a bug where the correct LLM error message was not propagated to the caller.

• Fixed an issue where Azure Managed Identity did not work for AI Transformer Plugins.

Release date 2024/11/04

Bugfix

• Fixed a bug where the correct LLM error message was not propagated to the caller.

Release date 2024/09/11

Feature

• AI plugins: retrieved latency data and pushed it to logs and metrics.

• allow AI plugin to read request from buffered file

• Kong AI Gateway (AI Proxy and associated plugin family) now supports all AWS Bedrock “Converse API” models.

• Kong AI Gateway (AI Proxy and associated plugin family) now supports the Google Gemini “chat” (generateContent) interface.

Bugfix

• Fixed a bug where cloud identity authentication was not used in ai-request-transformer and ai-response-transformer plugins.

• Fixed an issue for multi-modal inputs are not properly validated and calculated.

• Fixed a bug where Azure Managed-Identity tokens would never rotate
  in case of a network failure when authenticating.

Release date 2024/02/12

Feature

• Introduced the new AI Request Transformer plugin that enables passing mid-flight consumer requests to an LLM for transformation or sanitization.