AI Request Transformer

Release date 2025/03/27

Deprecation

• Deprecated preserve mode in config.route_type. Use config.llm_format instead. The preserve mode setting will be removed in a future release.

Feature

• Allow authentication to Bedrock services with assume roles in AWS.

Bugfix

• Fixed an issue where AI upstream URL trailing would be empty.

• • Fixed an issue where templates weren’t being resolved correctly.
  • The plugins now support nested fields.

Release date 2025/03/20

Bugfix

• Fixed issue of template not being resolved correctly and supported nested fields.

Release date 2025/01/28

Bugfix

• Reverted the analytics container key from “proxy” to “ai-proxy” to align with previous versions.

Release date 2024/12/12

Bugfix

• Fixed a bug where the correct LLM error message was not propagated to the caller.

• Fixed an issue where Azure Managed Identity did not work for AI Transformer Plugins.

Release date 2024/11/04

Bugfix

• Fixed a bug where the correct LLM error message was not propagated to the caller.

Release date 2024/09/11

Feature

• AI plugins: retrieved latency data and pushed it to logs and metrics.

• allow AI plugin to read request from buffered file

• Kong AI Gateway (AI Proxy and associated plugin family) now supports all AWS Bedrock “Converse API” models.

• Kong AI Gateway (AI Proxy and associated plugin family) now supports the Google Gemini “chat” (generateContent) interface.

Bugfix

• Fixed a bug where cloud identity authentication was not used in ai-request-transformer and ai-response-transformer plugins.

• Fixed an issue for multi-modal inputs are not properly validated and calculated.

• Fixed a bug where Azure Managed-Identity tokens would never rotate
  in case of a network failure when authenticating.

Release date 2024/02/12

Feature

• Introduced the new AI Request Transformer plugin that enables passing mid-flight consumer requests to an LLM for transformation or sanitization.