AI Semantic Response Guard Policy

The AI Semantic Response Guard AI Policy filters LLM responses based on semantic similarity to predefined rules, helping prevent unwanted or unsafe responses when serving /chat, /completions, or /embeddings requests through AI Gateway.

You can use a combination of allow and deny response rules to maintain integrity and compliance when returning responses from an LLM service.

The AI Policy analyzes the semantic content of the full LLM response before it is returned to the client. The matching behavior is as follows:

• If any deny_responses are set and the response matches a pattern in the deny list, the response is blocked with a 403 Forbidden.
• If any allow_responses are set, but the response matches none of the allowed patterns, the response is also blocked with a 403 Forbidden.
• If any allow_responses are set and the response matches one of the allowed patterns, the response is permitted.
• If both deny_responses and allow_responses are set, the deny condition takes precedence. A response that matches a deny pattern will be blocked, even if it also matches an allow pattern. If the response does not match any deny pattern, it must still match an allow pattern to be permitted.

To enforce these rules, the AI Semantic Response Guard Policy:

1. Disables streaming (stream=false) to ensure the full response body is buffered before analysis.
2. Intercepts the response body using the guard-buffered-response filter.
3. Extracts response text, supporting JSON parsing of multiple LLM formats and gzipped content.
4. Generates embeddings for the extracted text.
5. Searches the vector database (Redis or pgvector) against configured allow_responses or deny_responses.
6. Applies the decision rules described above.

If a response is blocked or if a system error occurs during evaluation, the AI Policy returns a 403 Forbidden to the client without exposing that the AI Semantic Response Guard blocked it.

A vector database stores and compares vector embeddings—numerical representations of text, prompts, documents, or other content. When you configure semantic features in AI Models or AI Policies, embeddings are generated and stored in the vector database so that incoming requests can be compared against the stored vectors to find semantically similar matches. For example, an incoming prompt is embedded and compared against cached prompt keys, model descriptions, document chunks, or allow/deny lists to determine semantic similarity.

AI Gateway semantic features support the following vector databases:

• Using vectordb.strategy: redis and parameters in vectordb.redis:
  • Redis with Redis Vector Search
  • Redis Cloud
  • Valkey: When you configure vectordb.strategy: redis, Kong Gateway queries the server and checks the server name field. If it detects Valkey request, it automatically uses the Valkey-specific driver.
  • Managed Redis with cloud authentication:
    • AWS ElastiCache (auth_provider: aws)
    • Azure Managed Redis (auth_provider: azure)
    • Google Cloud Memorystore (auth_provider: gcp)

    For configuration details, see Using cloud authentication with Redis.

• Using vectordb.strategy: pgvector and parameters in vectordb.pgvector:
  • PostgreSQL with pgvector

Configure vector database settings in AI Models and AI Policies to enable semantic similarity features.

Using cloud authentication with Redis

If your AI Policy uses a Redis datastore, you can authenticate to it with a cloud Redis provider. This allows you to rotate credentials without relying on static passwords.

The following providers are supported:

• AWS ElastiCache
• Azure Managed Redis
• Google Cloud Memorystore (with or without Valkey)

Each provider also supports an instance and cluster configuration.

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 10000
      username: $CLUSTER_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

Set config.proxy on this entity to route its outbound requests through an HTTP forward proxy. Use this in network-isolated deployments where AI Gateway cannot open direct connections to LLM providers or auxiliary services.

The proxy record is identical for AI Model, AI MCP Server, and supported AI Policy entities. Existing capabilities such as load balancing, health checking, streaming, WebSocket, and HTTP/2 continue to work when the proxy is active.

For the full field reference, traffic flow, and limitations, see Forward proxy support.