AI Semantic Prompt Guard Policy

The AI Semantic Prompt Guard Policy enforces prompt governance using semantic similarity matching. It compares incoming requests against your configured allow and deny lists, preventing misuse of text completion requests.

You can use a combination of allow and deny rules to maintain integrity and compliance when serving an LLM service using AI Gateway.

By default, the Policy only evaluates the latest message from the user role. To also match earlier turns in a multi-turn conversation, set config.rules.match_all_conversation_history to true. To also match messages from roles other than user, set config.rules.match_all_roles to true.

The matching behavior is as follows:

• If any deny prompts are set and the request matches a prompt in the deny list, the caller receives a 403 response.
• If any allow prompts are set, but the request matches none of the allowed prompts, the caller also receives a 403 response.
• If any allow prompts are set and the request matches one of the allow prompts, the request passes through to the LLM.
• If there are both deny and allow prompts set, the deny condition takes precedence over allow. Any request that matches a prompt in the deny list will return a 403 response, even if it also matches a prompt in the allow list. If the request doesn’t match a prompt in the deny list, then it must match a prompt in the allow list to be passed through to the LLM.

A vector database stores and compares vector embeddings—numerical representations of text, prompts, documents, or other content. When you configure semantic features in AI Models or AI Policies, embeddings are generated and stored in the vector database so that incoming requests can be compared against the stored vectors to find semantically similar matches. For example, an incoming prompt is embedded and compared against cached prompt keys, model descriptions, document chunks, or allow/deny lists to determine semantic similarity.

AI Gateway semantic features support the following vector databases:

• Using vectordb.strategy: redis and parameters in vectordb.redis:
  • Redis with Redis Vector Search
  • Redis Cloud
  • Valkey: When you configure vectordb.strategy: redis, Kong Gateway queries the server and checks the server name field. If it detects Valkey request, it automatically uses the Valkey-specific driver.
  • Managed Redis with cloud authentication:
    • AWS ElastiCache (auth_provider: aws)
    • Azure Managed Redis (auth_provider: azure)
    • Google Cloud Memorystore (auth_provider: gcp)

    For configuration details, see Using cloud authentication with Redis.

• Using vectordb.strategy: pgvector and parameters in vectordb.pgvector:
  • PostgreSQL with pgvector

Configure vector database settings in AI Models and AI Policies to enable semantic similarity features.

Using cloud authentication with Redis

If your AI Policy uses a Redis datastore, you can authenticate to it with a cloud Redis provider. This allows you to rotate credentials without relying on static passwords.

The following providers are supported:

• AWS ElastiCache
• Azure Managed Redis
• Google Cloud Memorystore (with or without Valkey)

Each provider also supports an instance and cluster configuration.

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 10000
      username: $CLUSTER_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

Set config.proxy on this entity to route its outbound requests through an HTTP forward proxy. Use this in network-isolated deployments where AI Gateway cannot open direct connections to LLM providers or auxiliary services.

The proxy record is identical for AI Model, AI MCP Server, and supported AI Policy entities. Existing capabilities such as load balancing, health checking, streaming, WebSocket, and HTTP/2 continue to work when the proxy is active.

For the full field reference, traffic flow, and limitations, see Forward proxy support.