AI Semantic Prompt Guard Policy

The AI Semantic Prompt Guard Policy enforces prompt governance using semantic similarity matching. It compares incoming requests against your configured allow and deny lists, preventing misuse of text completion requests.

You can use a combination of allow and deny rules to maintain integrity and compliance when serving an LLM service using AI Gateway.

By default, the Policy only evaluates the latest message from the user role. To also match earlier turns in a multi-turn conversation, set config.rules.match_all_conversation_history to true. To also match messages from roles other than user, set config.rules.match_all_roles to true.

The matching behavior is as follows:

• If any deny prompts are set and the request matches a prompt in the deny list, the request is blocked.
• If any allow prompts are set, but the request matches none of the allowed prompts, the request is also blocked.
• If any allow prompts are set and the request matches one of the allow prompts, the request passes through to the LLM.
• If there are both deny and allow prompts set, the deny condition takes precedence over allow. Any request that matches a prompt in the deny list is blocked, even if it also matches a prompt in the allow list. If the request doesn’t match a prompt in the deny list, then it must match a prompt in the allow list to be passed through to the LLM.

The AI Semantic Prompt Guard Policy responds to a blocked request according to its rejection mode. In the default none mode, the response is a 400 with the message bad request, which hides that a guardrail is in use.

Use config.rejection_mode to control how AI Gateway responds to a blocked request:

Use none or stealth for clients outside your organization, so responses don’t reveal how your guardrails are configured. Use verbose in development or for trusted internal clients, to see why a request was blocked.

Verbose response

In verbose mode, the response always has the status code 403 and the error code GUARDRAIL_BLOCKED. Clients can check the error.code field to tell a guardrail rejection apart from other errors:

• type: Always guardrail_rejected.
• plugin: The type of the Policy that blocked the request.
• reason: Why the request was blocked.
• code: Always GUARDRAIL_BLOCKED.
• detail: Additional context about the match. This field is only present when the Policy has details to report.

The AI Semantic Prompt Guard Policy evaluates the request before it reaches the LLM, so a blocked request returns the same error response whether or not the request sets stream to true.

For example, a request that matches a deny prompt returns the following response in verbose mode. The detail field contains the prompt that matched and its score, which is the distance between the request and that prompt (a lower score means a closer match):

{
  "error": {
    "type": "guardrail_rejected",
    "plugin": "ai-semantic-prompt-guard",
    "reason": "prompt pattern is blocked",
    "code": "GUARDRAIL_BLOCKED",
    "detail": {
      "matched_prompt": "How do I pick a lock?",
      "score": 0.12
    }
  }
}

To return this response, set config.rejection_mode to verbose:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "AI Semantic Prompt Guard - Verbose Rejection",
      "name": "ai-semantic-prompt-guard",
      "type": "ai-semantic-prompt-guard",
      "config": {
        "embeddings": {
          "auth": {
            "header_name": "Authorization",
            "header_value": "Bearer '$OPENAI_API_KEY'"
          },
          "model": {
            "provider": "openai",
            "name": "text-embedding-3-small"
          }
        },
        "vectordb": {
          "strategy": "redis",
          "distance_metric": "cosine",
          "threshold": 0.5,
          "dimensions": 1024,
          "redis": {
            "host": "'$REDIS_HOST'",
            "port": 6379
          }
        },
        "rules": {
          "deny_prompts": [
            "How do I pick a lock?"
          ]
        },
        "rejection_mode": "verbose"
      }
    }
    '

ai_gateway_policies:
  - ref: ai-semantic-prompt-guard
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: AI Semantic Prompt Guard - Verbose Rejection
    name: ai-semantic-prompt-guard
    type: ai-semantic-prompt-guard
    config:
      embeddings:
        auth:
          header_name: Authorization
          header_value: Bearer $OPENAI_API_KEY
        model:
          provider: openai
          name: text-embedding-3-small
      vectordb:
        strategy: redis
        distance_metric: cosine
        threshold: 0.5
        dimensions: 1024
        redis:
          host: !env REDIS_HOST
          port: 6379
      rules:
        deny_prompts:
        - How do I pick a lock?
      rejection_mode: verbose

To run the AI Semantic Prompt Guard Policy without blocking, set config.continue_on_detection to true. When a request matches, the Policy lets the request continue to the LLM. The caller receives the LLM’s normal response, so config.rejection_mode has no effect while this setting is enabled.

Unlike other AI guardrail Policies, the AI Semantic Prompt Guard Policy doesn’t populate the ai.proxy.guardrail_triggered object when this setting is enabled. For the fields that other guardrail Policies record, see the AI Gateway audit log reference.

Set continue_on_detection back to false to enforce the rules.

A vector database stores and compares vector embeddings—numerical representations of text, prompts, documents, or other content. When you configure semantic features in AI Models or AI Policies, embeddings are generated and stored in the vector database so that incoming requests can be compared against the stored vectors to find semantically similar matches. For example, an incoming prompt is embedded and compared against cached prompt keys, model descriptions, document chunks, or allow/deny lists to determine semantic similarity.

AI Gateway semantic features support the following vector databases:

• Using vectordb.strategy: redis and parameters in vectordb.redis:
  • Redis with Redis Vector Search
  • Redis Cloud
  • Valkey: When you configure vectordb.strategy: redis, Kong Gateway queries the server and checks the server name field. If it detects Valkey request, it automatically uses the Valkey-specific driver.
  • Managed Redis with cloud authentication:
    • AWS ElastiCache (auth_provider: aws)
    • Azure Managed Redis (auth_provider: azure)
    • Google Cloud Memorystore (auth_provider: gcp)

    For configuration details, see Using cloud authentication with Redis.

• Using vectordb.strategy: pgvector and parameters in vectordb.pgvector:
  • PostgreSQL with pgvector

Configure vector database settings in AI Models and AI Policies to enable semantic similarity features.

Redis version and module requirements

Kong’s semantic features use the Redis Query Engine (RediSearch) to create and search vector indexes, and the Redis JSON data type to store indexed documents. Your Redis or Valkey deployment must support both, or vector index creation fails.

• Self-hosted Redis: Use Redis Open Source 8.0 or later, which bundles the Redis Query Engine and JSON data type by default, or a distribution that adds them as modules, such as Redis Stack, Redis Enterprise, or Redis Cloud.
• Self-hosted Valkey: Use Valkey 8.1 or later with the valkey-search and valkey-json modules enabled. A vanilla Valkey build without these modules can’t be used for semantic features.
• AWS ElastiCache: ElastiCache doesn’t support custom Redis modules, so ElastiCache for Redis OSS can’t be used for semantic features at any version. Use ElastiCache for Valkey 8.2 or later, which has vector search built in natively.
• Azure Managed Redis: Enable the RediSearch and RedisJSON modules when you create the instance. Modules can only be selected at creation time and can’t be added afterward.
• Google Cloud Memorystore: Use Memorystore for Redis Cluster 8.0 or later or Memorystore for Valkey 8.0 or later, both of which include the required modules automatically. Standard (non-cluster) Memorystore for Redis instances, including the Basic tier, are frozen at Redis 7.2 and don’t support RediSearch or the JSON data type, so they can’t be used for semantic features.

Using cloud authentication with Redis

If your AI Policy uses a Redis datastore, you can authenticate to it with a cloud Redis provider. This allows you to rotate credentials without relying on static passwords.

The following providers are supported:

• AWS ElastiCache
• Azure Managed Redis
• Google Cloud Memorystore (with or without Valkey)

Each provider also supports an instance and cluster configuration.

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 10000
      username: $CLUSTER_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

Set config.proxy on this entity to route its outbound requests through an HTTP forward proxy. Use this in network-isolated deployments where AI Gateway cannot open direct connections to LLM providers or auxiliary services.

The proxy record is identical for AI Model, AI MCP Server, and supported AI Policy entities. Existing capabilities such as load balancing, health checking, streaming, WebSocket, and HTTP/2 continue to work when the proxy is active.

For the full field reference, traffic flow, and limitations, see Forward proxy support.