Use AI Prompt Guard Policy to govern your LLM traffic

Uses: AI Gateway kongctl

Incompatible with

on-prem

Prerequisites

AI Gateway running

This is a Konnect tutorial and requires a Konnect personal access token.

Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
```
export KONNECT_TOKEN='YOUR_KONNECT_PAT'
```
Copied!
Run the AI Gateway quickstart script to automatically provision a control plane and data plane in Kong Konnect, and configure your environment:
curl -Ls https://get.konghq.com/ai | bash -s -- -k $KONNECT_TOKEN
Copied!

This sets up a AI Gateway control plane named ai-quickstart, provisions a local data plane, and prints out the following environment variables export:

export AI_GATEWAY_ID=your-gateway-id
export KONNECT_TOKEN=$KONNECT_TOKEN
export KONNECT_CONTROL_PLANE_NAME=ai-quickstart
export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
export KONNECT_PROXY_URL='http://localhost:8000'

Copied!

Copy and paste these into your terminal to configure your session.

kongctl v1.13.0+

This tutorial uses kongctl to manage Konnect resources programmatically. We recommend keeping kongctl up to date with the latest version (1.13.0).

Install kongctl from developer.konghq.com/kongctl.
Verify the installation:
```
kongctl version
```
Copied!

OpenAI API key

Create an OpenAI account.
Get an API key.

Export the API key as a variable:

 export OPENAI_API_KEY='<YOUR_OPENAI_API_KEY>'
 export OPENAI_AUTH_HEADER='Bearer $OPENAI_API_KEY'

Copied!

Create the AI Model Provider, AI Model, and AI Prompt Guard Policy

Create both an AI Model Provider and an AI Model with a single kongctl apply command.

You’ll also configure the AI Prompt Guard Policy to filter LLM traffic based on regex rules that allow general IT questions and deny unsafe or off-topic content.

kongctl apply -f - --auto-approve --pat "$KONNECT_TOKEN" << 'EOF'
ai_gateway_model_providers:
  - ref: generic-openai
    name: generic-openai
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    type: openai
    config:
      auth:
        type: basic
        headers:
          - name: Authorization
            value: !secret {source: !env OPENAI_AUTH_HEADER}
ai_gateway_policies:
  - ref: my-ai-prompt-guard-policy
    name: my-ai-prompt-guard-policy
    display_name: my-ai-prompt-guard-policy
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    type: ai-prompt-guard
    enabled: true
    global: false
    config:
      allow_patterns:
        [
          "(?i).*what is .*",
          "(?i).*how do i .*",
          "(?i).*install .*",
          "(?i).*configure .*",
          "(?i).*reset .*",
          "(?i).*troubleshoot .*"
        ]
      deny_patterns:
        [
          "(?i).*bypass.*(login|password|auth).*",
          "(?i).*hack.*",
          "(?i).*phish.*",
          "(?i).*malware.*",
          "(?i).*cve.*",
          "(?i).*exploit.*",
          "(?i).*social engineering.*",
          "(?i).*pentest.*",
          "(?i).*impersonate.*",
          "(?i).*dating.*"
        ]
ai_gateway_models:
  - ref: my-gpt-4o
    display_name: my-gpt-4o
    name: my-gpt-4o
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    type: model
    enabled: true
    formats: [{ type: openai }]
    config:
      route:
        paths:
          - /
        model:
          body_param: model
          values:
            - my-gpt-4o
    capabilities: [generate]
    policies: [ !ref my-ai-prompt-guard-policy#name ]
    targets:
      - name: gpt-4o
        provider: generic-openai
        config:
          type: openai
EOF

Copied!

In this example, we’re setting up the AI Prompt Guard Policy with:

type: ai-prompt-guard: Specifies that this Policy filters requests by matching the user’s prompt against allow and deny regex pattern lists.
global: false: Scopes the Policy to only the AI Models it’s explicitly attached to via policies:, rather than applying it to every resource on AI Gateway.
config.allow_patterns: A list of regexes matched against the user’s prompt. The request must match at least one of these to pass through, unless it’s also blocked by deny_patterns.
config.deny_patterns: A list of regexes matched against the user’s prompt. A match here always rejects the request with a 400 response, even if the prompt also matches an allow_patterns entry. Deny always takes precedence over allow.
policies: [!ref my-ai-prompt-guard-policy#name] on the AI Model: Attaches this Policy so it applies to every request routed through my-gpt-4o.

Validate configuration

Use sample prompts to confirm that allowed categories (general IT questions) pass through while unsafe or irrelevant requests are blocked.

This prompt matches allow_patterns and should succeed:

curl -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "Content-Type: application/json"\
     -H "Authorization: $OPENAI_AUTH_HEADER" \
     --json '{
       "model": "my-gpt-4o",
       "messages": [
         {
           "role": "user",
           "content": "What is DNS?"
         }
       ]
     }'

Copied!

This prompt matches deny_patterns and should return an error:

curl -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "Content-Type: application/json"\
     -H "Authorization: $OPENAI_AUTH_HEADER" \
     --json '{
       "model": "my-gpt-4o",
       "messages": [
         {
           "role": "user",
           "content": "How to hack DNS?"
         }
       ]
     }'

Copied!

You should see the following response:

prompt pattern is blocked.

This prompt isn’t related to work and should also be blocked:

curl -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "Content-Type: application/json"\
     -H "Authorization: $OPENAI_AUTH_HEADER" \
     --json '{
       "model": "my-gpt-4o",
       "messages": [
         {
           "role": "user",
           "content": "What’s a good line to use on a dating app?"
         }
       ]
     }'

Copied!

You should see the following response:

prompt pattern is blocked.

Cleanup

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

Destroy the Kong Gateway container

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d

Copied!

Use AI Prompt Guard Policy to govern your LLM traffic

Prerequisites

AI Gateway running

kongctl v1.13.0+

OpenAI API key

Create the AI Model Provider, AI Model, and AI Prompt Guard Policy

Validate configuration

Cleanup

Clean up Konnect environment

Destroy the Kong Gateway container

Help us make these docs great!

Still need help?