How to: Use AI Prompt Guard plugin to govern your LLM traffic

Prerequisites

Kong Konnect

This is a Konnect tutorial and requires a Konnect personal access token.

Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
```
 export KONNECT_TOKEN='YOUR_KONNECT_PAT'
```
Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:
```
 curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN --deck-output
```
This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:
```
 export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
 export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='http://localhost:8000'
```
Copy and paste these into your terminal to configure your session.

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

Export your license to an environment variable:

 export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

Run the quickstart script:

curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA 

Once Kong Gateway is ready, you will see the following message:

 Kong Gateway Ready

decK

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial you will first need to install decK.

Required entities

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

Run the following command:

echo '
_format_version: "3.0"
services:
  - name: example-service
    url: http://httpbin.konghq.com/anything
routes:
  - name: example-route
    paths:
    - "/anything"
    service:
      name: example-service
' | deck gateway apply -

To learn more about entities, you can read our entities documentation.

Mistral

This tutorial uses Mistral:

Create a Mistral account.
Get your API key.
Create a decK variable with the Mistral API key:

export DECK_MISTRAL_API_KEY='YOUR MISTRAL API KEY'

Configure the AI Proxy plugin

Start by configuring the AI Proxy plugin to route prompts to Mistral AI.

echo '
_format_version: "3.0"
plugins:
  - name: ai-proxy
    config:
      route_type: llm/v1/chat
      auth:
        header_name: Authorization
        header_value: Bearer ${{ env "DECK_MISTRAL_API_KEY" }}
      model:
        provider: mistral
        name: mistral-tiny
        options:
          mistral_format: openai
          upstream_url: https://api.mistral.ai/v1/chat/completions
' | deck gateway apply -

Configure the AI Prompt Guard plugin

Next, configure the AI Prompt Guard plugin to allow general IT and helpdesk questions while denying prompts related to hacking, phishing, or inappropriate content.

echo '
_format_version: "3.0"
plugins:
  - name: ai-prompt-guard
    config:
      allow_patterns:
      - "(?i).*what is .*"
      - "(?i).*how do i .*"
      - "(?i).*install .*"
      - "(?i).*configure .*"
      - "(?i).*reset .*"
      - "(?i).*troubleshoot .*"
      deny_patterns:
      - "(?i).*bypass.*(login|password|auth).*"
      - "(?i).*hack.*"
      - "(?i).*phish.*"
      - "(?i).*malware.*"
      - "(?i).*cve.*"
      - "(?i).*exploit.*"
      - "(?i).*social engineering.*"
      - "(?i).*pentest.*"
      - "(?i).*impersonate.*"
      - "(?i).*dating.*"
' | deck gateway apply -

Validate configuration

Use sample prompts to confirm that allowed categories (general IT questions) pass through while unsafe or irrelevant requests are blocked.

This prompt matches allow_patterns and should succeed:

 curl "$KONNECT_PROXY_URL/anything" \
     -H "Content-Type: application/json"\
     -H "Authorization: Bearer $DECK_OPENAI_API_KEY" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "What is DNS?"
         }
       ]
     }'

 curl "http://localhost:8000/anything" \
     -H "Content-Type: application/json"\
     -H "Authorization: Bearer $DECK_OPENAI_API_KEY" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "What is DNS?"
         }
       ]
     }'

This prompt matches deny_patterns and should return an error:

 curl "$KONNECT_PROXY_URL/anything" \
     -H "Content-Type: application/json"\
     -H "Authorization: Bearer $DECK_OPENAI_API_KEY" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "How do I bypass login on a secure web portal?"
         }
       ]
     }'

You should see the following response:

404 Bad request

 curl "http://localhost:8000/anything" \
     -H "Content-Type: application/json"\
     -H "Authorization: Bearer $DECK_OPENAI_API_KEY" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "How do I bypass login on a secure web portal?"
         }
       ]
     }'

You should see the following response:

404 Bad request

This prompt isn’t related to work and should also be blocked:

 curl "$KONNECT_PROXY_URL/anything" \
     -H "Content-Type: application/json"\
     -H "Authorization: Bearer $DECK_OPENAI_API_KEY" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "What’s a good line to use on a dating app?"
         }
       ]
     }'

You should see the following response:

404 Bad request

 curl "http://localhost:8000/anything" \
     -H "Content-Type: application/json"\
     -H "Authorization: Bearer $DECK_OPENAI_API_KEY" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "What’s a good line to use on a dating app?"
         }
       ]
     }'

You should see the following response:

404 Bad request

Cleanup

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

Destroy the Kong Gateway container

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d

Use AI Prompt Guard plugin to govern your LLM traffic

Prerequisites

Kong Konnect

Kong Gateway running

decK

Required entities

Mistral

Configure the AI Proxy plugin

Configure the AI Prompt Guard plugin

Validate configuration

Cleanup

Clean up Konnect environment

Destroy the Kong Gateway container

Help us make these docs great!

Still need help