curl -X POST "$KONNECT_PROXY_URL/chat/completions" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a helpful assistant. Please repeat the following information back to me."
},
{
"role": "user",
"content": "My name is John Doe, my phone number is 123-456-7890."
}
],
"model": "my-gpt-4o"
}'Use AI PII Sanitizer plugin to protect sensitive data in responses
Incompatible with
on-prem
Related Documentation
Related Resources
Previous Versions of this page
TL;DR
Create an AI Model Provider and AI Model and add an AI PII Sanitizer policy in OUTPUT mode to automatically redact or replace sensitive data in the responses from the upstream LLM provider.
Prerequisites
AI Gateway running
This is a Konnect tutorial and requires a Konnect personal access token.
-
Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
-
Export your token to an environment variable:
export KONNECT_TOKEN='YOUR_KONNECT_PAT' -
Run the AI Gateway quickstart script to automatically provision a control plane and data plane in Kong Konnect, and configure your environment:
curl -Ls https://get.konghq.com/ai | bash -s -- -k $KONNECT_TOKEN
This sets up a AI Gateway control plane named ai-quickstart, provisions a local data plane, and prints out the following environment variables export:
export AI_GATEWAY_ID=your-gateway-id
export KONNECT_TOKEN=$KONNECT_TOKEN
export KONNECT_CONTROL_PLANE_NAME=ai-quickstart
export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
export KONNECT_PROXY_URL='http://localhost:8000'Copy and paste these into your terminal to configure your session.
kongctl v1.15.1+
This tutorial uses kongctl to manage Konnect resources programmatically. We recommend keeping kongctl up to date with the latest version (1.15.1).
- Install kongctl from developer.konghq.com/kongctl.
-
Verify the installation:
kongctl version
OpenAI API key
- Create an OpenAI account.
- Get an API key.
- Export the API key as a variable:
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>" export OPENAI_AUTH_HEADER="Bearer $OPENAI_API_KEY"
AI PII Anonymizer service access
Kong distributes these images publicly on Docker Hub, under the kong/ai-pii-service repository. No authentication is required to pull them.
To pull an image:
docker pull kong/ai-pii-service:TAGReplace TAG with the appropriate version and language code, such as:
docker pull kong/ai-pii-service:v0.2.2-enEach image includes a built-in NLP model. Check the AI Sanitizer documentation for more detail.
Start the Kong AI PII Sanitizer service
Make sure you have access to the AI PII service, then run the following command to start it locally with Docker:
docker run --rm -d -p 8080:8080 kong/ai-pii-service:v0.2.2-enCreate the AI Model Provider, AI Model, and AI PII Sanitizer Policy
Create both an AI Model Provider and an AI Model with a single kongctl apply command.
You’ll also configure the AI PII Sanitizer Policy to anonymize personal information.
kongctl apply -f - --auto-approve --pat "$KONNECT_TOKEN" << 'EOF'
ai_gateway_model_providers:
- ref: generic-openai
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: generic-openai
display_name: "generic-openai"
type: openai
config:
auth:
type: basic
headers:
- name: Authorization
value: !secret {source: !env OPENAI_AUTH_HEADER}
ai_gateway_policies:
- ref: my-ai-sanitizer-policy
name: my-ai-sanitizer-policy
display_name: "My AI sanitizer policy"
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
type: ai-sanitizer
enabled: true
global: false
config:
anonymize:
- all_and_credentials
sanitization_mode: OUTPUT
host: host.docker.internal
port: 8080
redact_type: placeholder
recover_redacted: false
stop_on_error: true
ai_gateway_models:
- ref: my-gpt-4o
display_name: my-gpt-4o
name: my-gpt-4o
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
type: model
enabled: true
formats: [{ type: openai }]
config:
route:
paths:
- /
model:
body_param: model
values:
- my-gpt-4o
capabilities: [generate]
policies: [ !ref my-ai-sanitizer-policy#name ]
targets:
- name: gpt-4o
provider: generic-openai
config:
type: openai
EOFValidate
Send a request that would normally include sensitive information in the response:
If configured correctly, the response should have sensitive output data replaced with placeholders:
Your name is PLACEHOLDER1, and your phone number is PLACEHOLDER2.