Configure dynamic Policy config with CEL in AI Gateway

Uses: AI Gateway Kong Identity Konnect API

Incompatible with

on-prem

Prerequisites

AI Gateway running

This is a Konnect tutorial and requires a Konnect personal access token.

Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
```
export KONNECT_TOKEN='YOUR_KONNECT_PAT'
```
Copied!
Run the AI Gateway quickstart script to automatically provision a control plane and data plane in Kong Konnect, and configure your environment:
curl -Ls https://get.konghq.com/ai | bash -s -- -k $KONNECT_TOKEN
Copied!

This sets up a AI Gateway control plane named ai-quickstart, provisions a local data plane, and prints out the following environment variables export:

export AI_GATEWAY_ID=your-gateway-id
export KONNECT_TOKEN=$KONNECT_TOKEN
export KONNECT_CONTROL_PLANE_NAME=ai-quickstart
export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
export KONNECT_PROXY_URL='http://localhost:8000'

Copied!

Copy and paste these into your terminal to configure your session.

Want a fully-hosted AI Gateway? See AI Gateway 2.0 on Dedicated Cloud Gateways.

OpenAI API key

Create an OpenAI account.
Get an API key.

Export the API key as a variable:

 export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
 export OPENAI_AUTH_HEADER="Bearer $OPENAI_API_KEY"

Copied!

Kong Identity directory

A directory is a regional collection of principals. A Konnect organization supports only one Kong Identity directory. Create a directory for the tutorial, or look up your existing one.

Create a directory for this tutorial:

DIRECTORY_ID=$(curl -X POST "https://us.api.konghq.com/v2/directories" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Content-Type: application/json" \
     --json '{
       "name": "kong-identity-directory",
       "description": "Directory for this tutorial",
       "allow_all_control_planes": true
     }' | jq -r ".id"
)

Copied!

If you run into 403 errors when trying to create a directory, the directory likely already exists. Look up your existing directory instead.

Look up an existing directory:

DIRECTORY_ID=$(curl -X GET "https://us.api.konghq.com/v2/directories" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" | jq -r ".data[0].id"
)

Copied!

Create a Principal

An AI Policy’s config can be computed per request from a CEL expression, for example, reading an attribute of the authenticated AI Consumer or Principal, instead of always using a fixed value. In this guide, you’ll create a Principal so that a custom limit can be applied to authenticated Principals. Create the Principal by sending a POST request to the /v2/directories/{directoryId}/principals endpoint:

PRINCIPAL_ID=$(curl -X POST "https://us.api.konghq.com/v2/directories/$DIRECTORY_ID/principals" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" \
     --json '{
       "display_name": "example-principal",
       "description": "Example principal"
     }' | jq -r ".id"
)

Copied!

This script:

Creates a Principal named example-principal
Saves the returned ID as $PRINCIPAL_ID

Set the rate_limit and partner_id metadata fields on the Principal so the Policy expressions can reference them:

curl -X PUT "https://us.api.konghq.com/v2/directories/$DIRECTORY_ID/principals/$PRINCIPAL_ID" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" \
     --json '{
       "display_name": "example-principal",
       "description": "Example principal",
       "metadata": {
         "rate_limit": 20,
         "partner_id": "acme-rockets"
       }
     }'

Copied!

Add key auth

Add an API key credential to the Principal so clients can authenticate with AI Gateway using key authentication.

The following example sets a system-generated key (v1) and stores the key secret as $KEY_SECRET:

KEY_SECRET=$(curl -X POST "https://us.api.konghq.com/v2/directories/$DIRECTORY_ID/principals/$PRINCIPAL_ID/api-keys" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" \
     --json '{
       "type": "v1"
     }' | jq -r ".secret"
)

Copied!

Get the directory name

DECK_DIRECTORY_NAME=$(curl -X GET "https://us.api.konghq.com/v2/directories" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" | jq -r ".data[0].name"
)

Copied!

Export the directory name so decK can read it during sync:

export DECK_DIRECTORY_NAME

Copied!

Create an AI Auth Strategy and AI Model Provider

Create a key-auth AI Auth Strategy with Kong Identity Principal mapping enabled, so clients authenticate with the API key but AI Gateway resolves them to the Principal created earlier. Principal mapping is mutually exclusive with identity_realms, so the strategy sets identity_realms to an empty list. Also create an AI Model Provider for OpenAI.

curl -X POST "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/auth-strategies" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Content-Type: application/json" \
     --json '{
       "name": "identity-principal",
       "display_name": "Identity Principal",
       "type": "key-auth",
       "config": {
         "identity_realms": [],
         "principals": {
           "enabled": true,
           "directory": "'$DECK_DIRECTORY_NAME'"
         }
       }
     }'

Copied!

curl -X POST "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/model-providers" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Content-Type: application/json" \
     --json '{
       "name": "generic-openai",
       "display_name": "generic-openai",
       "type": "openai",
       "config": {
         "auth": {
           "type": "basic",
           "headers": [
             {
               "name": "Authorization",
               "value": "'$OPENAI_AUTH_HEADER'"
             }
           ]
         }
       }
     }'

Copied!

Create a Rate Limiting Advanced Policy with CEL expressions

Create a Rate Limiting Advanced Policy with CEL expressions.

limit and custom_key are expressible config fields on the Rate Limiting Advanced Policy. Their CEL expressions go in the Policy’s top-level expressions object, alongside the static values in config:

identifier: principal, so the Policy keys and limits by the authenticated Principal.
A static config.limit and config.custom_key, used as the fallback whenever a request’s expressions can’t be evaluated.
expressions.limit and expressions.custom_key, reading the authenticated Principal’s rate_limit and partner_id metadata.

curl -X POST "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/policies" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Content-Type: application/json" \
     --json '{
       "name": "rate-limit-by-principal",
       "display_name": "Rate Limiting Advanced - Principal Metadata",
       "type": "rate-limiting-advanced",
       "enabled": true,
       "global": false,
       "config": {
         "identifier": "principal",
         "custom_key": "unknown-partner",
         "limit": [
           10
         ],
         "window_size": [
           60
         ],
         "strategy": "local"
       },
       "expressions": {
         "custom_key": "principal.metadata.partner_id",
         "limit": [
           "principal.metadata.rate_limit"
         ]
       }
     }'

Copied!

Create an AI Model

Create an AI Model that references the AI Auth Strategy, the AI Model Provider, and the Policy:

curl -X POST "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/models" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Content-Type: application/json" \
     --json '{
       "name": "my-gpt-4o-mini",
       "display_name": "my-gpt-4o-mini",
       "type": "model",
       "enabled": true,
       "formats": [
         {
           "type": "openai"
         }
       ],
       "access": {
         "auth_strategies": [
           "identity-principal"
         ]
       },
       "policies": [
         "rate-limit-by-principal"
       ],
       "config": {
         "route": {
           "paths": [
             "/"
           ],
           "model": {
             "body_param": "model",
             "values": [
               "my-gpt-4o-mini"
             ]
           }
         }
       },
       "capabilities": [
         "generate"
       ],
       "targets": [
         {
           "name": "gpt-4o-mini",
           "provider": "generic-openai",
           "config": {
             "type": "openai"
           }
         }
       ]
     }'

Copied!

Validate

Send a request with the API key stored in $KEY_SECRET:

curl -i -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "apikey: $KEY_SECRET"\
     -H "Content-Type: application/json" \
     --json '{
       "model": "my-gpt-4o-mini",
       "messages": [
         {
           "role": "user",
           "content": "What is the capital of France?"
         }
       ]
     }'

Copied!

Check the RateLimit-Limit response header:

RateLimit-Limit: 20

The limit is now 20, which expressions.limit reads from the Principal’s rate_limit metadata, instead of the static fallback value of 10 from config.limit.

To see the static fallback take over, remove the Principal’s rate_limit metadata:

curl -X PUT "https://us.api.konghq.com/v2/directories/$DIRECTORY_ID/principals/$PRINCIPAL_ID" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" \
     --json '{
       "display_name": "example-principal",
       "description": "Example principal",
       "metadata": {
         "partner_id": "acme-rockets"
       }
     }'

Copied!

AI Gateway caches a successfully authenticated Principal for the directory’s configured ttl_secs (5 minutes at minimum, 10 minutes by default). Wait at least 5 minutes after the previous request before continuing, or the next request will still show the cached Principal’s old metadata.

After the ttl has elapsed (5-10 minutes), send another request:

curl -i -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "apikey: $KEY_SECRET"\
     -H "Content-Type: application/json" \
     --json '{
       "model": "my-gpt-4o-mini",
       "messages": [
         {
           "role": "user",
           "content": "What is the capital of France?"
         }
       ]
     }'

Copied!

With rate_limit no longer present on the Principal, expressions.limit fails to evaluate, and AI Gateway falls back to the static config.limit value of 10:

RateLimit-Limit: 10

Cleanup

Clean up AI Gateway resources

To clean up all AI Gateway resources created in this guide, run:

curl -Ls https://get.konghq.com/ai | bash -s -- -d

Copied!

FAQs

What happens if a Principal is missing the metadata that an expression expects to find?

If the Principal is missing expected metadata, the expression fails to evaluate, and AI Gateway falls back to the field’s static config value for that request. AI Gateway doesn’t raise an error, and the Policy still runs normally.

Configure dynamic Policy config with CEL in AI Gateway

Prerequisites

AI Gateway running

OpenAI API key

Kong Identity directory

Create a Principal

Add key auth

Get the directory name

Create an AI Auth Strategy and AI Model Provider

Create a Rate Limiting Advanced Policy with CEL expressions

Create an AI Model

Validate

Cleanup

Clean up AI Gateway resources

FAQs

Help us make these docs great!

Still need help?