curl -X POST "$KONNECT_PROXY_URL/v1/chat/completions" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/json" \
--json '{
"model": "budget-chat",
"messages": [
{
"role": "user",
"content": "What is DNS?"
}
]
}'Enforce tiered AI budgets on AI Models with Kong Identity
Incompatible with
on-prem
Related Documentation
In This Series
Tags
Related Resources
Minimum Version
AI Gateway - 2.0
TL;DR
Create an AI Auth Strategy backed by Kong Identity, then a standard and a premium AI Model that both authenticate through it but deny different AI Consumer Groups. Attach an AI Rate Limiting Advanced Policy with a ceiling per tier plus a shared org pool and an individual cap: every matching ceiling is charged, and the lowest one binds no matter which model handles the request.
Prerequisites
Series Prerequisites
This page is part of the Enforce tiered AI budgets with Kong Identity series.
Complete the previous page, Set up a Kong Identity auth server for tiered AI budgets before completing this page.
kongctl v1.13.0+
This tutorial uses kongctl to manage Konnect resources programmatically. We recommend keeping kongctl up to date with the latest version (1.13.0).
- Install kongctl from developer.konghq.com/kongctl.
-
Verify the installation:
kongctl version
OpenAI API key
- Create an OpenAI account.
- Get an API key.
- Export the API key as a variable:
export OPENAI_API_KEY='<YOUR_OPENAI_API_KEY>' export OPENAI_AUTH_HEADER='Bearer $OPENAI_API_KEY'
A Kong Identity issuer and per-persona credentials
Complete Set up a Kong Identity auth server for tiered AI budgets first. You need its $ISSUER_URL and each persona’s client ID and secret.
Overview
Set up a Kong Identity auth server for tiered AI budgets issued each of five example callers a token carrying its tier, individual spend cap, shared org pool, and group membership as claims. This guide connects those tokens to AI Gateway:
- An AI Auth Strategy verifies each token and projects its claims onto request headers.
- A standard and a premium AI Model both check those headers but deny different callers.
- An AI Rate Limiting Advanced Policy enforces a spend ceiling per tier, plus the shared org pool and individual cap from the previous guide.
Create the AI Consumer Groups, AI Auth Strategy, AI Model Provider, and AI Models
Create the following:
- Two AI Consumer Groups,
contractorsandsuspended, thataccess.aclsdenies. - An AI Auth Strategy that verifies bearer tokens against your Kong Identity issuer.
- An AI Model Provider for the upstream OpenAI credentials.
- A standard and a premium AI Model that share a route and an AI Rate Limiting Advanced Policy, but differ in which groups
access.aclsdenies.
tokens_count_strategy: cost charges each request in dollars, so the same ceiling applies whether a request lands on the cheaper gpt-4o-mini or the pricier gpt-4o:
kongctl apply -f - --auto-approve --pat "$KONNECT_TOKEN" << 'EOF'
ai_gateway_consumer_groups:
- ref: contractors
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: contractors
name: contractors
- ref: suspended
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: suspended
name: suspended
ai_gateway_auth_strategies:
- ref: budget-identity
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: "Budget identity"
name: budget-identity
type: openid-connect
config:
issuer: !env ISSUER_URL
auth_methods: [bearer]
consumer_optional: true
consumer_groups_claim: [kong_groups]
consumer_groups_optional: true
upstream_headers_claims: [sub, budget_tier, budget_org, budget_cap]
upstream_headers_names: [x-budget-sub, x-budget-tier, x-budget-org, x-budget-cap]
cache_tokens_salt: budgets-cache-salt
ai_gateway_model_providers:
- ref: generic-openai
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: generic-openai
type: openai
config:
auth:
type: basic
headers:
- name: Authorization
value: !secret {source: !env OPENAI_AUTH_HEADER}
ai_gateway_policies:
- ref: budget-limits
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: budget-limits
display_name: "Budget limits"
type: ai-rate-limiting-advanced
enabled: true
global: false
config:
strategy: local
policies:
- match:
- { type: header, key: x-budget-tier, values: [baseline] }
- { type: header, key: x-budget-sub, partition_by: true }
window_type: sliding
limits: [{ limit: 0.05, window_size: 60, tokens_count_strategy: cost }]
- match:
- { type: header, key: x-budget-tier, values: ["2x"] }
- { type: header, key: x-budget-sub, partition_by: true }
window_type: sliding
limits: [{ limit: 0.10, window_size: 60, tokens_count_strategy: cost }]
- match:
- { type: header, key: x-budget-tier, values: ["4x"] }
- { type: header, key: x-budget-sub, partition_by: true }
window_type: sliding
limits: [{ limit: 0.20, window_size: 60, tokens_count_strategy: cost }]
- match:
- { type: header, key: x-budget-sub, partition_by: true }
window_type: sliding
limits: [{ limit: 0.20, window_size: 60, tokens_count_strategy: cost }]
- match:
- { type: header, key: x-budget-org, values: [live-balance], partition_by: false }
window_type: sliding
limits: [{ limit: 0.35, window_size: 60, tokens_count_strategy: cost }]
- match:
- { type: header, key: x-budget-cap, values: [strict] }
- { type: header, key: x-budget-sub, partition_by: true }
window_type: sliding
limits: [{ limit: 0.02, window_size: 60, tokens_count_strategy: cost }]
ai_gateway_models:
- ref: budget-chat
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: "Budget chat"
name: budget-chat
type: model
enabled: true
formats: [{ type: openai }]
capabilities: [generate]
access:
auth_strategies:
- !ref budget-identity#name
acls:
deny:
- suspended
policies:
- !ref budget-limits#name
config:
route:
paths: [/v1]
model:
body_param: model
values: [budget-chat]
targets:
- name: gpt-4o-mini
provider: generic-openai
config:
type: openai
input_cost: 0.15
output_cost: 0.60
- ref: budget-chat-premium
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: "Budget chat premium"
name: budget-chat-premium
type: model
enabled: true
formats: [{ type: openai }]
capabilities: [generate]
access:
auth_strategies:
- !ref budget-identity#name
acls:
deny:
- contractors
- suspended
policies:
- !ref budget-limits#name
config:
route:
paths: [/v1]
model:
body_param: model
values: [budget-chat-premium]
targets:
- name: gpt-4o
provider: generic-openai
config:
type: openai
input_cost: 2.50
output_cost: 10.00
EOFA request selects its model through the request body’s model field. The fourth policy in budget-limits matches on the subject header alone, so it always applies, and the lowest matching ceiling binds.
Validate
Obtain a bearer token for each persona using the client credentials grant shown in Set up a Kong Identity auth server for tiered AI budgets, then send requests through either model. None of the requests below set the x-budget-* headers themselves; budget-identity derives them internally from each token’s claims, before the request reaches budget-limits or the upstream model.
An unauthenticated request is rejected:
The request fails with 401 Unauthorized.
Carol (tier: "4x") authenticates and is charged against the 4x ceiling. Generate a token for her client and export it:
CAROL_ACCESS_TOKEN=$(curl -X POST "$ISSUER_URL/oauth/token" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "grant_type=client_credentials" \
-d "client_id=$CAROL_CLIENT_ID" \
-d "client_secret=$CAROL_CLIENT_SECRET" \
-d "scope=budgets-access" | jq -r ".access_token"
)Send the request with her token:
curl -X POST "$KONNECT_PROXY_URL/v1/chat/completions" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/json"\
-H "Authorization: Bearer $CAROL_ACCESS_TOKEN" \
--json '{
"model": "budget-chat",
"messages": [
{
"role": "user",
"content": "What is DNS?"
}
]
}'Carol isn’t in contractors or suspended, so the same token also reaches the premium model:
curl -X POST "$KONNECT_PROXY_URL/v1/chat/completions" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/json"\
-H "Authorization: Bearer $CAROL_ACCESS_TOKEN" \
--json '{
"model": "budget-chat-premium",
"messages": [
{
"role": "user",
"content": "What is DNS?"
}
]
}'budget-limits charges the same 4x ceiling either way, since the policy reads headers, not which model resolved the request. Only access.acls differs between the two models, and none of the five personas in this guide are in contractors, so this guide can’t demonstrate a request denied on budget-chat-premium but accepted on budget-chat. Create a client with a groups: "contractors" label the same way the prior guide creates the others, and it’s denied on budget-chat-premium while still reaching budget-chat.
Dave (tier: "4x", cap: "strict") authenticates the same way, but his individual cap and his tier ceiling are charged together, and the lower one binds:
DAVE_ACCESS_TOKEN=$(curl -X POST "$ISSUER_URL/oauth/token" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "grant_type=client_credentials" \
-d "client_id=$DAVE_CLIENT_ID" \
-d "client_secret=$DAVE_CLIENT_SECRET" \
-d "scope=budgets-access" | jq -r ".access_token"
)curl -X POST "$KONNECT_PROXY_URL/v1/chat/completions" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/json"\
-H "Authorization: Bearer $DAVE_ACCESS_TOKEN" \
--json '{
"model": "budget-chat",
"messages": [
{
"role": "user",
"content": "What is DNS?"
}
]
}'Dave’s tier: "4x" block alone would allow $0.20 per minute, the same as Carol. His cap: "strict" block is charged in parallel and caps him at $0.02 instead, so moving him to a higher tier later would never raise his ceiling. Only removing the cap label does.
Grace (tier: "4x", groups: "suspended") never reaches a budget:
GRACE_ACCESS_TOKEN=$(curl -X POST "$ISSUER_URL/oauth/token" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "grant_type=client_credentials" \
-d "client_id=$GRACE_CLIENT_ID" \
-d "client_secret=$GRACE_CLIENT_SECRET" \
-d "scope=budgets-access" | jq -r ".access_token"
)curl -X POST "$KONNECT_PROXY_URL/v1/chat/completions" \
--no-progress-meter --fail-with-body \
-H "Content-Type: application/json"\
-H "Authorization: Bearer $GRACE_ACCESS_TOKEN" \
--json '{
"model": "budget-chat",
"messages": [
{
"role": "user",
"content": "What is DNS?"
}
]
}'The request fails with 403 Forbidden. access.acls denies on the suspended group before the request reaches budget-limits, regardless of Grace’s tier.
If this request doesn’t return
403,budget-identitylikely isn’t projectingkong_groupsonto an ACL group Kong recognizes yet. Verifyconsumer_groups_claimandupstream_headers_namesactually took effect (see the FAQ above), and allow a few seconds after anykongctl applyfor the change to reach the data plane before retrying.
View usage in Observability
AI Gateway exports Gen AI metrics to Observability. View them in a dashboard built from a template:
- In the Konnect sidebar, click Observability.
- In the Observability sidebar, click Dashboards.
- From the Create dashboard dropdown menu, select “Create from template”.
- Click AI Gateway Template.
- Click Use template.
In the Gen AI model usage count tile, you’ll see 2 uses of gpt-4o-mini and 1 use of gpt-4o, reflecting Carol’s and Dave’s requests to budget-chat and Carol’s request to budget-chat-premium. Grace’s request never reaches a model, so it doesn’t appear here.
Cleanup
Clean up AI Gateway resources
To clean up all AI Gateway resources created in this guide, run:
curl -Ls https://get.konghq.com/ai | bash -s -- -dClean up Kong Identity resources
curl -X DELETE "https://us.api.konghq.com/v1/auth-servers/$AUTH_SERVER_ID?force=true" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN"\
-H "Content-Type: application/json"FAQs
Why do only two AI Consumer Groups exist when there are three tiers and a cap?
Tiers and the cap are matched from headers the AI Auth Strategy projects out of the token, not from AI Consumer Group entities, so a tier change never writes anything to AI Gateway. Only contractors and suspended need a matching AI Consumer Group, because access.acls compares against real group names.
Why do budget-identity’s config fields include consumer_groups_claim and upstream_headers_claims?
These project the kong_groups, budget_tier, budget_org, and budget_cap claims from the token onto x-budget-* request headers and the ACL groups AI Gateway checks. kongctl (1.12.0 and later) applies them directly as part of the AI Auth Strategy’s config, no separate step required.
What happens to a caller with no resolvable claim at all?
consumer_groups_optional: true means a missing or unrecognized group claim doesn’t fail the request. The fail-safe policy, matched on nothing but the subject header, then bounds that caller at the same ceiling as the top tier instead of leaving them unlimited.
Why does the fail-safe policy matter?
The AI Rate Limiting Advanced Policy has no enforcement to fall back on when nothing matches. If an expression upstream ever returns null instead of a literal default, the tier header goes missing and a caller with no matching policy is not rate limited at all. The fail-safe policy matches on the subject header alone, so it always applies, and caps that caller at the top tier’s ceiling instead of leaving them unbounded.