Model cost management in Kong AI Gateway

Cost is one of the primary signals AI governance acts on: budget enforcement, per-consumer spend limits, and chargeback all depend on an accurate cost figure for each request. AI Gateway calculates that figure for every request to a large language model (LLM) provider, matching what the provider actually bills. It feeds cost reporting and any policy that enforces spend against it, such as AI Rate Limiting Advanced.

This page explains how that cost figure feeds governance and reporting, and the pricing configuration behind it, so you can confirm your governance policies are enforcing against a number that reflects real provider spend.

AI Gateway prices a request starting from a base two-rate calculation (an input rate times input tokens, plus an output rate times output tokens), then layers in the following dimensions to match how LLM providers actually bill:

Pricing is configured per target on the AI Model entity’s targets, using the input_cost and output_cost fields. Three optional fields extend those scalars to cover the dimensions in Model pricing dimensions: cache_write_cost_list, context_window_factor, and service_tier_factor. If one of these fields is absent, the calculation falls back to the corresponding scalar. If the field is present but no entry matches a request, the calculation also falls back to the scalar.

cache_write_cost_list, context_window_factor, and service_tier_factor extend the AI Model target schema beyond input_cost and output_cost. Check the AI Model entity reference for more details.

Scalar fields

input_cost and output_cost price every target. Add cache_read_cost and cache_write_cost when the provider discounts cache reads or charges more for cache writes:

Cache-write pricing by TTL

Use cache_write_cost_list when a provider prices cache writes differently depending on how long the entry is kept, for example Anthropic’s 5-minute and 1-hour TTL tiers. Set a ttl and its cost per entry:

If a request’s TTL doesn’t match any entry, the model falls back to cache_write_cost.

Context-window pricing

Use context_window_factor when a provider re-prices an entire request once it crosses a context-size threshold, for example GPT-5.6’s rate change above 272K input tokens. Each entry pairs a token threshold with its input and output multipliers:

If multiple entries exist, the model applies the highest threshold that the request’s input tokens cross.

With a 2M-token request and tiers at above: 200k and above: 1m, the above: 1m tier applies.

Service-tier pricing

Use service_tier_factor when a provider offers a priority tier for lower latency or a flex tier for lower cost, and you want that price difference reflected in the calculated cost. The standard (default) tier is 1 and needs no configuration. Pair each tier name with its multiplier:

Example configuration

This example configures an AI Model target that bills:

• $4 per million input tokens and $24 per million output tokens, on the standard service tier.
• $0.4 per million cache-read tokens, a tenth of the input rate.
• $5 per million cache-write tokens by default, or $8 per million tokens for entries with a 1-hour TTL.
• Double the input rate and 1.5x the output rate, once a request’s input crosses 200K tokens.
• Double the resulting rate on the priority tier, or half on the flex tier.

# Scalar fields (always present)
input_cost: 4          # $ per 1M input tokens
output_cost: 24         # $ per 1M output tokens
cache_read_cost: 0.4    # $ per 1M cache-read tokens
cache_write_cost: 5     # $ per 1M cache-write tokens (fallback)

# Cache-write pricing by TTL
cache_write_cost_list:
  - ttl: 5m
    cost: 5
  - ttl: 1h
    cost: 8

# Context-window pricing
context_window_factor:
  - above: 200k         # threshold measured on input tokens
    input_factor: 2
    output_factor: 1.5

# Service-tier pricing (standard = 1, omitted)
service_tier_factor:
  - tier: priority
    factor: 2
  - tier: flex
    factor: 0.5

For a single request, AI Gateway calculates cost from the configured fields as follows:

cost = service_tier_factor * (
    context_window_factor_for_input * (
        input
        + cache_read
        + cache_write priced per its TTL tier
    )
    + context_window_factor_for_output * output
)

The calculation proceeds in four steps:

1. Price the input side:
   Sum normal input, cache reads, and cache writes. Normal input tokens are priced at input_cost and cache reads at cache_read_cost. Cache writes are priced by matching the request’s TTL against cache_write_cost_list. If no entry matches, cache_write_cost applies. Because cache pricing derives from the base input side, any discount or premium on input flows through to cache automatically.
2. Apply the context-window factor:
   If the request’s input-token count crosses a threshold in context_window_factor, the input side is multiplied by that tier’s input_factor and the output side by its output_factor. If no threshold is crossed, both factors are effectively 1. The threshold is always measured on input tokens and gates both factors together.
3. Price the output side:
   Output tokens are priced at output_cost and multiplied by the applicable output_factor. Reasoning or thinking tokens, where a provider produces them, are billed at the output rate.
4. Apply the service-tier multiplier:
   The factor for the request’s service tier scales the entire composed cost, input side and output side together. Because the input side already includes cache, the service-tier factor flows through to cache pricing as well. The standard tier uses a factor of 1.

Per-token rates are expressed per one million tokens, so the final figure divides accordingly. The result is the cost of the request.

AI Gateway prices a request entirely within the AI Model entity: every field in Model cost configuration lives on a target, not on any policy. AI Rate Limiting Advanced doesn’t define any pricing fields of its own. It consumes the cost figure the AI Model already calculated for the request, and enforces a limit against it when a limit’s strategy is set to cost.

This split means changing a model’s pricing (for example, updating input_cost after a provider rate change) never requires touching the AI Rate Limiting Advanced configuration, and changing a rate limit never requires touching the AI Model. The cost figure produced by the calculation in this page is reported by AI Gateway regardless of whether any rate-limiting policy is attached.