Metering & Billing provides a real-time event based usage metering to aggregate consumption over time precisely. It also provides deduplication and flexible usage attribution of events and consumers to billable customers.
Metering & Billing metering can help to track various usage:
Use Case
Description
AI Tokens
Metering AI Tokens consumed by LLMs and AI agents.
API Requests
Metering API Requests count and duration.
Compute
Metering runtime of VMs, CPUs, GPUs, etc.
Seats
Metering number of unique users over sessions.
You can meter Kong Gateway events, like API requests and LLM token usage, as well as generic events.
Each generic meter is comprised from the following attributes:
Event type: The event type that the meter is tracking. This is used to filter the events that are used to calculate the meter.
Value type: The JSON path to the property that contains the value to be metered. This is optional when the aggregation is COUNT.
Aggregation types determine how usage data is aggregated for generic meters.
You can configure the following aggregation types for generic meters:
Aggregation Type
Description
COUNT
The COUNT aggregation type counts the number of events that occur within a specific time window. This is often used for metrics that are inherently countable, such as the number of transactions processed or API calls made. The COUNT aggregation type doesn’t have the valueProperty.
SUM
The SUM aggregation type calculates the total sum of the metered values for a specific time window. SUM aggregates over the events valueProperty. This is useful for accumulating metrics like total LLM tokens used, total data transferred, or total time spent on a service.
UNIQUE COUNT
The UNIQUE_COUNT aggregation type counts the number of unique events. This is useful when events are unique by a specific field. The valueProperty defines the field that makes the ingested event unique. The property’s value in the ingested event must be a string or number.
LATEST
The LATEST aggregation type returns the latest value for a specific time window. This is useful for when you track the size of a resource on your own and report periodically the value of it to Metering & Billing. For example disk size, number of resources or seats. The latest aggregation takes the last value reported for the period.
MIN
The MIN aggregation type identifies the minimum value among the metered data points within a specific time window. This is useful for metrics where the lowest value is of interest, such as minimum available storage or minimum response time.
MAX
The MAX aggregation type identifies the maximum value among the metered data points within a specific time window. This is useful for metrics where the highest value is of interest, such as maximum load on a server or maximum transaction value.
Metering & Billing ingests Konnect API Gateway and LLM events automatically when they’re enabled. If you want to configure generic meters, you must use the CloudEvents format for event ingestion.
As CloudEvents is generic, here are some best practices for defining events in OpenMeter:
Name
Description
Examples
Subject (API Property: event.subject)
Subjects in OpenMeter are entities that consume resources you wish to meter. These can range from users, servers, and services to devices. The design of subjects is intentionally generic, enabling flexible application across various metering scenarios. Typically, a subject acts as a unique identifier within your system for a user or customer.
Customer ID or User ID
Hostname or IP address
Service or Application name
Device ID
Source Property (API Property: event.source)
The event’s source (e.g. the service name). As events are unique by id and source, set different sources if you report the same transaction in multiple applications.
my-service-name
my-application-name
Choosing Event ID
Events are unique by id and source properties for idempotency. OpenMeter deduplicates events by uniqueness. Therefore, picking an ID that makes the event unique and resilient to retries is important. For example, in the case of a metering API call, this can be the request ID. You can generate a new UUID if your application doesn’t have a unique identifier.
HTTP Request ID, typically in headers: Request-ID, X-Request-ID
LLM Chat Completion ID: id field in ChatGPT response
Workflow ID: like activity ID in Temporal
Generate UUID: Node.js, Python, Go
Data Property (API Property: event.data)
OpenMeter uses CloudEvents format’s data property to ingest values and group bys. Be sure to always include in this data property what the meter requires, like value property and group bys.
Always include value property for non-COUNT aggregations.
Always include group by properties.
Use string quotation for numbers to preserve precision like \"123\".
If you want to meter Kong AI Gateway LLM token usage, you can enable the built-in integration to meter usage in one click.
In most cases, AI applications want to count token usage for billing or cost control purposes. As a single AI interaction involves consuming multiple tokens, we define our generic meter with the SUM aggregation and report token usage in the data’s tokens property. As most LLMs charge differently for input, output and system prompts and different models it makes sense to add model and prompt type to the group by.
meters: - slug: tokens_total description: AI Token Usage # Filter events by type eventType: prompt aggregation: SUM # JSONPath to parse usage value valueProperty: $.tokens groupBy: # Model used: gpt4-turbo, etc. model: $.model # Prompt type: input, output, system type: $.type
If you want to meter Kong Gateway API requests, you can enable the built-in integration to meter usage in one click.
Products monetizing API usage may want to count the number of requests. With choosing the COUNT aggregation each event will increase the meter by one. For grouping we can add method and route. Note how we report the route template not the actual HTTP path to avoid differences around IDs and dynamic routes.
meters: - slug: api_requests_total description: API Requests # Filter events by type eventType: request aggregation: COUNT groupBy: # HTTP Method: GET, POST, etc. method: $.method # Route: /products/:product_id route: $.route
Similar to the API request, you can decide to track the request duration. This is basically how serverless products like AWS Lambda charge their customers. If you want to track both the request count and duration, you can check out our advanced example.
meters: - slug: api_request_duration description: API Request Duration # Filter events by type eventType: request aggregation: SUM # JSONPath to parse duration value valueProperty: $.duration_seconds groupBy: # HTTP Method: GET, POST, etc. method: $.method # Route: /products/:product_id route: $.route
In OpenMeter, a single event can move multiple meters if the event type matches. Let’s see an example of tracking an API request’s occurrence, execution duration, and network usage.
In some cases you want to count how many states a workflow or task went through as it progresses for example from created to in_progress and success. The challenge is that if you report a usage event for every state change and track state as a group by answering simple questions like how many workflows were in total would always require filtering by states like created, which is easy to forget and error-prone.
The recommended way to model states is to create separate meters per state.
Example meters for the imaginary Translate AI product that translates PDF documents between languages. For example, you can use an LLM like GPT-4 to translate a PDF document from German to English. For this use case, you want to track the number of pages, words, and LLM tokens used for each translation.
Meter to count the number of pages translated:
meters: - slug: pages_total description: Number of pages translated eventType: translate aggregation: SUM valueProperty: $.pages
Copied!
Meter to count the number of words translated:
meters: - slug: words_total description: Number of words translated eventType: translate aggregation: SUM valueProperty: $.words
Copied!
Meter to count the number of LLM tokens used:
meters: - slug: tokens_total description: Number of LLM tokens used eventType: translate aggregation: SUM valueProperty: $.tokens groupBy: model: $.model