Gen AI OpenTelemetry metrics reference

v3.14+ AI Gateway can export OpenTelemetry (OTLP) metrics for generative AI, MCP, and A2A traffic through the OpenTelemetry plugin. These metrics are aggregated time-series data points (counters, histograms) pushed to a configured OTLP metrics endpoint on a regular interval. They are separate from the per-request Gen AI span attributes emitted on traces.

For a step-by-step setup using an OpenTelemetry Collector, see Collect metrics, logs, and traces with the OpenTelemetry plugin. To visualize Gen AI traces in Jaeger, see Set up Jaeger with Gen AI OpenTelemetry.

Use these metrics to:

• Track LLM request latency and upstream provider processing time
• Monitor token consumption across providers, models, and consumers
• Measure time-to-first-token (TTFT) and inter-token latency (TPOT) for streaming responses
• Calculate AI request costs
• Observe MCP tool-call latency, error rates, and ACL decisions
• Monitor A2A agent request volume, duration, and task state transitions

To collect AI OTel metrics, enable the following settings:

Some metrics have additional requirements:

• gen_ai.server.request.duration and mcp.client.operation.duration require config.metrics.enable_latency_metrics set to true in the OpenTelemetry plugin.
• The error.type attribute on duration metrics requires config.metrics.enable_request_metrics set to true in the OpenTelemetry plugin.

These metrics follow the OpenTelemetry Gen AI semantic conventions. They capture request duration, upstream latency, token usage, and streaming performance.

gen_ai.client.operation.duration

Total time Kong Gateway spends processing a Gen AI operation, such as an LLM request. Kong Gateway acts as the client calling the Gen AI provider.

• Type: Histogram
• Unit: s (seconds)

Attributes:

gen_ai.server.request.duration

Time the LLM provider spends processing the request (upstream latency). Requires enable_latency_metrics set to true.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as gen_ai.client.operation.duration.

gen_ai.client.token.usage

Number of tokens consumed by the Gen AI operation. Each data point is labeled with a gen_ai.token.type attribute that identifies the token category.

• Type: Counter
• Unit: {token}

gen_ai.server.time_to_first_token

Time from when the model server receives the request until the first output token is generated. Relevant for streaming responses.

• Type: Histogram
• Unit: s (seconds)

gen_ai.server.time_per_output_token

Time between successive output tokens generated by the model server after the first token. Measures inter-token latency for streaming responses.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as gen_ai.server.time_to_first_token.

These metrics use the kong.gen_ai.* namespace and capture Kong-specific AI observability data, including cost tracking, cache and RAG latency, and AWS Guardrails processing time.

kong.gen_ai.llm.cost

Cost of AI requests. To populate this metric, define model.options.input_cost and model.options.output_cost in the AI Proxy or AI Proxy Advanced plugin configuration.

• Type: Counter
• Unit: {cost}

kong.gen_ai.cache.fetch.latency

Time to fetch a response from the semantic cache.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as kong.gen_ai.llm.cost.

kong.gen_ai.cache.embeddings.latency

Time to generate embeddings during cache operations.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as kong.gen_ai.llm.cost.

kong.gen_ai.rag.fetch.latency

Time to fetch data from a RAG (Retrieval-Augmented Generation) source.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as kong.gen_ai.llm.cost.

kong.gen_ai.rag.embeddings.latency

Time to generate embeddings for RAG operations.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as kong.gen_ai.llm.cost.

kong.gen_ai.aws.guardrails.latency

Time for AWS Guardrails to process a request.

• Type: Histogram
• Unit: s (seconds)

These metrics provide observability into MCP (Model Context Protocol) server interactions, including latency, response sizes, errors, and ACL decisions.

mcp.client.operation.duration

Duration of the MCP request as observed by the sender. Only available when the AI MCP Proxy plugin is in passthrough-listener mode (the upstream is an MCP server). Requires enable_latency_metrics set to true.

• Type: Histogram
• Unit: s (seconds)

mcp.server.operation.duration

Duration of the MCP request as observed by the receiver.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as mcp.client.operation.duration.

kong.gen_ai.mcp.response.size

Size of the MCP response body.

• Type: Histogram
• Unit: By (bytes)

kong.gen_ai.mcp.request.error.count

Number of MCP request errors.

• Type: Counter
• Unit: {error}

kong.gen_ai.mcp.acl.allowed

Number of MCP requests allowed by ACL rules.

• Type: Counter
• Unit: {request}

kong.gen_ai.mcp.acl.denied

Number of MCP requests denied by ACL rules.

• Type: Counter
• Unit: {request}

Attributes: Same as kong.gen_ai.mcp.acl.allowed.

These metrics provide observability into A2A (Agent-to-Agent) traffic, including request volume, latency, response sizes, and task state transitions.

kong.gen_ai.a2a.request.count

Total number of A2A requests.

• Type: Counter
• Unit: {request}

kong.gen_ai.a2a.request.duration

Duration of an A2A request.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as kong.gen_ai.a2a.request.count.

kong.gen_ai.a2a.response.size

Size of the A2A response body.

• Type: Histogram
• Unit: By (bytes)

Attributes: Same as kong.gen_ai.a2a.request.count.

kong.gen_ai.a2a.ttfb

Time to first byte for A2A streaming responses.

• Type: Histogram
• Unit: s (seconds)

Attributes: Same as kong.gen_ai.a2a.request.count.

kong.gen_ai.a2a.request.error.count

Number of A2A request errors.

• Type: Counter
• Unit: {error}

kong.gen_ai.a2a.task.state.count

Number of A2A task state transitions.

• Type: Counter
• Unit: {state}