---
title: "AI Gateway logs"
description: See where AI Gateway logs are located, the different log levels,
  and how to configure logs and log levels.
url: "/ai-gateway/ai-logs/"
canonical_url: "/ai-gateway/ai-logs/"
content_type: reference
min_version:
  ai-gateway: '2.0'
products:
- AI Gateway
tags:
- logging
- monitoring
canonical: true
works_on:
- konnect


---

# AI Gateway logs










Logging in AI Gateway allows you to see information, warnings, and errors about requests that are proxied by AI Gateway.

The information in this reference doc helps you understand and modify AI Gateway logs. You can also set [logging Policies](/ai-gateway/policies/?category=logging) to extend these capabilities by logging additional information or sending logs to another application.

## Where are AI Gateway logs located?

By default, you can view AI Gateway logs at `/usr/local/kong/logs/error.log`. If you're running an AI Gateway data plane in Docker, you can also view them from your Docker container.

## Log levels

By default, logs are set to the recommended `notice` level. If logs are too busy, you can increase the level to something like `warn`.


### `debug`
Description: Provides debug information about the Policy’s run loop and each individual Policy or other components. This should only be used during debugging. If this is enabled for extended periods of time, it can result in excess disk space consumption.

### `info` and `notice`
Description: Provides information about normal behavior, most of which can be ignored.

### `warn`
Description: Logs any abnormal behavior that doesn't result in dropped transactions but requires further investigation.

### `error`
Description: Used for logging errors that result in a request being dropped. For example, getting a `500` error. The rate of these logs must be monitored.

### `crit`
Description: Used when AI Gateway is working under critical conditions, affecting several clients. `crit` is the highest severity log level.




## Configure log levels

You can change log levels dynamically, without restarting AI Gateway, using the Admin API. Alternatively, you can configure log levels using the `log_level` parameter in the [`kong.conf` file](/gateway/configuration/), but this requires you to [restart AI Gateway](/how-to/restart-kong-gateway-container/).


### View current log level<sup>1</sup>
How to configure: [`/debug/node/log-level/`](/api/gateway/admin-ee/#/operations/get-debug-node-log-level/)

### Modify the log level for an individual AI Gateway node
How to configure: [`/debug/node/log-level/{logLevel}`](/api/gateway/admin-ee/#/operations/get-debug-node-log-level-log_level/)

### Change the log level of the AI Gateway cluster
How to configure: [`/debug/cluster/log-level/{loglevel}`](/api/gateway/admin-ee/#/operations/update-debug-cluster-log-level/)

### Keep the log level of new nodes added to the cluster in sync with other nodes in the cluster
How to configure: Change the [`log_level`](/gateway/configuration/#log-level) entry in `kong.conf` to `KONG_LOG_LEVEL`, and start every new node with the `KONG_LOG_LEVEL` env variable set.

### Change the log level of all control plane AI Gateway nodes
How to configure: [`/debug/cluster/control-planes-nodes/log-level/{loglevel}`](/api/gateway/admin-ee/#/operations/create-debug-cluster-control-planes-nodes-log-level)





> <sup>1</sup>: You can't change the log level of the data plane or DB-less nodes.


## Find specific client requests in logs

The `X-Kong-Request-Id` header contains a unique identifier for each client request. You can use this header to match specific requests to their corresponding error logs.

If AI Gateway returns an error by calling the PDK `kong.response.error`, the request ID will also be included in the response body generated by AI Gateway. In addition, any generated AI Gateway error log contains the same request ID with the format `request_id: xxx`. This can help with debugging because you can search for the header when the debug output is too long to fit in the response header.

This feature can be customized for upstreams and downstreams using the `headers` and `headers_upstream` configuration options in [`kong.conf`](/gateway/configuration/):



### Parameter: headers
parameter: headers
default_value: server_tokens, latency_tokens, X-Kong-Request-Id
description: |
  Comma-separated list of headers Kong should
  inject in client responses.
  
  Accepted values are:
  - `Server`: Injects `Server: kong/x.y.z`
    on Kong-produced responses (e.g., Admin
    API, rejected requests from auth plugin).
  - `Via`: Injects `Via: kong/x.y.z` for
    successfully proxied requests.
  - `X-Kong-Proxy-Latency`: Time taken
    (in milliseconds) by Kong to process
    a request and run all plugins before
    proxying the request upstream.
  - `X-Kong-Response-Latency`: Time taken
    (in milliseconds) by Kong to produce
    a response in case of, e.g., a plugin
    short-circuiting the request, or in
    case of an error.
  - `X-Kong-Upstream-Latency`: Time taken
    (in milliseconds) by the upstream
    service to send response headers.
  - `X-Kong-Admin-Latency`: Time taken
    (in milliseconds) by Kong to process
    an Admin API request.
  - `X-Kong-Upstream-Status`: The HTTP status
    code returned by the upstream service.
    This is particularly useful for clients to
    distinguish upstream statuses if the
    response is rewritten by a plugin.
  - `X-Kong-Request-Id`: Unique identifier of
    the request.
  - `X-Kong-Total-Latency` (v3.11+): Time elapsed
    (in milliseconds) between the first bytes
    being read from the client and the log
    write after the last bytes were sent to
    the client. Calculated as the difference
    between the current timestamp and the
    timestamp when the request was created.
  - `X-Kong-Third-Party-Latency` (v3.11+): Cumulative
    sum of all third-party latencies, including
    DNS resolution, HTTP client calls, Socket
    operations, and Redis operations.
  - `X-Kong-Client-Latency` (v3.11+): Time that Kong waits
    to receive headers and body from the client, and
    also how long Kong waits for the client to
    read/receive the response from Kong.
  - `server_tokens`: Same as specifying both
    `Server` and `Via`.
  - `latency_tokens`: Same as specifying
    `X-Kong-Proxy-Latency`,
    `X-Kong-Response-Latency`,
    `X-Kong-Admin-Latency`, and
    `X-Kong-Upstream-Latency`.
  - `advanced_latency_tokens` (v3.11+): Same as specifying
    `X-Kong-Proxy-Latency`,
    `X-Kong-Response-Latency`,
    `X-Kong-Admin-Latency`,
    `X-Kong-Upstream-Latency`.
    `X-Kong-Total-Latency`,
    `X-Kong-Third-Party-Latency`, and
    `X-Kong-Client-Latency`.
  
  In addition to these, this value can be set
  to `off`, which prevents Kong from injecting
  any of the above headers. Note that this
  does not prevent plugins from injecting
  headers of their own.
  
  Example: `headers = via, latency_tokens`

### Parameter: headers_upstream
parameter: headers_upstream
default_value: X-Kong-Request-Id
description: |
  Comma-separated list of headers Kong should
  inject in requests to upstream.
  
  At this time, the only accepted value is:
  - `X-Kong-Request-Id`: Unique identifier of
    the request.
  
  In addition, this value can be set
  to `off`, which prevents Kong from injecting
  the above header. Note that this
  does not prevent plugins from injecting
  headers of their own.



## Customize what AI Gateway logs

You may need to customize what AI Gateway logs. For instance, you may want to:
* Protect private information
* Comply with GDPR or other data protection regulations
* Remove instances of a specific piece of data from your logs, such as an email address

These changes can be made to AI Gateway's Nginx template and only affect the output of the Nginx access logs. This doesn't have any effect on AI Gateway's [logging Policies](/ai-gateway/policies/?category=logging).

Let's look at an example where you want to remove any instances of an email address from your AI Gateway logs. The email addresses may come through in different formats, for example `/servicename/v2/verify/alice@example.com` or `/v3/verify?alice@example.com`. To keep all of these formats from being added to the logs, you need to use a custom Nginx template.

Make a copy of AI Gateway's Nginx template, then edit it to add or remove the data you need. The following template shows an example configuration for removing email addresses from logs:

```nginx
# ---------------------
# custom_nginx.template
# ---------------------

worker_processes $; # can be set by kong.conf
daemon $;                     # can be set by kong.conf

pid pids/nginx.pid;                      # this setting is mandatory
error_log stderr $; # can be set by kong.conf



events {
    use epoll; # custom setting
    multi_accept on;
}

http {


    map $request_uri $keeplog {
        ~.+\@.+\..+ 0;
        ~/v1/invitation/ 0;
        ~/reset/v1/customer/password/token 0;
        ~/v2/verify 0;

        default 1;
    }
    log_format show_everything '$remote_addr - $remote_user [$time_local] '
        '$request_uri $status $body_bytes_sent '
        '"$http_referer" "$http_user_agent"';

    include 'nginx-kong.conf';
}
```

For this example, we're using the following:

* `map $request_uri $keeplog`: Maps a new variable called `keeplog`, which is dependent on values appearing in the `$request_uri`. Each line in the example starts with a `~` because this is what tells Nginx to use a regex when evaluating the line. This example looks for the following:
  - The first line uses a regex to look for any email address in the `x@y.z` format
  - The second line looks for any part of the URI that contains `/servicename/v2/verify`
  - The third line looks at any part of the URI that contains `/v3/verify`

    Because all of these patterns have a value of something other than `0`, if a request has any of those elements, it will not be added to the log.
* `log_format`: Sets the log format for what AI Gateway keeps in the logs. The contents of the log can be customized for your needs. For the purpose of this example, you can assign the new logs with the name `show_everything` and set everything to the AI Gateway default standards. To see the full list of options, refer to the [Nginx core module variables reference](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables).

Once you've adjusted the Nginx template for your environment, you need to tell AI Gateway to use the newly created log, `show_everything`.

To do this, alter the AI Gateway variable `proxy_access_log` by either editing `etc/kong/kong.conf` or using the environmental variable `KONG_PROXY_ACCESS_LOG` and adjust the default location:

```sh
proxy_access_log=logs/access.log show_everything if=$keeplog
```

Restart AI Gateway to apply changes with the `kong restart` command.

Now, any request made with an email address in it will no longer be logged.

## AI Gateway Policy logs

AI Gateway collects logs for the [AI Gateway Policies](/ai-gateway/policies/). This allows you to aggregate AI usage analytics across various providers.

Each log entry includes the following details:


### `ai.$POLICY_NAME.payload.request`
Description: The request payload.

### `ai.$POLICY_NAME.payload.response`
Description: The response payload.

### `ai.$POLICY_NAME.usage.prompt_token`
Description: The number of tokens used for prompting.

### `ai.$POLICY_NAME.usage.completion_token`
Description: The number of tokens used for completion.

### `ai.$POLICY_NAME.usage.total_tokens`
Description: The total number of tokens used.

### `ai.$POLICY_NAME.usage.cost`
Description: The total cost of the request (input and output cost).

### `ai.$POLICY_NAME.usage.time_per_token`
Description: The average time to generate an output token, in milliseconds.

### `ai.$POLICY_NAME.meta.request_model`
Description: The model used for the AI request.

### `ai.$POLICY_NAME.meta.provider_name`
Description: The name of the AI service provider.

### `ai.$POLICY_NAME.meta.response_model`
Description: The model used for the AI response.

### `ai.$POLICY_NAME.meta.plugin_id`
Description: The unique identifier of the Policy.

### `ai.$POLICY_NAME.meta.llm_latency`
Description: The time, in milliseconds, it took the LLM provider to generate the full response.

### `ai.$POLICY_NAME.cache.cache_status`
Description: The cache status. This can be `Hit`, `Miss`, `Bypass` or `Refresh`.

### `ai.$POLICY_NAME.cache.fetch_latency`
Description: The time, in milliseconds, it took to return a cache response.

### `ai.$POLICY_NAME.cache.embeddings_provider`
Description: For semantic caching, the provider used to generate the embeddings.

### `ai.$POLICY_NAME.cache.embeddings_model`
Description: For semantic caching, the model used to generate the embeddings.

### `ai.$POLICY_NAME.cache.embeddings_latency`
Description: For semantic caching, the time taken to generate the embeddings.




<!--
## Next steps
* [Debug AI Gateway with logs](/gateway/debug/)
-->



## Related Resources

- [Konnect logs](/dedicated-cloud-gateways/konnect-logs/)

- [Konnect platform audit logs](/konnect-platform/audit-logs/)

- [Logging Policies](/ai-gateway/policies/?category=logging)

- [Add Correlation IDs to AI Gateway logs](/how-to/add-correlation-ids-to-gateway-logs/)

- [AI Gateway audit log reference](/ai-gateway/ai-audit-log-reference/)

- [Monitor AI LLM metrics (Prometheus)](/ai-gateway/monitor-ai-llm-metrics/)

- [Gen AI OpenTelemetry metrics reference](/ai-gateway/ai-otel-metrics/)