Add metrics to a custom plugin

Open the handler.lua file you last edited in Consume external services, and register a counter and a gauge as module-level locals, above the plugin handler table:

local http  = require("resty.http")
local cjson = require("cjson.safe")

local MyPluginHandler = {
  PRIORITY = 1000,
  VERSION = "0.0.1",
}

local request_count = kong.metrics.counter("my_plugin.request.count", {
  description = "Number of requests processed by my-plugin",
  unit = "{request}",
})

local in_flight = kong.metrics.gauge("my_plugin.requests.in_flight", {
  description = "Number of requests currently being processed by my-plugin",
  unit = "{request}",
})

Note: Register each metric once, when the plugin module loads. Don’t call kong.metrics.counter() or kong.metrics.gauge() from a request phase. Only the record calls, like :add() and :record(), belong in access, response, or log.

In the handler.lua file, add an access function that increments the gauge as a request comes in, and a log function that increments the counter and decrements the gauge once the request finishes:

function MyPluginHandler:access(conf)
  in_flight:add(1)
end

function MyPluginHandler:log(conf)
  request_count:add(1, {
    status = kong.response.get_status(),
    method = kong.request.get_method(),
  })

  in_flight:add(-1)
end

Warning: The Metrics PDK doesn’t mask or redact attribute values. You’re responsible for what your plugin puts into an attributes table: don’t record sensitive data, like personally identifiable information, credentials, or tokens, as an attribute value.

The full handler.lua file now looks like this:

local http  = require("resty.http")
local cjson = require("cjson.safe")

local MyPluginHandler = {
  PRIORITY = 1000,
  VERSION = "0.0.1",
}

local request_count = kong.metrics.counter("my_plugin.request.count", {
  description = "Number of requests processed by my-plugin",
  unit = "{request}",
})

local in_flight = kong.metrics.gauge("my_plugin.requests.in_flight", {
  description = "Number of requests currently being processed by my-plugin",
  unit = "{request}",
})

function MyPluginHandler:response(conf)

  kong.log("response handler")

  local httpc = http.new()

  local res, err = httpc:request_uri("http://httpbin.konghq.com/anything", {
    method = "GET",
  })

  if err then
    return kong.response.error(500,
      "Error when trying to access third-party service: " .. err,
      { ["Content-Type"] = "text/html" })
  end

  local body_table, err = cjson.decode(res.body)

  if err then
    return kong.response.error(500,
      "Error when decoding third-party service response: " .. err,
      { ["Content-Type"] = "text/html" })
  end

  kong.response.set_header(conf.response_header_name, body_table.url)

end

function MyPluginHandler:access(conf)
  in_flight:add(1)
end

function MyPluginHandler:log(conf)
  request_count:add(1, {
    status = kong.response.get_status(),
    method = kong.request.get_method(),
  })

  in_flight:add(-1)
end

return MyPluginHandler

request_count only ever gets :add(1), because a counter must not decrease. in_flight uses :add(1) and :add(-1), because a gauge accepts negative deltas.

Custom metrics leave Kong Gateway only through the OpenTelemetry plugin. Set up a small local pipeline (using Kong Gateway, an OpenTelemetry Collector, and Prometheus) with Docker Compose so you can query the metrics you just added.

1. In your plugin project’s root directory, create the OpenTelemetry Collector configuration. It receives OTLP data from Kong Gateway and forwards it to Prometheus:

   cat <<'EOF' > otel-collector-config.yaml
   receivers:
     otlp:
       protocols:
         http:
           endpoint: 0.0.0.0:4318
   
   processors:
     batch:
   
   exporters:
     otlphttp/prometheus:
       endpoint: http://prometheus:9090/api/v1/otlp
   
   service:
     pipelines:
       metrics:
         receivers: [otlp]
         processors: [batch]
         exporters: [otlphttp/prometheus]
   EOF

   Copied!

2. Create the Prometheus configuration to enable Prometheus’s OTLP receiver:

   cat <<'EOF' > prometheus.yml
   storage:
     tsdb:
       out_of_order_time_window: 30m
   
   otlp:
     promote_resource_attributes:
       - service.name
   EOF

   Copied!

3. Create a declarative configuration file for Kong Gateway. The following configuration contains a Service and Route, my-plugin, and the OpenTelemetry plugin:

   cat <<'EOF' > kong.yml
   _format_version: "3.0"
   
   services:
     - name: example-service
       url: https://httpbin.konghq.com
       routes:
         - name: example-route
           paths:
           - "/anything"
           protocols:
           - http
           - https
   plugins:
     - name: my-plugin
       route: example-route
     - name: opentelemetry
       config:
         metrics:
           endpoint: "http://otel-collector:4318/v1/metrics"
           push_interval: 5
   EOF

   Copied!

4. Create the Docker Compose file:

   cat <<'EOF' > docker-compose.yaml
   services:
     kong:
       image: kong/kong-gateway:latest
       environment:
         KONG_DATABASE: "off"
         KONG_DECLARATIVE_CONFIG: /kong/declarative/kong.yml
         KONG_PLUGINS: bundled,my-plugin
         KONG_LUA_PACKAGE_PATH: /kong/plugins/?.lua;/kong/plugins/?/init.lua;;
       volumes:
         - ./kong.yml:/kong/declarative/kong.yml:ro
         - ./kong/plugins/my-plugin:/kong/plugins/my-plugin:ro
       ports:
         - "8000:8000"
         - "8001:8001"
   
     otel-collector:
       image: otel/opentelemetry-collector-contrib:latest
       command: ["--config=/etc/otel-collector-config.yaml"]
       volumes:
         - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml:ro
       ports:
         - "4318:4318"
   
     prometheus:
       image: prom/prometheus:latest
       command:
         - "--config.file=/etc/prometheus/prometheus.yml"
         - "--web.enable-otlp-receiver"
       volumes:
         - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
       ports:
         - "9090:9090"
   EOF

   Copied!

5. Start the pipeline:

   docker compose up

   Copied!

1. In a new terminal, send a few requests through the example Route:

   curl http://localhost:8000/anything

   Copied!

2. Open Prometheus at http://localhost:9090 and query the counter. Enter the following into the text box and click Execute:

   my_plugin_request_count_total

   Copied!

   You should see a series with a value equal to the number of requests you sent, labeled with status and method.

3. Now let’s query the gauge. Enter the following into the text box and click Execute:

   my_plugin_requests_in_flight

   Copied!

   The value returns to 0 between requests, since access and log add and subtract the same amount.

Note: If a metric doesn’t appear, check Kong Gateway’s error log with docker compose logs kong. If an input is invalid, the Metrics PDK logs an error and does nothing, rather than disrupting the request path. A registration or recording mistake logs an error instead of failing the request.