Monitor Kong Gateway with Prometheus using direct scraping

Deployment Platform
Related Documentation
TL;DR

Create a KongPlugin resource for the prometheus plugin, and apply the plugin using KongPluginBinding or the konghq.com/plugins annotation.

Prerequisites

If you don’t have a Konnect account, you can get started quickly with our onboarding wizard.

  1. The following Konnect items are required to complete this tutorial:
    • Personal access token (PAT): Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
  2. Set the personal access token as an environment variable:

    export KONNECT_TOKEN='YOUR KONNECT TOKEN'

If you want cert-manager to issue and rotate the admission and conversion webhook certificates, install cert-manager to your cluster and enable cert-manager integration by passing the following argument while installing, in the next step:

   --set global.webhooks.options.certManager.enabled=true

If you do not enable this, the chart will generate and inject self-signed certificates automatically. We recommend enabling cert-manager to manage the lifecycle of these certificates. Kong Operator needs a certificate authority to sign the certificate for mTLS communication between the control plane and the data plane. This is handled automatically by the Helm chart. If you need to provide a custom CA certificate, refer to the certificateAuthority section in the values.yaml of the Helm chart to learn how to create and reference your own CA certificate.

Apply a KongLicense. This assumes that your license is available in ./license.json

echo "
apiVersion: configuration.konghq.com/v1alpha1
kind: KongLicense
metadata:
 name: kong-license
rawLicenseString: '$(cat ./license.json)'
" | kubectl apply -f -
kubectl create namespace kong --dry-run=client -o yaml | kubectl apply -f -
echo '
kind: KonnectAPIAuthConfiguration
apiVersion: konnect.konghq.com/v1alpha1
metadata:
  name: konnect-api-auth
  namespace: kong
spec:
  type: token
  token: "'$KONNECT_TOKEN'"
  serverURL: us.api.konghq.com
' | kubectl apply -f -

Create the kong namespace:

kubectl create namespace kong

Create the kong namespace:

kubectl create namespace kong --dry-run=client -o yaml | kubectl apply -f -

Create the GatewayConfiguration, GatewayClass, and Gateway resources with basic configuration:

Setting spec.konnect.authRef attaches the data plane to Konnect, which manages its configuration:

echo '
apiVersion: gateway-operator.konghq.com/v2beta1
kind: GatewayConfiguration
metadata:
  name: gateway-configuration
  namespace: kong
spec:
  konnect:
    authRef:
      name: konnect-api-auth
      namespace: kong
  dataPlaneOptions:
    deployment:
      podTemplateSpec:
        spec:
          containers:
            - image: kong/kong-gateway:3.16
              name: proxy
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: gateway-class
spec:
  controllerName: konghq.com/gateway-operator
  parametersRef:
    group: gateway-operator.konghq.com
    kind: GatewayConfiguration
    name: gateway-configuration
    namespace: kong
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong
  namespace: kong
  annotations:
    gateway-operator.konghq.com/static-naming: "true"
spec:
  gatewayClassName: gateway-class
  listeners:
    - name: http
      port: 80
      protocol: HTTP' | kubectl apply -f -

The gateway-operator.konghq.com/static-naming: "true" annotation gives the resources Kong Operator generates for the Gateway static, predictable names: the KonnectGatewayControlPlane it creates for the Gateway is named after the Gateway, in this case kong. You can reference that control plane from other resources before it exists, and its name in Konnect is derived from the Gateway name.

echo '
apiVersion: gateway-operator.konghq.com/v2beta1
kind: GatewayConfiguration
metadata:
  name: gateway-configuration
  namespace: kong
spec:
  dataPlaneOptions:
    deployment:
      podTemplateSpec:
        spec:
          containers:
            - image: kong/kong-gateway:3.16
              name: proxy
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: gateway-class
spec:
  controllerName: konghq.com/gateway-operator
  parametersRef:
    group: gateway-operator.konghq.com
    kind: GatewayConfiguration
    name: gateway-configuration
    namespace: kong
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong
  namespace: kong
spec:
  gatewayClassName: gateway-class
  listeners:
    - name: http
      port: 80
      protocol: HTTP' | kubectl apply -f -

Monitoring your gateway is critical for understanding traffic patterns, latency, and system health. Kong Operator provides two ways to collect metrics with Prometheus:

  • Direct scraping: Directly scrapes standard Prometheus plugin metrics from the data plane Pods.
  • Enriched metrics: Uses the DataPlaneMetricsExtension resource to enrich metrics with Kubernetes metadata and re-expose them via Kong Operator’s metrics endpoint.

In this example, we’ll use direct scraping.

Enable the Prometheus plugin

Create a KongPlugin resource to enable the Prometheus plugin:

echo '
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: prometheus-global
  namespace: kong
plugin: prometheus
config:
  bandwidth_metrics: true
  latency_metrics: true
  status_code_metrics: true' | kubectl apply -f -

Apply the plugin globally

Create a KongPluginBinding resource and set spec.scope to GlobalInControlPlane:

echo '
apiVersion: configuration.konghq.com/v1alpha1
kind: KongPluginBinding
metadata:
  name: global-prometheus
  namespace: kong
spec:
  pluginRef:
    name: prometheus-global
  scope: GlobalInControlPlane
  controlPlaneRef:
    type: konnectNamespacedRef
    konnectNamespacedRef:
      name: kong' | kubectl apply -f -

Enable the plugin by annotating the Gateway resource with konghq.com/plugins: prometheus-global:

echo '
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong
  namespace: kong
  annotations:
    konghq.com/plugins: prometheus-global
spec:
  gatewayClassName: gateway-class
  listeners:
  - name: proxy
    port: 80
    protocol: HTTP' | kubectl apply -f -

Validate

You can verify that metrics are being collected by port-forwarding to the data plane Pod:

  1. Wait for the Gateway to be programmed:

    kubectl wait --for=condition=Programmed gateway/kong -n kong --timeout=300s
  2. Get the data plane Pod name:

    POD_NAME=$(kubectl get pods -n kong -o jsonpath='{.items[0].metadata.name}')
  3. Port forward the metrics port. Run the port-forward in the background so that the next command can use it:

    kubectl port-forward $POD_NAME 8100:8100 -n kong >/dev/null 2>&1 &
  4. Access the metrics:

    curl -s http://localhost:8100/metrics | grep -m1 kong_

Cleanup

Deleting the Gateway also deletes the control plane and data plane resources Kong Operator created for it. Deleting the kong-system namespace removes the operator:

kubectl delete -n kong --ignore-not-found \
  kongpluginbinding/global-prometheus \
  kongplugin/prometheus-global \
  gateway/kong \
  gatewayconfiguration/gateway-configuration
kubectl delete --ignore-not-found gatewayclass/gateway-class
kubectl delete --ignore-not-found namespace kong
helm uninstall kong-operator -n kong-system
kubectl delete --ignore-not-found namespace kong-system

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!