Set timeouts and retries for a Service

Deployment Platform
Related Documentation
Related Resources
Minimum Version
Kong Operator - 2.2
TL;DR

Annotate the Kubernetes Service with konghq.com/connect-timeout, konghq.com/read-timeout, konghq.com/write-timeout, and konghq.com/retries. Kong Gateway applies these settings when forwarding requests to the upstream service.

Prerequisites

If you don’t have a Konnect account, you can get started quickly with our onboarding wizard.

  1. The following Konnect items are required to complete this tutorial:
    • Personal access token (PAT): Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
  2. Set the personal access token as an environment variable:

    export KONNECT_TOKEN='YOUR KONNECT TOKEN'
  1. Add the Kong Helm charts:

    helm repo add kong https://charts.konghq.com
    helm repo update
  2. Install Kong Operator using Helm:

    helm upgrade --install kong-operator kong/kong-operator -n kong-system \
      --create-namespace \
      --set image.tag=2.2 \
      --set env.ENABLE_CONTROLLER_KONNECT=true
    helm upgrade --install kong-operator kong/kong-operator -n kong-system \
      --create-namespace \
      --set image.tag=2.2

    If you want cert-manager to issue and rotate the admission and conversion webhook certificates, install cert-manager to your cluster and enable cert-manager integration by passing the following argument while installing, in the next step:

    --set global.webhooks.options.certManager.enabled=true

    If you do not enable this, the chart will generate and inject self-signed certificates automatically. We recommend enabling cert-manager to manage the lifecycle of these certificates. Kong Operator needs a certificate authority to sign the certificate for mTLS communication between the control plane and the data plane. This is handled automatically by the Helm chart. If you need to provide a custom CA certificate, refer to the certificateAuthority section in the values.yaml of the Helm chart to learn how to create and reference your own CA certificate.

This tutorial doesn’t require a license, but you can add one using KongLicense. This assumes that your license is available in ./license.json.

echo "
apiVersion: configuration.konghq.com/v1alpha1
kind: KongLicense
metadata:
 name: kong-license
rawLicenseString: '$(cat ./license.json)'
" | kubectl apply -f -
kubectl create namespace kong --dry-run=client -o yaml | kubectl apply -f -
echo '
kind: KonnectAPIAuthConfiguration
apiVersion: konnect.konghq.com/v1alpha1
metadata:
  name: konnect-api-auth
  namespace: kong
spec:
  type: token
  token: "'$KONNECT_TOKEN'"
  serverURL: us.api.konghq.com
' | kubectl apply -f -
echo '
kind: KonnectGatewayControlPlane
apiVersion: konnect.konghq.com/v1alpha2
metadata:
  name: gateway-control-plane
  namespace: kong
spec:
  createControlPlaneRequest:
    name: gateway-control-plane
  konnect:
    authRef:
      name: konnect-api-auth
' | kubectl apply -f -

Create the kong namespace:

kubectl create namespace kong

Create the GatewayConfiguration, GatewayClass, and Gateway resources with basic configuration:

echo '
apiVersion: gateway-operator.konghq.com/v2beta1
kind: GatewayConfiguration
metadata:
  name: gateway-configuration
  namespace: kong
spec:
  dataPlaneOptions:
    deployment:
      podTemplateSpec:
        spec:
          containers:
            - image: kong/kong-gateway:3.14
              name: proxy
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: gateway-class
spec:
  controllerName: konghq.com/gateway-operator
  parametersRef:
    group: gateway-operator.konghq.com
    kind: GatewayConfiguration
    name: gateway-configuration
    namespace: kong
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong
  namespace: kong
spec:
  gatewayClassName: gateway-class
  listeners:
    - name: http
      port: 80
      protocol: HTTP' | kubectl apply -f -
  1. Run the following command to create a sample httpbin Service:

    kubectl apply -f https://developer.konghq.com/manifests/kic/httpbin-service.yaml -n kong
  2. Create an HTTPRoute resource:

    echo '
    apiVersion: gateway.networking.k8s.io/v1
    kind: HTTPRoute
    metadata:
      name: httpbin-route
      namespace: kong
      annotations:
        konghq.com/strip-path: "true"
        konghq.com/preserve-host: "false"
    spec:
      parentRefs:
        - name: kong
      rules:
        - matches:
            - path:
                type: PathPrefix
                value: /httpbin
          backendRefs:
            - name: httpbin
              kind: Service
              port: 80' | kubectl apply -f -

Timeout and retry settings control how long Kong Gateway waits for responses from your upstream services and how many times it retries failed requests. You configure these settings by annotating the Kubernetes Service that your HTTPRoute routes traffic to.

Annotate the Service

Annotate the httpbin Service with timeout and retry values:

kubectl annotate service httpbin -n kong \
  konghq.com/connect-timeout="3000" \
  konghq.com/read-timeout="5000" \
  konghq.com/write-timeout="5000" \
  konghq.com/retries="3"

The annotations have the following effects:

Annotation

Value

Effect

konghq.com/connect-timeout 3000 Kong Gateway waits up to 3 seconds to establish a TCP connection
konghq.com/read-timeout 5000 Kong Gateway waits up to 5 seconds for the upstream to send a response
konghq.com/write-timeout 5000 Kong Gateway waits up to 5 seconds when sending data upstream
konghq.com/retries 3 Failed requests are retried up to 3 times before returning an error

All timeout values are in milliseconds.

Validate

  1. Get the Gateway’s external IP address:

    export PROXY_IP=$(kubectl get gateway kong -n kong -o jsonpath='{.status.addresses[0].value}')
  2. Send a request that completes within the read timeout. The /delay/1 endpoint waits 1 second before responding:

    curl -s -o /dev/null -w "%{http_code}" $PROXY_IP/httpbin/delay/1

    The response code should be 200.

  3. Send a request that exceeds the read timeout. The /delay/10 endpoint waits 10 seconds:

    curl -s -o /dev/null -w "%{http_code}" $PROXY_IP/httpbin/delay/10

    The response code should be 504, indicating that the upstream did not respond within the configured read-timeout.

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!