Dynamically set upstream based on the authenticated caller with Datakit

TL;DR

Configure the OpenID Connect plugin with credential_claim to extract a token claim (for example, client_id) and set it as the authenticated credential. Then, configure the Datakit plugin to read that credential, map its value to a named Upstream entity, and override the backend for the request.

Prerequisites

This is a Konnect tutorial and requires a Konnect personal access token.

  1. Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.

  2. Export your token to an environment variable:

    export KONNECT_TOKEN='YOUR_KONNECT_PAT'
  3. Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:

    curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN \
         --deck-output

    This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:

    export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
    export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
    export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
    export KONNECT_PROXY_URL='http://localhost:8000'

    Copy and paste these into your terminal to configure your session.

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

  1. Export your license to an environment variable:

     export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'
  2. Run the quickstart script:

    curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA 

    Once Kong Gateway is ready, you will see the following message:

     Kong Gateway Ready

To complete this tutorial, install decK. We recommend keeping decK up to date with the latest version (1.65.1).

decK is a CLI tool for managing Kong Gateway declaratively with state files. This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance.

You can check your current decK version with deck version.

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

  1. Run the following command:

    echo '
    _format_version: "3.0"
    services:
      - name: example-service
        url: http://httpbin.konghq.com/anything
    routes:
      - name: example-route
        paths:
        - "/anything"
        service:
          name: example-service
        protocols:
        - http
        - https
    ' | deck gateway apply -

To learn more about entities, you can read our entities documentation.

This tutorial requires an identity provider (IdP). If you don’t have one, you can use Keycloak. The steps will be similar in other standard identity providers.

For this tutorial, you need three clients: one for the OpenID Connect plugin to use when validating tokens, and two representing the API callers that will be routed to different backends.

Install and run Keycloak

  1. Install Keycloak (version 26 or later) on your platform.

    For example, you can use the Keycloak Docker image. The following command attaches Keycloak to the same network as Kong Gateway so that the OIDC plugin can reach it. Open a new session in your terminal and run:

    docker run -p 127.0.0.1:8080:8080 \
      --name keycloak \
      --network kong-quickstart-net \
      -e KC_BOOTSTRAP_ADMIN_USERNAME=admin \
      -e KC_BOOTSTRAP_ADMIN_PASSWORD=admin \
      -e KC_HOSTNAME=http://localhost:8080 \
      quay.io/keycloak/keycloak start-dev

    The KC_HOSTNAME=http://localhost:8080 parameter ensures Keycloak always uses localhost:8080 as its token issuer regardless of which URL it’s accessed through. This is required because Kong Gateway runs inside Docker and accesses Keycloak via the container name keycloak:8080, but the iss claim in issued tokens must use localhost:8080 for the plugin to recognize them.

  2. Export your issuer URL, JWKS endpoint, and Keycloak host to environment variables. For example, using Docker and the default master realm:

    export DECK_ISSUER='http://localhost:8080/realms/master'
    export DECK_JWKS_ENDPOINT='http://keycloak:8080/realms/master/protocol/openid-connect/certs'
    export KEYCLOAK_HOST='localhost'

    Because we’re using Docker for this demo, we must configure a few networking parameters:

    • DECK_ISSUER and KEYCLOAK_HOST use localhost because that’s how you access Keycloak from your machine.
    • DECK_JWKS_ENDPOINT uses the container name keycloak because Kong Gateway runs inside Docker and reaches Keycloak over the shared kong-quickstart-net network.

    In your own setup, especially running outside of a container, you may not need DECK_JWKS_ENDPOINT.

  3. Open the admin console at default URL http://localhost:8080/admin/master/console/ and sign in with your username (admin) and password (admin).

Create the plugin client

This client is used by the OpenID Connect plugin to connect to Keycloak for token validation.

  1. In the sidebar, open Clients, then click Create client.
  2. Configure the client:

Section

Settings

General settings
  • Client type: OpenID Connect
  • Client ID: kong
Capability config
  • Toggle Client authentication to on
  • Make sure Service accounts roles is checked
  1. Open the Credentials tab.
  2. Set Client Authenticator to Client ID and Secret.
  3. Copy the Client Secret and export it along with the client ID:

    export DECK_CLIENT_ID='kong'
    export DECK_CLIENT_SECRET='YOUR-KONG-CLIENT-SECRET'

Create the caller clients

These clients represent the API callers. Each one authenticates to Keycloak with the client credentials grant. The client_id claim in each token is what Datakit uses to select the upstream.

Create the first caller client:

  1. In the sidebar, open Clients, then click Create client.
  2. Configure the client:

Section

Settings

General settings
  • Client type: OpenID Connect
  • Client ID: caller-a
Capability config
  • Toggle Client authentication to on
  • Make sure Service accounts roles is checked
  1. Open the Credentials tab.
  2. Set Client Authenticator to Client ID and Secret.
  3. Copy the Client Secret and export it:

    export CALLER_A_SECRET='YOUR-CALLER-A-SECRET'

Create the second caller client:

  1. In the sidebar, open Clients, then click Create client.
  2. Configure the client:

Section

Settings

General settings
  • Client type: OpenID Connect
  • Client ID: caller-b
Capability config
  • Toggle Client authentication to on
  • Make sure Service accounts roles is checked
  1. Open the Credentials tab.
  2. Set Client Authenticator to Client ID and Secret.
  3. Copy the Client Secret and export it:

    export CALLER_B_SECRET='YOUR-CALLER-B-SECRET'

Kong’s router runs before authentication, so it can’t directly route traffic based on who is making a request. This guide shows you how to solve that using two plugins working together in the access phase:

  1. OpenID Connect validates the bearer token and extracts a claim value (such as client_id) onto a virtual credential.
  2. Datakit reads that credential and maps its value to a named Upstream entity, overriding the backend for that specific request.

All callers share one Route and one Service. After authentication, the backend is selected dynamically based on the client’s identity.

This guide uses named Upstream entities, which preserves load balancing, health checks, and retries. If you don’t need those features and prefer to route directly to a host:port without a pool, see Route requests to different targets based on the authenticated caller.

Note: The OpenID Connect plugin has a higher static priority than Datakit, so it always runs first in the access phase. No explicit plugin ordering configuration is required.

Generate salt token

Starting with decK v1.59+, you need to set cache_tokens_salt to avoid regenerating session credentials during sync. Generate a salt token:

export DECK_TOKEN_SALT="$(openssl rand -base64 16)"
export DECK_TOKEN_SALT="$(openssl rand -base64 16)"

Add upstreams

Create the named Upstream entities that Datakit will route to. Each Upstream is a pool of targets, so load balancing, health checks, and retries are preserved when Datakit overrides the backend.

In this example, two callers route to separate upstreams and all other callers fall back to a default upstream.

In this example, all three upstreams point at httpbin.konghq.com so you can run the guide end to end. In production, each upstream would point at a different backend host.

echo '
_format_version: "3.0"
upstreams:
  - name: upstream-a
    targets:
    - target: httpbin.konghq.com:80
  - name: upstream-b
    targets:
    - target: httpbin.konghq.com:80
  - name: upstream-default
    targets:
    - target: httpbin.konghq.com:80
' | deck gateway apply -

Update the example-service Service so that its host points to upstream-default. This is the default backend; Datakit overrides it per request based on the authenticated caller.

echo '
_format_version: "3.0"
services:
  - name: example-service
    host: upstream-default
    port: 80
    protocol: http
' | deck gateway apply -

Enable the OpenID Connect plugin

Configure the OpenID Connect plugin to validate bearer tokens and extract the client_id claim as a virtual credential. The credential_claim field sets credential.id to the value of the named claim without requiring a Kong Consumer entity.

echo '
_format_version: "3.0"
plugins:
  - name: openid-connect
    service: example-service
    config:
      issuer: "${{ env "DECK_ISSUER" }}"
      using_pseudo_issuer: true
      jwks_endpoint: "${{ env "DECK_JWKS_ENDPOINT" }}"
      client_id:
      - "${{ env "DECK_CLIENT_ID" }}"
      client_secret:
      - "${{ env "DECK_CLIENT_SECRET" }}"
      client_auth:
      - client_secret_post
      auth_methods:
      - bearer
      credential_claim:
      - client_id
      consumer_optional: true
      cache_tokens_salt: "${{ env "DECK_TOKEN_SALT" }}"
' | deck gateway apply -

In this configuration:

  • issuer: The issuer URL used to validate the iss claim in incoming tokens. This must match the iss value Keycloak stamps into its tokens (localhost:8080).
  • using_pseudo_issuer: true: Disables OIDC discovery from the issuer URL. This is required here because Kong Gateway runs inside Docker and can’t reach localhost:8080 directly. The issuer value is still used to validate the iss claim in incoming tokens.
  • jwks_endpoint: The explicit URL Kong Gateway uses to fetch Keycloak’s signing keys. This uses the keycloak container name, which is reachable from Kong Gateway over the shared Docker network.
  • auth_methods: bearer: The plugin only accepts tokens in the Authorization: Bearer header.
  • credential_claim: [client_id]: Extracts the client_id claim from the validated token and places its value on credential.id.
  • consumer_optional: true: Prevents a 401 when no Consumer entity matches the credential. The plugin still validates the token, but it doesn’t require a Consumer to exist.

Enable the Datakit plugin

Configure the Datakit plugin to read the credential set by the OpenID Connect plugin and map it to an upstream name.

echo '
_format_version: "3.0"
plugins:
  - name: datakit
    service: example-service
    config:
      nodes:
      - name: GET_CREDENTIAL
        type: property
        property: kong.client.credential
      - name: PICK_UPSTREAM
        type: jq
        input: GET_CREDENTIAL
        jq: |
          {
            "caller-a": "upstream-a",
            "caller-b": "upstream-b"
          }[.id] // "upstream-default"
      - name: SET_UPSTREAM
        type: property
        property: kong.service.upstream
        input: PICK_UPSTREAM
      debug: true
' | deck gateway apply -

In this configuration:

  • GET_CREDENTIAL: Reads the kong.client.credential object that the OpenID Connect plugin populates. No input is connected because this is a read-only (get) operation.
  • PICK_UPSTREAM: Uses a jq map to look up the credential’s .id field and return the matching upstream name. The keys caller-a and caller-b match the client_id values in the tokens issued by the Keycloak clients you created in the prerequisites. The // "upstream-default" fallback handles any caller whose ID isn’t in the map.
  • SET_UPSTREAM: Writes the resolved upstream name to kong.service.upstream, overriding the backend for this request.
  • debug: true: Enables trace output for this tutorial. Remove it before using this configuration in production.

Validate the routing

To validate that routing via Datakit is working, retrieve access tokens from Keycloak using the client credentials grant, then send them as bearer tokens to verify that each caller is routed to the correct upstream.

All three upstreams point at the same host in this tutorial, so each request returns an HTTP 200 from httpbin.konghq.com. In the following requests, we’re setting the X-Datakit-Debug-Trace: true request header so that Datakit returns a JSON trace in the response body showing each node’s input and output.

  1. Fetch a token as caller-a :

    export TOKEN_A=$(curl -s -X POST http://$KEYCLOAK_HOST:8080/realms/master/protocol/openid-connect/token \
      -d "client_id=caller-a" \
      -d "client_secret=$CALLER_A_SECRET" \
      -d "grant_type=client_credentials" | jq -r .access_token)

    Send a request as caller-a:

    curl -si http://localhost:8000/anything \
      -H "Authorization: Bearer $TOKEN_A" \
      -H "X-Datakit-Debug-Trace: true"

    In the response body, find the complete event for each node and check:

    • GET_CREDENTIAL: value.value.id is caller-a.
    • PICK_UPSTREAM: value.value is upstream-a.
    • SET_UPSTREAM: value.value is upstream-a.
  2. Fetch a token as caller-b and send a request:

    export TOKEN_B=$(curl -s -X POST http://$KEYCLOAK_HOST:8080/realms/master/protocol/openid-connect/token \
      -d "client_id=caller-b" \
      -d "client_secret=$CALLER_B_SECRET" \
      -d "grant_type=client_credentials" | jq -r .access_token)

    Send a request as caller-b:

    curl -si http://localhost:8000/anything \
      -H "Authorization: Bearer $TOKEN_B" \
      -H "X-Datakit-Debug-Trace: true"

    PICK_UPSTREAM should resolve to upstream-b.

  3. Fetch a token as the kong client to confirm the fallback:

    export TOKEN_FALLBACK=$(curl -s -X POST http://$KEYCLOAK_HOST:8080/realms/master/protocol/openid-connect/token \
      -d "client_id=$DECK_CLIENT_ID" \
      -d "client_secret=$DECK_CLIENT_SECRET" \
      -d "grant_type=client_credentials" | jq -r .access_token)

    Send a request as the fallback client:

    curl -si http://localhost:8000/anything \
      -H "Authorization: Bearer $TOKEN_FALLBACK" \
      -H "X-Datakit-Debug-Trace: true"

    GET_CREDENTIAL should show value.value.id as kong, and PICK_UPSTREAM should be upstream-default because kong isn’t in the routing map.

Cleanup

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!