Authentication with the data plane proxy

To obtain a configuration from the control plane, a data plane proxy must authenticate itself. There are several authentication methods available.

On Kubernetes, a data plane proxy proves its identity with the Service Account Token that is mounted in every Pod.

If you don’t explicitly specify serviceAccountToken in a Deployment, the Pod runs with the default Service Account Token in the namespace. This means that the authentication scope is bound to a namespace, so any Pod in the namespace can authenticate as any other Pod in that namespace.

To enforce strict security per Deployment, every Deployment should use a unique Service Account Token. Users should also not be able to modify serviceAccountToken in a Deployment. You can enforce this with a policy engine such as OPA Gatekeeper.

Kubernetes itself doesn’t bound the Service Account Token to a mesh. For information on restricting which Pods can join a mesh, see data plane proxy membership.

On Universal, a data plane proxy must be explicitly configured with a unique data plane proxy token that proves its identity.

The data plane proxy token is a JWT token that contains:

• The expiration date of the token (required; defaults to 10 years if not specified)
• The mesh to which the data plane belongs (required)
• The name of the data plane proxy (optional)
• A list of allowed tags (optional)
• v2.13+ The workload label value (optional)

The data plane proxy token is signed by a signing key that is autogenerated when the mesh is created. Tokens are never stored in the control plane, only the signing keys used to verify tokens are stored. The signing key uses the RS256 signing algorithm (RSA with SHA-256).

You can check for the signing key using the following command:

kumactl get secrets

It returns something like this:

MESH      NAME                                    AGE
default   dataplane-token-signing-key-default-1   49s

Usage

You can generate a data plane proxy token using either the API or kumactl:

curl -XPOST \
  -H "Content-Type: application/json" \
  --data '{"name": "dp-echo-1", "mesh": "default", "tags": {"kuma.io/service": ["backend", "backend-admin"]}, "validFor": "720h"}' \
  http://localhost:5681/tokens/dataplane

kumactl generate dataplane-token \
  --name dp-echo-1 \
  --mesh default \
  --tag kuma.io/service=backend,backend-admin \
  --valid-for 720h > /tmp/kuma-dp-echo1-token

Workload label in tokens v2.13+

You can optionally bind a token to a specific workload by including the workload parameter. This provides additional authentication granularity beyond service tags.

curl -XPOST \
  -H "Content-Type: application/json" \
  --data '{"name": "dp-backend-1", "mesh": "default", "workload": "backend", "validFor": "720h"}' \
  http://localhost:5681/tokens/dataplane

kumactl generate dataplane-token \
  --name dp-backend-1 \
  --mesh default \
  --workload backend \
  --valid-for 720h > /tmp/kuma-dp-backend-1-token

When a token includes a workload value, the data plane proxy must have a matching kuma.io/workload label. This label can be set either:

• In the token itself (as shown above)
• Directly on the data plane proxy resource

If a data plane proxy connects with a workload token, it must have the kuma.io/workload label with a value matching the token. Connection attempts without the matching label are rejected.

If a MeshIdentity resource references the kuma.io/workload label in its SPIFFE ID path template (for example, /workload/{{ label "kuma.io/workload" }}), then:

• The data plane proxy must have the kuma.io/workload label
• The label can be provided either via the token or on the data plane proxy resource
• Connection attempts from data plane proxies selected by that MeshIdentity are denied if they lack the required label

This validation is enforced at connection time.

Store the token in a file and pass it when starting kuma-dp:

kuma-dp run \
  --cp-address=https://127.0.0.1:5678 \
  --dataplane-file=dp-backend.yaml \
  --dataplane-token-file=/tmp/kuma-dp-echo-1-token

You can also pass the data plane proxy token inline as the KUMA_DATAPLANE_RUNTIME_TOKEN environment variable.

Data plane proxy token boundary

You can generate a token by specifying a name, mesh, workload, and a list of tags. The control plane verifies the connecting data plane proxy resource against the token. The scope of a token depends on what you include at generation time:

• Only mesh: The token can be reused for all data planes in that mesh.
• mesh + tag (for example, kuma.io/service): The token can be used across all instances of a given service. You must include all services that the data plane proxy is responsible for. For example, if a Dataplane has two inbounds, one tagged kuma.io/service: backend and one tagged kuma.io/service: backend-admin, you must specify both values (--tag kuma.io/service=backend,backend-admin).
• mesh + name + tag (for example, kuma.io/service): The token is scoped to a single instance of a given service.
• v2.13+ mesh + workload: The token is bound to data plane proxies with a specific kuma.io/workload label value, providing finer-grained authentication than service tags.

Token revocation

Kong Mesh doesn’t keep a list of issued tokens. Whenever a single token is compromised, you can add it to the revocation list to invalidate it.

Every token has its own ID under the jti key. You can extract the ID from the token using jwt.io or the jwt-cli tool.

v2.10+ Authentication between the control plane and data planes is only checked at connection start. This means revoking a token after a data plane connects does not terminate the existing connection. The recommended action on token revocation is to restart either the control plane or the affected data planes.

To revoke tokens, specify a comma-separated list of revoked IDs in a secret named dataplane-token-revocations-default.

REVOCATIONS=$(echo '0e120ec9-6b42-495d-9758-07b59fe86fb9' | base64) && echo "apiVersion: v1
kind: Secret
metadata:
  name: dataplane-token-revocations-default
  namespace: kong-mesh-system
data:
  value: $REVOCATIONS
type: system.kuma.io/secret" | kubectl apply -f -

echo "
type: Secret
mesh: default
name: dataplane-token-revocations-default
data: " | kumactl apply --var revocations=$(echo '0e120ec9-6b42-495d-9758-07b59fe86fb9' | base64) -f -

Signing key rotation

If the signing key is compromised, you must rotate it. You must also rotate all the tokens that were signed by it.

1. Generate a new signing key. The signing key is stored as a secret named in the following format: dataplane-token-signing-key-{mesh}-{serialNumber}.

   When generating a new signing key, assign it a serial number greater than the current key’s serial number.

   Kubernetes Universal

   Get the current key’s serial number:

   kubectl get secrets -n kong-mesh-system --field-selector='type=system.kuma.io/secret'
   

   Copied!

   In this example, the highest serial number is 1:

   NAME                                 TYPE                    DATA   AGE
   dataplane-token-signing-key-default-1   system.kuma.io/secret   1      25m
   

   Copied!

   Generate a new signing key with a serial number of 2:

   TOKEN="$(kumactl generate signing-key)" && echo "
   apiVersion: v1
   data:
     value: $TOKEN
   kind: Secret
   metadata:
     name: dataplane-token-signing-key-default-2
     namespace: kong-mesh-system
   type: system.kuma.io/secret
   " | kubectl apply -f -
   

   Copied!

   Get the current key’s serial number:

   kumactl get secrets
   

   Copied!

   In this example, the highest serial number is 1:

   MESH      NAME                                    AGE
   default   dataplane-token-signing-key-default-1   49s
   

   Copied!

   Generate a new signing key with a serial number of 2:

   echo "
   type: Secret
   mesh: default
   name: dataplane-token-signing-key-default-2
   data: " | kumactl apply --var key=$(kumactl generate signing-key) -f -
   

   Copied!

2. Regenerate tokens. New tokens are automatically signed with the key that has the highest serial number, so they use the new signing key. At this point, tokens signed by either the new or old signing key are valid.

3. Remove the old signing key.

   Kubernetes Universal

   kubectl delete secret dataplane-token-signing-key-default-1 -n kong-mesh-system
   

   Copied!

   kumactl delete secret dataplane-token-signing-key-default-1 --mesh=default
   

   Copied!

   All new connections to the control plane now require tokens signed with the new signing key.

Token rotation

You can configure dynamic token reloading if:

• You need to generate a new token for a data plane proxy.
• You are using Service Account Token projection on Kubernetes.

Set the kuma-cp configuration property dpServer.auth.enableReloadableTokens to true. When enabled, kuma-dp detects changes to the token file, reloads the token, and uses the new value when establishing a new connection to kuma-cp. By default, enableReloadableTokens is enabled on Kubernetes.

Offline token issuing

In addition to the regular flow of generating signing keys, storing them in a secret, and using them to sign tokens on the control plane, Kong Mesh also offers offline signing of tokens.

In this flow, you can generate a pair of public and private keys and configure the control plane only with public keys for token verification. You can generate all the tokens without running the control plane.

The advantages of this mode are:

• It allows for easier and more reproducible deployments of the control plane, and it’s more in line with GitOps.
• It’s a potentially more secure setup, because the control plane doesn’t have access to the private keys.

Here’s how to use offline issuing:

1. Generate a pair of signing keys:

   kumactl generate signing-key --format=pem > /tmp/key-private.pem
   kumactl generate public-key --signing-key-path=/tmp/key-private.pem > /tmp/key-public.pem
   

   Copied!

   These commands generate standard RSA key of 2048 bits and outputs it in PEM-encoded format. You can use any external tool to generate a pair of keys. The result should look like this:

   cat /tmp/key-private.pem /tmp/key-public.pem 
   -----BEGIN RSA PRIVATE KEY-----
   MIIEpAIBAAKCAQEAsS61a79gC4mkr2Ltwi09ajakLyUR8YTkJWzZE805EtTkEn/r
   ...
   htKtzsYA7yGlt364IuDybrP+PlPMSK9cQAmWRRZIcBNsKOODkAgKFA==
   -----END RSA PRIVATE KEY-----
   -----BEGIN RSA PUBLIC KEY-----
   MIIBCgKCAQEAsS61a79gC4mkr2Ltwi09ajakLyUR8YTkJWzZE805EtTkEn/rL2u/
   ...
   se7sx2Pt/NPbWFFTMGVFm3A1ueTUoorW+wIDAQAB
   -----END RSA PUBLIC KEY----- 
   

2. Configure the control plane with the public key.

   Configure a control plane with the following settings:

   dpServer:
     authn:
       dpProxy:
         type: dpToken
         dpToken:
           enableIssuer: false # disable control plane token issuer that uses secrets
           validator:
             useSecrets: false # do not use signing key stored in secrets to validate the token
             publicKeys:
             - kid: "key-1"
               mesh: default
               key: |
                 -----BEGIN RSA PUBLIC KEY-----
                 MIIBCgKCAQEAsS61a79gC4mkr2Ltwi09ajakLyUR8YTkJWzZE805EtTkEn/rL2u/
                 ...
                 se7sx2Pt/NPbWFFTMGVFm3A1ueTUoorW+wIDAQAB
                 -----END RSA PUBLIC KEY-----
   

   Copied!

3. Use the private key to issue tokens offline:

   kumactl generate dataplane-token \
     --name dp-echo-1 \
     --mesh default \
     --tag kuma.io/service=backend,backend-admin \
     --valid-for 720h > /tmp/kuma-dp-echo1-token \
     --signing-key-path /tmp/key-private.pem \
     --kid key-1
   

   Copied!

   The command is the same as with online signing, but with two additional arguments:

   • --kid: The ID of the key that should be used to validate the token. This should match kid specified in the control plane configuration.
   • --signing-key-path: The path to a PEM-encoded private key.

   You can also use any external system that can issue JWT tokens using RS256 signing method with the following claims:

   • Name: The name of the data plane proxy.
   • Mesh: The name of the mesh.
   • Tags: A map of string to list of strings representing the tags.
   • Workload: The workload label value for the data plane proxy. v2.13+

Migration

You can use both offline and online issuing by setting dpServer.authn.dpProxy.dpToken.enableIssuer to true. You can use both secrets and public key static config validators by setting dpServer.authn.dpProxy.dpToken.validator.useSecrets to true.

Management

Token revocation works the same when using both online and offline issuing.

Signing key rotation works similarly:

1. Generate another pair of signing keys.
2. Configure a control plane with old and new public keys.
3. Regenerate all tokens with the new private key.
4. Remove the old public key from the configuration.

Multi-zone

In multi-zone mode, you can generate a data plane proxy token on either the global or a zone control plane. If your deployment pipeline generates tokens before starting the proxy, you can use the Zone CP so the Global CP isn’t a single point of failure. Signing key rotation and token revocation must be performed on the global control plane.

Workload label validation errors v2.13+

When using workload labels in tokens or MeshIdentity resources, you may encounter the following errors.

Missing required workload label

missing required label 'kuma.io/workload' - dataplane is selected by MeshIdentity 'backend-identity' with path template '/workload/{{ label "kuma.io/workload" }}'

Cause: A MeshIdentity resource uses {{ label "kuma.io/workload" }} in its SPIFFE ID path template, but the connecting data plane proxy lacks the kuma.io/workload label.

Solution: Add the kuma.io/workload label to your data plane proxy. You can do this by:

• Generating a token with the --workload parameter (the label will be set automatically from the token)
• Setting the label directly on the data plane proxy resource

Workload mismatch between token and data plane proxy

dataplane workload 'frontend' is not allowed with this token. Allowed workload in token is 'backend'

Cause: The data plane proxy token specifies a workload value, but the data plane proxy is configured with a different kuma.io/workload label value.

Solution: Ensure consistency between the token and data plane proxy configuration:

• Regenerate the token with the correct --workload value matching your data plane proxy
• Update the data plane proxy resource to use the workload value that matches the token
• If the data plane proxy resource explicitly sets a different workload label, remove it to use the token’s value

Missing workload label with workload token

dataplane has no workload label required by the token

Cause: The token was generated with a workload value, but the data plane proxy resource does not have the kuma.io/workload label.

Solution: The workload label should be automatically applied from the token. This error typically indicates a configuration issue. Verify:

• The token was generated correctly with the --workload parameter
• The token file is valid and not corrupted
• The data plane proxy is using the correct token file

You can disable authentication by setting KUMA_DP_SERVER_AUTH_TYPE to none.

Disabling authentication between the control plane and data plane proxies allows any data plane proxy to impersonate any service. This is not recommended in production environments.