Home / Kong Mesh
Edit
Report

Deploy mesh-scoped zone proxies

Uses: Kong Mesh
Incompatible with
konnect
Related Documentation
Mesh Documentation
In This Series
  1. Deploy mesh-scoped zone proxies
  2. Apply policies to mesh-scoped zone proxies
Related Resources
Multi-zone deployment
Minimum Version
Kong Mesh - 2.14
TL;DR
  1. Create a Mesh with spec.meshServices.mode: Exclusive and a MeshIdentity on the global control plane.
  2. Install each zone control plane with a kuma.meshes[] entry.
  3. Kong Mesh renders a per-mesh Deployment and Service for each role, and generates the Dataplane listeners automatically.

Prerequisites

Helm

You will need Helm, a package manager for Kubernetes.

minikube

This series requires minikube with the Docker driver to run three local Kubernetes clusters (one global and two zones).

Starting in Kong Mesh 2.14, zone ingresses and zone egresses are mesh-scoped. Declare them in your zone control plane values under kuma.meshes[]. Each entry creates a Deployment, Service, and Dataplane for that mesh, and the zone proxies carry per-mesh workload identities so policies can target them directly.

This guide walks through a three-cluster setup: a global control plane and two zone control planes, each deploying a zone ingress and zone egress through kuma.meshes[]. It uses the minikube Docker driver and one shared Docker bridge so the three clusters can reach each other directly through NodePort Services.

Start the global control plane cluster

  1. Export the profile names, the shared Docker network, the static node IPs, and the fixed KDS NodePort:

    export MZ_NETWORK=guide-mz-net
export GLOBAL_PROFILE=guide-mz-global
export ZONE1_PROFILE=guide-mz-zone-1
export ZONE2_PROFILE=guide-mz-zone-2
export GLOBAL_IP=192.168.240.11
export ZONE1_IP=192.168.240.21
export ZONE2_IP=192.168.240.31
export KDS_NODEPORT=30685

  2. Create the shared Docker bridge:

    docker network inspect $MZ_NETWORK >/dev/null 2>&1 || \
  docker network create --driver bridge --subnet 192.168.240.0/24 $MZ_NETWORK

    If Docker Desktop fails with failed complete: too many open files while starting a profile, run:

    docker run --rm --privileged alpine \
  sysctl -w fs.inotify.max_user_instances=8192 fs.inotify.max_user_watches=524288

    Then restart the affected profile.

  3. Create a new minikube cluster for the global control plane:

    minikube start -p $GLOBAL_PROFILE --driver=docker --network=$MZ_NETWORK --static-ip=$GLOBAL_IP

Deploy the global control plane

  1. Install the global control plane:

    helm repo add kong-mesh https://kong.github.io/kong-mesh-charts
helm repo update
helm install --kube-context $GLOBAL_PROFILE --create-namespace --namespace kong-mesh-system \
  --set kuma.controlPlane.mode=global \
  --set kuma.controlPlane.defaults.skipMeshCreation=true \
  --set kuma.controlPlane.globalZoneSyncService.type=NodePort \
  --set kuma.controlPlane.globalZoneSyncService.nodePort=$KDS_NODEPORT \
  kong-mesh kong-mesh/kong-mesh

    We’re skipping default mesh creation because we’ll apply a custom Mesh in the next step.

  2. Wait for the control plane to become ready:

    kubectl --context $GLOBAL_PROFILE -n kong-mesh-system wait \
  --for=condition=ready pod --selector=app=kong-mesh-control-plane --timeout=120s

  3. Export the KDS address that the zone control planes will connect to:

    export KDS_ADDRESS=grpcs://${GLOBAL_IP}:${KDS_NODEPORT}

    The global control plane is now reachable at ${GLOBAL_IP}:${KDS_NODEPORT} from the other two minikube clusters.

Create the mesh on the global control plane

Zone proxy listeners are only generated when the mesh uses the MeshService exclusive mode. Without this, the zone proxies install but produce no listeners.

  1. Create the mesh and allow all traffic:

    echo 'apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
  name: default
spec:
  meshServices:
    mode: Exclusive
---
apiVersion: kuma.io/v1alpha1
kind: MeshTrafficPermission
metadata:
  name: allow-all
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  rules:
    - default:
        allow:
          - spiffeID:
              type: Prefix
              value: "spiffe://default."' | kubectl --context $GLOBAL_PROFILE apply -f -

    This rule allows traffic from any workload identity whose SPIFFE trust domain starts with default.. That includes the mesh-scoped zone proxies and the demo workloads in both zones.

Create a MeshIdentity

Zone egress listeners need a workload identity to terminate mTLS for cross-zone traffic. Apply a MeshIdentity on the global control plane:

echo 'apiVersion: kuma.io/v1alpha1
kind: MeshIdentity
metadata:
  name: identity
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  selector:
    dataplane:
      matchLabels: {}
  spiffeID:
    trustDomain: "{{ .Mesh }}.{{ .Zone }}.mesh.local"
  provider:
    type: Bundled
    bundled:
      meshTrustCreation: Enabled
      insecureAllowSelfSigned: true
      certificateParameters:
        expiry: 24h
      autogenerate:
        enabled: true' | kubectl --context $GLOBAL_PROFILE apply -f -

The resource will sync to every zone automatically.

insecureAllowSelfSigned: true keeps the demo simple by using the bundled CA. For production, follow the MeshIdentity guide to integrate a SPIRE trust domain or an external CA.

Deploy zone-1 with mesh-scoped zone proxies

A kuma.meshes[] entry defines which mesh the zone proxies belong to. Each entry creates its own Deployment, Service, and ServiceAccount for the ingress and egress roles.

  1. Start the zone-1 cluster on the shared Docker bridge:

    minikube start -p $ZONE1_PROFILE --driver=docker --network=$MZ_NETWORK --static-ip=$ZONE1_IP

  2. Create the values file for zone-1:

    cat <<EOF > zone-1-values.yaml
kuma:
  controlPlane:
    mode: zone
    zone: zone-1
    kdsGlobalAddress: ${KDS_ADDRESS}
    tls:
      kdsZoneClient:
        skipVerify: true
  meshes:
    - name: default
      ingress:
        enabled: true
        service:
          type: NodePort
      egress:
        enabled: true
EOF

    Set the ingress Service type to NodePort so the other minikube clusters can reach it on the shared Docker network. Kubernetes assigns the port automatically, and MeshZoneAddress advertises the resulting ${ZONE1_IP}:<nodePort> address for you.

    To deploy zone proxies for additional meshes, append more entries to kuma.meshes.

  3. Install the zone control plane together with the zone ingress and egress for default:

    helm install --kube-context $ZONE1_PROFILE --create-namespace --namespace kong-mesh-system \
  -f zone-1-values.yaml \
  kong-mesh kong-mesh/kong-mesh

Deploy zone-2

  1. Start the zone-2 cluster on the shared Docker bridge:

    minikube start -p $ZONE2_PROFILE --driver=docker --network=$MZ_NETWORK --static-ip=$ZONE2_IP

  2. Create the values file for zone-2:

    cat <<EOF > zone-2-values.yaml
kuma:
  controlPlane:
    mode: zone
    zone: zone-2
    kdsGlobalAddress: ${KDS_ADDRESS}
    tls:
      kdsZoneClient:
        skipVerify: true
  meshes:
    - name: default
      ingress:
        enabled: true
        service:
          type: NodePort
      egress:
        enabled: true
EOF

    Zone-2 follows the same pattern. Kubernetes allocates the ingress NodePort, and MeshZoneAddress publishes the resulting ${ZONE2_IP}:<nodePort> endpoint.

  3. Install the zone control plane and its zone proxies:

    helm install --kube-context $ZONE2_PROFILE --create-namespace --namespace kong-mesh-system \
  -f zone-2-values.yaml \
  kong-mesh kong-mesh/kong-mesh

Propagate trust between zones

Each zone generates a MeshTrust containing its local CA bundle. The MeshIdentity controller appends a content hash to the trust name (for example, identity-xf4d5dz5c4w47645), so you should look it up by label rather than hardcoding the name. For cross-zone mTLS to work, each zone must trust the other zone’s CA. Publish each zone’s trust bundle to the global control plane so it syncs everywhere.

  1. Export zone-1’s trust bundle and apply it to the global CP:

    kubectl --context $ZONE1_PROFILE -n kong-mesh-system get meshtrust -o json | \
  jq '.items[] | select(.metadata.labels["kuma.io/origin"] == "zone") |
      .metadata.name = "trust-of-zone-1" |
      .metadata.labels["kuma.io/origin"] = "global" |
      del(.metadata.resourceVersion, .metadata.uid, .metadata.creationTimestamp, .metadata.generation, .metadata.ownerReferences)' | \
  kubectl --context $GLOBAL_PROFILE apply -f -

    The jq filter selects only resources where kuma.io/origin: zone, which is the trust the local zone created. Resources synced back from the global control plane carry kuma.io/origin: global and must be excluded, otherwise you would publish the wrong zone’s CA.

  2. Export zone-2’s trust bundle and apply it to the global CP:

    kubectl --context $ZONE2_PROFILE -n kong-mesh-system get meshtrust -o json | \
  jq '.items[] | select(.metadata.labels["kuma.io/origin"] == "zone") |
      .metadata.name = "trust-of-zone-2" |
      .metadata.labels["kuma.io/origin"] = "global" |
      del(.metadata.resourceVersion, .metadata.uid, .metadata.creationTimestamp, .metadata.generation, .metadata.ownerReferences)' | \
  kubectl --context $GLOBAL_PROFILE apply -f -

The global control plane syncs these trust bundles to all zones, enabling cross-zone certificate validation.

Verify cross-zone traffic

  1. Create the demo app configuration:

    cat <<'EOF' > demo.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: kong-mesh-demo
  labels:
    kuma.io/sidecar-injection: enabled
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: kong-mesh-demo
spec:
  selector:
    matchLabels:
      app: redis
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - name: redis
          image: "redis"
          ports:
            - name: tcp
              containerPort: 6379
          lifecycle:
            preStop:
              exec:
                command: ["/bin/sleep", "30"]
            postStart:
              exec:
                command:
                  - /bin/sh
                  - -c
                  - |
                    for i in $(seq 1 30); do
                      if /usr/local/bin/redis-cli ping >/dev/null 2>&1; then
                        /usr/local/bin/redis-cli set zone local && exit 0
                      fi
                      sleep 1
                    done
                    echo "Redis not ready after 30s, skipping postStart"
                    exit 0
---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: kong-mesh-demo
  labels:
    app: redis
spec:
  selector:
    app: redis
  ports:
  - protocol: TCP
    port: 6379
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app
  namespace: kong-mesh-demo
spec:
  selector:
    matchLabels:
      app: demo-app
  replicas: 1
  template:
    metadata:
      labels:
        app: demo-app
    spec:
      containers:
        - name: demo-app
          image: "kumahq/kuma-demo"
          env:
            - name: REDIS_HOST
              value: "redis.kong-mesh-demo.svc.cluster.local"
            - name: REDIS_PORT
              value: "6379"
            - name: APP_VERSION
              value: "1.0"
            - name: APP_COLOR
              value: "#efefef"
          ports:
            - name: http
              containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
  name: demo-app
  namespace: kong-mesh-demo
  labels:
    app: demo-app
spec:
  selector:
    app: demo-app
  ports:
  - protocol: TCP
    appProtocol: http
    port: 5000
EOF

  2. Deploy the Kong Mesh demo app into each zone:

    for ctx in $ZONE1_PROFILE $ZONE2_PROFILE; do
  kubectl --context $ctx apply -f demo.yaml
done

  3. Wait for the demo app to become ready in both zones:

    for ctx in $ZONE1_PROFILE $ZONE2_PROFILE; do
  kubectl --context $ctx -n kong-mesh-demo wait \
    --for=condition=available deployment --all --timeout=120s
done

  4. From a demo-app pod in zone-1, request the demo-app Service in zone-2 using its cross-zone hostname:

    kubectl --context $ZONE1_PROFILE -n kong-mesh-demo exec deploy/demo-app -c demo-app -- \
  wget -qO /dev/null http://demo-app.kong-mesh-demo.svc.zone-2.mesh.local:5000/

    The request leaves zone-1 through the zone egress, enters zone-2 through the zone ingress, and hits the demo-app pod there.

    If the request times out, re-check the MeshZoneAddress output and confirm the ingress Services still publish the expected NodePort values.

  5. Reset the stats counters on the zone-2 ingress:

    kubectl --context $ZONE2_PROFILE -n kong-mesh-system \
  exec deploy/kong-mesh-default-ingress -c kuma-sidecar -- \
  wget -qO- --post-data='' 'http://127.0.0.1:9902/reset_counters'

  6. Send the cross-zone request again:

    kubectl --context $ZONE1_PROFILE -n kong-mesh-demo exec deploy/demo-app -c demo-app -- \
  wget -qO /dev/null http://demo-app.kong-mesh-demo.svc.zone-2.mesh.local:5000/

  7. Inspect the ingress Envoy cluster stats:

    kubectl --context $ZONE2_PROFILE -n kong-mesh-system \
  exec deploy/kong-mesh-default-ingress -c kuma-sidecar -- \
  wget -qO- 'http://127.0.0.1:9902/stats?format=json&filter=demo-app' | \
  jq '.stats[] | select(.name | strings | test("upstream_rq_total"))'

    upstream_rq_total for the zone-2 demo-app ingress cluster should be greater than zero. The follow-up guide shows how to inspect zone-egress-specific traffic on top of this setup.

FAQs

How do I list the per-mesh Deployments and Services?

Query kong-mesh-system filtered by mesh label:

kubectl --context $ZONE1_PROFILE -n kong-mesh-system get deploy,svc -l kuma.io/mesh=default

You’ll see a kong-mesh-default-ingress and kong-mesh-default-egress Deployment, each with a matching Service. A second mesh produces another pair named after its mesh.

How do I confirm the Services carry the zone-proxy-type label?

 
kubectl --context $ZONE1_PROFILE -n kong-mesh-system get svc \
  -l k8s.kuma.io/zone-proxy-type -L k8s.kuma.io/zone-proxy-type

How do I confirm the ingress Services are NodePort?

Run the following for each zone and note the published port:

kubectl --context $ZONE1_PROFILE -n kong-mesh-system get svc kong-mesh-default-ingress \
  -o jsonpath='{.spec.type}{"\t"}{.spec.ports[0].nodePort}{"\n"}'
kubectl --context $ZONE2_PROFILE -n kong-mesh-system get svc kong-mesh-default-ingress \
  -o jsonpath='{.spec.type}{"\t"}{.spec.ports[0].nodePort}{"\n"}'

How do I inspect the MeshZoneAddress resources?

MeshZoneAddress resources are namespaced, so query kong-mesh-system explicitly:

kubectl --context $GLOBAL_PROFILE -n kong-mesh-system get meshzoneaddress -o json | \
  jq -r '.items[] |
    [
      .metadata.labels["kuma.io/zone"],
      .metadata.name,
      (.spec.address + ":" + (.spec.port | tostring))
    ] | @tsv'

The addresses should use the shared Docker bridge IPs and the published ingress NodePort values.

How do I view the zone proxy Dataplane resources?

 
kubectl --context $GLOBAL_PROFILE get dataplane -A
Next → 2. Apply policies to mesh-scoped zone proxies
Something wrong?
Report an Issue | Edit this Page
View all Kong documentation

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help?

Ask in our Slack Community
Contact Support