Home / How-to Guides
Edit
Report

Federate a zone control plane

Uses: Kong Mesh
Related Documentation
Mesh Documentation
Related Resources
Add a builtin gateway
Producer and Consumer policies
Multi-zone deployment
Minimum Version
Kong Mesh - 2.6
TL;DR
  1. Create a zone control plane and a global control plane in separate Kubernetes clusters.
  2. Copy resources from the zone control plane to the global control plane.
  3. Connect the two control planes by updating the zone control plane’s Helm deployment.

Prerequisites

Helm

You will need Helm, a package manager for Kubernetes.

Install kumactl

  1. Go to the Kong Mesh packages page to download and extract the installation archive for your OS, or download and extract the latest release automatically (Linux or macOS):

    curl -L https://developer.konghq.com/mesh/installer.sh | VERSION=2.13.0 sh -

  2. Add the Kong Mesh binaries directory to your path. By default, the directory is /kong-mesh-2.13.0/bin. You can use the following command to set the directory in your path for the current terminal window:

    export PATH=$PATH:$(pwd)/kong-mesh-2.13.0/bin

A running Kubernetes cluster

This guide requires a running Kubernetes cluster. If you already have a Kubernetes cluster running, you can skip this step. It can be a cluster running locally, like Docker, or in a public cloud like AWS EKS, GCP GKE, etc.

For example, if you are using minikube:

minikube start -p mesh-zone

Install Kong Mesh with demo configuration

  1. Install Kong Mesh:

    helm upgrade \
  --install \
  --create-namespace \
  --namespace kong-mesh-system \
  kong-mesh kong-mesh/kong-mesh
kubectl wait -n kong-mesh-system --for=condition=ready pod --selector=app=kong-mesh-control-plane --timeout=90s

  2. Apply the demo configuration:

    echo "
apiVersion: v1
kind: Namespace
metadata:
  labels:
    kuma.io/sidecar-injection: enabled
  name: kong-mesh-demo
---
apiVersion: v1
kind: Service
metadata:
  name: demo-app
  namespace: kong-mesh-demo
spec:
  ports:
  - appProtocol: http
    port: 5050
    protocol: TCP
    targetPort: 5050
  selector:
    app: demo-app
---
apiVersion: v1
kind: Service
metadata:
  name: demo-app-v1
  namespace: kong-mesh-demo
spec:
  ports:
  - appProtocol: http
    port: 5050
    protocol: TCP
    targetPort: 5050
  selector:
    app: demo-app
    version: v1
---
apiVersion: v1
kind: Service
metadata:
  name: demo-app-v2
  namespace: kong-mesh-demo
spec:
  ports:
  - appProtocol: http
    port: 5050
    protocol: TCP
    targetPort: 5050
  selector:
    app: demo-app
    version: v2
---
apiVersion: v1
kind: Service
metadata:
  name: kv
  namespace: kong-mesh-demo
spec:
  ports:
  - appProtocol: http
    port: 5050
    protocol: TCP
    targetPort: 5050
  selector:
    app: kv
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demo-app
    version: v1
  name: demo-app
  namespace: kong-mesh-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-app
      version: v1
  template:
    metadata:
      labels:
        app: demo-app
        version: v1
    spec:
      containers:
      - env:
        - name: OTEL_SERVICE_NAME
          value: demo-app
        - name: OTEL_EXPORTER_OTLP_ENDPOINT
          value: http://opentelemetry-collector.mesh-observability:4317
        - name: KV_URL
          value: http://kv.kong-mesh-demo.svc.cluster.local:5050
        - name: APP_VERSION
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        image: ghcr.io/kumahq/kuma-counter-demo:latest@sha256:daf8f5cffa10b576ff845be84e4e3bd5a8a6470c7e66293c5e03a148f08ac148
        name: app
        ports:
        - containerPort: 5050
          name: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demo-app
    version: v2
  name: demo-app-v2
  namespace: kong-mesh-demo
spec:
  replicas: 0
  selector:
    matchLabels:
      app: demo-app
      version: v2
  template:
    metadata:
      labels:
        app: demo-app
        version: v2
    spec:
      containers:
      - env:
        - name: OTEL_SERVICE_NAME
          value: demo-app
        - name: OTEL_EXPORTER_OTLP_ENDPOINT
          value: http://opentelemetry-collector.mesh-observability:4317
        - name: KV_URL
          value: http://kv.kong-mesh-demo.svc.cluster.local:5050
        - name: APP_VERSION
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        image: ghcr.io/kumahq/kuma-counter-demo:latest@sha256:daf8f5cffa10b576ff845be84e4e3bd5a8a6470c7e66293c5e03a148f08ac148
        name: demo-app
        ports:
        - containerPort: 5050
          name: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kv
  namespace: kong-mesh-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kv
  template:
    metadata:
      labels:
        app: kv
    spec:
      containers:
      - env:
        - name: OTEL_SERVICE_NAME
          value: kv
        - name: OTEL_EXPORTER_OTLP_ENDPOINT
          value: http://opentelemetry-collector.mesh-observability:4317
        - name: APP_VERSION
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        image: ghcr.io/kumahq/kuma-counter-demo:latest@sha256:daf8f5cffa10b576ff845be84e4e3bd5a8a6470c7e66293c5e03a148f08ac148
        name: app
        ports:
        - containerPort: 5050
          name: http
---
apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
  name: default
spec:
  meshServices:
    mode: Exclusive
  mtls:
    backends:
    - name: ca-1
      type: builtin
    enabledBackend: ca-1
---
apiVersion: kuma.io/v1alpha1
kind: MeshTrafficPermission
metadata:
  name: kv
  namespace: kong-mesh-demo
spec:
  from:
  - default:
      action: Allow
    targetRef:
      kind: MeshSubset
      tags:
        app: demo-app
        k8s.kuma.io/namespace: kong-mesh-demo
  targetRef:
    kind: Dataplane
    labels:
      app: kv" | kubectl apply -f -

Start a new Kubernetes cluster for the global control plane

We’ve created a zone control plane in the prerequisites, now we need a global control plane. The zone a global control planes can’t be in the same Kubernetes cluster, so we must start by creating a new cluster:

minikube start -p mesh-global

Use the minikube tunnel feature to provision local load balancer addresses:

nohup minikube tunnel -p mesh-global &

Deploy the global control plane

Run the following command to deploy a global control plane:

helm install --kube-context mesh-global --create-namespace --namespace kong-mesh-system \
--set kuma.controlPlane.mode=global \
--set kuma.controlPlane.defaults.skipMeshCreation=true \
kong-mesh kong-mesh/kong-mesh

We’ll skip the default mesh creation since we’ll bring the mesh from the zone control plane in the next steps.

Set up kumactl

Before we start migrating, we need to set up kumactl, which we’ll use to export resources.

  1. Run the following command to expose the control plane’s API server. We’ll need this to access kumactl:

    kubectl --context mesh-zone port-forward svc/kong-mesh-control-plane -n kong-mesh-system 5681:5681

  2. In a new terminal, check that kumactl is installed and that its directory is in your path:

    kumactl

    If the command is not found:

    1. Make sure that kumactl is installed

    2. Add the Kong Mesh binaries directory to your path:

      export PATH=$PATH:$(pwd)/kong-mesh-2.13.0/bin

  3. Export your admin token and add your control plane:

    export ZONE_USER_ADMIN_TOKEN=$(kubectl --context mesh-zone get secrets -n kong-mesh-system admin-user-token -o json | jq -r .data.value | base64 -d)
kumactl config control-planes add \
  --address http://localhost:5681 \
  --headers "authorization=Bearer $ZONE_USER_ADMIN_TOKEN" \
  --name "my-cp" \
  --overwrite

Copy resources from the zone control plane to the global control plane
  1. Export the external IP to use to access the global control plane: 
    export EXTERNAL_IP=host.minikube.internal

    If you’re not using minikube, you can find your external IP with this command:

    export EXTERNAL_IP=$(kubectl --context mesh-global get svc -n kong-mesh-system kong-mesh-global-zone-sync -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

  2. Export the zone control plane resources:

    kumactl export --profile federation-with-policies --format kubernetes > resources.yaml

  3. Apply the resources to the global control plane:

    kubectl apply --context mesh-global -f resources.yaml

Connect the control planes

Update the zone control plane’s Helm deployment to configure the connection to the global control plane:

helm upgrade --kube-context mesh-zone --namespace kong-mesh-system \
--set kuma.controlPlane.mode=zone \
--set kuma.controlPlane.zone=zone-1 \
--set kuma.ingress.enabled=true \
--set kuma.controlPlane.kdsGlobalAddress=grpcs://$EXTERNAL_IP:5685 \
--set kuma.controlPlane.tls.kdsZoneClient.skipVerify=true \
kong-mesh kong-mesh/kong-mesh

Validate

  1. To validate the federation, start by port-forwarding the API service from the global control plane to port 15681 to avoid collision with previous port-forward:

    kubectl --context mesh-global port-forward svc/kong-mesh-control-plane -n kong-mesh-system 15681:5681

  2. In a browser, go to http://127.0.0.1:15681/gui/ to see the GUI.

    You should see:

    • A zone in list of zones
    • Policies, including the MeshTrafficPermission that we applied in the prerequisites
    • Data plane proxies for the demo application that we installed in the prerequisites

    It can take some time for these to appear, if you don’t see them immediately, wait a few minutes and try again.

  3. Create the kong-mesh-demo namespace in the global control plane:

    kubectl --context mesh-global create namespace kong-mesh-demo

  4. Apply a policy on the global control plane:

    echo "apiVersion: kuma.io/v1alpha1
kind: MeshCircuitBreaker
metadata:
  name: demo-app-to-redis
  namespace: kong-mesh-demo
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Dataplane
    labels:
      app: demo-app
  to:
  - targetRef:
      kind: MeshService
      name: kv
    default:
      connectionLimits:
        maxConnections: 2
        maxPendingRequests: 8
        maxRetries: 2
        maxRequests: 2" | kubectl --context mesh-global apply -f -

  5. Check that the policy is applied on the zone control plane:

    kubectl get --context mesh-zone meshcircuitbreakers -A

    You should get the following result:

    NAMESPACE          NAME                                                TARGETREF KIND   TARGETREF NAME
kong-mesh-system   demo-app-to-redis-65xb45x2xfd5bf7f                  Dataplane        
kong-mesh-system   mesh-circuit-breaker-all-default                    Mesh             
kong-mesh-system   mesh-circuit-breaker-all-default-d6zfxc24v7449xfv   Mesh

Cleanup

Clean up kumactl control plane
  1. Get a list of control planes: 
    kumactl config control-planes list
  2. Remove the control planes that are no longer needed: 
    kumactl config control-planes remove --name my-cp

Clean up Kong Mesh resources

To clean up your environment, remove the Docker containers, network, temporary directory, and the control plane configuration. Run the following command:

minikube delete --profile mesh-zone
minikube delete --profile mesh-global
Something wrong?
Report an Issue | Edit this Page

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help

Ask in our Forum
Contact Support