Backend clusters

Uses: Event Gateway Konnect API Terraform

What is a backend cluster?

A backend cluster is an abstraction of a real Kafka cluster. It stores the connection and configuration details required for Kong Event Gateway to proxy traffic to Kafka.

Multiple Kafka clusters can be proxied through a single Kong Event Gateway. The Event Gateway control plane manages information such as:

Authentication credentials for connecting to Kafka clusters
TLS verification preferences
Metadata refresh intervals for fetching cluster information

 
flowchart LR
    
    A[Kafka 
    client]
    B[Listener 
    &#40TCP socket&#41
    + listener policies]
    C@{ shape: processes, label: "Virtual clusters
    + consume, produce, 
    and cluster policies"}
    D[Backend 
    cluster]
    E[Kafka 
    cluster]

    A --> B
    subgraph id1 [Event Gateway]
    B --> C 
    C --> D
    end
    D --> E

    style D stroke:#86e2cc

    style id1 rx:7,ry:7

Authentication

Authentication on the backend cluster defines how the proxy connects to the backend for capturing metadata (topics, consumer groups, and so on).

The backend cluster supports multiple authentication methods and can mediate authentication between clients and backend clusters. See Virtual cluster authentication to learn more.

Backend clusters support the following auth methods:

Auth method (`authentication.type`)	Description	Credential mediation types (`authentication.mediation`)
Anonymous	Doesn’t require clients to provide any authentication when connecting to the proxy.	None
SASL/PLAIN	Requires clients to provide a username and password. Accepts a hardcoded list of usernames and passwords, either as strings or environment variables.	`passthrough`, `terminate`
SASL/SCRAM-SHA-256	Requires clients to provide a username and password using SCRAM-SHA-256 hashing.	`passthrough`
SASL/SCRAM-SHA-512	Requires clients to provide a username and password using SCRAM-SHA-512 hashing.	`passthrough`

Set up a backend cluster

curl -X POST https://{region}.api.konghq.com/v1/event-gateways/{eventGatewayId}/backend-clusters \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "example-backend-cluster",
      "bootstrap_servers": [
        "host:9092"
      ],
      "authentication": {
        "type": "anonymous"
      },
      "insecure_allow_anonymous_virtual_cluster_auth": true,
      "tls": {
        "insecure_skip_verify": false
      }
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
virtualClusterId: The id of the Virtual Cluster.
eventGatewayId: The id of the Event Gateway.
eventGatewayListenerId: The id of the Event Gateway Listener.

See the Konnect Event Gateway API reference to learn about region-specific URLs and personal access tokens.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect-beta = {
      source  = "kong/konnect-beta"
    }
  }
}

provider "konnect-beta" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

resource "konnect_event_gateway_backend_cluster" "my_backend_cluster" {
  provider = konnect-beta
  bootstrap_servers = ["host:9092"]
  authentication = {
    type = "anonymous"
  }
  insecure_allow_anonymous_virtual_cluster_auth = true
  tls = {
    insecure_skip_verify = false
  }



  gateway_id = konnect_event_gateway.my_event_gateway.id
}

Copied!

The following creates a new backend cluster called example-backend-cluster with basic configuration:

In Konnect, navigate to Event Gateway in the sidebar.
Click an Event Gateway.
Navigate to Backend Clusters in the sidebar.
Click New backend cluster.
In the Name field, enter example-backend-cluster.
In the Bootstrap servers field, enter host:9092.
From the Authentication dropdown menu, select anonymous
Select the Allow anonymous authentication from virtual clusters checkbox.
Click Save.

Backend clusters

What is a backend cluster?

Authentication

Set up a backend cluster

Schema

Help us make these docs great!

Still need help