Create a backend cluster with authentication.type: sasl_plain and supply a username and password.
Install kafkactl. You’ll need it to interact with Kafka clusters.
If you don’t have a Konnect account, you can get started quickly with our onboarding wizard.
Set the personal access token as an environment variable:
export KONNECT_TOKEN='YOUR KONNECT TOKEN'
In this guide you’ll configure Event Gateway to connect to a secured Kafka cluster using SASL/PLAIN credentials.
flowchart LR
C[Kafka Client]
subgraph EG [" Event Gateway "]
VC1[sasl virtual cluster]
end
subgraph K [Kafka Cluster]
L1["SASL_PLAINTEXT :9082"]
end
C -->|anonymous| VC1
VC1 -->|SASL/PLAIN| L1
Create the JAAS configuration file that defines the SASL/PLAIN credentials:
cat <<'EOF' > kafka_server_jaas.conf
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="gateway"
password="gateway-secret"
user_gateway="gateway-secret";
};
EOF
Create the Docker Compose file:
cat <<'EOF' > docker-compose.yaml
name: kafka_cluster
networks:
kafka:
name: kafka_event_gateway
services:
kafka1:
image: apache/kafka:4.2.0
networks:
- kafka
container_name: kafka1
ports:
- "9094:9094"
environment:
KAFKA_NODE_ID: 0
KAFKA_PROCESS_ROLES: broker,controller
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_LISTENERS: INTERNAL://kafka1:9092,CONTROLLER://kafka1:9093,EXTERNAL://0.0.0.0:9094,SASL_PLAINTEXT://0.0.0.0:9082
KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka1:9092,EXTERNAL://localhost:9094,SASL_PLAINTEXT://kafka1:9082
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
KAFKA_CONTROLLER_QUORUM_VOTERS: 0@kafka1:9093,1@kafka2:9093,2@kafka3:9093
KAFKA_CLUSTER_ID: 'abcdefghijklmnopqrstuv'
KAFKA_LOG_DIRS: /tmp/kraft-combined-logs
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
volumes:
- ./kafka_server_jaas.conf:/etc/kafka/kafka_server_jaas.conf
kafka2:
image: apache/kafka:4.2.0
networks:
- kafka
container_name: kafka2
ports:
- "9095:9095"
environment:
KAFKA_NODE_ID: 1
KAFKA_PROCESS_ROLES: broker,controller
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_LISTENERS: INTERNAL://kafka2:9092,CONTROLLER://kafka2:9093,EXTERNAL://0.0.0.0:9095,SASL_PLAINTEXT://0.0.0.0:9082
KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka2:9092,EXTERNAL://localhost:9095,SASL_PLAINTEXT://kafka2:9082
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
KAFKA_CONTROLLER_QUORUM_VOTERS: 0@kafka1:9093,1@kafka2:9093,2@kafka3:9093
KAFKA_CLUSTER_ID: 'abcdefghijklmnopqrstuv'
KAFKA_LOG_DIRS: /tmp/kraft-combined-logs
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
volumes:
- ./kafka_server_jaas.conf:/etc/kafka/kafka_server_jaas.conf
kafka3:
image: apache/kafka:4.2.0
networks:
- kafka
container_name: kafka3
ports:
- "9096:9096"
environment:
KAFKA_NODE_ID: 2
KAFKA_PROCESS_ROLES: broker,controller
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_LISTENERS: INTERNAL://kafka3:9092,CONTROLLER://kafka3:9093,EXTERNAL://0.0.0.0:9096,SASL_PLAINTEXT://0.0.0.0:9082
KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka3:9092,EXTERNAL://localhost:9096,SASL_PLAINTEXT://kafka3:9082
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
KAFKA_CONTROLLER_QUORUM_VOTERS: 0@kafka1:9093,1@kafka2:9093,2@kafka3:9093
KAFKA_CLUSTER_ID: 'abcdefghijklmnopqrstuv'
KAFKA_LOG_DIRS: /tmp/kraft-combined-logs
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
volumes:
- ./kafka_server_jaas.conf:/etc/kafka/kafka_server_jaas.conf
EOF
The broker exposes a SASL_PLAINTEXT listener on port 9082 in the Docker network for Event Gateway connections, and a PLAINTEXT listener on ports 9094/9095/9096 for direct local access.
Start the cluster:
docker compose up -d
Run the quickstart script to provision a local data plane and configure your environment:
curl -Ls https://get.konghq.com/event-gateway | bash -s -- -k $KONNECT_TOKEN -N kafka_event_gateway
Copy the exported variable into your terminal:
export EVENT_GATEWAY_ID=your-gateway-id
This quickstart script is meant for demo purposes only, therefore it runs locally with most default parameters and a small number of exposed ports. If you want to run Kong Gateway as a part of a production-ready platform, set up your control plane and data planes through the Konnect UI, or using Terraform.
Create a kafkactl.yaml config file with contexts for direct Kafka access and the SASL virtual cluster:
cat <<EOF > kafkactl.yaml
contexts:
direct:
brokers:
- localhost:9094
- localhost:9095
- localhost:9096
sasl-vc:
brokers:
- localhost:19092
EOF
Create a test topic using the direct context:
kafkactl -C kafkactl.yaml --context direct create topic orders
Create a backend cluster with sasl_plain authentication and the credentials you want to use:
SASL_BACKEND_CLUSTER_ID=$(curl -X POST "https://us.api.konghq.com/v1/event-gateways/$EVENT_GATEWAY_ID/backend-clusters" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "sasl_backend_cluster",
"bootstrap_servers": [
"kafka1:9082",
"kafka2:9082",
"kafka3:9082"
],
"authentication": {
"type": "sasl_plain",
"username": "gateway",
"password": "gateway-secret"
},
"insecure_allow_anonymous_virtual_cluster_auth": true,
"tls": {
"enabled": false
}
}' | jq -r ".id"
)
Create a virtual cluster with anonymous authentication:
SASL_VC_ID=$(curl -X POST "https://us.api.konghq.com/v1/event-gateways/$EVENT_GATEWAY_ID/virtual-clusters" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "sasl_vc",
"destination": {
"id": "'$SASL_BACKEND_CLUSTER_ID'"
},
"dns_label": "sasl-vc",
"authentication": [
{
"type": "anonymous"
}
],
"acl_mode": "passthrough"
}' | jq -r ".id"
)
Run the following command to create a new listener:
SASL_LISTENER_ID=$(curl -X POST "https://us.api.konghq.com/v1/event-gateways/$EVENT_GATEWAY_ID/listeners" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "sasl_listener",
"addresses": [
"0.0.0.0"
],
"ports": [
"19092-19095"
]
}' | jq -r ".id"
)
Add a Forward to Virtual Cluster policy, which will forward requests based on a defined mapping to our virtual cluster:
curl -X POST "https://us.api.konghq.com/v1/event-gateways/$EVENT_GATEWAY_ID/listeners/$SASL_LISTENER_ID/policies" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"type": "forward_to_virtual_cluster",
"name": "forward_to_sasl_vc",
"config": {
"type": "port_mapping",
"advertised_host": "localhost",
"destination": {
"id": "'$SASL_VC_ID'"
}
}
}'
List the topics through the sasl-vc virtual cluster:
kafkactl -C kafkactl.yaml --context sasl-vc list topics
TOPIC PARTITIONS REPLICATION FACTOR
orders 1 1
Event Gateway authenticated to Kafka using the gateway SASL/PLAIN credentials and forwarded the metadata request successfully.
When you’re done experimenting with this example, clean up the resources:
If you created a new Event Gateway control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.
Stop and remove the containers:
docker-compose down
This will stop all services and remove the containers, but preserve your configuration files for future use.