With namespaces, you can preserve any naming systems that you have in place, and ensure they remain consistent.
Namespaces let you:
- Rewrite and enforce topics, consumer groups, and transaction IDs with a consistent prefix
- Expose topics and consumer groups through the virtual cluster
This allows you to expose clean, simple names to clients while maintaining organization on the backend.
For example, a virtual cluster exposes a topic named orders to the client.
Behind the scenes, this maps to team-a-orders on the actual Kafka cluster. The client doesn’t need to know about or manage the team-a- prefix.
This enables transparent multitenancy, where multiple teams can share the same Kafka cluster without needing to manually prefix every topic and consumer group name in their applications.
The following examples provide some common use cases for namespaces and show how to set them up.
The most common use case for namespaces is to automatically prefix read and create operations when interacting with topics and consumer groups.
This helps avoid overlapping from multiple tenants.
You can do this by setting a prefix on the virtual cluster:
curl -X POST "https://us.api.konghq.com/v1/event-gateways/$EVENT_GATEWAY_ID/virtual-clusters" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "example-virtual-cluster",
"destination": {
"name": "example-backend-cluster"
},
"authentication": [
{
"type": "anonymous"
}
],
"dns_label": "virtual-cluster-1",
"acl_mode": "passthrough",
"namespace": {
"mode": "hide_prefix",
"prefix": "my-prefix"
}
}'
In this example, the prefix my-prefix will be used for all consumer group and topics that connect to this virtual cluster.
Along with topics owned by a specific team, you can pull in a select group of additional topics.
This is useful when you want to:
- Consume topics owned by other teams
- Gradually migrate to a namespace while still using old topics temporarily
Here’s an example configuration using a glob pattern:
curl -X POST "https://us.api.konghq.com/v1/event-gateways/$EVENT_GATEWAY_ID/virtual-clusters" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "example-virtual-cluster",
"destination": {
"name": "example-backend-cluster"
},
"authentication": [
{
"type": "anonymous"
}
],
"dns_label": "virtual-cluster-1",
"acl_mode": "passthrough",
"namespace": {
"mode": "hide_prefix",
"prefix": "my-prefix",
"additional": {
"topics": [
{
"type": "glob",
"glob": "my-topic-*",
"conflict": "warn"
}
]
}
}
}'
These topics are accessed using their full unmodified names.
This example uses a glob expression to capture topics using name patterns.
You can also pass an exact list of topics as an array:
"topics": [
{
"type": "exact_list",
"list": [
{
"backend": "allowed_topic",
"backend": "another_allowed_topic"
}
]
}
]
You can access existing consumer groups to avoid migrating offsets:
curl -X POST "https://us.api.konghq.com/v1/event-gateways/$EVENT_GATEWAY_ID/virtual-clusters" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "example-virtual-cluster",
"destination": {
"name": "example-backend-cluster"
},
"authentication": [
{
"type": "anonymous"
}
],
"dns_label": "virtual-cluster-1",
"acl_mode": "passthrough",
"namespace": {
"mode": "hide_prefix",
"prefix": "my-prefix",
"additional": {
"consumer_groups": [
{
"type": "glob",
"glob": "my-app-*",
"conflict": "warn"
}
]
}
}
}'
End users of this virtual cluster can use their existing, unnamespaced consumer groups.
This example uses a glob expression to capture consumer groups using name patterns.
You can also pass an exact list of consumer groups as an array:
"consumer_groups": [
{
"type": "exact_list",
"list": [
{
"value": "foo",
"value": "bar"
}
]
}
]
