Convert JSON records to Avro before they reach the clusterv1.3+
Convert JSON records produced by a client to Avro before Event Gateway writes them to the backend cluster.
This example transcodes the target Avro schema directly in the policy using inline.
This change is irreversible. Use the produce phase when every downstream consumer of the topic reads Avro.
Prerequisites
- A Schema Validation Produce policy that parses the record value as JSON.
Environment variables
PARENT_POLICY_ID: The UUID of the parent Schema Validation policy.
curl -X POST https://{region}.api.konghq.com/v1/event-gateways/{eventGatewayId}/virtual-clusters/{virtualClusterId}/produce-policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "convert-produce-json-to-avro",
"type": "transcode",
"parent_policy_id": "'$PARENT_POLICY_ID'",
"config": {
"failure_mode": "reject",
"output_format": "avro",
"schema_source": {
"type": "inline",
"inline": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"age\",\"type\":\"long\"}]}\n"
}
}
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
virtualClusterId: Theidof the Virtual Cluster. -
eventGatewayId: Theidof the Event Gateway. -
eventGatewayListenerId: Theidof the Event Gateway Listener.
See the Konnect Event Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect-beta = {
source = "kong/konnect-beta"
}
}
}
provider "konnect-beta" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_event_gateway_produce_policy_transcode" "my_virtual_cluster_policy_transcode" {
provider = konnect-beta
type = "transcode"
parent_policy_id = var.parent_policy_id
config = {
failure_mode = "reject"
output_format = "avro"
schema_source = {
type = "inline"
inline = <<EOF
{"type":"record","name":"User","fields":[{"name":"name","type":"string"},{"name":"age","type":"long"}]}
EOF
}
}
virtual_cluster_id = konnect_event_gateway_virtual_cluster.my_virtual_cluster.id
gateway_id = konnect_event_gateway.my_event_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "parent_policy_id" {
type = string
}The following example creates a new transcode policy.
event_gateways:
- ref: eventGatewayName
name: eventGatewayName
virtual_clusters:
- ref: virtualClusterName
name: virtualClusterName
produce_policies:
- ref: convert-produce-json-to-avro
type: transcode
transcode:
name: convert-produce-json-to-avro
parent_policy_id: !env PARENT_POLICY_ID
config:
failure_mode: reject
output_format: avro
schema_source:
type: inline
inline: '{"type":"record","name":"User","fields":[{"name":"name","type":"string"},{"name":"age","type":"long"}]}
'Make sure to replace the following placeholders with your own values:
-
eventGatewayName: Thenameof your Event Gateway. -
virtualClusterName: Thenameof the Virtual Cluster.