Mask Fields Produce

The Mask Fields Produce policy redacts string fields of Kafka records during the produce phase. It runs on a record that a Schema Validation Produce policy parsed, so you must nest it under that policy.

Use this policy to redact a value before Event Gateway writes it to the backend cluster. This change is irreversible, as the cluster never receives the original value. To redact a value for the client that reads it, and keep the stored record unchanged, use the Mask Fields Consume policy instead.

Select the fields to redact as a static list of paths, or as a single expression that returns a list of paths. Each entry pairs a set of paths with the strategy that redacts them.

Common use cases for the Mask Fields Produce policy:

This policy runs in the produce phase, after schema validation parses the record.

1. A Kafka client produces a record and sends it to Event Gateway.
2. The parent Schema Validation policy parses the record value.
3. The Mask Fields Produce policy replaces the value of each selected field with a masked value.
4. Event Gateway sends the record to the backend cluster, which stores the masked value.

 
sequenceDiagram
  autonumber
  participant client as Client
  participant egw as Event Gateway
  participant broker as Event broker

  client->>egw: produce record
  egw->>egw: parse record (Schema Validation)
  egw->>egw: mask selected fields

  egw->>broker: send record with masked value
  

This policy masks string fields only:

• A selected field that is not present in the record is ignored.
• A selected field that is not a string is a policy failure. The failure_mode setting controls what happens in this situation.

Warning: Masking can break client-side validation. For example, a client that validates the format of an SSN can reject a masked value. Make sure your clients accept masked values before you enable this policy.

Each entry in mask_fields uses one of the following strategies:

The Mask Fields Produce policy supports the following failure modes:

Set the parent Schema Validation Produce policy to reject to use this policy.

The parent policy must not use passthrough or mark. With these failure modes, the parent sends the record without parsing it. The Mask Fields Produce policy then never runs, and the cluster stores the fields unmasked.

Instead of listing paths one by one, the paths entry can take a single expression that reads the record’s schema metadata and returns the tagged field paths. For example, with a Confluent Schema Registry, an expression can return every field path that carries a PII tag:

paths: 'record.value.schema.metadata.tags["PII"]'

This is the recommended way to mask fields. The mask follows the schema, so when the schema evolves and a new field gets tagged, it is masked without any policy change.

In this case, the schema becomes the input that decides which fields are masked, so anyone who can edit the schema tags can change the masking. Keep schema registry write access restricted to trusted roles.

See the full walkthrough in Mask Kafka message fields selected by schema registry tags.

This policy needs the original value, so the policy order is important when a field is also encrypted. Mask the fields before the Encrypt Fields policy encrypts them.

See the reference for nested policies for more detail.