Request Rule Validator

The Request Rule Validator policy validates the content of Kafka requests against a list of rules you define. Use it to enforce conventions that the ACLs policy can’t express, because ACLs decide whether a principal can perform an action, not what values the action carries.

For example, you can use the Request Rule Validator policy to:

• Limit the number of partitions or replicas a topic can have.
• Enforce a naming convention for topics or consumer groups.
• Require producers to acknowledge writes from all in-sync replicas.
• Block changes to broker-level configuration.
• Prevent overly permissive ACL bindings, like granting ALL or using a wildcard principal.

Each rule describes the valid state of a request as a CEL expression. When the expression evaluates to false, the rule’s configured action runs.

Common use cases for the Request Rule Validator policy:

This policy runs in the cluster phase, evaluating rules against the content of specific Kafka request types.

1. A Kafka client sends a request, for example to create a topic or produce a message.
2. Event Gateway evaluates every configured rule for that request type.
   • If a rule’s expression evaluates to true, the request satisfies that rule.
   • If a rule’s expression evaluates to false, Event Gateway runs the rule’s configured action.
3. If any rule configured with the reject action fails, Event Gateway responds with the Kafka POLICY_VIOLATION error code and doesn’t forward the request to the backend cluster.

 
sequenceDiagram
  autonumber
  participant client as Client
  participant egw as Event Gateway
  participant broker as Event broker

  client->>egw: request (e.g. CreateTopics)
  egw->>egw: evaluate rules for the request type

  alt If every reject rule passes

  egw->>broker: forward request

  else If a reject rule fails

  egw-x client: POLICY_VIOLATION
  end

  

Request types and available values

You configure rules per Kafka request type. Each request type exposes a different set of values to the rule’s CEL expression, matching the fields of that request. All rule expressions also have access to the authentication fields.

See the policy reference for the supported request types and the values available to each one.

Automatic topic creation

When at least one create_topics rule is configured, Event Gateway turns off automatic topic creation on Metadata requests. Otherwise, clients could create topics through Metadata requests and bypass the create_topics rules.

Batched requests

Some request types, like create_topics, carry a batch of items in a single request. Event Gateway evaluates every rule against each item independently. When one item in a batch violates a reject rule, only that item is rejected with POLICY_VIOLATION, while the rest of the batch proceeds normally.

Actions

Each rule configures one of two actions, which runs when the rule’s expression evaluates to false:

• reject: Fails the request, or the offending item in a batch, with the POLICY_VIOLATION error code.
• passthrough: Lets the request continue, but records the violation the same way reject does (metrics and logs on a debug level).

If a rule’s expression can’t be evaluated, for example because a value has an unexpected format, Event Gateway treats the result as false and runs the rule’s action.

Multiple rules and policies

You can configure more than one rule per request type, and more than one Request Rule Validator policy on a virtual cluster. Like other cluster policies, they run in the order they’re configured.

We recommend placing the Request Rule Validator policy after the ACLs policy, so an unauthorized client learns nothing about your rules before it’s rejected on authorization grounds.

A rule’s description is included in the error the client sees and in Event Gateway logs. Avoid including sensitive details, like internal system names or other rules you enforce, in a rule’s description if that information shouldn’t be visible to the client that triggered the violation.

The Request Rule Validator policy logs each rule evaluation at a different level:

• Passed rules are logged at the trace level.
• Rule violations are logged at the debug level.

By default, Event Gateway logs at the info level, which excludes these logs. Kafka usually has high traffic, and logging every evaluation could overwhelm the logging system. To show rule evaluations without raising the level for the whole gateway, set the debug level for the policy module only:

KEG__OBSERVABILITY__LOG_FLAGS=info,keg::kafka::proxy::transform::policies::request_rule_validator=debug