AI Lakera Guard

The AI Lakera Guard plugin evaluates requests and responses that pass through Kong to Large Language Models (LLMs). It uses the Lakera Guard SaaS service to detect safety policy violations and block unsafe content before it reaches upstream LLMs or returns to clients. The plugin supports multiple inspection modes and guards both inbound prompts and outbound model outputs.

The plugin inspects model traffic at three points in the LLM request lifecycle. Each phase pages data into memory, extracts content that Lakera Guard can evaluate, and sends that content to Lakera for inspection.

• Request phase: Inspection occurs before any data leaves the gateway toward the target LLM. The plugin buffers the full request body in memory, extracts the fields that the AI Lakera Guard plugin can evaluate, and sends them for inspection.
• Response phase (buffered): Inspection occurs before any byte is transmitted back toward the client. The plugin buffers the full upstream response in memory, extracts the response fields that Lakera Guard can evaluate, and inspects them. This occurs before Kong AI Gateway sends any part of the response back to the client.
• Response phase (per-frame): The plugin runs during streaming responses like Server-Sent Events. Kong processes the response in chunks, buffering each frame in memory as it arrives. When enough data is available to extract an evaluable segment, the plugin inspects that segment with Lakera Guard before forwarding the frame to the client.

The plugin inspects request and response bodies for routes that use supported model interaction formats. It skips inspection on response types that are not text responses based on Lakera Guard’s current product limitations.

You can use the logging capabilities of the AI Lakera Guard plugin to monitor the inspection process and understand the detected violations.

The plugin provides detailed logging and controls over how violations are reported:

• SaaS platform logging: All inspected requests, responses, and chats are made available on the Lakera SaaS platform.
• Kong AI Gateway logging: Kong logs all request and response Lakera request UUIDs to the standard logging subsystem.
• Unsupported logging outputs: Prometheus, Splunk, or OpenTelemetry.
• Logging outputs: HTTP-Log, File-Log, and TCP-Log.

By default, the plugin doesn’t tell clients why their request was blocked. However, this information is always logged to Kong AI Gateway logs for administrators.

To change this behavior, use reveal_failure_categories: true. If activated, you’ll receive a JSON response including a breakdown array that details the specific detector_type that caused the failure.

Standard logging subsystem example

When a request passes all guardrails, the log includes processing latency and the request UUID:

"ai": {
  "proxy": {
    "lakera-guard": {
      "input_processing_latency": 72,
      "lakera_service_url": "https://api.lakera.ai/v2/guard",
      "input_request_uuid": "a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d",
      "lakera_project_id": "project-1234567890"
    }
  }
}

Violations log example

When a request is blocked, the log captures the violation reason and detector details:

"ai": {
  "proxy": {
    "lakera-guard": {
      "input_processing_latency": 78,
      "lakera_service_url": "https://api.lakera.ai/v2/guard",
      "input_block_detail": [
        {
          "policy_id": "policy-4f8a9b2c-1d3e-4a5b-8c9d-0e1f2a3b4c5d",
          "detector_id": "detector-lakera-moderation-1-input",
          "project_id": "project-1234567890",
          "message_id": 3,
          "detected": true,
          "detector_type": "moderated_content/hate"
        }
      ],
      "input_request_uuid": "a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d",
      "input_block_reason": "moderated_content/hate",
      "lakera_project_id": "project-1234567890"
    }
  }
}