Custom policies

AI Gateway allows you to develop and deploy custom AI Policies. A custom AI policy has two parts, a schema and a plugin handler that implements the custom functionality.

Plugins consist of Lua modules interacting with request and response objects or network streams to implement arbitrary logic. Plugin development operates in the same way for both AI Gateway and Kong Gateway. We provide a Plugin Development Kit (PDK), a set of Lua functions that facilitate interactions between plugins, the Kong Gateway core, and other components.

A custom policy can be used in the same way as any other AI Policy and the name you set when creating the custom policy is the type used when creating an AI Policy entity. The schema you provide is used for validation in the same way as a built-in AI Policy.

This page describes how to run a custom policy you have already developed and manage its lifecycle.

You can deploy custom policies in two ways:

• streaming: streamed from a single control plane
• installed: direct installation on each data plane

Streamed policies

You can deploy a custom policy’s schema and plugin handler by uploading both to a single control plane. During configuration reconciliation, the control plane sends the plugin handler to the data plane in a payload. You can then reference it as a type in any AI Policy configuration.

Data planes must be started with KONG_CUSTOM_PLUGIN_STREAMING_ENABLED to accept custom policies from the control plane.

The same limitations as streaming Kong Gateway apply. For more information, see Streaming custom plugins.

Direct installation

First manually install the custom plugin handler on each data plane by following the installation guide.

Next upload the policy’s schema to the control plane. You can then reference it as a type in any AI Policy configuration.

You can manage the lifecycle of a custom policy using any of the following:

• Konnect UI
• AI Gateway API with the /v1/ai-gateways/{aiGatewayId}/custom-policies endpoint
• kongctl

Custom policy configuration

A custom AI policy configuration is defined by the following fields:

List custom policies

To get a paginated list of existing custom policies, use the /v1/ai-gateways/{aiGatewayId}/custom-policies endpoint:

curl -X GET "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/custom-policies" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Accept: application/json, application/problem+json"

Get a custom policy

To fetch a custom policy by name or id, use the /v1/ai-gateways/{aiGatewayId}/custom-policies/{name|id} endpoint:

curl -X GET "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/custom-policies/my-streaming-custom-policy" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Accept: application/json, application/problem+json"

Create a custom policy

To create a custom policy in streaming mode:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/custom-policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "my-streaming-custom-policy",
      "type": "streaming",
      "display_name": "Custom Policy - Streaming plugin",
      "schema": "'$LUA_SCHEMA'",
      "handler": "'$LUA_HANDLER'"
    }
    '

ai_gateway_custom_policies:
  - ref: my-streaming-custom-policy
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: my-streaming-custom-policy
    type: streaming
    display_name: Custom Policy - Streaming plugin
    schema: !env LUA_SCHEMA
    handler: !env LUA_HANDLER

To create a custom policy in installed mode:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/custom-policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "my-installed-custom-policy",
      "type": "installed",
      "display_name": "Custom Policy - Installed plugin",
      "schema": "'$LUA_SCHEMA'"
    }
    '

ai_gateway_custom_policies:
  - ref: my-installed-custom-policy
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: my-installed-custom-policy
    type: installed
    display_name: Custom Policy - Installed plugin
    schema: !env LUA_SCHEMA

Including a handler in installed mode or omitting it in streaming mode results in an error.

Update a custom policy

To update an existing custom policy, redeploy the custom_policy entity with the same name:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/custom-policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "my-streaming-custom-policy",
      "type": "streaming",
      "display_name": "Custom Policy - Streaming plugin (updated)",
      "schema": "'$LUA_SCHEMA'",
      "handler": "'$LUA_HANDLER'"
    }
    '

ai_gateway_custom_policies:
  - ref: my-streaming-custom-policy
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: my-streaming-custom-policy
    type: streaming
    display_name: Custom Policy - Streaming plugin (updated)
    schema: !env LUA_SCHEMA
    handler: !env LUA_HANDLER

The name field is immutable. You must include the current name when updating a custom policy.

Delete a custom policy

To delete a custom policy by name or id, use the /v1/ai-gateways/{aiGatewayId}/custom-policies/{name|id} endpoint:

curl -X DELETE "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/custom-policies/my-streaming-custom-policy" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Accept: application/json, application/problem+json"