AI Prompt Template Policy

The AI Prompt Template Policy lets you provide tuned AI prompts to users. Users only need to fill in the blanks with variable placeholders in the following format: {{variable}}.

This lets admins set up templates, which can then be used by anyone in the organization. It also allows admins to present an LLM as an API in its own right: for example, a bot that can provide software class examples and/or suggestions.

This Policy also sanitizes string inputs to ensure that JSON control characters are escaped, preventing arbitrary prompt injection.

When activated, the template restricts LLM usage to the predefined templates. They are defined in the following format:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "AI Prompt Template - Sample Template",
      "name": "ai-prompt-template",
      "type": "ai-prompt-template",
      "config": {
        "templates": [
          {
            "name": "sample-template",
            "template": "{\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"Explain to me what {{thing}} is.\"\n    }\n  ]\n}"
          }
        ]
      }
    }
    '

ai_gateway_policies:
  - ref: ai-prompt-template
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: AI Prompt Template - Sample Template
    name: ai-prompt-template
    type: ai-prompt-template
    config:
      templates:
      - name: sample-template
        template: |-
          {
            "messages": [
              {
                "role": "user",
                "content": "Explain to me what {{thing}} is."
              }
            ]
          }

When calling a template, replace the content of messages (llm/v1/chat) or prompt (llm/v1/completions) with a template reference, using the following format:

{
  "messages": "{template://sample-template}",
  "properties": {
    "thing": "gravity"
  }
}

By default, requests that don’t use a template are still be passed to the LLM. However, this can be configured using the config.allow_untemplated_requests parameter. If this parameter is set to false, requests that don’t use a template will return a 400 Bad Request response.