AI Rate Limiting Advanced: Request prompt function with Azure Managed Redis instance auth - Policy

Request prompt function with Azure Managed Redis instance auth

Protect your LLM services with rate limiting and Azure Managed Redis instance auth. The AI Rate Limiting Advanced Policy analyzes query costs and token responses to provide an enterprise-grade rate limiting strategy.

The following example uses request prompt rate limiting, which lets you rate limit requests based on a custom token.

Prerequisites

An existing AI Model.
A running Redis instance on an Azure Managed Redis instance with Entra authentication configured..
Port 10000, or your custom Redis port, is open and reachable from AI Gateway.
Add the user/service principal/identity to the “Microsoft Entra Authentication Redis user” list for the Azure Managed Redis instance..

Environment variables

INSTANCE_ADDRESS: The Azure Managed Redis instance address.
INSTANCE_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
AZURE_CLIENT_ID: The client ID of the Principal/Identity.
AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

Set up the policy

kongctl

policy.yaml

Copied!

ai_gateway_policies:
  - ref: request-prompt-count-azure-instance
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: request-prompt-count-azure-instance
    display_name: Request prompt function with Azure Managed Redis instance auth
    type: ai-rate-limiting-advanced
    enabled: true
    config:
      strategy: redis
      redis:
        host: !env INSTANCE_ADDRESS
        username: !env INSTANCE_USERNAME
        port: 10000
        cloud_authentication:
          auth_provider: azure
          azure_client_id: !env AZURE_CLIENT_ID
          azure_client_secret: !env AZURE_CLIENT_SECRET
          azure_tenant_id: !env AZURE_TENANT_ID
      sync_rate: 0
      policies:
      - match:
        - type: provider
          partition_by: true
          values:
          - cohere
        limits:
        - limit: 100
          window_size: 60
        - limit: 1000
          window_size: 3600
      request_prompt_count_function: |
        local header_count = tonumber(kong.request.get_header("x-prompt-count"))
        if header_count then
          return header_count
        end
        return 0
    global: false

Make sure to replace the following placeholders with your own values:

AI_GATEWAY_ID: The id of your AI Gateway.
!env INSTANCE_ADDRESS: The Azure Managed Redis instance address.
!env INSTANCE_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
!env AZURE_CLIENT_ID: The client ID of the Principal/Identity.
!env AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
!env AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "request-prompt-count-azure-instance",
      "display_name": "Request prompt function with Azure Managed Redis instance auth",
      "type": "ai-rate-limiting-advanced",
      "enabled": true,
      "config": {
        "strategy": "redis",
        "redis": {
          "host": "'$INSTANCE_ADDRESS'",
          "username": "'$INSTANCE_USERNAME'",
          "port": 10000,
          "cloud_authentication": {
            "auth_provider": "azure",
            "azure_client_id": "'$AZURE_CLIENT_ID'",
            "azure_client_secret": "'$AZURE_CLIENT_SECRET'",
            "azure_tenant_id": "'$AZURE_TENANT_ID'"
          }
        },
        "sync_rate": 0,
        "policies": [
          {
            "match": [
              {
                "type": "provider",
                "partition_by": true,
                "values": [
                  "cohere"
                ]
              }
            ],
            "limits": [
              {
                "limit": 100,
                "window_size": 60
              },
              {
                "limit": 1000,
                "window_size": 3600
              }
            ]
          }
        ],
        "request_prompt_count_function": "local header_count = tonumber(kong.request.get_header(\"x-prompt-count\"))\nif header_count then\n  return header_count\nend\nreturn 0\n"
      },
      "global": false
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
AIGatewayId: The id of the AI Gateway.

See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration:

resource "konnect_ai_gateway_policy" "my_policy" {
  name = "request-prompt-count-azure-instance"
  display_name = "Request prompt function with Azure Managed Redis instance auth"
  type = "ai-rate-limiting-advanced"
  enabled = true
  global = false
  config = jsonencode({
    strategy = "redis"

    redis = {
      host = var.instance_address
      username = var.instance_username
      port = 10000

      cloud_authentication = {
        auth_provider = "azure"
        azure_client_id = var.azure_client_id
        azure_client_secret = var.azure_client_secret
        azure_tenant_id = var.azure_tenant_id
      }
    }
    sync_rate = 0
    policies = [
      {
        match = [
          {
            type = "provider"
            partition_by = true
            values = ["cohere"]
          }        ]
        limits = [
          {
            limit = 100
            window_size = 60
          }, 

          {
            limit = 1000
            window_size = 3600
          }        ]
      }    ]
    request_prompt_count_function = <<EOF
local header_count = tonumber(kong.request.get_header("x-prompt-count"))
if header_count then
  return header_count
end
return 0
EOF
  })


  gateway_id = konnect_ai_gateway.my_ai_gateway.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "instance_address" {
  type = string
}
variable "instance_username" {
  type = string
}
variable "azure_client_id" {
  type = string
}
variable "azure_client_secret" {
  type = string
}
variable "azure_tenant_id" {
  type = string
}

Copied!

kongctl

policy.yaml

Copied!

ai_gateway_policies:
  - ref: request-prompt-count-azure-instance
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: request-prompt-count-azure-instance
    display_name: Request prompt function with Azure Managed Redis instance auth
    type: ai-rate-limiting-advanced
    enabled: true
    config:
      strategy: redis
      redis:
        host: !env INSTANCE_ADDRESS
        username: !env INSTANCE_USERNAME
        port: 10000
        cloud_authentication:
          auth_provider: azure
          azure_client_id: !env AZURE_CLIENT_ID
          azure_client_secret: !env AZURE_CLIENT_SECRET
          azure_tenant_id: !env AZURE_TENANT_ID
      sync_rate: 0
      policies:
      - match:
        - type: provider
          partition_by: true
          values:
          - cohere
        limits:
        - limit: 100
          window_size: 60
        - limit: 1000
          window_size: 3600
      request_prompt_count_function: |
        local header_count = tonumber(kong.request.get_header("x-prompt-count"))
        if header_count then
          return header_count
        end
        return 0
    global: false

Make sure to replace the following placeholders with your own values:

AI_GATEWAY_ID: The id of your AI Gateway.
!env INSTANCE_ADDRESS: The Azure Managed Redis instance address.
!env INSTANCE_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
!env AZURE_CLIENT_ID: The client ID of the Principal/Identity.
!env AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
!env AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "request-prompt-count-azure-instance",
      "display_name": "Request prompt function with Azure Managed Redis instance auth",
      "type": "ai-rate-limiting-advanced",
      "enabled": true,
      "config": {
        "strategy": "redis",
        "redis": {
          "host": "'$INSTANCE_ADDRESS'",
          "username": "'$INSTANCE_USERNAME'",
          "port": 10000,
          "cloud_authentication": {
            "auth_provider": "azure",
            "azure_client_id": "'$AZURE_CLIENT_ID'",
            "azure_client_secret": "'$AZURE_CLIENT_SECRET'",
            "azure_tenant_id": "'$AZURE_TENANT_ID'"
          }
        },
        "sync_rate": 0,
        "policies": [
          {
            "match": [
              {
                "type": "provider",
                "partition_by": true,
                "values": [
                  "cohere"
                ]
              }
            ],
            "limits": [
              {
                "limit": 100,
                "window_size": 60
              },
              {
                "limit": 1000,
                "window_size": 3600
              }
            ]
          }
        ],
        "request_prompt_count_function": "local header_count = tonumber(kong.request.get_header(\"x-prompt-count\"))\nif header_count then\n  return header_count\nend\nreturn 0\n"
      },
      "global": false
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
AIGatewayId: The id of the AI Gateway.

See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration:

resource "konnect_ai_gateway_policy" "my_policy" {
  name = "request-prompt-count-azure-instance"
  display_name = "Request prompt function with Azure Managed Redis instance auth"
  type = "ai-rate-limiting-advanced"
  enabled = true
  global = false
  config = jsonencode({
    strategy = "redis"

    redis = {
      host = var.instance_address
      username = var.instance_username
      port = 10000

      cloud_authentication = {
        auth_provider = "azure"
        azure_client_id = var.azure_client_id
        azure_client_secret = var.azure_client_secret
        azure_tenant_id = var.azure_tenant_id
      }
    }
    sync_rate = 0
    policies = [
      {
        match = [
          {
            type = "provider"
            partition_by = true
            values = ["cohere"]
          }        ]
        limits = [
          {
            limit = 100
            window_size = 60
          }, 

          {
            limit = 1000
            window_size = 3600
          }        ]
      }    ]
    request_prompt_count_function = <<EOF
local header_count = tonumber(kong.request.get_header("x-prompt-count"))
if header_count then
  return header_count
end
return 0
EOF
  })


  gateway_id = konnect_ai_gateway.my_ai_gateway.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "instance_address" {
  type = string
}
variable "instance_username" {
  type = string
}
variable "azure_client_id" {
  type = string
}
variable "azure_client_secret" {
  type = string
}
variable "azure_tenant_id" {
  type = string
}

Copied!

kongctl

policy.yaml

Copied!

ai_gateway_policies:
  - ref: request-prompt-count-azure-instance
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: request-prompt-count-azure-instance
    display_name: Request prompt function with Azure Managed Redis instance auth
    type: ai-rate-limiting-advanced
    enabled: true
    config:
      strategy: redis
      redis:
        host: !env INSTANCE_ADDRESS
        username: !env INSTANCE_USERNAME
        port: 10000
        cloud_authentication:
          auth_provider: azure
          azure_client_id: !env AZURE_CLIENT_ID
          azure_client_secret: !env AZURE_CLIENT_SECRET
          azure_tenant_id: !env AZURE_TENANT_ID
      sync_rate: 0
      policies:
      - match:
        - type: provider
          partition_by: true
          values:
          - cohere
        limits:
        - limit: 100
          window_size: 60
        - limit: 1000
          window_size: 3600
      request_prompt_count_function: |
        local header_count = tonumber(kong.request.get_header("x-prompt-count"))
        if header_count then
          return header_count
        end
        return 0
    global: false

Make sure to replace the following placeholders with your own values:

AI_GATEWAY_ID: The id of your AI Gateway.
!env INSTANCE_ADDRESS: The Azure Managed Redis instance address.
!env INSTANCE_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
!env AZURE_CLIENT_ID: The client ID of the Principal/Identity.
!env AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
!env AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "request-prompt-count-azure-instance",
      "display_name": "Request prompt function with Azure Managed Redis instance auth",
      "type": "ai-rate-limiting-advanced",
      "enabled": true,
      "config": {
        "strategy": "redis",
        "redis": {
          "host": "'$INSTANCE_ADDRESS'",
          "username": "'$INSTANCE_USERNAME'",
          "port": 10000,
          "cloud_authentication": {
            "auth_provider": "azure",
            "azure_client_id": "'$AZURE_CLIENT_ID'",
            "azure_client_secret": "'$AZURE_CLIENT_SECRET'",
            "azure_tenant_id": "'$AZURE_TENANT_ID'"
          }
        },
        "sync_rate": 0,
        "policies": [
          {
            "match": [
              {
                "type": "provider",
                "partition_by": true,
                "values": [
                  "cohere"
                ]
              }
            ],
            "limits": [
              {
                "limit": 100,
                "window_size": 60
              },
              {
                "limit": 1000,
                "window_size": 3600
              }
            ]
          }
        ],
        "request_prompt_count_function": "local header_count = tonumber(kong.request.get_header(\"x-prompt-count\"))\nif header_count then\n  return header_count\nend\nreturn 0\n"
      },
      "global": false
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
AIGatewayId: The id of the AI Gateway.

See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration:

resource "konnect_ai_gateway_policy" "my_policy" {
  name = "request-prompt-count-azure-instance"
  display_name = "Request prompt function with Azure Managed Redis instance auth"
  type = "ai-rate-limiting-advanced"
  enabled = true
  global = false
  config = jsonencode({
    strategy = "redis"

    redis = {
      host = var.instance_address
      username = var.instance_username
      port = 10000

      cloud_authentication = {
        auth_provider = "azure"
        azure_client_id = var.azure_client_id
        azure_client_secret = var.azure_client_secret
        azure_tenant_id = var.azure_tenant_id
      }
    }
    sync_rate = 0
    policies = [
      {
        match = [
          {
            type = "provider"
            partition_by = true
            values = ["cohere"]
          }        ]
        limits = [
          {
            limit = 100
            window_size = 60
          }, 

          {
            limit = 1000
            window_size = 3600
          }        ]
      }    ]
    request_prompt_count_function = <<EOF
local header_count = tonumber(kong.request.get_header("x-prompt-count"))
if header_count then
  return header_count
end
return 0
EOF
  })


  gateway_id = konnect_ai_gateway.my_ai_gateway.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "instance_address" {
  type = string
}
variable "instance_username" {
  type = string
}
variable "azure_client_id" {
  type = string
}
variable "azure_client_secret" {
  type = string
}
variable "azure_tenant_id" {
  type = string
}

Copied!

kongctl

policy.yaml

Copied!

ai_gateway_policies:
  - ref: request-prompt-count-azure-instance
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: request-prompt-count-azure-instance
    display_name: Request prompt function with Azure Managed Redis instance auth
    type: ai-rate-limiting-advanced
    enabled: true
    config:
      strategy: redis
      redis:
        host: !env INSTANCE_ADDRESS
        username: !env INSTANCE_USERNAME
        port: 10000
        cloud_authentication:
          auth_provider: azure
          azure_client_id: !env AZURE_CLIENT_ID
          azure_client_secret: !env AZURE_CLIENT_SECRET
          azure_tenant_id: !env AZURE_TENANT_ID
      sync_rate: 0
      policies:
      - match:
        - type: provider
          partition_by: true
          values:
          - cohere
        limits:
        - limit: 100
          window_size: 60
        - limit: 1000
          window_size: 3600
      request_prompt_count_function: |
        local header_count = tonumber(kong.request.get_header("x-prompt-count"))
        if header_count then
          return header_count
        end
        return 0
    global: true

Make sure to replace the following placeholders with your own values:

AI_GATEWAY_ID: The id of your AI Gateway.
!env INSTANCE_ADDRESS: The Azure Managed Redis instance address.
!env INSTANCE_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
!env AZURE_CLIENT_ID: The client ID of the Principal/Identity.
!env AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
!env AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "request-prompt-count-azure-instance",
      "display_name": "Request prompt function with Azure Managed Redis instance auth",
      "type": "ai-rate-limiting-advanced",
      "enabled": true,
      "config": {
        "strategy": "redis",
        "redis": {
          "host": "'$INSTANCE_ADDRESS'",
          "username": "'$INSTANCE_USERNAME'",
          "port": 10000,
          "cloud_authentication": {
            "auth_provider": "azure",
            "azure_client_id": "'$AZURE_CLIENT_ID'",
            "azure_client_secret": "'$AZURE_CLIENT_SECRET'",
            "azure_tenant_id": "'$AZURE_TENANT_ID'"
          }
        },
        "sync_rate": 0,
        "policies": [
          {
            "match": [
              {
                "type": "provider",
                "partition_by": true,
                "values": [
                  "cohere"
                ]
              }
            ],
            "limits": [
              {
                "limit": 100,
                "window_size": 60
              },
              {
                "limit": 1000,
                "window_size": 3600
              }
            ]
          }
        ],
        "request_prompt_count_function": "local header_count = tonumber(kong.request.get_header(\"x-prompt-count\"))\nif header_count then\n  return header_count\nend\nreturn 0\n"
      },
      "global": true
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
AIGatewayId: The id of the AI Gateway.

See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration:

resource "konnect_ai_gateway_policy" "my_policy" {
  name = "request-prompt-count-azure-instance"
  display_name = "Request prompt function with Azure Managed Redis instance auth"
  type = "ai-rate-limiting-advanced"
  enabled = true
  global = true
  config = jsonencode({
    strategy = "redis"

    redis = {
      host = var.instance_address
      username = var.instance_username
      port = 10000

      cloud_authentication = {
        auth_provider = "azure"
        azure_client_id = var.azure_client_id
        azure_client_secret = var.azure_client_secret
        azure_tenant_id = var.azure_tenant_id
      }
    }
    sync_rate = 0
    policies = [
      {
        match = [
          {
            type = "provider"
            partition_by = true
            values = ["cohere"]
          }        ]
        limits = [
          {
            limit = 100
            window_size = 60
          }, 

          {
            limit = 1000
            window_size = 3600
          }        ]
      }    ]
    request_prompt_count_function = <<EOF
local header_count = tonumber(kong.request.get_header("x-prompt-count"))
if header_count then
  return header_count
end
return 0
EOF
  })


  gateway_id = konnect_ai_gateway.my_ai_gateway.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "instance_address" {
  type = string
}
variable "instance_username" {
  type = string
}
variable "azure_client_id" {
  type = string
}
variable "azure_client_secret" {
  type = string
}
variable "azure_tenant_id" {
  type = string
}

Copied!

Attach the policy

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.

kongctl

model.yaml

Copied!

ai_gateway_models:
  - ref: my-gpt-4o
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: my-gpt-4o
    name: my-gpt-4o
    type: model
    capabilities:
    - generate
    formats:
    - type: openai
    targets:
    - name: gpt-4o
      provider: my-openai-account
      config:
        type: openai
    config:
      route:
        paths:
        - "/v1"
        model:
          body_param: model
          values:
          - my-gpt-4o
      logging:
        payloads: false
    policies:
    - !ref request-prompt-count-azure-instance#name

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.

curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/models/my-gpt-4o \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "my-gpt-4o",
      "name": "my-gpt-4o",
      "type": "model",
      "capabilities": [
        "generate"
      ],
      "formats": [
        {
          "type": "openai"
        }
      ],
      "policies": [
        "request-prompt-count-azure-instance"
      ],
      "targets": [
        {
          "name": "gpt-4o",
          "provider": "my-openai-account",
          "config": {
            "type": "openai"
          }
        }
      ],
      "config": {
        "route": {
          "paths": [
            "/v1"
          ],
          "model": {
            "body_param": "model",
            "values": [
              "my-gpt-4o"
            ]
          }
        },
        "logging": {
          "payloads": false
        }
      }
    }
    '

Copied!

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.

resource "konnect_ai_gateway_model" "my_model" {
  display_name = "my-gpt-4o"
  name = "my-gpt-4o"
  type = "model"
  capabilities = ["generate"]
  formats = [
    {
      type = "openai"
    }  ]
  targets = [
    {
      name = "gpt-4o"
      provider = "my-openai-account"

      config = {
        type = "openai"
      }
    }  ]
  config = jsonencode({

    route = {
      paths = ["/v1"]

      model = {
        body_param = "model"
        values = ["my-gpt-4o"]
      }
    }

    logging = {
      payloads = false
    }
  })
  policies = [konnect_ai_gateway_policy.my_policy.name]


  gateway_id = konnect_ai_gateway.my_ai_gateway.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "instance_address" {
  type = string
}
variable "instance_username" {
  type = string
}
variable "azure_client_id" {
  type = string
}
variable "azure_client_secret" {
  type = string
}
variable "azure_tenant_id" {
  type = string
}

Copied!

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.

kongctl

consumer.yaml

Copied!

ai_gateway_consumers:
  - ref: mobile-app-production
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: Mobile App - Production
    name: mobile-app-production
    type: api-key
    policies:
    - !ref request-prompt-count-azure-instance#name

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.

curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/consumers/mobile-app-production \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "Mobile App - Production",
      "name": "mobile-app-production",
      "type": "api-key",
      "policies": [
        "request-prompt-count-azure-instance"
      ]
    }
    '

Copied!

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.

resource "konnect_ai_gateway_consumer" "my_consumer" {
  display_name = "Mobile App - Production"
  name = "mobile-app-production"
  type = "api-key"
  policies = [konnect_ai_gateway_policy.my_policy.name]


  gateway_id = konnect_ai_gateway.my_ai_gateway.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "instance_address" {
  type = string
}
variable "instance_username" {
  type = string
}
variable "azure_client_id" {
  type = string
}
variable "azure_client_secret" {
  type = string
}
variable "azure_tenant_id" {
  type = string
}

Copied!

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.

kongctl

consumer_group.yaml

Copied!

ai_gateway_consumer_groups:
  - ref: internal-teams
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: Internal Teams
    name: internal-teams
    policies:
    - !ref request-prompt-count-azure-instance#name

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.

curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/consumer-groups/internal-teams \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "Internal Teams",
      "name": "internal-teams",
      "policies": [
        "request-prompt-count-azure-instance"
      ]
    }
    '

Copied!

After creating the policy, add request-prompt-count-azure-instance to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.

resource "konnect_ai_gateway_consumer_group" "my_consumer_group" {
  display_name = "Internal Teams"
  name = "internal-teams"
  policies = [konnect_ai_gateway_policy.my_policy.name]


  gateway_id = konnect_ai_gateway.my_ai_gateway.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "instance_address" {
  type = string
}
variable "instance_username" {
  type = string
}
variable "azure_client_id" {
  type = string
}
variable "azure_client_secret" {
  type = string
}
variable "azure_tenant_id" {
  type = string
}

Copied!

AI Rate Limiting Advanced Policy

Request prompt function with Azure Managed Redis instance auth

Prerequisites

Environment variables

Set up the policy

Attach the policy

Help us make these docs great!

Still need help?