Configure the AI LLM as Judge policy
Evaluate responses by assigning a correctness score for AI-assisted learning and assessment.
Prerequisites
-
You have a working OpenAI API key.
-
An existing AI Model.
Environment variables
OPENAI_API_KEY: Your OpenAI API key
ai_gateway_policies:
- ref: configure-ai-llm-as-judge
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: configure-ai-llm-as-judge
display_name: Configure the AI LLM as Judge policy
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
global: falseMake sure to replace the following placeholders with your own values:
-
AI_GATEWAY_ID: Theidof your AI Gateway. -
!env OPENAI_API_KEY: Your OpenAI API key
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "configure-ai-llm-as-judge",
"display_name": "Configure the AI LLM as Judge policy",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only — no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3
},
"global": false
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "configure-ai-llm-as-judge"
display_name = "Configure the AI LLM as Judge policy"
type = "ai-llm-as-judge"
enabled = true
global = false
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}ai_gateway_policies:
- ref: configure-ai-llm-as-judge
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: configure-ai-llm-as-judge
display_name: Configure the AI LLM as Judge policy
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
global: falseMake sure to replace the following placeholders with your own values:
-
AI_GATEWAY_ID: Theidof your AI Gateway. -
!env OPENAI_API_KEY: Your OpenAI API key
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "configure-ai-llm-as-judge",
"display_name": "Configure the AI LLM as Judge policy",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only — no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3
},
"global": false
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "configure-ai-llm-as-judge"
display_name = "Configure the AI LLM as Judge policy"
type = "ai-llm-as-judge"
enabled = true
global = false
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}ai_gateway_policies:
- ref: configure-ai-llm-as-judge
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: configure-ai-llm-as-judge
display_name: Configure the AI LLM as Judge policy
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
global: falseMake sure to replace the following placeholders with your own values:
-
AI_GATEWAY_ID: Theidof your AI Gateway. -
!env OPENAI_API_KEY: Your OpenAI API key
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "configure-ai-llm-as-judge",
"display_name": "Configure the AI LLM as Judge policy",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only — no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3
},
"global": false
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "configure-ai-llm-as-judge"
display_name = "Configure the AI LLM as Judge policy"
type = "ai-llm-as-judge"
enabled = true
global = false
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}ai_gateway_policies:
- ref: configure-ai-llm-as-judge
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: configure-ai-llm-as-judge
display_name: Configure the AI LLM as Judge policy
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
global: trueMake sure to replace the following placeholders with your own values:
-
AI_GATEWAY_ID: Theidof your AI Gateway. -
!env OPENAI_API_KEY: Your OpenAI API key
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "configure-ai-llm-as-judge",
"display_name": "Configure the AI LLM as Judge policy",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only — no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3
},
"global": true
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "configure-ai-llm-as-judge"
display_name = "Configure the AI LLM as Judge policy"
type = "ai-llm-as-judge"
enabled = true
global = true
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only — no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}After creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
ai_gateway_models:
- ref: my-gpt-4o
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: my-gpt-4o
name: my-gpt-4o
type: model
capabilities:
- generate
formats:
- type: openai
targets:
- name: gpt-4o
provider: my-openai-account
config:
type: openai
config:
route:
paths:
- "/v1"
model:
body_param: model
values:
- my-gpt-4o
logging:
payloads: false
policies:
- !ref configure-ai-llm-as-judge#nameAfter creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/models/my-gpt-4o \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "my-gpt-4o",
"name": "my-gpt-4o",
"type": "model",
"capabilities": [
"generate"
],
"formats": [
{
"type": "openai"
}
],
"policies": [
"configure-ai-llm-as-judge"
],
"targets": [
{
"name": "gpt-4o",
"provider": "my-openai-account",
"config": {
"type": "openai"
}
}
],
"config": {
"route": {
"paths": [
"/v1"
],
"model": {
"body_param": "model",
"values": [
"my-gpt-4o"
]
}
},
"logging": {
"payloads": false
}
}
}
'After creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
resource "konnect_ai_gateway_model" "my_model" {
display_name = "my-gpt-4o"
name = "my-gpt-4o"
type = "model"
capabilities = ["generate"]
formats = [
{
type = "openai"
} ]
targets = [
{
name = "gpt-4o"
provider = "my-openai-account"
config = {
type = "openai"
}
} ]
config = jsonencode({
route = {
paths = ["/v1"]
model = {
body_param = "model"
values = ["my-gpt-4o"]
}
}
logging = {
payloads = false
}
})
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}After creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.
ai_gateway_consumers:
- ref: mobile-app-production
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: Mobile App - Production
name: mobile-app-production
type: api-key
policies:
- !ref configure-ai-llm-as-judge#nameAfter creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/consumers/mobile-app-production \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "Mobile App - Production",
"name": "mobile-app-production",
"type": "api-key",
"policies": [
"configure-ai-llm-as-judge"
]
}
'After creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.
resource "konnect_ai_gateway_consumer" "my_consumer" {
display_name = "Mobile App - Production"
name = "mobile-app-production"
type = "api-key"
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}After creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.
ai_gateway_consumer_groups:
- ref: internal-teams
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: Internal Teams
name: internal-teams
policies:
- !ref configure-ai-llm-as-judge#nameAfter creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/consumer-groups/internal-teams \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "Internal Teams",
"name": "internal-teams",
"policies": [
"configure-ai-llm-as-judge"
]
}
'After creating the policy, add configure-ai-llm-as-judge to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.
resource "konnect_ai_gateway_consumer_group" "my_consumer_group" {
display_name = "Internal Teams"
name = "internal-teams"
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}