Return the judge score in a response header
Configure the AI LLM as Judge policy to return the score it assigns in a response header. Clients and downstream policies can read the score from the header without parsing the response body.
Prerequisites
-
An OpenAI API key.
-
An existing AI Model.
Environment variables
OPENAI_API_KEY: Your OpenAI API key
kongctl
policy.yaml
Copied!
ai_gateway_policies:
- ref: llm-as-judge-score-header
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: llm-as-judge-score-header
display_name: Return the judge score in a response header
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
inject_score_header: true
score_header_name: X-Judge-Score
global: falseMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "llm-as-judge-score-header",
"display_name": "Return the judge score in a response header",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only: no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3,
"inject_score_header": true,
"score_header_name": "X-Judge-Score"
},
"global": false
}
'Copied!
Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Copied!
Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "llm-as-judge-score-header"
display_name = "Return the judge score in a response header"
type = "ai-llm-as-judge"
enabled = true
global = false
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
inject_score_header = true
score_header_name = "X-Judge-Score"
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}Copied!
This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}Copied!
kongctl
policy.yaml
Copied!
ai_gateway_policies:
- ref: llm-as-judge-score-header
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: llm-as-judge-score-header
display_name: Return the judge score in a response header
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
inject_score_header: true
score_header_name: X-Judge-Score
global: falseMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "llm-as-judge-score-header",
"display_name": "Return the judge score in a response header",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only: no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3,
"inject_score_header": true,
"score_header_name": "X-Judge-Score"
},
"global": false
}
'Copied!
Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Copied!
Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "llm-as-judge-score-header"
display_name = "Return the judge score in a response header"
type = "ai-llm-as-judge"
enabled = true
global = false
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
inject_score_header = true
score_header_name = "X-Judge-Score"
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}Copied!
This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}Copied!
kongctl
policy.yaml
Copied!
ai_gateway_policies:
- ref: llm-as-judge-score-header
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: llm-as-judge-score-header
display_name: Return the judge score in a response header
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
inject_score_header: true
score_header_name: X-Judge-Score
global: falseMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "llm-as-judge-score-header",
"display_name": "Return the judge score in a response header",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only: no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3,
"inject_score_header": true,
"score_header_name": "X-Judge-Score"
},
"global": false
}
'Copied!
Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Copied!
Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "llm-as-judge-score-header"
display_name = "Return the judge score in a response header"
type = "ai-llm-as-judge"
enabled = true
global = false
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
inject_score_header = true
score_header_name = "X-Judge-Score"
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}Copied!
This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}Copied!
kongctl
policy.yaml
Copied!
ai_gateway_policies:
- ref: llm-as-judge-score-header
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: llm-as-judge-score-header
display_name: Return the judge score in a response header
type: ai-llm-as-judge
enabled: true
config:
prompt: |
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
http_timeout: 60000
https_verify: true
ignore_assistant_prompts: true
ignore_system_prompts: true
ignore_tool_prompts: true
sampling_rate: 1
llm:
auth:
allow_override: false
header_name: Authorization
header_value: Bearer $OPENAI_API_KEY
logging:
log_payloads: true
log_statistics: true
model:
name: gpt-4o
provider: openai
options:
temperature: 2
max_tokens: 5
top_p: 1
cohere:
embedding_input_type: classification
route_type: llm/v1/chat
message_countback: 3
inject_score_header: true
score_header_name: X-Judge-Score
global: trueMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "llm-as-judge-score-header",
"display_name": "Return the judge score in a response header",
"type": "ai-llm-as-judge",
"enabled": true,
"config": {
"prompt": "You are a strict evaluator. You will be given a request and a response.\nYour task is to judge whether the response is correct or incorrect. You must\nassign a score between 1 and 100, where: 100 represents a completely correct\nand ideal response, 1 represents a completely incorrect or irrelevant response.\nYour score must be a single number only: no text, labels, or explanations.\nUse the full range of values (e.g., 13, 47, 86), not just round numbers like\n10, 50, or 100. Be accurate and consistent, as this score will be used by another\nmodel for learning and evaluation.\n",
"http_timeout": 60000,
"https_verify": true,
"ignore_assistant_prompts": true,
"ignore_system_prompts": true,
"ignore_tool_prompts": true,
"sampling_rate": 1,
"llm": {
"auth": {
"allow_override": false,
"header_name": "Authorization",
"header_value": "Bearer '$OPENAI_API_KEY'"
},
"logging": {
"log_payloads": true,
"log_statistics": true
},
"model": {
"name": "gpt-4o",
"provider": "openai",
"options": {
"temperature": 2,
"max_tokens": 5,
"top_p": 1,
"cohere": {
"embedding_input_type": "classification"
}
}
},
"route_type": "llm/v1/chat"
},
"message_countback": 3,
"inject_score_header": true,
"score_header_name": "X-Judge-Score"
},
"global": true
}
'Copied!
Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Copied!
Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "llm-as-judge-score-header"
display_name = "Return the judge score in a response header"
type = "ai-llm-as-judge"
enabled = true
global = true
config = jsonencode({
prompt = <<EOF
You are a strict evaluator. You will be given a request and a response.
Your task is to judge whether the response is correct or incorrect. You must
assign a score between 1 and 100, where: 100 represents a completely correct
and ideal response, 1 represents a completely incorrect or irrelevant response.
Your score must be a single number only: no text, labels, or explanations.
Use the full range of values (e.g., 13, 47, 86), not just round numbers like
10, 50, or 100. Be accurate and consistent, as this score will be used by another
model for learning and evaluation.
EOF
http_timeout = 60000
https_verify = true
ignore_assistant_prompts = true
ignore_system_prompts = true
ignore_tool_prompts = true
sampling_rate = 1
llm = {
auth = {
allow_override = false
header_name = "Authorization"
header_value = "Bearer var.openai_api_key"
}
logging = {
log_payloads = true
log_statistics = true
}
model = {
name = "gpt-4o"
provider = "openai"
options = {
temperature = 2
max_tokens = 5
top_p = 1
cohere = {
embedding_input_type = "classification"
}
}
}
route_type = "llm/v1/chat"
}
message_countback = 3
inject_score_header = true
score_header_name = "X-Judge-Score"
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}Copied!
This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}Copied!
After creating the policy, add llm-as-judge-score-header to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
kongctl
model.yaml
Copied!
ai_gateway_models:
- ref: my-gpt-4o
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: my-gpt-4o
name: my-gpt-4o
type: model
capabilities:
- generate
formats:
- type: openai
targets:
- name: gpt-4o
provider: my-openai-account
config:
type: openai
config:
route:
paths:
- "/v1"
model:
body_param: model
values:
- my-gpt-4o
logging:
payloads: false
policies:
- !ref llm-as-judge-score-header#nameAfter creating the policy, add llm-as-judge-score-header to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/models/my-gpt-4o \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "my-gpt-4o",
"name": "my-gpt-4o",
"type": "model",
"capabilities": [
"generate"
],
"formats": [
{
"type": "openai"
}
],
"policies": [
"llm-as-judge-score-header"
],
"targets": [
{
"name": "gpt-4o",
"provider": "my-openai-account",
"config": {
"type": "openai"
}
}
],
"config": {
"route": {
"paths": [
"/v1"
],
"model": {
"body_param": "model",
"values": [
"my-gpt-4o"
]
}
},
"logging": {
"payloads": false
}
}
}
'Copied!
After creating the policy, add llm-as-judge-score-header to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
resource "konnect_ai_gateway_model" "my_model" {
display_name = "my-gpt-4o"
name = "my-gpt-4o"
type = "model"
capabilities = ["generate"]
formats = [
{
type = "openai"
} ]
targets = [
{
name = "gpt-4o"
provider = "my-openai-account"
config = {
type = "openai"
}
} ]
config = jsonencode({
route = {
paths = ["/v1"]
model = {
body_param = "model"
values = ["my-gpt-4o"]
}
}
logging = {
payloads = false
}
})
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}Copied!
This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}Copied!
After creating the policy, add llm-as-judge-score-header to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.
kongctl
consumer.yaml
Copied!
ai_gateway_consumers:
- ref: mobile-app-production
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: Mobile App - Production
name: mobile-app-production
type: api-key
policies:
- !ref llm-as-judge-score-header#nameAfter creating the policy, add llm-as-judge-score-header to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/consumers/mobile-app-production \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "Mobile App - Production",
"name": "mobile-app-production",
"type": "api-key",
"policies": [
"llm-as-judge-score-header"
]
}
'Copied!
After creating the policy, add llm-as-judge-score-header to the policies array of your AI Consumer. The tool examples update an existing AI Consumer with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer.
resource "konnect_ai_gateway_consumer" "my_consumer" {
display_name = "Mobile App - Production"
name = "mobile-app-production"
type = "api-key"
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}Copied!
This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}Copied!
After creating the policy, add llm-as-judge-score-header to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.
kongctl
consumer_group.yaml
Copied!
ai_gateway_consumer_groups:
- ref: internal-teams
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: Internal Teams
name: internal-teams
policies:
- !ref llm-as-judge-score-header#nameAfter creating the policy, add llm-as-judge-score-header to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/consumer-groups/internal-teams \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "Internal Teams",
"name": "internal-teams",
"policies": [
"llm-as-judge-score-header"
]
}
'Copied!
After creating the policy, add llm-as-judge-score-header to the policies array of your AI Consumer Group. The tool examples update an existing AI Consumer Group with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Consumer Group.
resource "konnect_ai_gateway_consumer_group" "my_consumer_group" {
display_name = "Internal Teams"
name = "internal-teams"
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}Copied!
This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.
variable "openai_api_key" {
type = string
}Copied!