Allow and deny custom User-Agents
Allow a client built on the Go HTTP library, and block any User-Agent header that contains scraper, in any case.
In this example:
- The
allowpattern letsGo-http-clientthrough, even though it’s on the built-in list. - The
denypattern blocks anyUser-Agentthat containsscraper. The(?i)prefix makes the match case-insensitive.
ai_gateway_policies:
- ref: allow-and-deny-custom-user-agents
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: allow-and-deny-custom-user-agents
display_name: Bot Detection with custom patterns
type: bot-detection
enabled: true
config:
allow:
- Go-http-client
deny:
- "(?i)scraper"
global: falseMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "allow-and-deny-custom-user-agents",
"display_name": "Bot Detection with custom patterns",
"type": "bot-detection",
"enabled": true,
"config": {
"allow": [
"Go-http-client"
],
"deny": [
"(?i)scraper"
]
},
"global": false
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "allow-and-deny-custom-user-agents"
display_name = "Bot Detection with custom patterns"
type = "bot-detection"
enabled = true
global = false
config = jsonencode({
allow = ["Go-http-client"]
deny = ["(?i)scraper"]
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}ai_gateway_policies:
- ref: allow-and-deny-custom-user-agents
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: allow-and-deny-custom-user-agents
display_name: Bot Detection with custom patterns
type: bot-detection
enabled: true
config:
allow:
- Go-http-client
deny:
- "(?i)scraper"
global: falseMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "allow-and-deny-custom-user-agents",
"display_name": "Bot Detection with custom patterns",
"type": "bot-detection",
"enabled": true,
"config": {
"allow": [
"Go-http-client"
],
"deny": [
"(?i)scraper"
]
},
"global": false
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "allow-and-deny-custom-user-agents"
display_name = "Bot Detection with custom patterns"
type = "bot-detection"
enabled = true
global = false
config = jsonencode({
allow = ["Go-http-client"]
deny = ["(?i)scraper"]
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}ai_gateway_policies:
- ref: allow-and-deny-custom-user-agents
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: allow-and-deny-custom-user-agents
display_name: Bot Detection with custom patterns
type: bot-detection
enabled: true
config:
allow:
- Go-http-client
deny:
- "(?i)scraper"
global: falseMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "allow-and-deny-custom-user-agents",
"display_name": "Bot Detection with custom patterns",
"type": "bot-detection",
"enabled": true,
"config": {
"allow": [
"Go-http-client"
],
"deny": [
"(?i)scraper"
]
},
"global": false
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "allow-and-deny-custom-user-agents"
display_name = "Bot Detection with custom patterns"
type = "bot-detection"
enabled = true
global = false
config = jsonencode({
allow = ["Go-http-client"]
deny = ["(?i)scraper"]
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}ai_gateway_policies:
- ref: allow-and-deny-custom-user-agents
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
name: allow-and-deny-custom-user-agents
display_name: Bot Detection with custom patterns
type: bot-detection
enabled: true
config:
allow:
- Go-http-client
deny:
- "(?i)scraper"
global: trueMake sure to replace the following placeholders with your own values:
AI_GATEWAY_ID: Theidof your AI Gateway.
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"name": "allow-and-deny-custom-user-agents",
"display_name": "Bot Detection with custom patterns",
"type": "bot-detection",
"enabled": true,
"config": {
"allow": [
"Go-http-client"
],
"deny": [
"(?i)scraper"
]
},
"global": true
}
'Make sure to replace the following placeholders with your own values:
-
region: Geographic region where your Kong Konnect is hosted and operates. -
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account. -
AIGatewayId: Theidof the AI Gateway.
See the Konnect AI Gateway API reference to learn about region-specific URLs and personal access tokens.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "$KONNECT_TOKEN"
server_url = "https://us.api.konghq.com/"
}Add the following to your Terraform configuration:
resource "konnect_ai_gateway_policy" "my_policy" {
name = "allow-and-deny-custom-user-agents"
display_name = "Bot Detection with custom patterns"
type = "bot-detection"
enabled = true
global = true
config = jsonencode({
allow = ["Go-http-client"]
deny = ["(?i)scraper"]
})
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}After creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
ai_gateway_models:
- ref: my-gpt-4o
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: my-gpt-4o
name: my-gpt-4o
type: model
capabilities:
- generate
formats:
- type: openai
targets:
- name: gpt-4o
provider: my-openai-account
config:
type: openai
config:
route:
paths:
- "/v1"
model:
body_param: model
values:
- my-gpt-4o
logging:
payloads: false
policies:
- !ref allow-and-deny-custom-user-agents#nameAfter creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/models/my-gpt-4o \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "my-gpt-4o",
"name": "my-gpt-4o",
"type": "model",
"capabilities": [
"generate"
],
"formats": [
{
"type": "openai"
}
],
"policies": [
"allow-and-deny-custom-user-agents"
],
"targets": [
{
"name": "gpt-4o",
"provider": "my-openai-account",
"config": {
"type": "openai"
}
}
],
"config": {
"route": {
"paths": [
"/v1"
],
"model": {
"body_param": "model",
"values": [
"my-gpt-4o"
]
}
},
"logging": {
"payloads": false
}
}
}
'After creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI Model. The tool examples update an existing AI Model with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Model.
resource "konnect_ai_gateway_model" "my_model" {
display_name = "my-gpt-4o"
name = "my-gpt-4o"
type = "model"
capabilities = ["generate"]
formats = [
{
type = "openai"
} ]
targets = [
{
name = "gpt-4o"
provider = "my-openai-account"
config = {
type = "openai"
}
} ]
config = jsonencode({
route = {
paths = ["/v1"]
model = {
body_param = "model"
values = ["my-gpt-4o"]
}
}
logging = {
payloads = false
}
})
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}After creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI Agent. The tool examples update an existing AI Agent with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Agent.
ai_gateway_agents:
- ref: kongair-flight-booking-agent
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: KongAir Flight Booking Agent
name: kongair-flight-booking-agent
type: a2a
access:
acls:
allow:
- internal-teams
config:
url: https://booking-agent.internal.kongair.com
route:
paths:
- "/kongair-flight-booking"
logging:
payloads: false
max_payload_size: 1048576
policies:
- !ref allow-and-deny-custom-user-agents#nameAfter creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI Agent. The tool examples update an existing AI Agent with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Agent.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/agents/kongair-flight-booking-agent \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "KongAir Flight Booking Agent",
"name": "kongair-flight-booking-agent",
"type": "a2a",
"access": {
"acls": {
"allow": [
"internal-teams"
]
}
},
"policies": [
"allow-and-deny-custom-user-agents"
],
"config": {
"url": "https://booking-agent.internal.kongair.com",
"route": {
"paths": [
"/kongair-flight-booking"
]
},
"logging": {
"payloads": false,
"max_payload_size": 1048576
}
}
}
'After creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI Agent. The tool examples update an existing AI Agent with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI Agent.
resource "konnect_ai_gateway_agent" "my_agent" {
display_name = "KongAir Flight Booking Agent"
name = "kongair-flight-booking-agent"
type = "a2a"
access = {
acls = {
allow = ["internal-teams"]
}
}
config = jsonencode({
url = "https://booking-agent.internal.kongair.com"
route = {
paths = ["/kongair-flight-booking"]
}
logging = {
payloads = false
max_payload_size = 1048576
}
})
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}After creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI MCP Server. The tool examples update an existing AI MCP Server with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI MCP Server.
ai_gateway_mcp_servers:
- ref: petstore-mcp
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: Petstore API
name: petstore-mcp
type: conversion-listener
enabled: true
access:
acl_attribute_type: consumer
acls:
allow: []
default_tool_acls:
deny: []
config:
url: http://host.docker.internal:8080/api/v3
route:
paths:
- "/petstore"
logging:
payloads: false
server:
timeout: 60000
tools:
- name: get-pets-by-status
description: Find pets by status
method: GET
path: "/petstore/pet/findByStatus"
parameters:
- name: status
in: query
required: true
schema:
type: string
enum:
- available
- pending
- sold
description: Status value to filter pets by
- name: get-pet-by-id
description: Get pet by ID
method: GET
path: "/petstore/pet/{petId}"
parameters:
- name: petId
in: path
required: true
schema:
type: integer
description: ID of the pet to retrieve
policies:
- !ref allow-and-deny-custom-user-agents#nameAfter creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI MCP Server. The tool examples update an existing AI MCP Server with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI MCP Server.
curl -X PUT https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/mcp-servers/petstore-mcp \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "Petstore API",
"name": "petstore-mcp",
"type": "conversion-listener",
"enabled": true,
"policies": [
"allow-and-deny-custom-user-agents"
],
"access": {
"acl_attribute_type": "consumer",
"acls": {
"allow": []
},
"default_tool_acls": {
"deny": []
}
},
"config": {
"url": "http://host.docker.internal:8080/api/v3",
"route": {
"paths": [
"/petstore"
]
},
"logging": {
"payloads": false
},
"server": {
"timeout": 60000
}
},
"tools": [
{
"name": "get-pets-by-status",
"description": "Find pets by status",
"method": "GET",
"path": "/petstore/pet/findByStatus",
"parameters": [
{
"name": "status",
"in": "query",
"required": true,
"schema": {
"type": "string",
"enum": [
"available",
"pending",
"sold"
]
},
"description": "Status value to filter pets by"
}
]
},
{
"name": "get-pet-by-id",
"description": "Get pet by ID",
"method": "GET",
"path": "/petstore/pet/{petId}",
"parameters": [
{
"name": "petId",
"in": "path",
"required": true,
"schema": {
"type": "integer"
},
"description": "ID of the pet to retrieve"
}
]
}
]
}
'After creating the policy, add allow-and-deny-custom-user-agents to the policies array of your AI MCP Server. The tool examples update an existing AI MCP Server with the policy attached: replace the entity configuration in the example with your own, as the update replaces the entire AI MCP Server.
resource "konnect_ai_gateway_mcp_server" "my_mcp_server" {
display_name = "Petstore API"
name = "petstore-mcp"
type = "conversion-listener"
enabled = true
access = {
acl_attribute_type = "consumer"
acls = {
allow = []
}
default_tool_acls = {
deny = []
}
}
tools = [
{
name = "get-pets-by-status"
description = "Find pets by status"
method = "GET"
path = "/petstore/pet/findByStatus"
parameters = [
{
name = "status"
in = "query"
required = true
schema = {
type = "string"
enum = ["available", "pending", "sold"]
}
description = "Status value to filter pets by"
} ]
},
{
name = "get-pet-by-id"
description = "Get pet by ID"
method = "GET"
path = "/petstore/pet/{petId}"
parameters = [
{
name = "petId"
in = "path"
required = true
schema = {
type = "integer"
}
description = "ID of the pet to retrieve"
} ]
} ]
config = jsonencode({
url = "http://host.docker.internal:8080/api/v3"
route = {
paths = ["/petstore"]
}
logging = {
payloads = false
}
server = {
timeout = 60000
}
})
policies = [konnect_ai_gateway_policy.my_policy.name]
gateway_id = konnect_ai_gateway.my_ai_gateway.id
}