AI Proxy: Configure AI Proxy for Claude Code with Google Vertex AI - Plugin

Configure AI Proxy for Claude Code with Google Vertex AI

v3.13+Set up the AI Proxy plugin to work with Claude Code, using Google Vertex AI with Gemini 2.5 Flash model. For a detailed guide on how to use Google Vertex AI with Claude Code see this guide

Prerequisites

Google Cloud Platform account
Vertex AI API enabled
Service account with Vertex AI permissions

Environment variables

GCP_SERVICE_ACCOUNT_JSON: The GCP service account JSON credentials for authentication.
VERTEX_API_ENDPOINT: The Vertex AI API endpoint (for example, us-east5-aiplatform.googleapis.com).
GCP_PROJECT_ID: The GCP project ID.
GCP_LOCATION_ID: The GCP location/region ID (for example, us-east5).

Set up the plugin

Add this section to your kong.yaml configuration file:

kong.yaml

Copied!

_format_version: "3.0"
plugins:
  - name: ai-proxy
    config:
      llm_format: anthropic
      route_type: llm/v1/chat
      logging:
        log_payloads: false
        log_statistics: true
      auth:
        allow_override: false
        gcp_use_service_account: true
        gcp_service_account_json: ${{ env "DECK_GCP_SERVICE_ACCOUNT_JSON" }}
      model:
        provider: gemini
        name: gemini-2.5-flash
        options:
          anthropic_version: vertex-2023-10-16
          gemini:
            api_endpoint: ${{ env "DECK_VERTEX_API_ENDPOINT" }}
            project_id: ${{ env "DECK_GCP_PROJECT_ID" }}
            location_id: ${{ env "DECK_GCP_LOCATION_ID" }}

Make the following request:

curl -i -X POST http://localhost:8001/plugins/ \
    --header "Accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make the following request:

curl -X POST https://{region}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
controlPlaneId: The id of the control plane.

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

echo "
apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
  name: ai-proxy
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/tags: ''
  labels:
    global: 'true'
config:
  llm_format: anthropic
  route_type: llm/v1/chat
  logging:
    log_payloads: false
    log_statistics: true
  auth:
    allow_override: false
    gcp_use_service_account: true
    gcp_service_account_json: '$GCP_SERVICE_ACCOUNT_JSON'
  model:
    provider: gemini
    name: gemini-2.5-flash
    options:
      anthropic_version: vertex-2023-10-16
      gemini:
        api_endpoint: '$VERTEX_API_ENDPOINT'
        project_id: '$GCP_PROJECT_ID'
        location_id: '$GCP_LOCATION_ID'
plugin: ai-proxy
" | kubectl apply -f -

Copied!

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_ai_proxy" "my_ai_proxy" {
  enabled = true

  config = {
    llm_format = "anthropic"
    route_type = "llm/v1/chat"

    logging = {
      log_payloads = false
      log_statistics = true
    }

    auth = {
      allow_override = false
      gcp_use_service_account = true
      gcp_service_account_json = var.gcp_service_account_json
    }

    model = {
      provider = "gemini"
      name = "gemini-2.5-flash"

      options = {
        anthropic_version = "vertex-2023-10-16"

        gemini = {
          api_endpoint = var.vertex_api_endpoint
          project_id = var.gcp_project_id
          location_id = var.gcp_location_id
        }
      }
    }
  }
  tags = []

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "gcp_location_id" {
  type = string
}

Copied!

Add this section to your kong.yaml configuration file:

kong.yaml

Copied!

_format_version: "3.0"
plugins:
  - name: ai-proxy
    service: serviceName|Id
    config:
      llm_format: anthropic
      route_type: llm/v1/chat
      logging:
        log_payloads: false
        log_statistics: true
      auth:
        allow_override: false
        gcp_use_service_account: true
        gcp_service_account_json: ${{ env "DECK_GCP_SERVICE_ACCOUNT_JSON" }}
      model:
        provider: gemini
        name: gemini-2.5-flash
        options:
          anthropic_version: vertex-2023-10-16
          gemini:
            api_endpoint: ${{ env "DECK_VERTEX_API_ENDPOINT" }}
            project_id: ${{ env "DECK_GCP_PROJECT_ID" }}
            location_id: ${{ env "DECK_GCP_LOCATION_ID" }}

Make sure to replace the following placeholders with your own values:

serviceName|Id: The id or name of the service the plugin configuration will target.

Make the following request:

curl -i -X POST http://localhost:8001/services/{serviceName|Id}/plugins/ \
    --header "Accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

serviceName|Id: The id or name of the service the plugin configuration will target.

Make the following request:

curl -X POST https://{region}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/services/{serviceId}/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
controlPlaneId: The id of the control plane.
serviceId: The id of the service the plugin configuration will target.

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: ai-proxy
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/tags: ''
config:
  llm_format: anthropic
  route_type: llm/v1/chat
  logging:
    log_payloads: false
    log_statistics: true
  auth:
    allow_override: false
    gcp_use_service_account: true
    gcp_service_account_json: '$GCP_SERVICE_ACCOUNT_JSON'
  model:
    provider: gemini
    name: gemini-2.5-flash
    options:
      anthropic_version: vertex-2023-10-16
      gemini:
        api_endpoint: '$VERTEX_API_ENDPOINT'
        project_id: '$GCP_PROJECT_ID'
        location_id: '$GCP_LOCATION_ID'
plugin: ai-proxy
" | kubectl apply -f -

Copied!

Next, apply the KongPlugin resource by annotating the service resource:

kubectl annotate -n kong service SERVICE_NAME konghq.com/plugins=ai-proxy

Copied!

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_ai_proxy" "my_ai_proxy" {
  enabled = true

  config = {
    llm_format = "anthropic"
    route_type = "llm/v1/chat"

    logging = {
      log_payloads = false
      log_statistics = true
    }

    auth = {
      allow_override = false
      gcp_use_service_account = true
      gcp_service_account_json = var.gcp_service_account_json
    }

    model = {
      provider = "gemini"
      name = "gemini-2.5-flash"

      options = {
        anthropic_version = "vertex-2023-10-16"

        gemini = {
          api_endpoint = var.vertex_api_endpoint
          project_id = var.gcp_project_id
          location_id = var.gcp_location_id
        }
      }
    }
  }
  tags = []

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
  service = {
    id = konnect_gateway_service.my_service.id
  }
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "gcp_location_id" {
  type = string
}

Copied!

Add this section to your kong.yaml configuration file:

kong.yaml

Copied!

_format_version: "3.0"
plugins:
  - name: ai-proxy
    route: routeName|Id
    config:
      llm_format: anthropic
      route_type: llm/v1/chat
      logging:
        log_payloads: false
        log_statistics: true
      auth:
        allow_override: false
        gcp_use_service_account: true
        gcp_service_account_json: ${{ env "DECK_GCP_SERVICE_ACCOUNT_JSON" }}
      model:
        provider: gemini
        name: gemini-2.5-flash
        options:
          anthropic_version: vertex-2023-10-16
          gemini:
            api_endpoint: ${{ env "DECK_VERTEX_API_ENDPOINT" }}
            project_id: ${{ env "DECK_GCP_PROJECT_ID" }}
            location_id: ${{ env "DECK_GCP_LOCATION_ID" }}

Make sure to replace the following placeholders with your own values:

routeName|Id: The id or name of the route the plugin configuration will target.

Make the following request:

curl -i -X POST http://localhost:8001/routes/{routeName|Id}/plugins/ \
    --header "Accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

routeName|Id: The id or name of the route the plugin configuration will target.

Make the following request:

curl -X POST https://{region}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/routes/{routeId}/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
controlPlaneId: The id of the control plane.
routeId: The id of the route the plugin configuration will target.

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: ai-proxy
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/tags: ''
config:
  llm_format: anthropic
  route_type: llm/v1/chat
  logging:
    log_payloads: false
    log_statistics: true
  auth:
    allow_override: false
    gcp_use_service_account: true
    gcp_service_account_json: '$GCP_SERVICE_ACCOUNT_JSON'
  model:
    provider: gemini
    name: gemini-2.5-flash
    options:
      anthropic_version: vertex-2023-10-16
      gemini:
        api_endpoint: '$VERTEX_API_ENDPOINT'
        project_id: '$GCP_PROJECT_ID'
        location_id: '$GCP_LOCATION_ID'
plugin: ai-proxy
" | kubectl apply -f -

Copied!

Next, apply the KongPlugin resource by annotating the httproute or ingress resource:

kubectl annotate -n kong httproute  konghq.com/plugins=ai-proxy

Copied!

kubectl annotate -n kong ingress  konghq.com/plugins=ai-proxy

Copied!

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_ai_proxy" "my_ai_proxy" {
  enabled = true

  config = {
    llm_format = "anthropic"
    route_type = "llm/v1/chat"

    logging = {
      log_payloads = false
      log_statistics = true
    }

    auth = {
      allow_override = false
      gcp_use_service_account = true
      gcp_service_account_json = var.gcp_service_account_json
    }

    model = {
      provider = "gemini"
      name = "gemini-2.5-flash"

      options = {
        anthropic_version = "vertex-2023-10-16"

        gemini = {
          api_endpoint = var.vertex_api_endpoint
          project_id = var.gcp_project_id
          location_id = var.gcp_location_id
        }
      }
    }
  }
  tags = []

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
  route = {
    id = konnect_gateway_route.my_route.id
  }
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "gcp_location_id" {
  type = string
}

Copied!

Add this section to your kong.yaml configuration file:

kong.yaml

Copied!

_format_version: "3.0"
plugins:
  - name: ai-proxy
    consumer: consumerName|Id
    config:
      llm_format: anthropic
      route_type: llm/v1/chat
      logging:
        log_payloads: false
        log_statistics: true
      auth:
        allow_override: false
        gcp_use_service_account: true
        gcp_service_account_json: ${{ env "DECK_GCP_SERVICE_ACCOUNT_JSON" }}
      model:
        provider: gemini
        name: gemini-2.5-flash
        options:
          anthropic_version: vertex-2023-10-16
          gemini:
            api_endpoint: ${{ env "DECK_VERTEX_API_ENDPOINT" }}
            project_id: ${{ env "DECK_GCP_PROJECT_ID" }}
            location_id: ${{ env "DECK_GCP_LOCATION_ID" }}

Make sure to replace the following placeholders with your own values:

consumerName|Id: The id or name of the consumer the plugin configuration will target.

Make the following request:

curl -i -X POST http://localhost:8001/consumers/{consumerName|Id}/plugins/ \
    --header "Accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

consumerName|Id: The id or name of the consumer the plugin configuration will target.

Make the following request:

curl -X POST https://{region}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/consumers/{consumerId}/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
controlPlaneId: The id of the control plane.
consumerId: The id of the consumer the plugin configuration will target.

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: ai-proxy
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/tags: ''
config:
  llm_format: anthropic
  route_type: llm/v1/chat
  logging:
    log_payloads: false
    log_statistics: true
  auth:
    allow_override: false
    gcp_use_service_account: true
    gcp_service_account_json: '$GCP_SERVICE_ACCOUNT_JSON'
  model:
    provider: gemini
    name: gemini-2.5-flash
    options:
      anthropic_version: vertex-2023-10-16
      gemini:
        api_endpoint: '$VERTEX_API_ENDPOINT'
        project_id: '$GCP_PROJECT_ID'
        location_id: '$GCP_LOCATION_ID'
plugin: ai-proxy
" | kubectl apply -f -

Copied!

Next, apply the KongPlugin resource by annotating the KongConsumer resource:

kubectl annotate -n kong kongconsumer CONSUMER_NAME konghq.com/plugins=ai-proxy

Copied!

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_ai_proxy" "my_ai_proxy" {
  enabled = true

  config = {
    llm_format = "anthropic"
    route_type = "llm/v1/chat"

    logging = {
      log_payloads = false
      log_statistics = true
    }

    auth = {
      allow_override = false
      gcp_use_service_account = true
      gcp_service_account_json = var.gcp_service_account_json
    }

    model = {
      provider = "gemini"
      name = "gemini-2.5-flash"

      options = {
        anthropic_version = "vertex-2023-10-16"

        gemini = {
          api_endpoint = var.vertex_api_endpoint
          project_id = var.gcp_project_id
          location_id = var.gcp_location_id
        }
      }
    }
  }
  tags = []

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
  consumer = {
    id = konnect_gateway_consumer.my_consumer.id
  }
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "gcp_location_id" {
  type = string
}

Copied!

Add this section to your kong.yaml configuration file:

kong.yaml

Copied!

_format_version: "3.0"
plugins:
  - name: ai-proxy
    consumer_group: consumerGroupName|Id
    config:
      llm_format: anthropic
      route_type: llm/v1/chat
      logging:
        log_payloads: false
        log_statistics: true
      auth:
        allow_override: false
        gcp_use_service_account: true
        gcp_service_account_json: ${{ env "DECK_GCP_SERVICE_ACCOUNT_JSON" }}
      model:
        provider: gemini
        name: gemini-2.5-flash
        options:
          anthropic_version: vertex-2023-10-16
          gemini:
            api_endpoint: ${{ env "DECK_VERTEX_API_ENDPOINT" }}
            project_id: ${{ env "DECK_GCP_PROJECT_ID" }}
            location_id: ${{ env "DECK_GCP_LOCATION_ID" }}

Make sure to replace the following placeholders with your own values:

consumerGroupName|Id: The id or name of the consumer group the plugin configuration will target.

Make the following request:

curl -i -X POST http://localhost:8001/consumer_groups/{consumerGroupName|Id}/plugins/ \
    --header "Accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

consumerGroupName|Id: The id or name of the consumer group the plugin configuration will target.

Make the following request:

curl -X POST https://{region}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/consumer_groups/{consumerGroupId}/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "llm_format": "anthropic",
        "route_type": "llm/v1/chat",
        "logging": {
          "log_payloads": false,
          "log_statistics": true
        },
        "auth": {
          "allow_override": false,
          "gcp_use_service_account": true,
          "gcp_service_account_json": "'$GCP_SERVICE_ACCOUNT_JSON'"
        },
        "model": {
          "provider": "gemini",
          "name": "gemini-2.5-flash",
          "options": {
            "anthropic_version": "vertex-2023-10-16",
            "gemini": {
              "api_endpoint": "'$VERTEX_API_ENDPOINT'",
              "project_id": "'$GCP_PROJECT_ID'",
              "location_id": "'$GCP_LOCATION_ID'"
            }
          }
        }
      },
      "tags": []
    }
    '

Copied!

Make sure to replace the following placeholders with your own values:

region: Geographic region where your Kong Konnect is hosted and operates.
KONNECT_TOKEN: Your Personal Access Token (PAT) associated with your Konnect account.
controlPlaneId: The id of the control plane.
consumerGroupId: The id of the consumer group the plugin configuration will target.

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: ai-proxy
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/tags: ''
config:
  llm_format: anthropic
  route_type: llm/v1/chat
  logging:
    log_payloads: false
    log_statistics: true
  auth:
    allow_override: false
    gcp_use_service_account: true
    gcp_service_account_json: '$GCP_SERVICE_ACCOUNT_JSON'
  model:
    provider: gemini
    name: gemini-2.5-flash
    options:
      anthropic_version: vertex-2023-10-16
      gemini:
        api_endpoint: '$VERTEX_API_ENDPOINT'
        project_id: '$GCP_PROJECT_ID'
        location_id: '$GCP_LOCATION_ID'
plugin: ai-proxy
" | kubectl apply -f -

Copied!

Next, apply the KongPlugin resource by annotating the KongConsumerGroup resource:

kubectl annotate -n kong kongconsumergroup CONSUMERGROUP_NAME konghq.com/plugins=ai-proxy

Copied!

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "$KONNECT_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Copied!

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_ai_proxy" "my_ai_proxy" {
  enabled = true

  config = {
    llm_format = "anthropic"
    route_type = "llm/v1/chat"

    logging = {
      log_payloads = false
      log_statistics = true
    }

    auth = {
      allow_override = false
      gcp_use_service_account = true
      gcp_service_account_json = var.gcp_service_account_json
    }

    model = {
      provider = "gemini"
      name = "gemini-2.5-flash"

      options = {
        anthropic_version = "vertex-2023-10-16"

        gemini = {
          api_endpoint = var.vertex_api_endpoint
          project_id = var.gcp_project_id
          location_id = var.gcp_location_id
        }
      }
    }
  }
  tags = []

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
  consumer_group = {
    id = konnect_gateway_consumer_group.my_consumer_group.id
  }
}

Copied!

This example requires the following variables to be added to your manifest. You can specify values at runtime by setting TF_VAR_name=value.

variable "gcp_location_id" {
  type = string
}

Copied!

AI Proxy

Configure AI Proxy for Claude Code with Google Vertex AI

Prerequisites

Environment variables

Set up the plugin

Help us make these docs great!

Still need help