Vertex AI provider

You can proxy requests to Gemini Vertex AI models through AI Gateway using the AI Proxy and AI Proxy Advanced plugins. This reference documents all supported AI capabilities, configuration requirements, and provider-specific details needed for proper integration.

AI Gateway automatically routes requests to the appropriate Gemini Vertex API endpoints. The following table shows the upstream paths used for each capability.

The following tables show the AI capabilities supported by Gemini Vertex provider when used with the AI Proxy or the AI Proxy Advanced plugin.

Set the plugin’s route_type based on the capability you want to use. See the tables below for supported route types.

Text generation

Support for Gemini Vertex basic text generation capabilities including chat, completions, and embeddings:

Advanced text generation

Support for Gemini Vertex function calling to allow Gemini Vertex models to use external tools and APIs:

Processing

Support for Gemini Vertex file operations, batch operations, assistants, and response handling:

¹ Gemini Vertex does not have a dedicated Files API. File storage uses Google Cloud Storage, similar to AWS S3.

Image

Support for Gemini Vertex image generation and editing capabilities:

For requests with large payloads, consider increasing config.max_request_body_size to three times the raw binary size.

Supported image sizes and formats vary by model. Refer to your provider’s documentation for allowed dimensions and requirements.

Video

Support for Gemini Vertex video generation capabilities:

For requests with large payloads (video generation), consider increasing config.max_request_body_size to three times the raw binary size.

The base URL is https://aiplatform.googleapis.com/, where {route_type_path} is determined by the capability.

AI Gateway uses this URL automatically. You only need to configure a URL if you’re using a self-hosted or Gemini Vertex-compatible endpoint, in which case set the upstream_url plugin option.

By default, the AI Proxy plugin uses OpenAI-compatible request formats. Set config.llm_format to a native format to use Gemini Vertex-specific APIs and features.

The following native Gemini Vertex APIs are supported:

To use Gemini Vertex with AI Gateway, configure the AI Proxy or AI Proxy Advanced.

Here’s a minimal configuration for chat completions:

_format_version: "3.0"
plugins:
  - name: ai-proxy
    config:
      route_type: llm/v1/chat
      model:
        provider: gemini
        name: gemini-2.0-flash-exp
        options:
          gemini:
            api_endpoint: Bearer ${{ env "DECK_GCP_API_ENDPOINT" }}
            project_id: Bearer ${{ env "DECK_GCP_PROJECT_ID" }}
            location_id: Bearer ${{ env "DECK_GCP_LOCATION_ID" }}
      auth:
        gcp_use_service_account: true
        gcp_service_account_json: Bearer ${{ env "DECK_GCP_SERVICE_ACCOUNT_JSON" }}

curl -i -X POST http://localhost:8001/plugins/ \
    --header "Accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "route_type": "llm/v1/chat",
        "model": {
          "provider": "gemini",
          "name": "gemini-2.0-flash-exp",
          "options": {
            "gemini": {
              "api_endpoint": "Bearer '$GCP_API_ENDPOINT'",
              "project_id": "Bearer '$GCP_PROJECT_ID'",
              "location_id": "Bearer '$GCP_LOCATION_ID'"
            }
          }
        },
        "auth": {
          "gcp_use_service_account": true,
          "gcp_service_account_json": "Bearer '$GCP_SERVICE_ACCOUNT_JSON'"
        }
      }
    }
    '

curl -X POST https://{region}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "route_type": "llm/v1/chat",
        "model": {
          "provider": "gemini",
          "name": "gemini-2.0-flash-exp",
          "options": {
            "gemini": {
              "api_endpoint": "Bearer '$GCP_API_ENDPOINT'",
              "project_id": "Bearer '$GCP_PROJECT_ID'",
              "location_id": "Bearer '$GCP_LOCATION_ID'"
            }
          }
        },
        "auth": {
          "gcp_use_service_account": true,
          "gcp_service_account_json": "Bearer '$GCP_SERVICE_ACCOUNT_JSON'"
        }
      }
    }
    '

echo "
apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
  name: ai-proxy
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: 'true'
config:
  route_type: llm/v1/chat
  model:
    provider: gemini
    name: gemini-2.0-flash-exp
    options:
      gemini:
        api_endpoint: Bearer $GCP_API_ENDPOINT
        project_id: Bearer $GCP_PROJECT_ID
        location_id: Bearer $GCP_LOCATION_ID
  auth:
    gcp_use_service_account: true
    gcp_service_account_json: Bearer $GCP_SERVICE_ACCOUNT_JSON
plugin: ai-proxy
" | kubectl apply -f -

resource "konnect_gateway_plugin_ai_proxy" "my_ai_proxy" {
  enabled = true

  config = {
    route_type = "llm/v1/chat"

    model = {
      provider = "gemini"
      name = "gemini-2.0-flash-exp"

      options = {

        gemini = {
          api_endpoint = "Bearer var.gcp_api_endpoint"
          project_id = "Bearer var.gcp_project_id"
          location_id = "Bearer var.gcp_location_id"
        }
      }
    }

    auth = {
      gcp_use_service_account = true
      gcp_service_account_json = "Bearer var.gcp_service_account_json"
    }
  }

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
}

variable "gcp_api_endpoint" {
  type = string
}

For more configuration options and examples, see:

• AI Proxy examples
• AI Proxy Advanced examples

• Set up AI Proxy Advanced with Vertex AI in Kong Gateway View →
• Set up AI Proxy with Vertex AI in Kong Gateway View →
• Route Claude CLI traffic through AI Gateway and Vertex AI View →