Use Gemini's googleSearch tool with AI Proxy Advanced in Kong AI Gateway

Deployment Platform
Tags
#ai
Minimum Version
Kong Gateway - 3.13
TL;DR

Configure the AI Proxy Advanced plugin with the Gemini provider and gemini-3-pro-preview model, then declare the googleSearch tool in your requests using the OpenAI tools array.

Prerequisites

This is a Konnect tutorial and requires a Konnect personal access token.

  1. Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.

  2. Export your token to an environment variable:

     export KONNECT_TOKEN='YOUR_KONNECT_PAT'
    
  3. Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:

     curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN --deck-output
    

    This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:

     export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
     export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
     export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
     export KONNECT_PROXY_URL='http://localhost:8000'
    

    Copy and paste these into your terminal to configure your session.

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

  1. Export your license to an environment variable:

     export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'
    
  2. Run the quickstart script:

    curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA 
    

    Once Kong Gateway is ready, you will see the following message:

     Kong Gateway Ready
    

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial, install decK version 1.43 or later.

This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance. We recommend upgrading your decK installation to take advantage of this tool.

You can check your current decK version with deck version.

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

  1. Run the following command:

    echo '
    _format_version: "3.0"
    services:
      - name: example-service
        url: http://httpbin.konghq.com/anything
    routes:
      - name: example-route
        paths:
        - "/anything"
        service:
          name: example-service
    ' | deck gateway apply -
    

To learn more about entities, you can read our entities documentation.

Before you begin, you must get the following credentials from Google Cloud:

  • Service Account Key: A JSON key file for a service account with Vertex AI permissions
  • Project ID: Your Google Cloud project identifier
  • API Endpoint: The global Vertex AI API endpoint https://aiplatform.googleapis.com

After creating the key, convert the contents of modelarmor-admin-key.json into a single-line JSON string. Escape all necessary characters — quotes (") and newlines (\n) — so that it becomes a valid one-line JSON string. Then export your credentials as environment variables:

export DECK_GCP_SERVICE_ACCOUNT_JSON="<single-line-escaped-json>"
export DECK_GCP_SERVICE_ACCOUNT_JSON="your-service-account-json"
export DECK_GCP_PROJECT_ID="your-project-id"

To complete this tutorial, you’ll need Python (version 3.7 or later) and pip installed on your machine. You can verify it by running:

python3
python3 -m pip --version

Install the OpenAI SDK:

pip install openai

Configure the plugin

First, configure AI Proxy Advanced to use the gemini-3-pro-preview model via Vertex AI:

echo '
_format_version: "3.0"
plugins:
  - name: ai-proxy-advanced
    config:
      genai_category: text/generation
      targets:
      - route_type: llm/v1/chat
        logging:
          log_payloads: false
          log_statistics: true
        model:
          provider: gemini
          name: gemini-3-pro-preview
          options:
            gemini:
              api_endpoint: aiplatform.googleapis.com
              project_id: "${{ env "DECK_GCP_PROJECT_ID" }}"
              location_id: global
        auth:
          allow_override: false
          gcp_use_service_account: true
          gcp_service_account_json: "${{ env "DECK_GCP_SERVICE_ACCOUNT_JSON" }}"
' | deck gateway apply -

Use the OpenAI SDK with googleSearch

Gemini 3 models support built-in tools including googleSearch, which allows the LLM to retrieve current information from the web. Unlike OpenAI function calling, Gemini’s built-in tools work automatically. The model decides when to use search based on the query, and integrates results directly into the response. For more information, see Gemini Built-in Tools.

To enable the googleSearch tool, add it to the tools array in your request. The tool declaration tells Gemini it has access to web search. Gemini uses this capability when the query requires current information.

Create a Python script to test the googleSearch tool:

cat << 'EOF' > google-search.py
#!/usr/bin/env python3
"""Test Gemini 3 googleSearch tool via Kong AI Gateway"""
from openai import OpenAI
import json
client = OpenAI(
    base_url="http://localhost:8000/anything",
    api_key="ignored"
)
print("Testing Gemini 3 googleSearch tool")
print("=" * 50)
print("\n=== Step 1: Current weather data ===")
response = client.chat.completions.create(
    model="gemini-3-pro-preview",
    messages=[
        {"role": "user", "content": "What's the current weather in San Francisco?"}
    ],
    tools=[
        {"googleSearch": {}}
    ]
)
content = response.choices[0].message.content
print(f"Response includes current data: {'✓' if '2025' in content else '✗'}")
print(f"\n{content}\n")
print("\n=== Step 2: Search with JSON output ===")
response = client.chat.completions.create(
    model="gemini-3-pro-preview",
    messages=[
        {"role": "user", "content": "Find the top 3 AI conferences in 2025. Return as JSON with name, date, location fields."}
    ],
    tools=[
        {"googleSearch": {}}
    ],
    response_format={"type": "json_object"}
)
content = response.choices[0].message.content
if content.startswith("```"):
    lines = content.split("\n")
    content_clean = "\n".join(lines[1:-1])
else:
    content_clean = content
try:
    parsed = json.loads(content_clean)
    print(f"✓ Valid JSON response")
    print(f"  Type: {type(parsed).__name__}")
    if isinstance(parsed, list):
        print(f"  Items: {len(parsed)}")
except Exception as e:
    print(f"Parse result: {e}")
print(f"\n{content}\n")
print("\n=== Step 3: Query without search need ===")
response = client.chat.completions.create(
    model="gemini-3-pro-preview",
    messages=[
        {"role": "user", "content": "What is 2+2?"}
    ],
    tools=[
        {"googleSearch": {}}
    ]
)
content = response.choices[0].message.content
print(f"Simple answer: {content}\n")
print("=" * 50)
print("Complete")
EOF

This script goes through three scenarios:

  1. Current data query: Asks for real-time weather information. Gemini uses search to retrieve current data.
  2. Structured output with search: Requests conference information formatted as JSON. Combines search with structured output.
  3. Query without search need: Asks a simple math question. Gemini answers directly without using search.

The OpenAI SDK sends requests to Kong AI Gateway using the OpenAI chat completions format. The tools array declares available capabilities. Kong AI Gateway transforms the OpenAI-format request into Gemini’s native format, forwards it to Vertex AI, and converts the response back to OpenAI format. Search results appear directly in the response content, not as separate tool_calls objects.

Run the script:

python3 google-search.py

Example output:

Testing Gemini 3 googleSearch tool
==================================================

=== Test 1: Current Weather Data ===
Response includes current data: ✓

As of 1:30 AM PST on Thursday, December 11, 2025, the weather in San Francisco is clear with a temperature of 46°F (8°C).

Here are the details:
*   Feels Like: 43°F (6°C)
*   Humidity: 91%
*   Wind: NNE at 7-8 mph
*   Forecast: Expect sunny skies later today with a high near 56°F to 58°F.


=== Test 2: Search with JSON Output ===
✓ Valid JSON response
  Type: list
  Items: 3
```json
[
  {
    "name": "CVPR 2025",
    "date": "June 11–15, 2025",
    "location": "Nashville, Tennessee, USA"
  },
  {
    "name": "ICML 2025",
    "date": "July 13–19, 2025",
    "location": "Vancouver, Canada"
  },
  {
    "name": "NeurIPS 2025",
    "date": "December 2–7, 2025",
    "location": "San Diego, California, USA"
  }
]
```


=== Test 3: Query Without Search Need ===
Simple answer: 2 + 2 is 4.

==================================================
Complete

The first test shows current weather data with a specific timestamp, confirming that Gemini used search. The second test returns structured JSON with conference information. The third test demonstrates that Gemini answers simple questions directly without using search, even when the tool is available.

Cleanup

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d

FAQs

The googleSearch tool requires Kong Gateway 3.13 or later.

Gemini’s googleSearch is a built-in capability that Gemini uses automatically when needed. It does not create explicit tool_calls objects in the response. Search results are integrated directly into the response content.

No. Gemini decides when to use search based on the query. Including the googleSearch tool declaration gives Gemini the capability, but it only uses search when the query requires current information.

Yes. You can combine tools: [{"googleSearch": {}}] with response_format: {"type": "json_object"} to get search results formatted as JSON.

Something wrong?

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!