Home / How-to Guides
Edit
Report

Install AI Gateway 2.x on-prem using Docker Compose

Uses: AI Gateway deck
Incompatible with
konnect
Related Documentation
AI Gateway Documentation decK Documentation
Related Resources
Configure AI Gateway on-prem
Install Kong Gateway using Docker Compose
Minimum Version
AI Gateway - 2.2
TL;DR

Create the example Docker Compose file and run docker compose up -d.

Prerequisites

decK   v1.69.0+

To complete this tutorial, install decK. We recommend keeping decK up to date with the latest version (1.69.0).

decK is a CLI tool for managing Kong Gateway declaratively with state files. This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance.

You can check your current decK version with deck version.

Docker Compose

This guide requires Docker installed on your system.

Set up the Docker Compose file

Run the following command to create docker-compose.yml:

cat <<EOF > docker-compose.yml

networks:
  kong:
    name: kong
    driver: bridge

# Image used by every Kong service in this file
x-kong-image: &kong-image
  image: kong/kong-ai-gateway:2.2.0

# Common environment variables used by Kong services (migrations and gateway)
x-kong-db-config: &kong-db-config
  KONG_DATABASE: postgres             # Use Postgres as the backing database
  KONG_PG_HOST: db                    # Hostname of the Postgres service
  KONG_PG_DATABASE: kong              # Name of the database to connect to
  KONG_PG_USER: kong                  # Database username
  KONG_PG_PASSWORD: kong              # Database password
  KONG_PASSWORD: password             # Admin GUI password (required for RBAC)
  KONG_LICENSE_DATA: "\${KONG_LICENSE_DATA}"  # Kong Enterprise license passed via environment variable

x-kong-container-common: &kong-container-common
  healthcheck:
    test: [ "CMD", "kong", "health" ]
    interval: 3s
    timeout: 3s
    retries: 10
    start_period: 3s
  restart: on-failure
  tmpfs:
    - /tmp

services:

  db:
    image: postgres:13
    networks:
      - kong
    environment:
      POSTGRES_DB: kong
      POSTGRES_USER: kong
      POSTGRES_PASSWORD: kong
    healthcheck:                      # Ensure the DB is ready before starting dependent services
      test: [ "CMD", "pg_isready", "-U", "kong" ]
      interval: 5s
      timeout: 5s
      retries: 10
    restart: on-failure

  kong-migrations:
    <<: *kong-image
    networks:
      - kong
    command: "kong migrations bootstrap"
    depends_on:
      db:
        condition: service_healthy    # Wait until Postgres is up and healthy
    environment:
      <<: *kong-db-config

  kong-gateway:
    <<: [ *kong-container-common, *kong-image ]
    deploy:
      replicas: 1
    mem_limit: 2g
    cpus: 4
    networks:
      - kong
    user: "\${KONG_USER:-kong}"
    ports:
      - 8000:8000   # Proxy HTTP
      - 8443:8443   # Proxy HTTPS
      - 8001:8001   # Admin API
      - 8002:8002   # Kong Manager (Admin GUI)
      - 8003:8003   # Dev Portal GUI
      - 8004:8004   # Dev Portal API
    depends_on:
      db:
        condition: service_healthy
      kong-migrations:
        condition: service_completed_successfully  # Start only after bootstrap has succeeded
    environment:
      <<: *kong-db-config
      KONG_ANONYMOUS_REPORTS: "off"
      KONG_PROXY_LISTEN: "0.0.0.0:8000,0.0.0.0:8443 ssl"
      KONG_ADMIN_LISTEN: "0.0.0.0:8001"
      KONG_ADMIN_GUI_LISTEN: "0.0.0.0:8002"
      KONG_PORTAL_GUI_LISTEN: "0.0.0.0:8003"
      KONG_PORTAL_API_LISTEN: "0.0.0.0:8004"
      KONG_LOG_LEVEL: "debug"
      KONG_PLUGINS: "bundled"
EOF

Start AI Gateway

Set your Enterprise license in the KONG_LICENSE_DATA environment variable:

export KONG_LICENSE_DATA='YOUR LICENSE DATA'

Then, start the containers:

docker compose up -d

Validate

You can validate AI Gateway is running using cURL against the Admin API:

curl -X GET "http://localhost:8001/" \
     --no-progress-meter --fail-with-body

This will return an HTTP/1.1 200 OK response.

Configure your on-prem AI Gateway 2.x

For AI Gateway 2.x running on-prem, you’ll use AI Gateway 2.x decK configuration files and deck ai sync to apply them. deck ai sync converts the AI Gateway 2.x AI entities and policies into compatible Kong Gateway 3.x configuration with plugins, Services, and Routes.

You can convert any kongctl AI Gateway 2.x configuration to decK by removing the fields that only apply to a Konnect-managed control plane:

  • Drop each entity’s ref field.
  • Drop each entity’s ai_gateway: !lookup {id: ...} field. On-prem, there’s only one AI Gateway, so entities don’t need to reference one by ID.
  • Rename the top-level entity collections from their kongctl names (ai_gateway_model_providers, ai_gateway_models, and so on) to their decK names (model_providers, models). See Configure AI Gateway on-prem for the full set of entity names decK expects.

See AI Gateway 2.x concepts for how the entity model maps to the underlying Kong Gateway primitives.

For example, here’s the AI Model Provider and AI Model from the AI Gateway get started guide written as a decK configuration file instead of a kongctl one. Set the DECK_OPENAI_API_KEY environment variable to your OpenAI API key, then apply the file with deck ai sync:

export DECK_OPENAI_API_KEY='Bearer YOUR OPENAI API KEY'

This example substitutes $DECK_OPENAI_API_KEY directly into the YAML using the shell, instead of decK’s own ${{ env "..." }} templating. That templating is the documented way to avoid hardcoding secrets (see Managing sensitive data), but it doesn’t resolve when an AI Gateway 2.x entity file is applied with deck ai sync/deck file ai2kong: the literal, unresolved string ends up in the generated plugin config instead of the real key.

cat <<EOF | deck ai sync --yes -
model_providers:
  - name: generic-openai
    display_name: "generic-openai"
    type: openai
    config:
      auth:
        type: basic
        headers:
          - name: Authorization
            value: $DECK_OPENAI_API_KEY

models:
  - name: my-gpt-4o
    display_name: "my-gpt-4o"
    type: model
    capabilities:
      - generate
    formats:
      - type: openai
    policies: []
    config:
      route:
        paths:
          - /v1
        protocols:
          - http
          - https
        model:
          body_param: model
          values:
            - my-gpt-4o
    targets:
      - name: gpt-4o
        provider: generic-openai
        config:
          type: openai
EOF

This example omits --kong-addr, since it defaults to http://localhost:8001, which is already where this Docker Compose file exposes the Admin API. If your AI Gateway isn’t reachable at that address, or needs authentication, see Configuring on-prem authentication.

Validate your configuration

Send a chat request to verify your setup:

curl -X POST "http://localhost:8000/v1/chat/completions" \
     --no-progress-meter --fail-with-body \
     -H "Accept: application/json" \
     -H "Content-Type: application/json" \
     -H "Authorization: $DECK_OPENAI_API_KEY" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "Say this is a test!"
         }
       ],
       "model": "my-gpt-4o"
     }'

Cleanup

Destroy the AI Gateway containers

 
docker compose down
Something wrong?
Report an Issue | Edit this Page
View all Kong documentation

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help?

Ask in our Slack Community
Contact Support