AI Vaults

You must store secrets like API keys and authentication tokens somewhere secure instead of embedding them directly in your configurations. An AI Vault entity lets you register an external secret backend (AWS Secrets Manager, HashiCorp Vault, environment variables, or others) so that AI Model Providers, AI Auth Strategies, AI Models, AI Agents, and AI MCP Servers can reference secrets instead of storing them as literal values.

An AI Vault entity stores the connection configuration and credentials needed to reach your secret backend. When other entities reference a secret, AI Gateway:

1. Looks up the vault at request time
2. Retrieves the actual secret value
3. Uses it for authentication or configuration.

AI Vaults can be created and managed through:

• Konnect UI
• AI Gateway API: /ai-gateways/{aiGatewayId}/vaults
• kongctl

For configuration examples and step-by-step setup instructions, see Set up an AI Vault.

Each AI Vault selects one of the supported secret backends:

• Konnect Config Store
• Environment variables
• AWS Secrets Manager
• Google Secret Manager
• Azure Key Vault
• CyberArk Conjur
• HashiCorp Vault

The connection details vary per backend; the Konnect UI surfaces the relevant fields based on the backend you choose.

AI Vault references can be used in sensitive fields across your AI Gateway entities:

Any field marked as supporting vault references can accept a secret reference instead of a literal value.

To reference a secret stored in a vault, use the syntax:

{vault://vault-name/secret-key}

Where:

• vault-name is the name field of the vault you created
• secret-key is the identifier of the secret within that vault (exact format depends on the backend)

For example, if you created a vault named prod-aws-vault and stored an OpenAI API key under the key openai-api-key, reference it as:

{vault://prod-aws-vault/openai-api-key}

Here’s how you’d use that reference in an AI Model Provider entity:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/model-providers \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "OpenAI Production",
      "name": "openai-prod",
      "type": "openai",
      "config": {
        "auth": {
          "type": "basic",
          "headers": [
            {
              "name": "Authorization",
              "value": "{vault://prod-aws-vault/openai-api-key}"
            }
          ]
        }
      }
    }
    '

ai_gateway_model_providers:
  - ref: openai-prod
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: OpenAI Production
    name: openai-prod
    type: openai
    config:
      auth:
        type: basic
        headers:
        - name: Authorization
          value: "{vault://prod-aws-vault/openai-api-key}"

The entire field value must be the vault reference string. You cannot use partial references like Bearer {vault://...}. The field itself must be exactly {vault://vault-name/secret-key}.

At request time, AI Gateway resolves the reference by looking up the vault name, retrieving the secret value, and using it for authentication or configuration.

Pick a backend matching your infrastructure and secret management strategy. Cloud-native deployments can use their platform’s secret service (aws, gcp, azure), enterprises can use dedicated secret management systems (conjur, hcv), and smaller deployments can use env (environment variables) or konnect (built-in Config Store).

Cloud-backed vault types (aws, gcp, azure, conjur, hcv) cache resolved secrets so AI Gateway doesn’t hit the backend on every request. This reduces latency and vault load. The env backend doesn’t cache because environment-variable lookups are local.

If your vault becomes unreachable, AI Gateway can continue using recently-cached secrets for a grace period, keeping your system operational during brief vault outages. This allows you to maintain service continuity even when secret infrastructure is temporarily unavailable.

Cache duration and grace periods are tunable per vault, allowing you to balance between fresh secrets (shorter cache times) and reduced vault requests (longer cache times). The default settings work for most deployments; adjust only if your secret rotation strategy or vault reliability requires custom behavior.

Unlike the other backends, the konnect type doesn’t connect out to an external secret manager. It stores secrets directly in Konnect, in a Config Store: a named container of key-value secrets that you create and populate through its own API, separate from the AI Vault entity itself.

A konnect-type AI Vault doesn’t hold any secret values. It only references a Config Store by ID through config.config_store_id. The Config Store holds the actual secrets.

Secret values are write-only. Once stored, AI Gateway never returns the value again, only the secret’s key and timestamps.

Manage Config Stores

Config Stores are managed through the AI Gateway API:

• Config Store: /ai-gateways/{aiGatewayId}/config-stores
• Config Store secrets: /ai-gateways/{aiGatewayId}/config-stores/{configStoreIdOrName}/secrets

Both support full create, list, get, update, and delete operations. Deleting a Config Store that still has secrets fails unless you pass ?force=true, which cascades the delete to all secrets in that Config Store.

Create a Config Store and add a secret

The following example creates a Config Store:

curl -X POST "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/config-stores" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Content-Type: application/json"\
     -H "Accept: application/json, application/problem+json" \
     --json '{
       "name": "prod-secrets"
     }'

Add a secret to the Config Store:

curl -X POST "https://us.api.konghq.com/v1/ai-gateways/$AI_GATEWAY_ID/config-stores/$CONFIG_STORE_ID/secrets" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN"\
     -H "Content-Type: application/json"\
     -H "Accept: application/json, application/problem+json" \
     --json '{
       "key": "openai-api-key",
       "value": "sk-my-openai-key"
     }'

Reference the Config Store from a konnect-type AI Vault

Create a konnect-type AI Vault that points at the Config Store’s id:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/vaults \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "prod-config-store-vault",
      "description": "Vault backed by the built-in Konnect Config Store.",
      "type": "konnect",
      "config": {
        "config_store_id": "'$CONFIG_STORE_ID'"
      }
    }
    '

ai_gateway_vaults:
  - ref: prod-config-store-vault
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: prod-config-store-vault
    description: Vault backed by the built-in Konnect Config Store.
    type: konnect
    config:
      config_store_id: !secret {source: !env CONFIG_STORE_ID}

Reference the secret the same way as any other AI Vault:

{vault://prod-config-store-vault/openai-api-key}

The following example registers an environment-variable AI Vault that resolves references against process environment variables prefixed with KONG_.

AI Vault doesn’t accept a display_name field. Only name and description identify a vault. If you include display_name when creating an AI Vault, AI Gateway silently ignores it.

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/vaults \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "prod-env-vault",
      "description": "Vault for production secrets sourced from environment variables.",
      "type": "env",
      "config": {
        "prefix": "KONG_"
      }
    }
    '

ai_gateway_vaults:
  - ref: prod-env-vault
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: prod-env-vault
    description: Vault for production secrets sourced from environment variables.
    type: env
    config:
      prefix: KONG_