Amazon SageMaker provider

You can proxy requests to Amazon SageMaker AI models through AI Gateway by creating AI Model Provider and AI Model entities. This reference documents all supported AI capabilities, configuration requirements, and provider-specific details needed for proper integration.

AI Gateway automatically routes requests to the appropriate Amazon SageMaker API endpoints. The following table shows the upstream paths used for each capability.

The following tables show the AI capabilities supported by the Amazon SageMaker provider when configuring AI Models.

By default, AI Gateway uses the path templates shown in the tables below (e.g., /chat/completions, /embeddings, etc.). To customize these paths, configure the config.paths field in your AI Model entity. Custom paths take the form {configured_path}/{template_path} — for example, if you set a custom path of /v2, requests to /embeddings would be routed to /v2/embeddings.

Text generation

Support for Amazon SageMaker text generation capabilities:

The base URL is https://runtime.sagemaker.{region}.amazonaws.com.

AI Gateway uses this URL automatically. You only need to configure a URL if you’re using a self-hosted or Amazon SageMaker-compatible endpoint, in which case set the upstream_url option in your AI Model configuration.

To use Amazon SageMaker with AI Gateway, configure a new AI Model Provider. You can then access supported AI Models from Amazon SageMaker.

Here’s a minimal configuration for chat completions:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/model-providers \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "SageMaker Production",
      "name": "my-sagemaker-account",
      "type": "sagemaker",
      "config": {
        "auth": {
          "type": "sagemaker",
          "aws": {
            "access_key_id": "'$AWS_ACCESS_KEY_ID'",
            "secret_access_key": "'$AWS_SECRET_ACCESS_KEY'"
          }
        }
      }
    }
    '

ai_gateway_model_providers:
  - ref: my-sagemaker-account
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: SageMaker Production
    name: my-sagemaker-account
    type: sagemaker
    config:
      auth:
        type: sagemaker
        aws:
          access_key_id: !env AWS_ACCESS_KEY_ID
          secret_access_key: !secret {source: !env AWS_SECRET_ACCESS_KEY}

For Amazon SageMaker, set auth to sagemaker and provide static IAM user credentials under aws, or omit them to fall back to the default AWS credentials provider chain (EC2 instance profiles, environment variables, and so on):

• access_key_id (optional): AWS access key ID for static IAM user credentials. Overrides the AWS_ACCESS_KEY_ID environment variable.
• secret_access_key (optional): AWS secret access key paired with access_key_id. Overrides the AWS_SECRET_ACCESS_KEY environment variable.
• session_token (optional): AWS session token for temporary credentials. Overrides the AWS_SESSION_TOKEN environment variable.

Amazon SageMaker can also use basic auth instead. See Outbound authentication on the AI Model Provider entity page for the full list of auth types.

Only the generate capability is supported for Amazon SageMaker targets.

A target is an entry in the targets array on the AI Model entity, not the AI Model Provider. The target name is the name of your SageMaker endpoint. Beyond the common target options (name, provider, weight), a target routing to Amazon SageMaker supports these config fields, grouped under aws and target:

The following target routes to a SageMaker endpoint in us-east-1 and selects a specific model, variant, and container on a multi-model endpoint:

targets:
  - name: my-sagemaker-endpoint
    provider: my-sagemaker-account
    config:
      type: sagemaker
      aws:
        region: us-east-1
      target:
        model: my-model.tar.gz
        variant: production-variant-1
        container_hostname: container-1