Monetize LLM traffic in Konnect

Before you configure Metering & Billing, you can set up a Consumer, Kong Air. Consumers let you identify the client that’s interacting with Kong Gateway. Later in this guide, you’ll be mapping this Consumer to a customer in Metering & Billing and assigning them to a Premium plan. Doing this allows you map existing Consumers that are already consuming your APIs to customers to make them billable.

echo '
_format_version: "3.0"
consumers:
  - username: kong-air
    keyauth_credentials:
    - key: hello_world
' | deck gateway apply -

To connect LLM usage to the Consumer, you’ll need to configure an authentication plugin. In this tutorial, we’ll use Key Authentication. This will require the Consumer to use an API key to access any Kong Gateway Services.

Configure the Key Auth plugin on the Service:

echo '
_format_version: "3.0"
plugins:
  - name: key-auth
    service: example-service
    config:
      key_names:
      - apikey
' | deck gateway apply -

To set up AI Proxy with OpenAI, specify the model and set the appropriate authentication header. To collect meters, you must also enable log_payloads and log_statistics.

In this example, we’ll use the gpt-4o model:

echo '
_format_version: "3.0"
plugins:
  - name: ai-proxy
    config:
      route_type: llm/v1/chat
      auth:
        header_name: Authorization
        header_value: Bearer ${{ env "DECK_OPENAI_API_KEY" }}
      model:
        provider: openai
        name: gpt-4o
      logging:
        log_payloads: true
        log_statistics: true
' | deck gateway apply -

In Metering & Billing, meters track and record the consumption of a resource or service over time.

1. In the Konnect sidebar, click Metering & Billing.
2. For AI Gateway Tokens, click Enable Related API Gateways.
3. Select the quickstart control plane.
4. Click Enable 1 Gateway.

You will see quickstart in the list of available meters.

Meters collect raw usage data, but features make that data billable. Without a feature, usage is tracked but not invoiced. Now that you’re metering LLM token usage, you need to label that as something you want to price or govern.

In this guide, you’ll create a feature for the example-service you created in the prerequisites.

1. In the Konnect sidebar, click Metering & Billing.
2. In the Metering & Billing sidebar, click Product Catalog.
3. Click Create Feature.
4. In the Name field, enter ai-token.
5. From the Meter dropdown menu, select “AI Gateway Tokens”.
6. Click Add group by filter. The group by filter ensures you only bill for LLM tokens from a specific provider.
7. From the Group by dropdown menu, select “Provider”.
8. From the Operator dropdown menu, select “Equals”.
9. In the Value dropdown menu, enter openai.
10. Click Add group by filter.
11. From the Group by dropdown menu, select “type”.
12. From the Operator dropdown menu, select “Equals”.
13. In the Value dropdown menu, enter request.
14. Click Save.

Plans are the core building blocks of your product catalog. They are a collection of rate cards that define the price and access of a feature.

A rate card describes price and usage limits or access control for a feature or item. Rate cards are made up of the associated feature, price, and optional usage limits or access control for the feature, called entitlements.

In this section, you’ll create a Premium plan that charges customers based on the AI token usage at a rate of $0.00002 per use.

1. In the Konnect sidebar, click Metering & Billing.
2. In the Metering & Billing sidebar, click Product Catalog.
3. Click the Plans tab.
4. Click Create Plan.
5. In the Name field, enter Token.
6. In the Billing cadence dropdown menu, select “1 month”.
7. Click Save.
8. Click Add Rate Card.
9. From the Feature dropdown menu, select “ai-token”.
10. Click Next Step.
11. From the Pricing model dropdown menu, select “Usage Based”.

12. In the Price per unit field, enter 1.

    We’re using $1 here to make it easy to see the cost changes in the customer invoice. Be sure to change this price in a production instance to match your own pricing model.

13. Click Next Step.
14. Select Boolean.
15. Click Save Rate Card.
16. Click Publish Plan.
17. Click Publish.

Customers are the entities who pay for the consumption. In many cases, it’s equal to your Consumer. Here you are going to create a customer and map our Consumer to it.

1. In the Konnect sidebar, click Metering & Billing.
2. In the Metering & Billing sidebar, click Billing.
3. Click Create Customer.
4. In the Name field, enter Kong Air.
5. In the Include usage from dropdown, select “kong-air”.
6. Click Save.
7. Click the Subscriptions tab.
8. Click Create a Subscription.
9. From the Subscribed Plan dropdown, select “Token”.
10. Click Next Step.
11. Click Start Subscription.

You can run the following command to test the that the Kong Air Consumer is invoiced correctly:

curl -X POST "$KONNECT_PROXY_URL/anything" \
     --no-progress-meter --fail-with-body  \
     -H "Accept: application/json"\
     -H "Content-Type: application/json"\
     -H "apikey: hello_world" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a mathematician"
         },
         {
           "role": "user",
           "content": "What is 1+1?"
         }
       ]
     }'

This will generate AI LLM token usage that will be captured by Metering & Billing.

1. In the Konnect sidebar, click Metering & Billing.
2. In the Metering & Billing sidebar, click Billing.
3. Click the Invoices tab.
4. Click Kong Air.
5. Click the Invoicing tab.
6. Click Preview Invoice.

You’ll see in Lines that ai-token is listed and was used once. In this guide, you’re using the sandbox for invoices. To deploy your subscription in production, configure a payments integration in Metering & Billing > Settings.