Route requests to the model Switchyard selects
Configure the AI Routing Provider policy in enforce mode.
The policy sends each request to the AI Model for the target that Switchyard selects.
If Switchyard is slow, unreachable or returns a target you haven’t listed, the policy falls back to default_target.
Prerequisites
-
A running NVIDIA Switchyard decision service that the data plane can reach.
-
The AI Routing Provider custom policy registered, and the data plane running with
KONG_UNTRUSTED_LUA=on. -
An existing AI Model for each target, with no policies attached, and an AI Model that clients call.