Share the latest user prompt with Switchyard
Configure the AI Routing Provider policy to send Switchyard the most recent user message. By default the policy redacts every message before the decision request leaves the gateway, so Switchyard sees only roles and message sizes. Share the latest user prompt when your Switchyard route needs the text to classify the request.
Prerequisites
-
A running NVIDIA Switchyard decision service that the data plane can reach.
-
The AI Routing Provider custom policy registered, and the data plane running with
KONG_UNTRUSTED_LUA=on. -
An existing AI Model for each target, with no policies attached, and an AI Model that clients call.