The following are an overview of the general steps you should take to enable, monitor, and use Bot Detector:
- Enable it on a public Dedicated Cloud Gateway control plane.
- Monitor detections for any false positives or valid traffic that is marked as blocked.
- (Optional) Allow list IPs with the IP Restrictions plugin as a complement to Bot Detector.
- Create any custom rules from your observations while Bot Detector is in monitoring mode.
- After you’ve gained confidence in monitoring mode, switch Bot Detector to blocking mode.
Bot Detector is enabled per control plane.
Because Bot Detector is scoped to the control plane, not the organization, this allows you to enable it on a single control plane to evaluate it, leave your other control planes untouched, and expand once you’re satisfied with the results.
Enable Bot Detector for a control plane by sending a POST request to the /bot-gline-manager/waap/{cpId}/enabled endpoint:
curl -X POST "https://us.api.konghq.com/bot-gline-manager/waap/$CONTROL_PLANE_ID/enabled" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"enabled": true,
"block_mode": false
}'
Setting block_mode to false starts the control plane in monitoring mode.
- In the Konnect sidebar, click API Gateway.
- Click Control planes.
- Click the Dedicated Cloud Gateway control plane you want to enable Bot Detector for.
- Click the Bot Detector tab.
- Click Enable bot detector.
Once enabled, a control plane’s Bot Detector starts in monitoring mode.
After Bot Detector processes traffic in monitoring mode, it will populate the overview and detections dashboards with what it would have done to the traffic (passthrough or block).
Bot Detector provides two views for reviewing traffic:
- Overview: An aggregate dashboard showing status, mode, active rules, and request totals (monitored and blocked) over a selectable time range.
- Detections: A filterable log of every evaluated request, including the path, action taken, timestamp, IP, user agent, and matched rule. You can promote any detection directly into a passthrough rule using the action menu if it turns out to be a false positive.
If while you are in monitoring mode, you see traffic that is marked incorrectly as Block or Passthrough, you can create custom rules to block, monitor, or allow (passthrough) that traffic.
Additionally, if there are certain IPs you know need to passthrough (for example, an IP of a partner company), you can create rules for those.
These rules match on one or more conditions (IP, path, user agent, or JA4 fingerprint) and take precedence over Kong-managed rules.
You can specify multiple match types per rule, but all conditions on a rule must match for the rule to apply.
The following table describes which match types you can set and some example values:
|
Match type
|
Description
|
Exact match example
|
Regex example
|
|
IP address
|
Matches a single client IP address.
|
203.0.113.5
|
Not supported
|
|
CIDR range
|
Matches a range of client IP addresses in CIDR notation.
|
203.0.113.0/24
|
Not supported
|
|
JA4
|
Matches a client’s JA4 TLS fingerprint.
|
t13d1516h2_8daaf6152771_02713d6af862
|
t13d1516h2_.*
|
|
User agent
|
Matches the request’s User-Agent header.
|
curl/8.7.1
|
(?i)(bot|crawler)
|
|
Path
|
Matches the request path.
|
/health
|
^/api/v[0-9]+/users$
|
Path, user agent, and JA4 conditions also have an Operator setting to choose whether the value should match exactly or by regular expression.
IP address and CIDR range conditions only support exact matches.
-
List your existing rules by sending a GET request to the /bot-gline-manager/waap/{cpId}/rules endpoint:
curl -X GET "https://us.api.konghq.com/bot-gline-manager/waap/$CONTROL_PLANE_ID/rules" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN"
-
Create a rule by sending a POST request to the /bot-gline-manager/waap/{cpId}/rule endpoint. The following example creates a passthrough rule that matches a single path:
curl -X POST "https://us.api.konghq.com/bot-gline-manager/waap/$CONTROL_PLANE_ID/rule" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "Allow health check path",
"action": "passthrough",
"priority": 100,
"value": {
"application_rules": {
"paths": [
{
"match": "equals",
"value": "/health"
}
]
}
}
}'
- Set
"action" to "block" or "monitor" to create those rule types instead.
- Set
"match" to "regex" to match a path, user agent, or JA4 fingerprint using a regular expression instead of an exact match.
- To match on an IP address or CIDR range, use
"network_rules" with "ips" or "cidrs" instead of "application_rules".
- Add more entries to
paths, user_agents, ja4, ips, or cidrs to require multiple conditions on the same rule. All conditions on a rule must match for the rule to apply.
"priority" determines the order your rules are evaluated in relative to each other (higher integers runs sooner). It doesn’t affect the position of Kong-managed rules, which always run after all your rules.- To write an advanced rule instead of
application_rules/network_rules, set expression to a raw ATC expression. For example, http.path == "/health".
- In the Konnect sidebar, click API Gateway.
- Click Control planes.
- Click the Dedicated Cloud Gateway control with Bot Detector enabled.
- Click the Bot Detector tab.
- Click the Rules tab.
- Click New rule.
- For the General settings, do the following:
- In the Name field, enter a name for the rule.
- In the Priority field, enter a priority. Higher-priority rules are evaluated first, and the first matching rule takes precedence. For example, a rule with a priority of
100 will run before a rule with a priority of 50. Kong-managed rules always run after your rules.
- For the Action settings, select one of the following:
- Block: Denies requests that match the conditions.
- Monitor: Records matching requests for analytics without blocking them.
- Passthrough: Allows requests that match the conditions.
- For the Match conditions settings, select one of the following:
- Basic: Build conditions using IP, CIDR, user agent, path, and JA4 matches. Do the following:
- From the Match type dropdown menu, select a match type.
- From the Operator dropdown menu, select whether to match the value exactly or by regular expression. This option isn’t available for IP address or CIDR range match types.
- In the Value field, enter the value you want to match.
-
(Optional) To add more match conditions to a rule, click Add another condition and repeat the previous three steps.
When multiple conditions are configured on a rule, all conditions must match for a rule to apply.
- Advanced: Write a custom expression using ATC expression syntax in the Rule expression field. For example,
http.path == "/health" matches a specific path.
- Click Save.
You can switch from monitoring to blocking mode after you’ve established confidence in the results in monitoring mode:
- Make sure your Gateway Services, Routes, and clients are configured correctly.
- Verify that good traffic is labelled as
Passthrough and unwanted traffic is labeled as Block.
While Bot Detector is enabled, you can switch between monitoring and block mode at any time.
Mode changes propagate to data planes on their next pull cycle (which can be up to five minutes).
No data plane restart or redeploy is required.
Send a POST request to the same /bot-gline-manager/waap/{cpId}/enabled endpoint, setting block_mode to true:
curl -X POST "https://us.api.konghq.com/bot-gline-manager/waap/$CONTROL_PLANE_ID/enabled" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"enabled": true,
"block_mode": true
}'
Continue monitoring traffic to ensure traffic is flowing in the way you expect.
To turn Bot Detector off entirely for the control plane, set enabled to false:
curl -X POST "https://us.api.konghq.com/bot-gline-manager/waap/$CONTROL_PLANE_ID/enabled" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"enabled": false
}'
After it’s enabled, a control plane’s Bot Detector starts in monitoring mode.
To switch the mode to blocking, click Enable blocking.
Continue monitoring traffic to ensure traffic is flowing in the way you expect.
To turn Bot Detector off entirely, select “Disable bot detector” from the Actions dropdown menu.