You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1b4577f
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: plugins/rust/python-package/rate_limiter/README.md
+9-5Lines changed: 9 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3
> Author: ContextForge Contributors
4
4
5
-
Enforces global per-user, per-user/per-server, tenant, and tool rate limits across `tool_pre_invoke` and `prompt_pre_fetch` hooks. Supports pluggable counting algorithms (fixed window, sliding window, token bucket), an in-process memory backend (single-instance), and a Redis backend (shared across all gateway instances).
5
+
Enforces global per-user, tenant, and tool/prompt-name rate limits across `tool_pre_invoke` and `prompt_pre_fetch` hooks, with an additional per-user/per-server quota for tool invocations. Supports pluggable counting algorithms (fixed window, sliding window, token bucket), an in-process memory backend (single-instance), and a Redis backend (shared across all gateway instances).
6
6
7
7
## Runtime Requirements
8
8
@@ -13,7 +13,7 @@ This plugin depends on `cpex>=0.1.0,<0.2` and imports hook models from `cpex.fra
13
13
| Hook | When it runs |
14
14
|---|---|
15
15
|`tool_pre_invoke`| Before every tool call — checks `by_user`, `by_user_per_server`, `by_tenant`, `by_tool`|
16
-
|`prompt_pre_fetch`| Before every prompt fetch — checks `by_user`, `by_user_per_server`, `by_tenant`, `by_tool`|
16
+
|`prompt_pre_fetch`| Before every prompt fetch — checks `by_user`, `by_tenant`, `by_tool` using the prompt name|
17
17
18
18
If any configured dimension is exceeded, the plugin returns a violation with HTTP 429. All requests include `X-RateLimit-*` headers. The most restrictive active dimension is surfaced (e.g. if both user and tenant limits are active, the one closest to exhaustion is reported).
19
19
@@ -28,7 +28,7 @@ If any configured dimension is exceeded, the plugin returns a violation with HTT
28
28
mode: enforce # enforce | permissive | disabled
29
29
config:
30
30
by_user: "300/m"# per-user limit across all servers
31
-
by_user_per_server: "60/m"# per-user limit for each MCP server
31
+
by_user_per_server: "60/m"# per-user tool limit for each upstream MCP catalog server
32
32
by_tenant: "300/m"# shared limit across all users in a tenant
33
33
by_tool: # per-tool overrides (applied on top of by_user)
34
34
search: "10/m"
@@ -56,7 +56,7 @@ If any configured dimension is exceeded, the plugin returns a violation with HTT
@@ -77,7 +77,9 @@ If any configured dimension is exceeded, the plugin returns a violation with HTT
77
77
78
78
**Omitting a dimension** (e.g. no `by_tenant`) means that dimension is unlimited — no counter is tracked for it.
79
79
80
-
`by_user` and `by_user_per_server` are independent and may be enabled together. For example, `by_user: "300/m"` plus `by_user_per_server: "60/m"` gives each user at most 60 requests per server and 300 requests across all servers. This prevents one user from multiplying their effective quota without relying on the shared `by_tenant` ceiling. When `by_user_per_server` is configured, requests without a safe non-empty `server_id` are blocked with `RATE_LIMIT_CONTEXT_MISSING` (HTTP 503) rather than bypassing the limit.
80
+
`by_user` and `by_user_per_server` are independent and may be enabled together. For example, `by_user: "300/m"` plus `by_user_per_server: "60/m"` gives each user at most 60 tool calls per catalog server and 300 combined tool calls and prompt fetches within the same tenant context. This prevents one user from multiplying their effective quota without relying on the shared `by_tenant` ceiling. When `by_user_per_server` is configured, tool calls without a safe non-empty `server_id` are blocked with `RATE_LIMIT_CONTEXT_MISSING` (HTTP 503) rather than bypassing the limit.
81
+
82
+
Prompt fetches neither require a catalog `server_id` nor consume `by_user_per_server` counters, even when a virtual-server ID is present. They still enforce `by_user`, `by_tenant`, and prompt-name overrides in `by_tool`. These counters remain shared with tool invocations within the same tenant context.
Version `0.1.12` fixes prompt retrieval when `by_user_per_server` is enabled. The per-server quota now applies only to tool invocations; prompt fetches retain existing global and prompt-name limits. Counter keys and tool identity validation are unchanged. Upgrade every replica to `0.1.12` or later before enabling per-server tool quotas with prompt hooks registered.
279
+
276
280
Version `0.1.11` adds the optional `by_user_per_server` dimension. Existing configurations and global `by_user` Redis keys remain unchanged. Enabling the new field creates fresh user/server counters; deploy `0.1.11` to every replica before enabling it so all replicas enforce the same dimensions. Use the stable catalog UUID carried in `GlobalContext.server_id`; display names or slugs can reset quotas when renamed.
277
281
278
282
Version `0.1.7` is a **breaking change** for any existing consumer reading rate-limit metadata:
0 commit comments