Skip to content

fix(tools): bound every unbounded MCP response - #887

Merged
tirth8205 merged 2 commits into
mainfrom
feat/tool-token-budgets
Aug 19, 2026
Merged

fix(tools): bound every unbounded MCP response#887
tirth8205 merged 2 commits into
mainfrom
feat/tool-token-budgets

Conversation

@tirth8205

Copy link
Copy Markdown
Owner

Issue #849 reported one tool returning 247k tokens inside a workflow documented as "5 tool calls, 800 tokens total". PR #853 capped that one tool. Nobody had checked the other 29.

Eleven were unbounded, several on the default path, which is worse than #849: that one at least needed a large diff to trigger. These fire on a bare call with no arguments.

Independently verified, same graph before and after

Measured on a fresh full build of this repository (284 files, 5,764 nodes, 47,638 edges), serialized exactly as FastMCP does and counted with tiktoken cl100k_base:

tool, default arguments before after cut
list_communities 339,235 12,045 96.5%
refactor dead_code 73,890 9,901 86.6%
refactor suggest 50,484 4,808 90.5%

A single default list_communities_tool call returned more tokens than most models can hold in context at all, against a documented budget of 800.

Worst-case paths were worse still: get_review_context 4.7M, get_surprising_connections 1.29M, get_hub_nodes 556k, get_community 536k, get_architecture_overview 625k.

Notable causes

Approach

Three shared helpers in tools/_common.py give every tool the contract #853 established: total always reports the untruncated count, truncated marks the cut, and the summary says "showing N of M". Validation matches query.py exactly, rejecting bools and values below 1 with the same message. Where a count alone cannot bound a response, a shared budget does: get_flow and get_affected_flows spend a step budget, get_review_context and detect_changes spend a source-line budget. Ceilings that depend on payload size vary by detail_level, mirroring how query.py caps minimal mode at five results.

Regression guard

tests/test_token_budget.py, 103 tests, offline, one shared fixture graph. It pins the contract three ways, because any one alone can be gamed:

  1. Per-tool default and worst-case token budgets, recorded as data with arguments and reasoning inline.
  2. test_hard_ceilings_bind asserts truncated list lengths equal the ceiling constants, so a cap that silently stops applying changes a length.
  3. test_ceiling_constants_are_not_raised asserts the constants themselves, since a token budget can be masked by its own headroom.

Plus test_budget_table_covers_every_registered_tool, so a new @mcp.tool() cannot ship without a budget entry. Verified it fails on regression: raising _MAX_MEMBERS to 100000 produces 2 failures.

Four holes deliberately left open

tools/query.py was being modified concurrently by the uncertainty work (#884, #885), so these are reported rather than fixed here, each with a follow-up owed:

  1. get_impact_radius, 3.4M tokens. changed_nodes and edges ignore max_results; only impacted_nodes is capped. Worse, max_results is not exposed on the MCP signature in main.py at all, so no client can bound it.
  2. find_large_functions, 737k. limit neither validated nor capped.
  3. traverse_graph, 385k. token_budget neither validated nor capped, and the check uses a len/4 heuristic applied after the entry is built.
  4. semantic_search_nodes, 145k. limit neither validated nor capped.

The budget test asserts their default budgets and skips only their worst case, listing each in a QUERY_OWNED_UNBOUNDED set with the reason, so a regression in normal use is still caught.

Docs updated: LLM-OPTIMIZED-REFERENCE.md, docs/COMMANDS.md (new Result Bounds section; every signature there was stale, missing even #853's parameters) and docs/FEATURES.md.

Full suite 2872 passed, 9 skipped, 2 xpassed. ruff and mypy clean.

Relates to #849.

tirth8205 and others added 2 commits August 19, 2026 20:14
#849 found get_affected_flows returning ~247k tokens inside a workflow
documented as "5 tool calls, 800 tokens total". PR #853 capped that one
tool. Measuring all 30 registered tools against a real 5.6k-node graph
found the same class of bug in ten more places, several of them on the
default path:

  list_communities            206,858 tokens with DEFAULT arguments
  get_community               134,781 default / 535,618 with members
  get_architecture_overview   625,012 in standard mode
  refactor dead_code           47,312 / suggest 38,246
  detect_changes               46,089 for a ONE-file diff
  get_surprising_connections 1,287,174 at top_n=10**6
  get_hub_nodes              555,848 / get_bridge_nodes 317,202
  get_review_context        4,720,622 on a whole-repo diff

PR #853's own fix was also only half a fix: standard mode carries a full
steps list per flow (~980 tokens each), so its 50-flow cap still produced
~49k tokens, and max_flows=0 disabled the limit entirely.

Every list-returning tool now follows one contract, the one #853
established: `total` (or a per-list `*_total`) always reports the
untruncated count, `truncated` marks the cut, and the summary line says
how many of how many are shown. Bounds are validated the way query.py
validates max_results - booleans rejected, values below 1 rejected.
detail_level="minimal" was added to the analysis and refactor tools,
projecting to the same compact field sets their siblings use.

Where a count alone cannot bound a response, a shared budget does:
get_flow and get_affected_flows spend a step budget, get_review_context
and detect_changes spend a source-line budget. Ceilings that depend on
payload size depend on detail_level, mirroring query.py capping
minimal-mode results at five.

Two behaviour changes are deliberate and their #853 tests are updated in
this commit: get_affected_flows standard mode now caps at 25 flows
(minimal at 500), and max_flows=0 keeps its "no caller limit" meaning
while still obeying the ceiling - an escape hatch that returns a quarter
of a million tokens is the bug, not a feature.

Default behaviour stays backward compatible in shape: an existing caller
passing nothing still gets a valid response, just bounded.

tests/test_token_budget.py records the measured budget table as
reviewable data and pins it three ways: per-tool default and worst-case
token ceilings, exact truncated list lengths against the imported
ceiling constants, and the ceiling constants themselves. Removing a cap,
raising a ceiling, or adding an unbounded field fails it. The fixture
graph builds once per module and the whole file runs in ~7s, offline.

Reported, not fixed: code_review_graph/tools/query.py is owned elsewhere
and four of its tools remain unbounded - get_impact_radius (3.4M tokens;
changed_nodes and edges ignore max_results, which is not even exposed on
the MCP signature), find_large_functions (737k), traverse_graph (385k)
and semantic_search_nodes, whose limit/token_budget are neither
validated nor capped. Their default budgets are still asserted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012fHfGDiZedoxjpKzanHri3
@github-actions

Copy link
Copy Markdown

code-review-graph review

Overall risk: 0.75 (HIGH) — 70 changed function(s)/class(es), 0 affected flow(s), 48 test gap(s)

Risk-scored changes

Risk Level Symbol Location Tested
0.75 high code_review_graph/main.py::get_surprising_connections_tool code_review_graph/main.py:949 no
0.75 high code_review_graph/tools/analysis_tools.py::get_surprising_connections_func code_review_graph/tools/analysis_tools.py:198 no
0.65 medium code_review_graph/tools/analysis_tools.py::_project code_review_graph/tools/analysis_tools.py:33 no
0.65 medium code_review_graph/tools/community_tools.py::_cap_members code_review_graph/tools/community_tools.py:30 no
0.65 medium code_review_graph/tools/refactor_tools.py::_project code_review_graph/tools/refactor_tools.py:37 no
0.65 medium code_review_graph/tools/review.py::_project code_review_graph/tools/review.py:63 no
0.60 medium code_review_graph/tools/review.py::_extract_relevant_lines code_review_graph/tools/review.py:316 no
0.55 medium code_review_graph/main.py::detect_changes_tool code_review_graph/main.py:677 no
0.55 medium code_review_graph/tools/analysis_tools.py::get_hub_nodes_func code_review_graph/tools/analysis_tools.py:38 no
0.55 medium code_review_graph/tools/analysis_tools.py::get_bridge_nodes_func code_review_graph/tools/analysis_tools.py:88 no

Test gaps

  • code_review_graph/main.py::get_review_context_tool (code_review_graph/main.py:289)
  • code_review_graph/main.py::get_flow_tool (code_review_graph/main.py:509)
  • code_review_graph/main.py::get_affected_flows_tool (code_review_graph/main.py:543)
  • code_review_graph/main.py::list_communities_tool (code_review_graph/main.py:575)
  • code_review_graph/main.py::get_community_tool (code_review_graph/main.py:611)
  • ...and 43 more without direct tests

Token savings: this graph-backed report used ~45,602 fewer tokens (~80%) than reading every changed file in full (estimated, chars/4 approximation).


Powered by code-review-graph — local-first analysis; no code leaves the CI runner.

@tirth8205
tirth8205 merged commit c39106e into main Aug 19, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant