Skip to content

feat(query): say why a result set is empty instead of returning a bare 0 - #884

Merged
tirth8205 merged 1 commit into
mainfrom
feat/uncertainty-on-empty-results
Aug 19, 2026
Merged

feat(query): say why a result set is empty instead of returning a bare 0#884
tirth8205 merged 1 commit into
mainfrom
feat/uncertainty-on-empty-results

Conversation

@tirth8205

Copy link
Copy Markdown
Owner

A zero from the graph is currently indistinguishable from an absence in the code. query_graph_tool(pattern="callers_of", target="X") returns result_count: 0 whether X has no callers, X is not indexed, the graph is stale, or the language cannot express that relationship statically. Agents read the zero as proof and act on it. Issues #314, #850, #851 and #819 all describe this failure, and #851 states it directly: a zero with no uncertainty marker is the dangerous direction.

Change

Adds one optional response key, confidence, a short string present only when the result list is empty. First match wins:

  1. Not indexed - target not indexed: no node matching 'X', so this 0 is not evidence that none exist
  2. Stale by commit - graph built at an older commit than HEAD
  3. Stale by mtime - the file changed after the last build (a matching commit says nothing about uncommitted edits, so both are checked)
  4. Known language gap - e.g. php container-resolved and constructor-injected calls are not statically traced, so callers can be missing (#850, #851)
  5. Confirmed absence - 'X' is indexed and the graph is current, so this 0 is a real absence, degrading to graph currency unverified when currency could not be established rather than over-claiming

The gap table lives as data in the new uncertainty.py, keyed by language and by query pattern, so a container-resolution note appears on callers_of and tests_for but not on file_summary. Wired into query_graph, get_impact_radius and semantic_search_nodes. No tool signature changes.

Token cost, measured

response today with marker delta
query_graph empty, standard 392 ch 522 ch +130 ch (~33 tok)
query_graph empty, minimal 381 ch 511 ch +130 ch (~33 tok)
unindexed target, minimal 220 ch 325 ch +105 ch (~27 tok)
semantic_search zero hits 159 ch 272 ch +113 ch (~29 tok)
any response with results +0 bytes, key absent

Worst case is 156 characters. The alternative behaviour, an agent that distrusts the zero and greps the repository, costs orders of magnitude more, so this is a net saving. A test asserts the exact absence of the key on non-empty responses so the budget cannot regress.

Accuracy of the gap table

Every claimed gap was verified against the parser with live parse runs rather than assumed:

Tests

50 new tests in tests/test_uncertainty.py: unknown target versus real absence, per-language and per-pattern firing, stale detection by commit and by mtime, silent degradation on failure, length cap under hostile unicode and control characters, and the key-absence budget guard.

Full suite 2759 passed, 5 skipped, 2 xpassed. ruff and mypy clean.

Relates to #314, #850, #851, #819.

`query_graph(pattern="callers_of", target="X")` returned `result_count: 0`
with no signal, and agents read that as "X has no callers". A zero can also
mean the graph cannot see the answer: X was never indexed, the graph is
behind the working tree, or X's language has a known static-analysis blind
spot. Reading it the first way either produces a wrong decision or sends the
agent off to grep the repository, which costs thousands of tokens.

Empty results from query_graph, get_impact_radius, and semantic_search_nodes
now carry a single flat `confidence` string computed in priority order:
target not indexed, graph stale, known language/pattern gap, or a positively
confirmed real absence. The gap table lives in the new uncertainty module as
data, is scoped per pattern so a container-resolution caveat never lands on
file_summary, and every entry was verified against the parser as it stands
today (#819 PHP include/require, #850/#851 PHP container and constructor
injection, #343 npm-aliased imports, #592 Java Spring AOP, plus JS/TS
callbacks and route registration, Go structural interface satisfaction, C# DI
and Python getattr/registry dispatch).

Token cost is the constraint, so the field is emitted only when the result
list is empty — responses that carry results are unchanged — and is capped at
140 characters, measured at +105..+130 chars (~27-33 tokens) on a typical
empty response. That is a saving, not a cost, against the fallback it
prevents.

Everything reaching the client goes through _sanitize_name, has whitespace
collapsed so a crafted node name cannot forge extra lines, and is clipped to
the cap; qualified names drop to their symbol rather than keeping a truncated
directory prefix. Any failure computing the marker degrades to omitting it.

Refs #314

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012fHfGDiZedoxjpKzanHri3
@github-actions

Copy link
Copy Markdown

code-review-graph review

Overall risk: 0.75 (HIGH) — 52 changed function(s)/class(es), 0 affected flow(s), 15 test gap(s)

Risk-scored changes

Risk Level Symbol Location Tested
0.75 high code_review_graph/tools/query.py::query_graph code_review_graph/tools/query.py:249 yes
0.65 medium code_review_graph/uncertainty.py::_clip code_review_graph/uncertainty.py:175 no
0.65 medium code_review_graph/uncertainty.py::_clean code_review_graph/uncertainty.py:182 no
0.65 medium code_review_graph/uncertainty.py::_fragment code_review_graph/uncertainty.py:194 no
0.65 medium code_review_graph/uncertainty.py::_bounded code_review_graph/uncertainty.py:199 no
0.65 medium code_review_graph/uncertainty.py::_interpolated code_review_graph/uncertainty.py:204 no
0.65 medium code_review_graph/uncertainty.py::_interpolated_target code_review_graph/uncertainty.py:228 no
0.65 medium code_review_graph/uncertainty.py::not_indexed_note code_review_graph/uncertainty.py:239 yes
0.65 medium code_review_graph/uncertainty.py::_staleness code_review_graph/uncertainty.py:279 no
0.60 medium code_review_graph/uncertainty.py::gap_note code_review_graph/uncertainty.py:159 yes

Test gaps

  • code_review_graph/uncertainty.py::LanguageGap (code_review_graph/uncertainty.py:58)
  • code_review_graph/uncertainty.py::_clip (code_review_graph/uncertainty.py:175)
  • code_review_graph/uncertainty.py::_clean (code_review_graph/uncertainty.py:182)
  • code_review_graph/uncertainty.py::_fragment (code_review_graph/uncertainty.py:194)
  • code_review_graph/uncertainty.py::_bounded (code_review_graph/uncertainty.py:199)
  • ...and 10 more without direct tests

Token savings: this graph-backed report used ~31,427 fewer tokens (~81%) than reading every changed file in full (estimated, chars/4 approximation).


Powered by code-review-graph — local-first analysis; no code leaves the CI runner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant