fix(sem,cli): TypeScript bodyless declarations, test-file demotion, bare-locator windows - #80
Closed
suhaanthayyil wants to merge 22 commits into
Closed
fix(sem,cli): TypeScript bodyless declarations, test-file demotion, bare-locator windows#80suhaanthayyil wants to merge 22 commits into
suhaanthayyil wants to merge 22 commits into
Conversation
Measured over 178 gated benchmark pairs, graph calls per session are invariant (1.00/1.03/1.00 across baseline bands <10/10-19/>=20) while displaced greps are not (1.35/2.16/3.74), so the call is a fixed toll with variable repayment: the no-op cohort (n=43) costs 1.165 CI [1.073,1.264] at +1.21 turns, the displacing cohort (n=135) costs 0.938 CI [0.889,0.990] at -2.07 turns, corr(turn delta, log ratio) = +0.735. No static payload feature predicts a no-op call (AUC 0.553-0.690 at a 28% base rate), so selective skipping is not implementable and pre-delivery must be universal. ENTIRE_GRAPH_PRESEARCH (alias EG_PRESEARCH) names a file holding the payload computed before the agent started; search echoes it byte-for-byte and returns before the profile, repo, cache and index are touched. An unreadable path is an error rather than a silent live query.
Two printed bodies of one file with a small hole between them are one region as far as the reader is concerned, and the hole is what it spends a turn closing. Measured across 113 Opus search payloads: 45 (40%) carry at least two ranked hits in one file within 40 lines (R30Oeg 12/25, R30OegBEST 12/30, R30OegV9 9/29, C15opeg_r1 12/29). Read side: of 53 answered pre-edit Read calls only 41 are ranged, and of those only 22 (53.7%) overlap the payload snippet at all, with the payload covering a mean 18.2% of the requested span - so nominal 62.0% read coverage is worth ~33%. Suite level: the tool makes +19.6% MORE Read calls than the no-tool baseline while total tool calls fall 14.5% (n=55 paired instances). Call by call: fluentd-3328 had gold at rank 1 (in_tail.rb:395-425, score 135.8, complete-symbol, "Top hit is exact") and still cost 1.64x, because ranks 1/2/3 were three disjoint spans of one file (349-360, 395-425, 427-443) and turn 2 was `sed -n '200,270p'; sed -n '330,470p'` - 6.9 kB re-reading a contiguous superset of what it already had. pylint-7080 turn 2 re-fetches rank 2 verbatim with `cat`. Near hits are now folded into one span carrying the verbatim text of the whole range, headed `path:START-END [contains ranks i,j,k - contiguous, nothing elided]`, because the reason the agent re-reads is that it cannot tell whether the hole hides something. Filling gaps adds bytes, and under a fixed ceiling added bytes evict the last ranked hit, so three bounds are enforced: a 40-line gap cap, an 80-line span cap, and a cost check of the whole prospective ranking against --max-context-bytes before each merge is committed - a bridge that would not fit simply does not happen. Only complete-symbol results take part (the one signal every renderer prints in full, so a merged header can never name a range whose text is absent), and merging never reorders: a run's survivor is its best-ranked member, so the rank at which a file first appears can only improve. (cherry picked from commit daa9b95d7911f35abbd0edfefac14bbea5cbed47)
Ported from eg-consolidated 024f25a. FocusLine is computed for every hit and has always been in the JSON; the text payload printed only the snippet range, so a hit on a 27-line method said 'the answer is somewhere in 135-161'. Asked what one change to the returned payload would let it finish in the fewest calls, a Sonnet agent that had just fixed apache/lucene-13170 with this tool named exactly this -- its only non-essential call was a Read to pin down line 151. Re-reading a file the payload already printed is 10.1% of post-payload tool calls across 30 measured sessions. Header, not an inline marker on the snippet line: agents copy snippet text verbatim as an edit's old_string anchor, so decorating a body line would break the anchor. Entire-Checkpoint: 2893461866fb
Adds EG_SEARCH_SESSION (the per-task state file the one-shot CLI needs to count its own calls) and EG_MAX_SEARCHES (default 1, 0 disables). Past the cap, search returns the first call's payload verbatim under a one-line header instead of running a new query. Measured: sessions with >=4 graph calls (n=16) cost 1.148 of the no-tool baseline against 0.975 for one-call sessions (n=70), diff +0.173, bootstrap CI [+0.019,+0.324]. Same gradient per config: 1.00 calls/session -> -12.7%, 1.17 -> -0.7%, 1.63 -> +5.1%, 4.45 -> worst retrieval of the set. (cherry picked from commit 823cc5e43e7e30bb121e0502c27d4f5e50f3e3b4)
…levers
Editability — the share of needed edits whose replaced text is present
VERBATIM in the payload — is a different measurement from recall, and the
two diverge. On the R30PUB benchmark payloads the gold file is ranked for 8
of 12 gold files while the gold LINES are printed for none of them.
The misses are not budget misses. They are the three conditions the
opportunistic body upgrade applies:
- searchEnclosableSymbolKind excludes containers, so a hit inside a class
body, a constant table or a type declaration resolves to no callable and
keeps a six-line window;
- defaultSearchEnclosureMaxLines (160) drops a longer callable back to a
window, which is the case where reading the file back costs the most;
- allocateSearchSnippets only evaluates demotion cuts down to
minInt(headRanks, len(results)), so at --top-k 5 with headRanks 5 the
only cut it can consider is "demote nothing" and the entire body budget
is the 10 kB growth allowance.
--full-unit-top N bypasses all three for the first N ranks: the unit is
resolved as the callable first and then as any enclosing symbol, bounded
only by a 400-line safety cap, and the allocator seats it before it prices
anything else, demoting the tail to locators to pay for it.
--max-context-bytes still wins — when the ceiling cannot hold the unit with
the whole tail reduced to locators, forced ranks hand their ranked snippets
back deepest-first, so rank 1 yields last. N >= 2 reaches rank 2 only while
its score is within 15% of rank 1's. A clipped unit reports
"elided lines X-Y (unit continues)" on its own line after the body, never
inline: body text is copied verbatim as an Edit anchor.
--edit-site-bodies gives the SAME-CONCEPT LITERAL block's EDIT sites their
source — the enclosing unit when resolvable, else +/-12 lines — capped at 60
lines and 3 kB per site and 8 sites, with the block's cap raised to 12 kB
while bodies are attached. CONSUMER and DOC sites stay file:line-only: a
consumer is listed precisely so an agent can decide not to open it.
Also fixes two cases where the renderer discarded source the allocator had
already paid for: a head-window result past the second rank was printed as a
bare locator (measured on fmtlib__fmt-2457, 1,709 B seated and thrown away),
and a forced unit could reach a rank below that tier.
Both flags default to off. Verified on eight benchmark instances that the
default payload is byte-identical to the previous binary; with the flags on,
verbatim source in the payload rises 21.2 kB -> 34.0 kB (top 2) and gold
files present as a locator with no source at all go 2 -> 0.
JSON additions are additive only: SearchResult.unit_start_line/unit_end_line
and SearchLiteralHit.body plus its line ranges.
…unit
With the unit levers on, the payload's remaining editability misses are not
truncated units — they are the WRONG unit of the right file. Measured across
eight R30PUB instances: 8 of 12 code-gold files ranked, 0 carried the gold
hunk verbatim, and in every ranked case the printed unit and the gold unit
were different callables of the same file. The commonest shape is a thin
public entry point ranked while the gold sits in the helper it calls:
redis__redis-10095 ranked src/t_list.c:577-581 lpopCommand (3 lines)
gold src/t_list.c:501 in popGenericCommand (489-535)
edge lpopCommand --CALLS--> popGenericCommand
One outgoing CALLS edge away, in the same file, already in the graph.
search_related.go excludes outgoing CALLS deliberately (:41-43) on the
grounds that such cases "are already reached as co-members of the same
unit". They are not: searchRelatedUnitCoMembers requires an identical
ContainerID and returns nothing at all once the unit holds more than
searchRelatedUnitMemberLimit (12) members, and src/t_list.c has ~40
top-level functions, so that route switches off entirely and no route in
the block can produce the site. The exclusion's reasoning still holds for
the four-slot block it was written about, so this is a separate gated route
and the related-sites rotation is untouched.
--callee-hop resolves the head's outgoing CALLS and admits up to three
callees as candidate fix sites, ordered by same-file then query overlap then
edge confidence. It fires for rank 1, plus rank 2 under the same
searchFullUnitGapRatio test --full-unit-top uses, and fires regardless of the
anchor's unit size — which is where the co-member limit turns off. Admission
requires a resolved same-repo callable whose file passes the ranker's own
file-class and test-artifact priors, and a cross-file callee must have a
query word in its NAME: without that the first cross-file hop on
redis__redis-10095 was zcalloc. Same-file callees are admitted
unconditionally, because popGenericCommand shares no word with an issue
phrased "LPOP ... null array reply".
Bodies follow the other levers rather than inventing a third policy: source
when --full-unit-top or --edit-site-bodies is on, a declaration locator
otherwise. Funding order is head, then callee bodies paid for by TERSIFYING
the tail (never dropping it, so no file loses its only mention), then fewer
callees; --max-context-bytes is exact at every step. Entries carry a
callee-hop signal and stats.callee_hop_sites so payloads stay attributable,
and print no ranked score because they were admitted by an edge.
Measured on the same eight instances: strict gold-hunk-verbatim goes 0/12 to
1/12 (redis__redis-10095 now carries the whole popGenericCommand body
including the exact replaced line), the hop fires on 3 of 8, and the default
payload is byte-identical to 6da0291 on all eight.
…ctive A 50-instance replay found --full-unit-top net-NEGATIVE: gold coverage in the losing cohort fell 28% -> 16% against the control. Per-instance forensics found three independent defects, all in funding rather than retrieval. 1. DESTRUCTIVE FUNDING. The forced pass demoted the tail to locators before the non-forced head was funded, so a lower rank that had been carrying a complete gold-covering body lost it. On redis__redis-11734 the control gave rank 4 the whole bitposCommand (src/bitops.c:882-1008, complete-symbol) and the forced run emitted `4. src/bitops.c:882 bitposCommand` with no source, losing three gold hunks to a 61-line header window at rank 2. Forced units are now seated ON TOP OF the allocation the same inputs produce with the flag off. That control plan is a floor: a forced unit may only spend budget the control left unused, plus the snippet bytes of ranks the text renderer prints as bare locators anyway. Nothing a reader would have seen is ever reduced. When the whole unit will not fit, the widest clipped form that does is seated, and only above searchForcedUnitMinLines; otherwise the pre-existing allocation stands. 2. UNSPENT BUDGET. Independently, 0 of 50 replayed payloads used even half the 24 kB ceiling (median 4,557 B), and redis__redis-11734 rendered a 127-line body as a 34 B locator with 18 kB free: --max-snippet-lines was acting as a ceiling on expansion instead of a floor on context. Under any payload-shape lever the allocator now plans an enclosure for EVERY ranked hit and spends the remaining ceiling in rank order, so a ranked hit never renders as a bare locator while its unit (or a >= 20-line window) would still fit. The pool is the caller's own --top-k, not a hidden second pool: gold units that sit at rank 6-8 are reached by raising --top-k, which keeps one visible contract. 3. HOP PRIORITY. --callee-hop entries were seated as full bodies and evicted ranked bodies outright. They are now the lowest-priority claim in the payload: they spend only what no ranked hit claimed, and they degrade themselves — full body, then a >= 20-line clip, then a locator — rather than reduce a ranked hit. The related-sites block gets the same guard: its own docstring says it reclaims redundant tail LOCATORS, but it was offering body-carrying results for displacement, which is how bitposCommand's body was dropped to seat four caller locators. Also size-aware admission for the container bypass. A non-callable unit is forced only under searchForcedContainerMaxLines: that keeps the win it was built for (rubocop-13680's 5-line config/default.yml section) and refuses the loss (docusaurus-9183's DocusaurusConfig expanding 6 -> 273 lines). Head windows over declaration-only regions cap at searchDeclarationWindowLines, since a prototype block has no body a wider window would explain. Measured on 10 instances at 24576 B, gold replaced-lines present verbatim: control 9/15, +L1/L2 11/15, +hop 12/15, +hop at --top-k 12 14/15. Every rank present without flags is still present under every flag combination, no payload exceeds the ceiling, no symbol-less bare locator appears while half the budget is free, and the default payload is byte-identical to 6da0291.
…ite's package A Go module routinely declares one type name per sibling package — prometheus/prometheus has 14 `Discovery` types with a `refresh` method, one per discovery/<provider>/. Same-name candidate selection preferred a same-file declaration and then took the first match in index order, so `d.refresh(ctx)` in discovery/puppetdb/puppetdb_test.go bound to discovery/azure's Discovery.refresh and no traversal from the test reached the code under test. The same first-match rule sent NewDiscovery's `conf.Port`/`conf.RefreshInterval` reads to azure's SDConfig. Rank candidates by path proximity instead: same file, then same directory (a Go package IS a directory), then the nearest shared module-path prefix. Ties keep input order, so an unscoped ambiguity resolves exactly as before.
…ugh them tree-sitter's Go interface body produced no member symbols, so an interface was a dead end in the call graph: hashicorp/terraform's communicator.Communicator had ZERO incoming CALLS even though every provisioner drives its remote work through it, and `comm.Disconnect()` in runScripts bound to nothing. Three parts: - emit a method symbol per interface method requirement (`method_elem`, and `method_spec` for older grammars), qualified under its interface; - type a receiver whose declared type is package-qualified by an in-module import (`comm communicator.Communicator`) — parameterVarTypes requires a bare capitalised type name, so an interface-typed parameter previously carried no receiver type at all and fell through to the globally-unique-name tier, which cannot fire when the interface and its implementations share a method name; - carry a call that landed on a requirement through to the concrete methods that satisfy it. Go satisfaction is implicit, so it is recovered structurally: a type qualifies when it declares a method for every requirement. A one-requirement interface only carries through when the called name is unique in the workspace (`Close`/`String`/`Error` are everywhere), and past 8 satisfying types the interface edge stands alone. The carry-through runs once over the finished relation list rather than at each typed-receiver tier, so a call reached through a parameter, a returned type or a constructor chain is treated identically. All edges are additive; none are rewritten.
A C++ specialization is named after the PRIMARY template, not after what it specializes: fmtlib/fmt's `template <typename Char, typename... T> struct formatter<tuple_join_view<Char, T...>, Char>` is indexed with the qualified name `formatter` — the most generic name in the repository, shared by dozens of declarations — so no name signal could ever rank it. Its signature carried tuple_join_view but only earned the 1.5x signature tier. Add the leading specialization argument, and the compound `Name<Argument>` form, as searchable ALIASES on the specialization and on its members. Aliases are the existing additive mechanism and already score as an exact name match, so the symbol keeps its name, its qualified name and — decisively — its compound-v1 ID, which is derived from the qualified name; renaming to `formatter<tuple_join_view>` would have re-keyed every specialization in every indexed repository. Only the leading argument is used. Partial specializations discriminate on it by convention and carry the primary template's own parameters (`Char`, an `enable_if_t` guard) in the trailing positions, so aliasing the rest would bind a declaration to its template parameters.
…efusing
Session-level forensics of 19 paid sessions found that the tool's most
expensive failures were refusals to answer a live query, not wrong answers.
Two messages account for both measured blow-ups, and in each case the agent
paid to do by hand what the tool had declined to do.
FIX A — "Ambiguous symbol %q matched N definitions; rerun with the selector
printed beside the one you mean." Measured +$2.22 on projectlombok__lombok-3486,
78 pre-edit operations spent manually disambiguating two same-named methods
whose locations the tool had already printed. Ambiguity means the graph found
MORE than it was asked for, which is information rather than an error, so the
listing now says "%q matches N definitions; all are listed, the first 2 with
source", keeps every locator and its narrowing selector, and appends a compact
body (<= 40 lines, elision noted) for the top matches. The rerun instruction is
gone, and both ambiguity tests now assert its ABSENCE.
FIX B — "No symbols matched %q" / "(no symbol named X)". Measured +$1.23 on
phpoffice__phpspreadsheet-3570: both live queries came back empty and shell
greps went 18 -> 39, because the query spelled the name in a different but
obvious way. An exact miss now degrades down a fuzzy ladder ordered by how much
it assumes — case-insensitive, then separator-insensitive (flattenSingleValue ==
flatten_single_value), then identifier-token subset in either direction, then
substring — and returns the best 3 candidates with locators plus a body for the
first. Every fuzzy answer is labelled with the rung that produced it, so a
caller is never misled into thinking it got what it asked for.
Both live on the one resolver and the two renderers that def, neighbors and
impact share (resolveFocusSymbolsOrFuzzy, writeDisambiguationListing,
writeFuzzyMatchListing), so no verb can drift back to refusing, and callers/
callees inherit the fuzzy focus resolution for free.
The substring rung requires 8 characters. def has a standing invariant that a
short fragment must never resolve to a longer name ("dele" is not "deletion")
and that invariant is right; the rungs above already cover every spelling case
this fix was built for, so nothing real depends on relaxing it.
Responses gain fuzzy_match, fuzzy_match_kind and match_bodies (additive).
…fuse tests The fuzzy resolver sorted candidates by how well each matched, and then both buildNeighborResponse and defIndex.resolve re-sorted them by file path — which buried the correct answer. On phpspreadsheet-3570 a query for flatten_single_value listed a `Single` class sharing one token above Functions.flattenSingleValue, and printed that class's body. File/line order is right for an EXACT match, where several definitions of one name are equally valid answers and a stable positional order is the honest presentation. For a fuzzy answer the order IS the answer, so it is now preserved. Verified on the real checkout: the method comes back first, with its body. Tests cover both fixes end to end — every spelling of the ladder, the standing def invariant that a four-letter fragment resolves to nothing, --file/--kind narrowing surviving a fuzzy name, the tokenizer across camel/snake/kebab/::/. spellings and capital runs, and both renderers asserting the absence of the rerun instruction. Also pins that pure case difference is already an exact match (resolveFocusSymbols compares with EqualFold), so the case rung is not mistaken for the thing that makes case work.
…VERIFY line Three tool-side fixes from session forensics. 1. ELISION CONTRACT (correctness). On briannesbitt__carbon-2752 the payload ranked Comparison.php #1 tagged full-unit,complete-symbol while the printed body elided lines 630-1125 — and the focus line, the region the edit had to land in, was 989. The signal was false, which is worse than a small body: an agent that trusts it edits the wrong place, and one that does not trust it re-reads the file, so the bytes bought nothing either way. The cause was the anchor. A clip anchored at the unit's START shows the declaration and drops whatever the hit was about, which for a 1,200-line unit is everything that matters. clipSearchUnitToCap now always centres on the focus, a clipped forced unit gets a focus window of +/-60 rather than a 400-line anchored slab, and widenSearchResultToEnclosure withholds BOTH complete-body signals whenever the printed span does not contain the focus — tested against the incoming focus, before the clamp rewrites it and hides the violation. The invariant is enforced at the one place signals are attached, so no caller can reintroduce it. 2. DEGENERATE IMPACT. An anchor with no relations still printed a focus line, a blast radius reading all zeros and one empty header per relation kind, which looks like an answer and is replayed on every later turn. Measured over 8 sessions: all 3 retrieval-miss instances produced exactly that (carbon-2752 isLongYear 0/0, prometheus Config.ScrapeConfigs 0/0/0, three.js build-only) and 0 of the 5 payload-hit instances did. Such a result now emits one machine-readable line, "IMPACT DEGENERATE: <anchor> has <reason>", for either of two reasons: no callers, callees or type consumers, or every relation (or, for an unresolved anchor, every definition) landing in built or vendored output. Co-change and sibling counts deliberately do not rescue it — they are heuristics, and a payload carrying only those has still said nothing about what the change breaks. The verdict is computed once on the finished response so text and JSON (degenerate, degenerate_reason) cannot disagree. 3. VERIFY CONTRACT. The emitted command now carries what to do with it: run it once after editing; if it fails, fix the code and re-run the same command; do not build an alternative harness, hand-assemble a classpath, or re-verify by reverting and reapplying. Each clause is a measured churn mode — post-edit verify churn cost +$4.11 across three sessions, and on gson-1014 a hand-built javac/JUnit classpath was 55.6% of the session's output tokens. When VERIFY is emitted is unchanged; only the contract text is added, and its cap rises from 320 to 640 bytes to hold it. Default no-flags search payload differs from 6da0291 by exactly the two added VERIFY contract lines on all eight replay instances, and by nothing else.
…block suppression Turn-level forensics of 12 real sessions: payloads got zero references, the VERIFY command was unrunnable or non-covering in 12/12 and chasing it cost ~35% of turns, impact asserted authority on the wrong symbol in 11/12, and payloads were 45-75% noise. F1 VERIFY. Kept our narrow->suite ladder, the contract note and the cap; added a third rung (single-file parse check: php -l, ruby -c, node --check, python -m py_compile) and a fourth (a residual floor line, so absence is never silent). Cherry-picked from the sibling branch: missing-runner detection, which ANNOTATES and never suppresses; the cost exclusion that keeps the note from deleting the block through a byte budget; the recovered-test-name family, so Go/Cargo narrow to a case name the test file actually declares; and CMake, which is why C and C++ had no tier at all. Universality: depth 8 -> 32 (monorepo test trees are 9-14 segments deep and the walk terminating early is indistinguishable from "no build system"); PREFIX affixes (test_*.py, test_*.rb, Test<Stem>) and test/<->spec/ swaps in both directions; suite-Node reads devDependencies like the narrow tier already did. Every command reports its tier on the evidence line and in stats.verify_tier, and --verify-prefix decorates after any `cd &&` while "VERIFY: " stays byte-identical at line start. VERIFY is promoted out of the droppable suffix set: it now rides in the agent-format prefix, yields only to a ranked location, and gets first refusal on leftover bytes when it has to. F3 Suppression. The degenerate-impact gate counts co-change as actionable (superseding round 4, which excluded it). The SAME-CONCEPT LITERAL block is emitted only when at least one EDIT site is LIVE CODE — nushell's three EDIT sites were commented-out code, so the block sent the agent to patch a comment. Completeness prints a fraction only when the denominator is real: "35 of 0 files failed to parse" (three.js) and "6 of 0" (terraform) are numbers that cannot be true and they discredit every other count on the line. F4 LOW CONFIDENCE now also fires mechanically on a near-exact top1/top2 tie (lombok-3486 shipped clean at delta 0.0100 and cost $2.22) and on a top hit whose own content is a doc comment, an example-usage list or a deprecated forwarder. The tie test is ABSOLUTE, not relative: read as 5% it fired on 56 of 56 calibration queries whose target provably exists, and a marker that always fires teaches the reader to ignore it. F5 Payload diet: the default text payload prints at most three full bodies and renders the rest as locators. Bodies past the third were never referenced in any of the 12 sessions. Ranks the caller forced with --full-unit-top or --callee-hop are exempt, and the diet writes the locator itself because writeTextSearchResult must keep refusing to collapse source the allocator paid for.
…E-FIX status line F4 GUARD-AWARE DERIVATION. fmt-2457's covering test `ranges_test.join_tuple` lives inside `#ifdef FMT_RANGES_TEST_ENABLE_JOIN`, so a correctly shaped `ctest -R ranges-test` compiles the case OUT and passes while exercising nothing. A command that silently verifies nothing is worse than the suite command it displaced, because it reports success. The guard is now discovered by a line scan (`#ifdef X`, `#if defined(X)`, `#[cfg(feature = "x")]`) that tracks balanced `#endif`, so a guard that closed before the test is never attributed to it, and Rust attributes only bind to an adjacent item. A C/C++ macro guard is satisfied from the CMake configure step (-DCMAKE_CXX_FLAGS=-D<MACRO>); a Rust feature guard becomes `--features x` but only when the manifest declares it, because an undeclared feature is a hard cargo error. Where the guard cannot be satisfied the command still ships and the derivation SAYS so, and either way the guard is printed as its own `guarded by:` line. The satisfaction rule is deliberately strict: a guard is only acted on when the test declaration was identified UNIQUELY. `TEST(ranges_test, join_tuple)` and `TEST(ranges_test, c_style_array)` both contain the suite name verbatim, so a payload whose symbol is the suite matched every case in the file — and an early version defined FMT_RANGES_TEST_ENABLE_C_STYLE_ARRAY for a case guarded by ENABLE_JOIN. Defining the wrong macro is the worst outcome available: it claims coverage the command does not have. Ambiguous now means unsatisfiable, reported. Test-line resolution also only considers DECLARATION-shaped lines (gtest/catch macros, Go/Rust functions, Python/Ruby methods, JS specs) and ranks them by how much of the name they spell out, instead of taking the first substring match. F3-RENDER. --verify-prefix-status <text> renders one `PRE-FIX:` line under the command, verbatim, capped at 200 bytes, newlines collapsed so a wrapper cannot inject a second line that parses as a VERIFY line. The binary deliberately does not interpret the text: the harness validates the pristine tree, and a status the tool invented would be a claim about a run it never made. The command cap rises 640 -> 768: the guard clause plus its provenance is a fixed addition, and at 640 it degraded the derivation away on exactly the instance the guard work was for. The define is on the command line either way; the derivation is what lets a reader judge it.
…s to low-value paths A ranked hit can match entirely inside prose. On preactjs__preact-3010 the gold file arrived as `src/diff/children.js:19`, a bodyless locator sitting in the JSDoc block of `diffChildren`, while the three body slots went to `benches/src/keyed-children/index.js` and `karma.conf.js`. The payload had located the right file and then described it by quoting four lines of `@param` text. Two changes, both default-on and both about RENDERING rather than ranking: - A hit whose focus line is a comment is re-anchored to the first code line that comment documents (the declaration below it, or a nearby enclosing unit) before enclosure planning runs. From there it is an ordinary code hit: the planner looks for a callable at the code line, the hit competes for a body slot at its code location, and the SAME-CONCEPT LITERAL block mines program text instead of `@param` prose. The matched line stays visible as `focus=19->26`, so the move is never silent. - Bodies from benchmark, demo, example and build-config paths yield their slot to real source, and charge neither a body slot nor a rank-tier position — the same reason a sectioned-away non-code hit does not charge its rank. A payload whose only program text IS the benchmark still prints it. Measured over 20 instances at the benchmark's own flags, 14 payloads are byte-identical and all 6 that move are correct comment-to-code re-anchors.
The re-anchor could take source away instead of adding it. On vuejs__core-11870 `arrayInstrumentations.ts:10->12` gained a complete body, became the third body in rank order, and pushed `runtime-core/src/helpers/renderList.ts:54-107` — 54 lines of the production helper the issue is about — out of the payload as a bare locator. Same invariant seatForcedSearchUnits enforces for forced units, now applied to re-anchored hits on both currencies: - Bytes: the re-anchored enclosures are planned, priced, and accepted only if the resulting allocation still shows every other rank everything the control allocation (the same plan with those enclosures blanked) showed it. - Slots: the text renderer counts the ordinary hits' demand on the three-body diet first; a re-anchor-gained body takes what is left. A denied hit keeps its re-anchored `focus=`, which is the larger part of the win and costs nothing. "Re-anchor-gained" is decided by that control comparison, not by `CommentFocusLine > 0`: most re-anchored hits already carried a body, and treating those as debtors turned two 30-line method bodies into locators on fluent__fluentd-4655 and facebook__docusaurus-9183. Verified over 30 replay instances: zero bodies lost or narrowed except the intended low-value-path demotions.
…ulti-query def
Transcript forensics of 12 sessions: the tool's rare real invocations were
fumbled by output ergonomics rather than by wrong answers.
1. NUMBERED, NEVER-TRUNCATED NAVIGATION BODIES. def/callers/neighbors bodies now
print each line in a right-aligned line-number gutter (" 989→ …"), and the
per-body cap rises 40 -> 400 lines with an actionable note when it binds:
"…continues to line N — rerun with --from M", plus a new `def --from N` so the
note is real. Measured on carbon: the agent called def, got an unnumbered
body, piped it through `head -80` — which cut AT the bug line — and abandoned
the tool for 87 turns of grep. A navigation answer that stops mid-symbol is
worse than no answer because it looks complete.
The SEARCH payload's bodies stay UNNUMBERED. That is not an inconsistency: an
agent copies those verbatim as an Edit anchor and a gutter breaks the anchor,
while a navigation answer wants coordinates. Two outputs, two rules, both
pinned by test.
`def` also now prints the SOURCE of each declaration under its card. The card
answers "what can I do with this"; agents call def to see the code, and the
card alone was never the whole answer.
2. FOLLOW-UP VERB ON BODYLESS HITS. Every search hit rendered without a body
gains " [body: def <symbol>]". Measured: redis's agent hand-sed-ranged
exactly such a locator and fmt's agent blind-Read a 200-line window off
another — both had the location and neither knew the tool could hand them the
body. A hit with no symbol name gets no suffix, because naming a verb that
cannot be run is worse than silence.
3. MULTI-QUERY def. `def A B C` answers each name in sequence under a `== name ==`
separator, on ONE index build. Agents batch shell calls under the prompt's
batching rule (laravel chained three greps into one Bash call) and a one-name
tool cannot compete with that.
Byte cost of the locator suffix, measured on the default payload: 0 B on
redis-10095 (no bodyless named hits), 50 B over 2 hits on fmt-2457, 48 B over 2 on
preact-3010 — ~24 B per affected hit and nothing on hits that carry a body. The
default search payload is otherwise unchanged.
…st output
Post-last-edit confirmation is 5.8 messages per session, the largest asymmetric
pot at 5.0 messages, and every prior attempt to shrink it freed budget the agent
immediately re-spent on more verification (transfer efficiency 0.2-0.35). A
cheaper way to run tests therefore cannot close the phase; only a verdict with
nothing left to re-spend on can, and that needs three properties raw test output
does not have.
A DELTA, not a state. `--record-baseline` captures {test id -> pass|fail|error}
on the pristine tree; `--pre-edit-baseline` diffs against it afterwards. That
turns "3 tests fail" — which invites a run to find out whether they failed
before — into three named classes, one of which is PRE-EXISTING FAILURES,
labelled as not caused by the change. Without that class a caller reads a red
suite and investigates a failure that predates the edit.
A VERDICT, not evidence. One line: PASS on zero regressions and at least one
newly-passing test, REGRESSION with every id listed (the ids are the actionable
part), or NO EFFECT. Regressions outrank fixes, because a green test going red is
the fact worth acting on.
NO RAW OUTPUT, EVER. Nothing the runner printed reaches the caller — forwarding
even an excerpt reopens the loop the verb exists to close. Ids are forwarded,
text is not, and setup output is discarded before parsing so an install log
cannot contribute test ids.
Parsers (table-driven, one function each, tested against captured samples in
testdata/verify): pytest, jest/vitest, cargo test, go test, phpunit, rspec,
minitest, maven/gradle surefire, ctest. A parser that is not confident returns
NOTHING rather than a partial set — a half-read set would manufacture verdicts
out of its own parse failures, since a test missing from one side of a diff reads
as a regression. An unrecognised format degrades to an exit-code-only verdict and
says that it is coarse.
Bounds: id lists cap at 20 with a count; the whole verdict caps at --max-bytes
(2048 default), enforced from the end so the VERDICT clause and its count always
survive — and when even that clause is too wide its id list yields, never the cap.
Timeouts mirror the harness's ecosystem split (900s compiled, 300s interpreted).
The verb writes nothing and mutates nothing beyond what the caller's own command
does.
FAIRNESS: this is tool capability — it runs a supplied command and adjudicates
the result. The "verification is complete" sentence is a statement about the DATA
(a zero-regression, at-least-one-fix delta is by construction complete), not
advice about how to behave, so the control arm's harness-side stub can print the
same wording.
…are-locator windows Three independent retrieval/extraction fixes, each with a regression test proven to fail without the change. All measured on rep30 (SWE-bench Multilingual) with a separate --cache-dir per binary -- the index cache is tree-keyed, not binary-keyed, so a shared cache silently returns the first build's results. 1. TYPESCRIPT BODYLESS DECLARATIONS (parser.go, +54) `entityFromNode` gated `function_signature` and `method_signature` to Dart, on the grounds that TypeScript uses those nodes for interface members. That is only half true: tree-sitter-typescript emits `function_signature` for EVERY bodyless function declaration -- overload signatures, `declare function`, ambients, `.d.ts` -- and `method_signature` for class-body overloads as well as interface members. All were silently dropped. vue's `renderList.ts` (five overloads + one implementation) exposed 1 symbol covering only the implementation's lines; lines 10-53, the entire typed surface, were unindexed. After: 6 symbols. Narrowly gated. `method_signature` becomes a symbol ONLY inside a `class_body` (`typeScriptClassBodySignature`). Interface and type-literal members stay inventory-only on purpose: they declare a contract, not a definition, and emitting them makes every `x.m()` call name-resolvable to a bodyless interface member. Not hypothetical -- extending this to interface_body/object_type members fabricates a CALLS edge in TestTypeScriptNamespaceCallSkipsParameterReceiverRoots, where a parameter named `B` shadows a namespace and `B.parse()` then binds to `Client.parse` instead of resolving to nothing. Attribution check: symbol output is BYTE-IDENTICAL to the pre-fix binary on eight non-TS repositories (django, axum, gson, carbon, bat, caddy, fmt, rubocop). 2. TEST-FILE DEMOTION (search_testrank.go, search_confidence.go) 3 of the 10 most expensive rep30 instances had a TEST FILE as the rank-1 hit (bat -> tests/integration_tests.rs, hugo-12204 -> render_hook_integration_test.go, fmt -> test/ranges-test.cc). When the payload points at a test instead of the code, the injected location is wrong AND the derived VERIFY command scopes to the wrong place. Demotion, not exclusion -- tests stay available, they just stop taking rank 1. Measured n=30: rank-1-is-a-test 3 -> 1, gold rank unchanged 13/30, VERIFY commands changed 0/30, file lists moved on 2/30. Also carries an independent real bug fix: the confidence gap was computed signed rather than absolute. 3. BARE-LOCATOR WINDOWS (search_locator_window.go, cli/search.go) A hit with no resolvable symbol shipped as a naked `path:line` with no body and no name, so the agent had to spend a turn opening the file to find out what it was. 19 of 30 payloads contained at least one. After: 1. Text-renderer only -- ranking provably untouched, top-5 byte-identical on 30/30. Cost: +1.83% payload bytes. VERIFICATION - `go test ./internal/sem/ ./internal/cli/` green (sem 52.4s, cli 4.8s). - Each fix has a regression test verified to FAIL with the production file stashed and PASS restored. - Collateral: non-JS/TS instances re-searched, top-5 unchanged. NOT INCLUDED, deliberately. A fourth fix (same-basename path prior) was held: it buys one gold@5 but DOWNGRADES a correctly-scoped VERIFY on preact from a narrow `npx mocha test/browser/render.test.js` to a whole-suite `npm test`, taking the correctly-scoped subset 7 -> 6. A fifth (coverage-mirror route) was dropped on its own author's no-ship: VERIFY coverage 17/30 -> 17/30, and 2 of its 3 new covering-test entries named the wrong test. NO TOKEN CLAIM is attached to any of these. Three are payload-quality changes and ten such levers have already failed as causal token levers on this benchmark; these are correctness fixes, measured on retrieval metrics only. Entire-Checkpoint: 3da7aa60f961
An issue body's reproduction link carries the web application's own state, and
that state mints tokens which outrank the words the caller actually wrote.
?file=/index.js the codesandbox / StackBlitz "open this file" parameter.
Tokenizes to `index.js`, earns a code-like weight AND a path
signal on EVERY index.js in the repository.
#fragment anchor state; never a name in the tree.
Scheme, host and path are KEPT: a GitHub blob URL names a real in-repo path and
that is genuine signal. lucene's and sklearn's issue URLs do exactly this.
MEASURED, n=30 rep30, one canonical query recipe, SEPARATE --cache-dir per binary
(the index cache is tree-keyed, not binary-keyed), determinism control 30/30
byte-identical:
gold found in the delivered payload 23/30 -> 24/30
JS/TS stratum 3/4 -> 4/4
vuejs-11870 position 5 -> 2
preact-3010 absent -> 5
non-JS/TS, all 26 instances: ZERO gold ranks moved
Rejected alternatives, both measured:
Stripping the WHOLE URL scores better on the benchmark (mean rank 2.04 vs 2.25,
gold@1 +2) but is a product regression: its fallback only fires when ZERO terms
survive, so a query naming a genuine code URL literal
("where do we hardcode https://api.example.com/v1/ingest") returns NO RESULTS.
It wins the benchmark partly by deleting real evidence. Not shipped.
A declaration-class prior was also rejected: no coverage gain, it cancels part
of this fix (preact's reachable gold IS a .d.ts, which it demotes out of the
payload), it was measured on the wrong base commit, and its constant was
calibrated from gold-patch content -- and still failed its own purpose.
NO TOKEN CLAIM. This is retrieval hygiene with a reproduced mechanism, measured on
retrieval metrics only. It does NOT demonstrably fix the JS/TS token deficit:
preact's recovered file is src/index.d.ts (a one-line `| bigint` union change)
while the behavioural site src/diff/children.js remains unreachable -- `grep -rni
bigint` over the repo returns 0 hits, because the gold patch introduces the word.
Eleven-plus retrieval/payload metrics have already failed to convert into token
savings on this benchmark; treat this as correctness, not a lever.
Entire-Checkpoint: 60e2a884c54c
Collaborator
Author
suhaanthayyil
added a commit
that referenced
this pull request
Aug 7, 2026
…wo regressions it caused (#82) Supersedes #80 (unmergeable, forked before #75/#76/#78). Contains #79's TypeScript work plus the fixes below. **Three ways search could answer with nothing, silently** — each turns a benchmark cell into one that never reaches the graph, and none was visible in the harness. Includes hardening `EG_PRESEARCH`: it read a pre-delivered payload with no length check, so a zero-byte file returned 0 bytes on stdout with exit 0. **Two regressions the TS overload extraction introduced, both fixed here in separately revertable commits:** - `800473c` — an overload set now resolves to its implementation. Before: 3 same-name declarations made every imported call `0.62 name_only "ambiguous imported call"`, which `shallowCallRelationRetained` filters in the fast profile, so `impact --profile fast` returned `IMPACT DEGENERATE: no callers` against a live caller — a false negative asserted as fact. After: `0.86 import_resolved`, `Blast radius: 1 caller`. Default and `--profile full` behaviour unchanged. - `e812688` — the implementation keeps its bare compound-v1 ID. Adding overload symbols pushed the shared base ID over the `baseCounts[id] > 1` threshold, so the *implementation* gained a `#sig:` suffix while `stable_id_version` still reported `compound-v1` — pinned IDs downstream would break with no version signal. **On the ID-churn dispute:** an earlier check reported the churn was already fixed at this head. It was not — that result came from the tree-keyed index cache. Without a separate `--cache-dir`, whichever binary runs first answers for both: main-then-PR82 prints the bare ID, PR82-then-main prints `#sig:`, same repo, same tree, opposite verdicts purely from ordering. Re-verified with a pristine build and a fresh cache dir per binary. Regression tests for all three, each proven fail-without/pass-with. `mise run check` exit 0 (fmt, vet, race, statusline 151/151, build). Schema untouched, `stable_id_version` untouched, no egress added. Known and deliberately left: `impact --symbol X` with no `--file` still lists all 3 definitions (degraded UX, not a false negative) — collapsing it needs a serialized field and schema 1.x is frozen.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://entire.io/gh/entireio/entire-graph/trails/34
Three independent retrieval/extraction fixes. Each has a regression test proven to fail without the change and pass with it (production file stashed, test run, restored, run again — both outcomes captured). All measured on rep30 / SWE-bench Multilingual with a separate
--cache-dirper binary, because the index cache is tree-keyed rather than binary-keyed and a shared cache silently returns the first build's results.1. TypeScript bodyless declarations —
parser.go(+54)entityFromNodegatedfunction_signatureandmethod_signatureto Dart, on the grounds that TypeScript uses those nodes for interface members. That is only half true. tree-sitter-typescript emitsfunction_signaturefor every bodyless function declaration — overload signatures,declare function, ambients,.d.ts— andmethod_signaturefor class-body overloads as well as interface members. All were silently dropped.vue's
renderList.ts(five overloads + one implementation) exposed 1 symbol, covering only the implementation's lines. Lines 10–53 — the entire typed surface — were unindexed, so no query phrased in terms of the declared parameter types could retrieve it. After: 6 symbols.Narrowly gated.
method_signaturebecomes a symbol only inside aclass_body(newtypeScriptClassBodySignature). Interface and type-literal members stay inventory-only on purpose: they declare a contract, not a definition, and emitting them makes everyx.m()call name-resolvable to a bodyless interface member. That is not hypothetical — extending this tointerface_body/object_typemembers fabricates a CALLS edge inTestTypeScriptNamespaceCallSkipsParameterReceiverRoots, where a parameter namedBshadows a namespace andB.parse()then binds toClient.parseinstead of correctly resolving to nothing.Attribution check: symbol output is byte-identical to the pre-fix binary on eight non-TypeScript repositories (django, axum, gson, carbon, bat, caddy, fmt, rubocop), so the change cannot perturb other languages.
2. Test-file demotion —
search_testrank.go,search_confidence.go3 of the 10 most expensive rep30 instances had a test file as the rank-1 hit (bat →
tests/integration_tests.rs, hugo-12204 →render_hook_integration_test.go, fmt →test/ranges-test.cc). When the payload points at a test instead of the code, the injected location is wrong and the derived VERIFY command scopes to the wrong place.Demotion, not exclusion — tests stay retrievable, they just stop taking rank 1.
Measured n=30: rank-1-is-a-test 3 → 1, gold rank unchanged 13/30, VERIFY commands changed 0/30, file lists moved on 2/30. Also carries an independent real bug fix: the confidence gap was computed signed where it should be absolute.
3. Bare-locator windows —
search_locator_window.go,cli/search.goA hit with no resolvable symbol shipped as a naked
path:line— no body, no name — so the agent had to spend a turn opening the file just to learn what it was. 19 of 30 payloads contained at least one; after, 1.Text-renderer only: ranking is provably untouched, top-5 byte-identical on 30/30. Cost is +1.83% payload bytes.
Verification
go test ./internal/sem/ ./internal/cli/— green (sem 52.4s, cli 4.8s)Deliberately not included
npx mocha test/browser/render.test.jsto a whole-suitenpm test, taking the correctly-scoped subset 7 → 6. Bad trade on a benchmark where VERIFY scope is the largest token lever. Worth revisiting once thecoverage_noteloss is understood.No token claim
Three of these are payload-quality changes, and ten such levers have already failed as causal token levers on this benchmark. These are correctness fixes and are measured on retrieval metrics only. Any savings claim would need its own paired cell.
Supersedes #79 (TypeScript fix alone, based on an older commit).