fix(sem): extract TypeScript bodyless declarations (overloads, ambients) - #79
fix(sem): extract TypeScript bodyless declarations (overloads, ambients)#79suhaanthayyil wants to merge 5 commits into
Conversation
Measured over 178 gated benchmark pairs, graph calls per session are invariant (1.00/1.03/1.00 across baseline bands <10/10-19/>=20) while displaced greps are not (1.35/2.16/3.74), so the call is a fixed toll with variable repayment: the no-op cohort (n=43) costs 1.165 CI [1.073,1.264] at +1.21 turns, the displacing cohort (n=135) costs 0.938 CI [0.889,0.990] at -2.07 turns, corr(turn delta, log ratio) = +0.735. No static payload feature predicts a no-op call (AUC 0.553-0.690 at a 28% base rate), so selective skipping is not implementable and pre-delivery must be universal. ENTIRE_GRAPH_PRESEARCH (alias EG_PRESEARCH) names a file holding the payload computed before the agent started; search echoes it byte-for-byte and returns before the profile, repo, cache and index are touched. An unreadable path is an error rather than a silent live query.
Two printed bodies of one file with a small hole between them are one region as far as the reader is concerned, and the hole is what it spends a turn closing. Measured across 113 Opus search payloads: 45 (40%) carry at least two ranked hits in one file within 40 lines (R30Oeg 12/25, R30OegBEST 12/30, R30OegV9 9/29, C15opeg_r1 12/29). Read side: of 53 answered pre-edit Read calls only 41 are ranged, and of those only 22 (53.7%) overlap the payload snippet at all, with the payload covering a mean 18.2% of the requested span - so nominal 62.0% read coverage is worth ~33%. Suite level: the tool makes +19.6% MORE Read calls than the no-tool baseline while total tool calls fall 14.5% (n=55 paired instances). Call by call: fluentd-3328 had gold at rank 1 (in_tail.rb:395-425, score 135.8, complete-symbol, "Top hit is exact") and still cost 1.64x, because ranks 1/2/3 were three disjoint spans of one file (349-360, 395-425, 427-443) and turn 2 was `sed -n '200,270p'; sed -n '330,470p'` - 6.9 kB re-reading a contiguous superset of what it already had. pylint-7080 turn 2 re-fetches rank 2 verbatim with `cat`. Near hits are now folded into one span carrying the verbatim text of the whole range, headed `path:START-END [contains ranks i,j,k - contiguous, nothing elided]`, because the reason the agent re-reads is that it cannot tell whether the hole hides something. Filling gaps adds bytes, and under a fixed ceiling added bytes evict the last ranked hit, so three bounds are enforced: a 40-line gap cap, an 80-line span cap, and a cost check of the whole prospective ranking against --max-context-bytes before each merge is committed - a bridge that would not fit simply does not happen. Only complete-symbol results take part (the one signal every renderer prints in full, so a merged header can never name a range whose text is absent), and merging never reorders: a run's survivor is its best-ranked member, so the rank at which a file first appears can only improve. (cherry picked from commit daa9b95d7911f35abbd0edfefac14bbea5cbed47)
Ported from eg-consolidated 024f25a. FocusLine is computed for every hit and has always been in the JSON; the text payload printed only the snippet range, so a hit on a 27-line method said 'the answer is somewhere in 135-161'. Asked what one change to the returned payload would let it finish in the fewest calls, a Sonnet agent that had just fixed apache/lucene-13170 with this tool named exactly this -- its only non-essential call was a Read to pin down line 151. Re-reading a file the payload already printed is 10.1% of post-payload tool calls across 30 measured sessions. Header, not an inline marker on the snippet line: agents copy snippet text verbatim as an edit's old_string anchor, so decorating a body line would break the anchor. Entire-Checkpoint: 2893461866fb
Adds EG_SEARCH_SESSION (the per-task state file the one-shot CLI needs to count its own calls) and EG_MAX_SEARCHES (default 1, 0 disables). Past the cap, search returns the first call's payload verbatim under a one-line header instead of running a new query. Measured: sessions with >=4 graph calls (n=16) cost 1.148 of the no-tool baseline against 0.975 for one-call sessions (n=70), diff +0.173, bootstrap CI [+0.019,+0.324]. Same gradient per config: 1.00 calls/session -> -12.7%, 1.17 -> -0.7%, 1.63 -> +5.1%, 4.45 -> worst retrieval of the set. (cherry picked from commit 823cc5e43e7e30bb121e0502c27d4f5e50f3e3b4)
`entityFromNode` gated `function_signature` and `method_signature` to Dart on the grounds that TypeScript uses those node types for interface members. That is only half true: tree-sitter-typescript emits `function_signature` for EVERY bodyless function declaration -- overload signatures, `declare function`, ambients, `.d.ts` -- and `method_signature` for class-body overload declarations as well as interface members. All of them were silently dropped. Consequence: vue's `renderList.ts` (five overload signatures + one implementation) exposed a single symbol covering only the implementation's lines, so the typed half of the file was invisible to search and no query phrased in terms of the declared parameter types could retrieve it. Measured 1 symbol before, 6 after. The fix is additive and narrowly gated: - `function_signature` in TypeScript becomes a function (or a method when scoped). An overload set is where the parameter types and generics live; the implementation's own signature is usually the erased `(source: any, ...)` catch-all. - `method_signature` becomes a method ONLY inside a `class_body`, via the new `typeScriptClassBodySignature` helper. Interface and type-literal members stay inventory-only on purpose: they declare a contract, not a definition, and emitting them would make every `x.m()` call name-resolvable to a bodyless interface member. That is not hypothetical -- extending this to interface_body/object_type members fabricates a CALLS edge in TestTypeScriptNamespaceCallSkipsParameterReceiverRoots, where a parameter named `B` shadows a namespace and `B.parse()` then binds to `Client.parse` instead of resolving to nothing. Dart behaviour is unchanged; the existing Dart paths are reached only when the language is not TypeScript. Verified: full `internal/sem` suite green (1976 tests). Symbol output is byte-identical to the pre-fix binary on eight non-TypeScript repositories (django, axum, gson, carbon, bat, caddy, fmt, rubocop), so the change is attributable to TypeScript alone. Entire-Checkpoint: 3eabe9fc89ff
|
Closing as superseded by #82, with a caveat worth recording. Superseded: #82 contains this PR's four search commits by identical SHA (8d464ce, 5511d6f, afdb54e, afd5a1c) and folds the TypeScript fix into 19b0769. Two reasons not to merge this one on its own, both measured on a 2-file TS repro:
The TypeScript regression tests here are genuine and verified fail-without/pass-with — they live on in #82. Separately: the same TS change causes compound-v1 ID churn and a fast-profile CALLS loss in both this PR and #82. Being fixed on #82 before it merges. |
…wo regressions it caused (#82) Supersedes #80 (unmergeable, forked before #75/#76/#78). Contains #79's TypeScript work plus the fixes below. **Three ways search could answer with nothing, silently** — each turns a benchmark cell into one that never reaches the graph, and none was visible in the harness. Includes hardening `EG_PRESEARCH`: it read a pre-delivered payload with no length check, so a zero-byte file returned 0 bytes on stdout with exit 0. **Two regressions the TS overload extraction introduced, both fixed here in separately revertable commits:** - `800473c` — an overload set now resolves to its implementation. Before: 3 same-name declarations made every imported call `0.62 name_only "ambiguous imported call"`, which `shallowCallRelationRetained` filters in the fast profile, so `impact --profile fast` returned `IMPACT DEGENERATE: no callers` against a live caller — a false negative asserted as fact. After: `0.86 import_resolved`, `Blast radius: 1 caller`. Default and `--profile full` behaviour unchanged. - `e812688` — the implementation keeps its bare compound-v1 ID. Adding overload symbols pushed the shared base ID over the `baseCounts[id] > 1` threshold, so the *implementation* gained a `#sig:` suffix while `stable_id_version` still reported `compound-v1` — pinned IDs downstream would break with no version signal. **On the ID-churn dispute:** an earlier check reported the churn was already fixed at this head. It was not — that result came from the tree-keyed index cache. Without a separate `--cache-dir`, whichever binary runs first answers for both: main-then-PR82 prints the bare ID, PR82-then-main prints `#sig:`, same repo, same tree, opposite verdicts purely from ordering. Re-verified with a pristine build and a fresh cache dir per binary. Regression tests for all three, each proven fail-without/pass-with. `mise run check` exit 0 (fmt, vet, race, statusline 151/151, build). Schema untouched, `stable_id_version` untouched, no egress added. Known and deliberately left: `impact --symbol X` with no `--file` still lists all 3 definitions (degraded UX, not a false negative) — collapsing it needs a serialized field and schema 1.x is frozen.
https://entire.io/gh/entireio/entire-graph/trails/33
The bug
entityFromNodegated bothfunction_signatureandmethod_signatureto Dart, with the comment that TypeScript uses those node types for interface members. That is only half true.tree-sitter-typescript emits
function_signaturefor every bodyless function declaration — overload signatures,declare function, ambients,.d.ts— andmethod_signaturefor class-body overload declarations as well as interface members. All of them were silently dropped from the graph.Concretely, vue's
renderList.tshas five overload signatures plus one implementation. It exposed 1 symbol, covering only the implementation's lines. Lines 10–53 — the entire typed surface — were unindexed, so no query phrased in terms of the declared parameter types could retrieve the file. After the fix: 6 symbols.The fix
Additive,
+54lines, no deletions, narrowly gated:function_signaturein TypeScript becomes afunction(or amethodwhen scoped). An overload set is where the parameter types and generics live; the implementation's own signature is usually the erased(source: any, ...)catch-all.method_signaturebecomes amethodonly inside aclass_body, via the newtypeScriptClassBodySignaturehelper.Interface and type-literal members deliberately stay inventory-only. They declare a contract, not a definition, and emitting them would make every
x.m()call name-resolvable to a bodyless interface member. That is not hypothetical — extending this case tointerface_body/object_typemembers fabricates a CALLS edge inTestTypeScriptNamespaceCallSkipsParameterReceiverRoots, where a parameter namedBshadows a namespace andB.parse()then binds toClient.parseinstead of correctly resolving to nothing.Dart behaviour is unchanged — the existing Dart paths are reached only when the language is not TypeScript.
Verification
internal/semsuite green (1976 tests).parser_typescript_regression_test.go(+124 lines) covering overload sets,declare function, class-body overloads, and the interface-member exclusion.Scope note
Repo-wide symbol gain is modest (vuejs +1.0%, docusaurus +2.1%) — this is a correctness fix for a whole class of declarations, not a broad indexing win. Its practical effect is concentrated in TypeScript codebases that use overloads and ambient declarations heavily.