Skip to content

fix(sem): extract TypeScript bodyless declarations (overloads, ambients) - #79

Closed
suhaanthayyil wants to merge 5 commits into
mainfrom
fix/typescript-bodyless-declaration-extraction
Closed

fix(sem): extract TypeScript bodyless declarations (overloads, ambients)#79
suhaanthayyil wants to merge 5 commits into
mainfrom
fix/typescript-bodyless-declaration-extraction

Conversation

@suhaanthayyil

@suhaanthayyil suhaanthayyil commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

https://entire.io/gh/entireio/entire-graph/trails/33

The bug

entityFromNode gated both function_signature and method_signature to Dart, with the comment that TypeScript uses those node types for interface members. That is only half true.

tree-sitter-typescript emits function_signature for every bodyless function declaration — overload signatures, declare function, ambients, .d.ts — and method_signature for class-body overload declarations as well as interface members. All of them were silently dropped from the graph.

Concretely, vue's renderList.ts has five overload signatures plus one implementation. It exposed 1 symbol, covering only the implementation's lines. Lines 10–53 — the entire typed surface — were unindexed, so no query phrased in terms of the declared parameter types could retrieve the file. After the fix: 6 symbols.

The fix

Additive, +54 lines, no deletions, narrowly gated:

  • function_signature in TypeScript becomes a function (or a method when scoped). An overload set is where the parameter types and generics live; the implementation's own signature is usually the erased (source: any, ...) catch-all.
  • method_signature becomes a method only inside a class_body, via the new typeScriptClassBodySignature helper.

Interface and type-literal members deliberately stay inventory-only. They declare a contract, not a definition, and emitting them would make every x.m() call name-resolvable to a bodyless interface member. That is not hypothetical — extending this case to interface_body/object_type members fabricates a CALLS edge in TestTypeScriptNamespaceCallSkipsParameterReceiverRoots, where a parameter named B shadows a namespace and B.parse() then binds to Client.parse instead of correctly resolving to nothing.

Dart behaviour is unchanged — the existing Dart paths are reached only when the language is not TypeScript.

Verification

  • Full internal/sem suite green (1976 tests).
  • New regression test parser_typescript_regression_test.go (+124 lines) covering overload sets, declare function, class-body overloads, and the interface-member exclusion.
  • Attribution check: symbol output is byte-identical to the pre-fix binary on eight non-TypeScript repositories — django, axum, gson, carbon, bat, caddy, fmt, rubocop. So the change is attributable to TypeScript alone and cannot perturb other languages.

Scope note

Repo-wide symbol gain is modest (vuejs +1.0%, docusaurus +2.1%) — this is a correctness fix for a whole class of declarations, not a broad indexing win. Its practical effect is concentrated in TypeScript codebases that use overloads and ambient declarations heavily.

suhaanthayyil and others added 5 commits August 1, 2026 18:55
Measured over 178 gated benchmark pairs, graph calls per session are invariant
(1.00/1.03/1.00 across baseline bands <10/10-19/>=20) while displaced greps are
not (1.35/2.16/3.74), so the call is a fixed toll with variable repayment: the
no-op cohort (n=43) costs 1.165 CI [1.073,1.264] at +1.21 turns, the displacing
cohort (n=135) costs 0.938 CI [0.889,0.990] at -2.07 turns, corr(turn delta, log
ratio) = +0.735. No static payload feature predicts a no-op call (AUC
0.553-0.690 at a 28% base rate), so selective skipping is not implementable and
pre-delivery must be universal.

ENTIRE_GRAPH_PRESEARCH (alias EG_PRESEARCH) names a file holding the payload
computed before the agent started; search echoes it byte-for-byte and returns
before the profile, repo, cache and index are touched. An unreadable path is an
error rather than a silent live query.
Two printed bodies of one file with a small hole between them are one region as
far as the reader is concerned, and the hole is what it spends a turn closing.

Measured across 113 Opus search payloads: 45 (40%) carry at least two ranked
hits in one file within 40 lines (R30Oeg 12/25, R30OegBEST 12/30, R30OegV9
9/29, C15opeg_r1 12/29). Read side: of 53 answered pre-edit Read calls only 41
are ranged, and of those only 22 (53.7%) overlap the payload snippet at all,
with the payload covering a mean 18.2% of the requested span - so nominal 62.0%
read coverage is worth ~33%. Suite level: the tool makes +19.6% MORE Read calls
than the no-tool baseline while total tool calls fall 14.5% (n=55 paired
instances). Call by call: fluentd-3328 had gold at rank 1 (in_tail.rb:395-425,
score 135.8, complete-symbol, "Top hit is exact") and still cost 1.64x, because
ranks 1/2/3 were three disjoint spans of one file (349-360, 395-425, 427-443)
and turn 2 was `sed -n '200,270p'; sed -n '330,470p'` - 6.9 kB re-reading a
contiguous superset of what it already had. pylint-7080 turn 2 re-fetches rank
2 verbatim with `cat`.

Near hits are now folded into one span carrying the verbatim text of the whole
range, headed `path:START-END [contains ranks i,j,k - contiguous, nothing
elided]`, because the reason the agent re-reads is that it cannot tell whether
the hole hides something.

Filling gaps adds bytes, and under a fixed ceiling added bytes evict the last
ranked hit, so three bounds are enforced: a 40-line gap cap, an 80-line span
cap, and a cost check of the whole prospective ranking against
--max-context-bytes before each merge is committed - a bridge that would not
fit simply does not happen. Only complete-symbol results take part (the one
signal every renderer prints in full, so a merged header can never name a range
whose text is absent), and merging never reorders: a run's survivor is its
best-ranked member, so the rank at which a file first appears can only improve.

(cherry picked from commit daa9b95d7911f35abbd0edfefac14bbea5cbed47)
Ported from eg-consolidated 024f25a. FocusLine is computed for every hit and has always been in
the JSON; the text payload printed only the snippet range, so a hit on a 27-line method said 'the
answer is somewhere in 135-161'. Asked what one change to the returned payload would let it finish
in the fewest calls, a Sonnet agent that had just fixed apache/lucene-13170 with this tool named
exactly this -- its only non-essential call was a Read to pin down line 151. Re-reading a file the
payload already printed is 10.1% of post-payload tool calls across 30 measured sessions.

Header, not an inline marker on the snippet line: agents copy snippet text verbatim as an edit's
old_string anchor, so decorating a body line would break the anchor.

Entire-Checkpoint: 2893461866fb
Adds EG_SEARCH_SESSION (the per-task state file the one-shot CLI needs to
count its own calls) and EG_MAX_SEARCHES (default 1, 0 disables). Past the
cap, search returns the first call's payload verbatim under a one-line
header instead of running a new query.

Measured: sessions with >=4 graph calls (n=16) cost 1.148 of the no-tool
baseline against 0.975 for one-call sessions (n=70), diff +0.173, bootstrap
CI [+0.019,+0.324]. Same gradient per config: 1.00 calls/session -> -12.7%,
1.17 -> -0.7%, 1.63 -> +5.1%, 4.45 -> worst retrieval of the set.

(cherry picked from commit 823cc5e43e7e30bb121e0502c27d4f5e50f3e3b4)
`entityFromNode` gated `function_signature` and `method_signature` to Dart on the
grounds that TypeScript uses those node types for interface members. That is only
half true: tree-sitter-typescript emits `function_signature` for EVERY bodyless
function declaration -- overload signatures, `declare function`, ambients, `.d.ts`
-- and `method_signature` for class-body overload declarations as well as interface
members. All of them were silently dropped.

Consequence: vue's `renderList.ts` (five overload signatures + one implementation)
exposed a single symbol covering only the implementation's lines, so the typed half
of the file was invisible to search and no query phrased in terms of the declared
parameter types could retrieve it. Measured 1 symbol before, 6 after.

The fix is additive and narrowly gated:

- `function_signature` in TypeScript becomes a function (or a method when scoped).
  An overload set is where the parameter types and generics live; the implementation's
  own signature is usually the erased `(source: any, ...)` catch-all.
- `method_signature` becomes a method ONLY inside a `class_body`, via the new
  `typeScriptClassBodySignature` helper. Interface and type-literal members stay
  inventory-only on purpose: they declare a contract, not a definition, and emitting
  them would make every `x.m()` call name-resolvable to a bodyless interface member.
  That is not hypothetical -- extending this to interface_body/object_type members
  fabricates a CALLS edge in TestTypeScriptNamespaceCallSkipsParameterReceiverRoots,
  where a parameter named `B` shadows a namespace and `B.parse()` then binds to
  `Client.parse` instead of resolving to nothing.

Dart behaviour is unchanged; the existing Dart paths are reached only when the
language is not TypeScript.

Verified: full `internal/sem` suite green (1976 tests). Symbol output is
byte-identical to the pre-fix binary on eight non-TypeScript repositories
(django, axum, gson, carbon, bat, caddy, fmt, rubocop), so the change is
attributable to TypeScript alone.

Entire-Checkpoint: 3eabe9fc89ff
@suhaanthayyil

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by #82, with a caveat worth recording.

Superseded: #82 contains this PR's four search commits by identical SHA (8d464ce, 5511d6f, afdb54e, afd5a1c) and folds the TypeScript fix into 19b0769. git diff pr79 pr82 -- internal/sem/parser_typescript_regression_test.go is empty and the parser.go hunks are byte-identical; #82's parser.go is a strict superset (it also carries the Go method_elem/method_spec case).

Two reasons not to merge this one on its own, both measured on a 2-file TS repro:

  1. It carries the unhardened EG_PRESEARCH path — internal/cli/search.go:254-261 reads a pre-delivered payload with no length check. EG_PRESEARCH=<zero-byte file> entire-graph search --query x returns 0 bytes on stdout, exit 0. fix(cli): three ways search could answer with nothing, silently (supersedes #80) #82 fixes this (8a4ffb3, search.go:310-312).
  2. Merging it first makes feat: ship search, compact snapshots, explain, and prose-memory results #84 conflict in internal/sem/search.go (the mergeSameFileSearchSpans call lands in the same SearchRepository stage feat: ship search, compact snapshots, explain, and prose-memory results #84 edits), adding a third source of the fix(cli): three ways search could answer with nothing, silently (supersedes #80) #82feat: ship search, compact snapshots, explain, and prose-memory results #84 conflict while contributing nothing fix(cli): three ways search could answer with nothing, silently (supersedes #80) #82 lacks.

The TypeScript regression tests here are genuine and verified fail-without/pass-with — they live on in #82.

Separately: the same TS change causes compound-v1 ID churn and a fast-profile CALLS loss in both this PR and #82. Being fixed on #82 before it merges.

suhaanthayyil added a commit that referenced this pull request Aug 7, 2026
…wo regressions it caused (#82)

Supersedes #80 (unmergeable, forked before #75/#76/#78). Contains #79's TypeScript work plus the fixes below.

**Three ways search could answer with nothing, silently** — each turns a benchmark cell into one that never reaches the graph, and none was visible in the harness. Includes hardening `EG_PRESEARCH`: it read a pre-delivered payload with no length check, so a zero-byte file returned 0 bytes on stdout with exit 0.

**Two regressions the TS overload extraction introduced, both fixed here in separately revertable commits:**

- `800473c` — an overload set now resolves to its implementation. Before: 3 same-name declarations made every imported call `0.62 name_only "ambiguous imported call"`, which `shallowCallRelationRetained` filters in the fast profile, so `impact --profile fast` returned `IMPACT DEGENERATE: no callers` against a live caller — a false negative asserted as fact. After: `0.86 import_resolved`, `Blast radius: 1 caller`. Default and `--profile full` behaviour unchanged.
- `e812688` — the implementation keeps its bare compound-v1 ID. Adding overload symbols pushed the shared base ID over the `baseCounts[id] > 1` threshold, so the *implementation* gained a `#sig:` suffix while `stable_id_version` still reported `compound-v1` — pinned IDs downstream would break with no version signal.

**On the ID-churn dispute:** an earlier check reported the churn was already fixed at this head. It was not — that result came from the tree-keyed index cache. Without a separate `--cache-dir`, whichever binary runs first answers for both: main-then-PR82 prints the bare ID, PR82-then-main prints `#sig:`, same repo, same tree, opposite verdicts purely from ordering. Re-verified with a pristine build and a fresh cache dir per binary.

Regression tests for all three, each proven fail-without/pass-with. `mise run check` exit 0 (fmt, vet, race, statusline 151/151, build). Schema untouched, `stable_id_version` untouched, no egress added.

Known and deliberately left: `impact --symbol X` with no `--file` still lists all 3 definitions (degraded UX, not a false negative) — collapsing it needs a serialized field and schema 1.x is frozen.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant