docs: LINEAGE.md — the row-by-row ledger of where the ideas come from - #2
Merged
Merged
Conversation
Almost nothing here is a new idea; the combination and the constraints
are. "Inspired by the whole field" is unfalsifiable and "we invented
ranked code maps" is false, so this is the table instead.
Every row: the work, its link, the one-line lesson taken, and the real
flag or source file where that lesson lives. Three tables, two honest
tiers:
1. Classic papers (6) — McCabe 1976, Okapi BM25 1994, Personalized
PageRank 1998, HITS 1999, Louvain 2008, RRF
2009. The ranking and quality machinery.
2. Modern research (21) — every paper traceable to a shipped decision,
including the ones that argued AGAINST a
feature: the similar-snippet result is why no
similarity output exists, the nested-data
serialization result is why --format=columnar
REFUSES non-tabular verbs, and the coupling
metrics have no independent validation, which
is why --deps/--arch label them a heuristic
and never let them gate.
3a. Folded tools (27) — a named, specific lesson each. Six are
vendored code and say so.
3b. Surveyed (225) — the labelled landscape, 26 categories, with
the prose stating twice that surveyed is NOT
borrowed-from. Folded tools are not repeated
there, so nothing is counted twice.
The honesty rule is the point: a work is folded only if the lesson fits
in one sentence AND points at something real. Works cited during the
research that could not be traced to a decision of their own are NAMED at
the end of section 2 and at the end of 3a rather than padded into the
tables — the near misses (a scoped-snippet view never shipped, an
elision idea whose source has no citable link, a head-to-head arm that
taught nothing) are more informative than a bigger number would be.
Every --flag named here is verified against the shipped binary's --help;
deckcheck.sh scans this file and would fail on a fabricated one.
RED FIRST. Arm (E) was written and proven to go red before the numbers
were allowed to agree: with README.md temporarily claiming 31
repositories against LINEAGE.md's enumerated 27, (E2) failed loudly and
named both sides. A second mutation — padding one surveyed row's n from 4
to 9 — was caught by BOTH (E2) (survey total 225 -> 230) and (E3) (the
row's declared n no longer equals the names it lists). Only then was the
true count written.
README gains "Standing on the whole field" after Measured: lessons folded
from 27 repositories and 27 papers, from a labelled survey of 225 tools,
linking docs/LINEAGE.md. Plus the nav link and a Documentation-table row.
readmedriftcheck.sh gains arm (E), five sub-arms, in the file's existing
A-D style:
E1 the tables derive to plausible sizes at all — a renamed heading or a
table turned into a bullet list would otherwise derive 0/0/0, and
0 == 0 against a stale README is a green-while-inert pass, the exact
failure mode this file's own header warns about for deckcheck.
E2 the README sentence must equal the derived counts.
E3 every surveyed row's n column must equal the number of names in that
same row — so the summed column cannot be quietly padded.
E4 mutation control for E2, exactly as (C) is (B)'s: a copy of the
README with a wrong repository count must be SEEN to disagree.
Without it, E2 passing proves two numbers were read, never that a
wrong one would be caught.
E5 LINEAGE.md's own header prose vs its own tables. A document whose
summary disagrees with its rows is worse than one that never
summarised.
Derivation is structural, not textual: markdown body rows scoped by
section heading, with the survey total summed from a per-row column so
each row is auditable on its own. The count extractor strips `*` and
anchors on the WORDS, so both files' different bolding, and any rewrap,
work through one regex.
The count class has bitten this suite twice already (the flag count in
arms A-D; the docs/README.md entry count). This is the same discipline
applied to three more advertised numbers.
The docs/ index gains LINEAGE.md ("Anyone asking what is actually new
here") and its header count goes seven -> eight. The count was verified
against the actual table rows rather than incremented on trust — this
exact class has been wrong here before, and ripwirepubliccheck arm 6b
only proves every docs/ file is LISTED, never that the prose count above
the list is right.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe change adds ChangesLineage documentation and exposure
Lineage validation
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/LINEAGE.md`:
- Around line 136-163: Deduplicate the survey inventory so each tool is counted
once: reconcile comby across its two categories, avoid separately counting Aider
when its repo-map capability is folded in, and avoid separately counting Cody
when Sourcegraph/Cody is folded in. Update the table totals and all derived
README and header counts to reflect the resulting unique-tool survey, or
explicitly rename entries to distinguish genuinely separate tools.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c7d99cf0-7fe4-40a4-a293-616f7c877ae5
📒 Files selected for processing (4)
README.mddocs/LINEAGE.mddocs/README.mdtest/readmedriftcheck.sh
…s they move
Row-level corrections, each checked against the binary or the source rather than
against the research record's intent:
- pdqsort was a folded row claiming "the ranking sorts". It is vendored, licensed,
and linked by NO shipped target: sortutil.h sorts with a radix pass plus a
std::sort fallback, and the only consumer of the pdqsort wrapper
(src/infra/fastSort.h) is a benchmark no CMake target compiles. Moved to the
near-miss prose, which now says exactly that. Folded drops by one.
- octocode gets the row it was denied. The old prose said the record "names it
without a citable link" — it is a public repository and the record names it
twice. Its lesson (self-describing elision) is what shown_<noun>=/<noun>_capped=
implements. Folded rises by one. Net: 27 repositories, unchanged by coincidence,
not by steering.
- --around claimed "stays shallow by default". cli.h:67 sets aroundDepth = 2, and
the record marks that landing PARTIAL ("should stay shallow" — aspirational).
The row now says two hops, names --around-depth=1 as the paper's form, and calls
the default an open question instead of a shipped fact.
- LARGER claimed the mechanism landed. It was measured (+0.41pp, 95% LB +0.00pp)
and REJECTED by the acceptance gate; --anchor is absent from --help and refuses.
Rewritten as the rejection story the §2 preamble promises, with the two opt-out
flags corrected: --no-route is query-shape routing, --no-mention-boost is the
mention anchor (both verified in --help).
- §3a and §3b overlapped on Aider, Cody, RepoGraph and octocode, and comby was in
two §3b rows. All five removed from the survey table; the affected rows' n
recounted. Surveyed: 225 -> 220.
- The header framed 27 as a subset of 225. The two sets are disjoint BY
CONSTRUCTION, so they add: reframed in both this document and the README.
- ctags row described a <s t= n= p=> shape the binary does not emit. The real row
is <s t= n= id= k=> inside an <f p=> file element, with no line number.
- Zoekt row pointed at src/lexindex.h, which contains the word "trigram" zero
times; src/search.h contains it 47 times. Pointer dropped.
- Serena row attributed counts_floor= to resolve.h; it lives in graphlegend.h.
- BM25 was cited as "Robertson & Sparck Jones 1994, doi:10.1561/1500000019". That
DOI is Robertson & Zaragoza 2009. Year and DOI now belong to the right works.
- CCFinder cited a course-hosted PDF; now the canonical Kamiya et al. 2002 DOI.
- The three non-peer-reviewed sources in §2 (a vendor spec, a book, a practitioner
article) are labelled as such in-row instead of sitting among papers unmarked.
Counts re-derive to 27 / 27 / 220. readmedriftcheck E1-E5 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…etic E1-E5 proved the three advertised counts were ARITHMETIC restatements of the tables. They cannot see a count that is arithmetically perfect and false about a SET, which is what "27 folded plus 220 surveyed" was: the addition is only valid if the two tables are disjoint, and nothing checked that they were. RED-FIRST, against the pre-fix docs/LINEAGE.md at 4ccf149: E6 (folded vs surveyed) aider 3a Aider cody 3a Cody repograph 2 RepoGraph E7 (within the survey) comby Both arms also carry a live mutation control on every run — E6 plants a folded name into a surveyed row, E7 plants a duplicate — so a green result proves the comparison can fail, not merely that two lists were read. Name matching is key-set, not string equality, because the two tables spell the same tool differently on purpose: a folded row uses the project's own name ("[Sourcegraph / Cody]", "[aider repo-map]") and the survey uses the catalogue's ("Cody", "Aider"). A whole-cell compare sees no overlap and passes — that is precisely how Aider and Cody survived review. Each folded entry therefore expands to its normalised whole name, its slash-alternatives, and (for §3a repository rows only) its four-plus-character words. §2 is excluded from the word split: its first column is prose, and splitting "Metric feedback into the loop" into keys would invite the false positive. It is not needed either — RepoGraph is the whole cell. E8 is the cheap half of the honesty rule the document states in its own header: a folded lesson must be "pointed at a real flag or source file", so the 30 paths it points at must exist. The literal path-CONTAINS-a-row-identifier form was considered and rejected: a row's lesson is English prose that shares no tokens with the source it cites, so containment would be a false-positive generator. Also hardened counts_from(): it flattens the file before matching. The header claimed a rewrap "cannot silently disarm the arm", but a line break between "of" and "220" made E2 report "sentence not found" — loud, but for the wrong reason, and one reflow from being deleted as broken. Now the claim is true. readmedriftcheck: 12 arms, ALL PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Said "three rows" for what is three SOURCES across two rows — the vendor spec is one row, the book and the practitioner article share the component-coupling row. Exactly the count-vs-set slip the rest of this round was about. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The rewritten RepoGraph row said "capped at two hops, never unbounded" while fixing a different false claim in the same row. src/cli.h:1609 registers --around-depth= as a plain positive integer up to kIntFlagMax, and --around-depth=3 returns 940 symbols where depth 1 returns 31. There is no cap; there is a default of 2. The row now separates the three facts it kept conflating: the flag EXPOSES the paper's one-hop form, the DEFAULT is 2, and nothing ENFORCES shallowness. Adds the measured cost of the second hop (31 -> 283 symbols on --around=buildGraph) so the open question about the default has a number attached to it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Aug 19, 2026
…instrument, not the floor
The ranking lane scored the LIVE source tree, so it measured this repository's own source
growth as if it were ranking quality: every wave that adds load-bearing symbols displaces
the lane's own gold. That is the identical defect the recall lane retired on 2026-08-07,
one level down — and its half of that entry ("Ranking lane unchanged, live: its labels are
code symbols and its floors carry wide margins") did not survive contact.
THREE INDEPENDENT MEASUREMENTS, none of them a floor argument, said so before this commit:
1. The convergence-disclosure lane's residual (docs/EVALS.md §6 probe 4). Three-cell
control: base binary / base tree 75.0% (MRR 0.694); base binary / wave tree 71.9%
(0.660); wave binary / wave tree 71.9% (0.660). The BASE binary scores the wave
number, so all of the -3.1pp is corpus. Displacers named by inspection: on "pagerank
power iteration" ranks 2-5 and 9 of the wave tree are RankDisclosure/renderDisclosure
(src/prconverge.h), RankedGraph/rankGraphTeleport (src/graph.h) and PageRankRun.
2. The wave-2 adversarial verifier's follow-up F, reached independently and repeated in
its 2026-08-19 addendum: recalibrate by freezing the lane's corpus (the mechanism
already exists for the recall lane), NOT by lowering the floor a fourth time.
3. The subtoken merge window's 2x2 (2026-08-19). Ranking lenient r@5, n=32, one query =
3.125pp: the parser-65 and parser-66 binaries both score 71.9% on the parent tree and
both score 68.8% on the merged tree. The tokenizer contributes exactly zero; the whole
step is one label flipping because that round consolidated the body of its own gold
(rw::subtokens, src/lexical.h, rank 3 -> 9).
WHAT IS FROZEN: bench/recalleval/snapshot.srcpack + srcsnapshot.lock — 1422 files @
7a3194b, the whole crawlable tree rather than src/ alone, because --for's universe is every
indexed file (BM25 statistics, the PageRank call graph and the fixture path tiers are all
properties of the corpus as a whole). Stored gzip-compressed behind the crawler-inert
.srcpack extension (~32 MB of text, 6.7 MB packed) and unpacked into its own temp root per
run, exactly as the doc pack is. make_snapshot.py now owns both corpora rather than gaining
a near-duplicate sibling; --corpus is REQUIRED for --freeze and deliberately has no default,
so a refresh can only move the corpus whose floors the same commit re-measures.
THE FREEZE MOVED NOTHING, measured rather than argued. At 7a3194b the frozen root reproduces
the live root exactly: lenient r@5 71.9% (23/32), lenient MRR 0.660, strict r@1/r@5
53.1/65.6, mrr_strict 0.598, pollution@5 0.0%, adversarial-class pollution 0.0%, and all
four CLASS rows (name 100.0 / concept 66.7 / task 42.9 / adversarial 80.0). Stronger than
the aggregates: all 32 PER-QUERY rank vectors are byte-identical between the live and frozen
roots. No tolerance band is claimed because none is needed — what the pack drops relative to
the live root (third_party/, docs/captures/) the crawl already prunes by name at BOTH roots.
FLOORS AND LABELS ARE UNCHANGED. r@5 70, MRR 0.55, pollution 5%, adversarial 8% are the same
values as before, now measured against a fixed corpus where they finally mean what they say.
labels_ranking.tsv is untouched: option 2 of the merge window's §7 (recalibrate the label)
was NOT taken. The available fourth floor-lowering is refused — it would spend the ratchet
on an artifact and buy one round.
AND IT IS NOT TUNED TO ADMIT ANYTHING. The corpus is pinned at 7a3194b, which PREDATES the
subtoken change: the frozen rw::subtokens is its PRE-consolidation body, the one whose
lexical evidence the query was labelled against. That is what makes the lane an instrument
for a pending merge rather than a verdict on it, and it cuts both ways — a genuine ranker
regression now has nowhere to hide behind corpus drift.
Also here, all consequences of the above:
* test/nulbytecheck.sh gains a .srcpack allowlist row with its reason. It failed once
naming the file, which is that gate's designed way of admitting a new binary type; the
pack is binary because it is COMPRESSED, and every byte inside it is a copy of a tracked
text file the sweep already scans at its own path.
* recallevalcheck check #0 now verifies BOTH locks and check #2 asserts each lane
announced its OWN pinned commit, so neither lane can silently fall back to the live root.
* snapshot.lock's comment header is regenerated to name the new recipe. Comment only:
a re-freeze at the same pin leaves snapshot.mdpack byte-identical and source_commit /
files / corpus_sha256 unchanged, which is also the proof that the refactored selection
rule is equivalent to the predicate it replaced.
Gates: test/recallevalcheck.sh ALL PASS (ranking 71.9% on the frozen corpus), nulbytecheck
ALL PASS, ripwirepubliccheck ALL PASS, full pargates -j 6 rc=0, --quality-delta regressions=0
acked=0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Aug 20, 2026
…ment Two items from the stack-graphs recon lane (#1 RefRole::Type use-sites, #2 namespaceCompatible candidate filtering), result-free: baseline edges=13755 ambiguous=5466 re-derived on this lane's own tree, the six success criteria, the failure criteria that revert each item, and the pre-fix reference binary the gates are recorded red against. Item #3 of that report (depth-2 receiver-chain resolution) is out of scope for this round and is not registered here.
joyful-ii-V-I
added a commit
that referenced
this pull request
Aug 20, 2026
…e predicate #1 RefRole::Type. usesVisitNode accepted exactly one node kind — bare `identifier` — so a `type_identifier` (a type named in a signature, a declaration or a template argument) was captured by nothing at all. It now emits a reference with the new RefRole::Type, which joins Read/Write/ Import/Extends on the NEVER-in-the-CSR list: buildGraph admits Call and Macro only, and the value-uses pass is RICH-family only, so the default ranked map is byte-identical (verified against the pre-fix binary on the same tree). The accept set is deliberately NARROWER than isBaseTypeNode, which the recon report proposed reusing verbatim: that table carries `identifier` (already the value path — reusing it would re-label every read as a type) and four CONTAINER kinds whose own name segment is a type_identifier child (reusing them would emit two rows for one mention, and C#'s qualified_name is a dotted VALUE node). Two guards keep the rows honest: a type's own DEFINITION is not a use of itself, and a base clause keeps role=extends rather than double-counting as a type mention. Measured on this tree, --uses counts 0 -> nonzero: IngestResult 0->416 ScipOverlay 0->7 AffResult 0->5 VarSpan 1->4 ShadowEvidence 0->2 Narrower 0->1 Narrower is 1 rather than 14 because its other mentions are the SCOPE segment of Narrower::appendUint, which the pre-existing qualified-segment rule excludes for values too; the legend now states that limit. Extraction changed => kParserVer 66->67 and quality.h's kIngestParserVerMirror 66->67 in this commit. #2 namespaceCompatible( RefRole, SymKind ) beside langCompatible: the one statement of which definition KINDS a role may bind to. Call stays UN-narrowed permanently (C++ Foo(x) is legitimately a constructor). Applied as a stable order-preserving filter over the assembled candidate set in buildGraph, where it is a PROVABLE no-op today (that loop admits only Call and Macro, and a Macro ref's name is uniquely a macro by construction) — it is there as the seam a future round would edit, gated so the edit cannot be silent. The implementors builder's isClassLike now routes through it instead of restating it. Where it actually bites is contextratio's all-roles resolution: that is what keeps a new type mention from spraying across a same-named free function. Legend, --uses help text and the Config comment restate role=type, including the two disclosed limits (C/C++/ObjC only; plain leaf spellings only).
joyful-ii-V-I
added a commit
that referenced
this pull request
Aug 20, 2026
… its own criterion #1 (RefRole::Type) meets all three registered criteria. --uses counts on this tree go 0 -> 416 (IngestResult), 2 -> 308 (Symbol), 0 -> 37 (RawRef) and so on; the contract half lands stronger than registered — not merely edges=/ambiguous= unchanged, but the whole default map BYTE-IDENTICAL between the pre-fix binary and this one on the same tree, for two independent reasons (buildGraph admits Call|Macro only, and the value-uses pass is RICH-family only). #2 (namespaceCompatible) FAILS criterion #2a: ambiguous= is unchanged 5468 -> 5468. Recorded as a proof rather than a weak result — the resolve loop admits only Call (un-narrowed by the hard constraint) and Macro (uniquely a macro by construction), so a namespace filter there can never remove a candidate. The registered failure criterion says that reverts the item, and it does: the predicate stays as the single statement of the rule and as the executable form of the Call-un-narrowed constraint, and is NOT published as an ambiguity reduction. Three recon-report claims corrected on the record: Extends was already kind-filtered; the 'real and unconditional' amb= effect does not exist; and of the six verbs said to gain type-only dependents, only --uses and the MCP uses/find_referencing_symbols actually can — the rest are LEAN and CSR-driven. Cost measured: RICH cache blob +2.5% (15.48 -> 15.86 MB), LEAN unchanged. Scope disclosed: contextratio deliberately skips role=type so the §9 calibrated thresholds are not moved as a side effect. Battery at bf67225: gates=437 pass=434 skip=2 fail=1, the one failure being ripwirepubliccheck arm 3, verified red at the clean baseline.
joyful-ii-V-I
added a commit
that referenced
this pull request
Aug 31, 2026
…urface, one emitter, two dialects The design's gap #2: --slice (and its rung-2 flow, and now its line seed) was CLI-only, so an MCP-only agent had no path to the variable story. The new `slice` read verb (tools/list 30->31) mirrors the CLI contract refusal-for-refusal: - fields: symbol (SYM | SYM:VAR | file:name[:VAR] | @file:LINE[:VAR]), var, flow=back|fwd|both, depth (1..32 through the shared banded reader, static_assert-tied to slice.h's named band). symbol+var and the SYM:VAR spelling answer byte-identically. - mcpverbs.h sliceText owns resolution + every refusal in the MCP dialect (not-found with the at-diagnosis clause, ambiguity listing the spellings that pick one, served-language gate, unknown-var locals listing, flow misuse) and calls the SAME sliceBundleText the CLI does — the payload is byte-identical to --slice on the same root/spec, seed disclosure (seed=/var_from=/seed_vars=) included. - single-root by table (kMcpSingleRootVerbs: a per-definition on-disk re-parse); NOT batch-served (kBatchExcludedCount 16->17, the stanza prose names it — the sweep is a follow-up, disclosed rather than implied). - the catalog description carries the honesty clauses verbatim: name-based, intra-procedural, line-granular, the served-language list, the seed grammar. Gate: test/mcpslicecheck.sh (advertise/validate/serve/refusals/ determinism/single-root/CLI-byte-parity), red-first — 22 FAILs on both the pre-lane binary and the commit-1 binary — listed in test/regression.sh same commit (gate-count pins 494->495). Count pins moved with the catalog: mcpverbscheck/mcpcontractcheck/mcptoolprunecheck (27->28, 30->31), ARCHITECTURE, the ripwire-mcp skill (31 = 16 read + 12 flagship-reflex + 3 edit) and its reference's stale 26. EVALS addendum extended with the MCP half. ASan+LSan clean on both slice gates; quality-delta clean with the dispatch-table growth acked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 5, 2026
…t-check; the --index-out post-command reports sizes, not an ls -l owner column showcasecapturecheck (D) named four uncovered flags at the wave-2 head: --allow-dirty (L2 added its case), --grep-after/--grep-before (uncovered since the 2026-09-04 capture — no generator case existed, so the close regen alone would not have cleared them) and --no-post-check (L8's flag, no case). Two cases added: the asymmetric -B/-A context spelling on a three-hit grep window, and the edit receipt's opt-out beside the insert-after case whose caption now names the folded verification. The --index-out case's post-command (ls -la | grep ci_index) was two of ripwirepubliccheck arm 5's three owner-column lines and would have RE-LEAKED at the close regen (lane-L2.md found-not-fixed #2); it now prints the two artifacts' byte sizes. No capture regenerated here — the orchestrator regenerates docs/captures + docs/COMMANDS.md at the close. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 5, 2026
…oom, --tree, --external-surface; gate defaultceilingcheck
Capture-audit 2026-09-04, lane L7, plan item P4 (§5a decision 4 accepted: the first rows stay, the tail is
windowed and disclosed). Lens 8 measured the defaults on this tree: --pr-context=main~1 659,824 B (~165K tokens,
217 files), --zoom 433,867 B, --tree 187,209 B, --around=SYM 61,891 B, --external-surface 67,862 B led by sh
builtins. Usage data (§5a): --pr-context=REF 0 uses, --around 9 in 20 days.
--pr-context budgeted BY DEFAULT: kPrDefaultBudgetTokens = 8000 (budget_tokens= budget_default="1" on the
root; --token-budget/--max-tokens set it explicitly). The trim ladder runs first (detail, never
files); when even the structural floor of every changed file exceeds it, the FILES are windowed in
blast-radius order (files_shown= + the quintet, next="--pr-context[=REF] --offset=N"); --offset/
--limit page them (pr-context joins the paging set). PrBudget replaces the size_t budget param
(same arity). The shaping guard lets this one verb both page and shape by budget; --top-k on it
is its kShapingVerbs notice, not the paging refusal.
--around default depth 1 (root depth= discloses; --around-depth=2 restores): 63,471 → 6,297 B here
--zoom levels_shown="2" of levels= (a module AT the cut carries children=; --zoom-levels=N, 0 = all)
over the 40 largest top modules (shown=/capped=/total=/next_offset=, next="--zoom --offset=40"):
444,200 → 8,063 B here
--tree the 80 best files (kTreeRowCap; --limit raises): 189,209 → 11,818 B; the plan said 400 — 400 rows
are 54,160 B on this repo, which the same plan's ≤12,000 B gate forbids; 80 ≈ 11.5 KB
--external-surface 100 rows (kExternalSurfaceRowCap) and the sh BUILTINS dropped and counted
(builtins_excluded=; externalnames.h kShellBuiltinNames — POSIX + bash builtins, never grep/sed/git,
which ARE the surface); --include-builtins keeps them: 68,437 → 5,569 B
Every legend defines its new attributes where the reader meets them; every cut carries next=.
Two new flags (--zoom-levels=, --include-builtins; kTotalFlagArms 207 → 209), each refusing alone naming its verb.
Gate test/defaultceilingcheck.sh (regression.sh), RED first on the wave-2 binary (30 rows): each ≤ 12,000 B at
defaults on this repo (pr-context: est_tokens ≤ 8000 on a 120-file fixture, the file window firing and --offset=N
continuing it), every cut disclosed with capped="1" + a next= that runs, the restoring flags restoring, the first 30
rows of --tree/--external-surface byte-identical to the uncapped listing's, every --around row a depth-2 row.
Re-pins, each to the new contract: prbudgetcheck #1 (default budget disclosed) / #2 (large budget: no
budget_default=) / #3 (a file cut is disclosed, never silent); treecheck (default = min(80,total) rows, "drops
nothing" on --limit=1000000); usescheck 5b (the per-language split asserted with --include-builtins; the default's
drop asserted beside it); w3fixlegendcheck §7/§11 (the default external-surface/tree roots are disclosed windows);
seedboundscheck (depth 1); mcpcontractcheck TWIN (--pr-context: no twin); the --max-tokens/--token-budget guard
sentences (pr-context named, budgetpolicycheck's parser shape kept); README flag/gate counts (179 / 537).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 5, 2026
…swept, all three dialects + the MCP twin (M3)
The lane's brief carried this as an open CLI defect ("--for's compact bundle overshoots an explicit
--token-budget with no over_ceiling= whenever the ladder admits one more row on its CHARGED arithmetic and
the exempt root width rides over"; fornotesbudgetcheck's 1600 rung has a hard upper edge at ~1610). Measured
first, per rule 10 — a plan is claims, the binary is the truth: on the lane's base binary it does NOT
reproduce. `02b4ff57` (verify-wave2 F2, "over_ceiling= is a property of the document, not of a rung") landed
the rule four hours before this round opened, and the sweep below finds zero silent overshoots in 333 runs.
So this commit ships the GATE the finding was owed and no feature code: the property is family-wide, the
rung a single sample lands on is exactly what let the defect survive four re-anchors of the middle rung, and
nothing in the battery swept it.
THE DECISION, with the arithmetic (METHODOLOGY §9 #2 and #4; the brief's two options).
The overshoot is not a bug in the ladder's fit check — it is the unit spread between the ceiling the ladder
fits BYTES against and the rate est_tokens prices them at, and the spread is the head-row rule made explicit:
shaping budget = N x kMinBytesPerToken(2.36) x kBudgetHeadroom(0.90) = 2.124 N bytes
conformance bar = N x kMinBytesPerToken(2.36) x kCeilingFirstEntryTolerance(1.15) = 2.714 N bytes
printed price = bytes / kBytesPerTokenDefault(2.50)
A document sitting exactly on the conformance bar therefore prices at 2.714/2.50 = 1.086 N tokens: a
STRUCTURAL 8.6% labelled overshoot, before the exempt root width (confidence=/margin_pct=, at=, the adaptive
note, the auto/compact legend, the tail legend) rides on top of it. Measured maximum across the swept band:
11.1% (budget 900, est_tokens 1000) — 8.6 pp of tolerance plus ~2.5 pp of exempt width.
kCeilingFirstEntryTolerance IS the head row: a signature is not divisible, so the first entry is emitted
whole and a bundle whose head is large lands over the bare ceiling with nothing left to trim. Option 1 of
the brief — charge the exempt width (and the tolerance) against the budget — would pay for a disclosure by
trimming <sigs> below what the DEFAULT regime selects, the inversion forbudgetmonotoncheck was written to
forbid, i.e. it would bound the HEAD to fund a label. §9 #2 says a ceiling bounds the tail, never the head.
So option 2 stands, and is what F2 implemented: over_ceiling="1" on every rung whose est_tokens exceeds the
stated budget, in the same unit the root prints budget_tokens= and est_tokens= in, absent meaning inside it.
No rung is re-derived and no --for byte gate moves; the arithmetic above is now written down where the next
lane to touch the ladder will read it.
GATE 1 — fornotesbudgetcheck arm 6 (placed before arm 4, which deletes the notes the rungs need).
Sweeps --token-budget 900..2000 step 10 across all THREE emitting dialects (XML full legend, XML
--legend=compact, --json) and asserts, per rung: est_tokens <= budget OR over_ceiling on the root.
RED on 77004e5 (the commit before F2, built in a throwaway clone):
FAIL sweep 900..2000 step 10 x 3 dialects: 103 SILENT overshoot(s) — first: budget=900 (xml)
est_tokens=914, 14 over, NO over_ceiling
GREEN on this tree:
PASS sweep 900..2000 step 10 x 3 dialects (333 runs): every overshoot is LABELLED (167 of 333 rungs
ran over the stated ceiling, every one carrying over_ceiling)
167 of 333 is not noise around an edge case — half the band overshoots by design, which is precisely why
the label is the contract and why a swept gate, not a rung, is what holds it.
GATE 2 — mcpforparitycheck arm (5), the same rule stated for the twin. A caller who states a ceiling gets
one of exactly two honest answers: the price is SERVED (est_tokens= on the root, then <= budget_tokens or
over_ceiling="1"), or the price is DECLARED ABSENT (est_tokens named in this dialect's lens= list, the
posture this twin holds today — shaped by the server's payload byte cap, not priced against the caller's
ceiling). Silence is the third answer and the arm forbids it. Also asserts H9's budget_tokens= echo. The
disjunction is written so lane F's F5 (pricing this bundle rather than declaring it) lands THROUGH this gate
rather than needing it rewritten. Green today on both clauses' declared-absent branch.
No new test file, so test/regression.sh is unchanged (manifestcheck).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 5, 2026
…absent (F5) The MCP twin carried `lens="churn,amp,tested,est_tokens"` and a legend sentence saying "this bundle is capped by the server, not priced". Declaring an absence is honest about a hole; it is not an answer — an agent handed no number cannot budget the call it just paid for, and the CLI twin has priced its own root since V1's N1 (capture-audit merge-wave2 "found, not fixed" #2, which also recorded that pricing it here is feasible because the bundle is fully rendered in memory before it is returned). It now serves the number, through serialize.h's pricedRootAttr + spliceRootAttrs — the same ≤4-pass fixpoint and the same splicer --handoff, --pack-task and --from-trace already price through, so there is no second estimator that could disagree with the CLI's. Measured on this repo, task "rank graph teleport": MCP 8,692 B / est_tokens=3,477 = 2.49 B/tok; CLI 9,702 B / est_tokens=3,881 = 2.49 B/tok — the same estimator over each dialect's own (deliberately different) bytes. Charged AFTER the sigs budget, deliberately: a disclosure's contract is disclosure only, and charging its ~18 bytes against the ranked head would drop a served row to pay for the attribute that describes the head — the exemption mcpConfidenceExemptBytes already makes a few lines above, for the same reason. The declaration goes with it: `lens="churn,amp,tested"`, and the legend clause replaces "and the CLI root's est_tokens= (this bundle is capped by the server, not priced)" with "est_tokens= prices this bundle in tokens". An attribute named as NOT MEASURED while the root carries it is worse than either state alone, because a reader believes the declaration. Net effect on the header: 38 B SMALLER than before. Gate: test/estchargecheck.sh arm #16 — this gate's own byte-rate band (2.00–3.20 B/tok for markup), applied to the MCP bundle and to the CLI bundle for the same task, plus the two declaration arms. It probes for a `legend` argument and passes `legend:"full"` only if the server accepts one, so it neither depends on today's default dialect nor breaks when lane M lands the field. band15 gained an optional arm-tag parameter rather than being duplicated. RED pre-fix: "<ctx> root carries no est_tokens=", "still declares est_tokens absent (lens= "churn,amp,tested,est_tokens") while the root serves it", "legend still says the bundle is 'not priced'". GREEN: ALL PASS. Verified: estchargecheck mcpforparitycheck mcpattrparitycheck mcpmanifestcheck mcpclidiffcheck mcpcontractcheck mcpverbscheck mcpframehonestycheck mcpflagshipcheck batchcheck mcptranchecheck mcptoolprunecheck legendcoveragecheck compactlegendcheck jsonparitycheck w3fixlegendcheck w3fixbudgetcheck fornotesbudgetcheck droppedpositivecheck mcpreadloopcheck — all rc=0, no gate needed re-pinning. Determinism ×2 byte-identical, xmllint clean, ASan/UBSan/LSan clean on the MCP path. --edit-check: forTaskText status="unchanged". --quality-delta regressions=0 gating=0 (4 rows acked, this lane's own). NOT taken here, and named in lane-F.md: with est_tokens served, MCP `for` with an explicit budget_tokens can now disclose over_ceiling="1" the way the CLI does. That is lane M's M3 (overshoot never silent) and is left to it rather than widened into this commit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 5, 2026
…JSON dialect or on the MCP twin verify-wave1 finding R1. The XML --for root has carried the whole contract since F2 — budget_tokens="N" naming the ceiling, est_tokens="N" pricing the document, over_ceiling="1" whenever the second exceeds the first. Its two machine siblings each held half of it. --for --json carried NEITHER budget_tokens nor the same predicate. It labelled on bundle_bytes > ceilingAllowanceBytes (N x kMinBytesPerToken 2.36 x kCeilingFirstEntryTolerance 1.15 = 2.714 N) while printing est_tokens = bytes / kBytesPerTokenDefault (2.50), so every document in 2.50 N < bytes <= 2.714 N priced OVER the stated ceiling and said nothing — an 8.6%-wide silent band that any bundle saturating the ladder crosses — and with no budget_tokens key on the root a caller could not run the comparison itself either. The MCP `for` twin was silent for a different reason: lane F's F5 moved it from "est_tokens DECLARED ABSENT" to "est_tokens SERVED" in the same wave that landed lane M's gate for exactly that landing, and the label never followed the price. Four of mcpforparitycheck arm (5)'s five rungs are RED on 4b72243 — budget_tokens=900 -> est_tokens=1997, 1097 over, no over_ceiling. It could not have been otherwise: the legend sentence was a constant local to runForLens. ONE unit and one wording, three surfaces: · rw::kOverCeilingLegend hoisted to serialize.h beside pricedRootAttr, whose number it qualifies; verbs_for.h binds its local name to it, mcpverbs.h now splices the same string. · forLensJsonBudgetStanza / forLensJsonOverCeiling (free functions, the forLensNotesStanza and forLensJsonTailStanza precedent in the same file) — the second is called by BOTH the envelope fixpoint's 20-byte reservation and the emitted key, so the two can no longer be written as separately drifting comparisons, which is how both halves of §C2's own bug got in. The byte clause is kept, not replaced: bytes past the allowance always price past the budget (2.714 / 2.50 = 1.086), so it costs nothing and dropping it would narrow what the key means. · rw::priceForTaskRoot lifted whole out of forTaskText (337 LOC, ccx 51): price, and on an overshoot splice the shared legend clause, re-price, and label. DISCLOSE, NOT CHARGE (METHODOLOGY §9 #2, and lane M's §M3 arithmetic): the conformance bar is 2.714 N bytes against a 2.50 B/tok price, so a bundle that genuinely fills its byte budget prices ~8.6% past the token ceiling BEFORE the exempt root width. Trimming a ranked row to fund the label that describes the ranked rows bounds the head to pay for the tail's disclosure. No rung is re-derived and no --for byte gate moves. GATES FIRST, all three RED on 4b72243 and GREEN here: · fornotesbudgetcheck arm 6, band re-anchored 900..2000 -> 700..2000 step 10 (393 runs). The old lower edge sat ABOVE the only place this fixture's JSON dialect can be silent: its bundle saturates at est_tokens=765, so the silent rungs are 710..760 and the old band had none of them. Measured 660..920 x 3 dialects: six silent rungs, all --json. Widening DOWN rather than adding a fourth dialect, because the gap was never a missing dialect — it was assuming one band's rungs exercise every dialect's saturation point. Second assertion added beside it: every rung ECHOES its ceiling (131 runs never named it before). RED 700..2000 x 3: 6 SILENT overshoot(s) — first budget=710 (json) est=765, 55 over 700..2000 x 3: 131 run(s) never named the stated ceiling — first budget=700 (json) GREEN 393 runs, 211 labelled overshoots, every rung echoes its ceiling · jsonparitycheck, new R1 arm on src/ at budgets 880/900/1500 — budget_tokens on BOTH roots, est_tokens served in both, and est > N implies the label in each dialect's own spelling. The XML side is asserted too: a parity that reads one side cannot tell a fix from a regressed reference. 880 is carried BECAUSE this corpus sits in the silent band there today; it can go vacuous as src/ grows (never falsely red) and the generated-fixture sweep is what still holds. RED 3x "budget_tokens=N rides the XML root and is ABSENT from the JSON root" + "budget=880 json — est_tokens=886 exceeds the stated ceiling with NO over_ceiling" GREEN all 15 · mcpforparitycheck arm (5), lane M's, unchanged — its disjunction was written so F5's landing would go THROUGH it. RED 4/5 rungs on the base, GREEN 5/5 here. Determinism x2 + xmllint on --for --token-budget=1600 (full and --legend=compact), determinism x2 + json.tool on --json. ASan/UBSan clean on the --json budget path at 880/900/1600, on the XML twin and on the MCP verb. --edit-check: emitForLensJson / forTaskText / runForLens all `unchanged`, 0 incompatible callers; the three new symbols are `new-symbol`. --quality-delta gating=0 regressions=0 (9 rows acked, all mine, with the reason in the ledger). Re-run green: forrankordercheck, budgetpolicycheck, bodydialectcheck, bundleidcheck, mcpattrparitycheck, legendcoveragecheck, attrvocabcheck, jsoncheck, estchargecheck, fordisclosurecheck, w3fixbudgetcheck, mcpmanifestcheck, versioncheck. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
…— disclose it `ripwire . --for=Q --detail=30 --max-tokens=300` printed `max_tokens="300"` on the <ctx> root and delivered `est_tokens="2640"` — 8.8x the ceiling it named — with no over_ceiling= anywhere. Reported by YogevKr as #61, reproduced verbatim on 2026-09-09, the third of three honesty-class issues from that reporter. The flag was never inert: it shapes the body count, and `<bodies capped="1">` disclosed THAT cut honestly right beside the silence. What --max-tokens does not do is bound the document — verbs_for.h turns it into detailBodyBudget, a budget over the BODIES alone, so the header, signatures, legend and symbol table are never charged against it. METHODOLOGY §9 #6 states the defect in one sentence: "a ceiling attribute names the ceiling actually applied." DISCLOSURE, NOT ENFORCEMENT — and the argument, because this is a §9 decision. §9 #2: "when a ceiling would cut something above the cliff, compress first, move prose into attributes second, and if it still does not fit, exceed the ceiling with over_ceiling="1" rather than drop the row that would have terminated the search." Thirty small functions totalling ~3.4K tokens, complete, ARE the terminating answer. A rung that trimmed them to fit 300 tokens would make the tool worse and would still have been perfectly honest, so no rung was added. The ladder was applied first and came up empty on this shape: the reporter measured --legend=compact at 8,761 B / est 2,974, still far past 300. What was missing is the VERDICT the default map has computed since §F5 (main.cpp, maxTokensFit.isOverCeiling) — the "as the default map does" the reporter's own Expected behavior cites. THE OPEN QUESTION, decided explicitly: --max-tokens is NOT converged onto the whole bundle. The tool already has a flag that means "bound the document" (--token-budget, the reporter's verified workaround: est 902 at --token-budget=1000) and one that means "shape the map" (--max-tokens). Convergence is allowed under one-step-smart-defaults and new-tool-no-compat-debt, but it is a DEFAULT change whose effect is to CUT rows, which is the direction §9 #1 says the data does not support — the budget flag "trims a ranking from the tail and cannot know which row would have ended the search". The two meanings stay, and both are now documented on --max-tokens and --detail in --help. WHAT CHANGED - src/verbs_for.h forLensOverCeiling: the XML dialect's over-ceiling predicate, a free function beside its JSON twin (forLensJsonOverCeiling) for the same reason that one is — runForLens is one of the largest bodies in the file and this is a contract of its own. Same rule, same unit, same attribute budget_tokens already answers to (packtask.h F2): over_ceiling="1" whenever est_tokens exceeds a ceiling the root states. Reused, not re-derived. The label is decided INSIDE the existing est_tokens fixpoint, so its own 17 bytes and its legend clause are charged — a disclosure that made est_tokens wrong is the one place that error matters most. - src/serialize.h: the legend sentence for a max_tokens-keyed verdict, beside the budget_tokens one, plus overCeilingLegendFor so no surface picks the wording by hand. Keyed on which ceilings the ROOT CARRIES, not on which one fired, so the choice cannot be made stale by the fixpoint it rides inside. A budget-only document is byte-identical to before. - src/cli.h / docs/COMMANDS.md (regenerated): --max-tokens and --detail=N now state what the flag bounds, what it does not, and which flag bounds the document. MEASURED, this repo's fixture of 30 tiny TS functions: --max-tokens=300 goes 9,647 -> 9,714 B (the 67-byte disclosure) and now reads `max_tokens="300" est_tokens="3589" over_ceiling="1"`; --max-tokens=8000 fits and is byte-identical at 14,968 B with no attribute. GATE FIRST (non-negotiable #1). test/formaxtokenscheck.sh, written before the code and red on the pre-fix binary at 6 of 9 band points plus the named reproduction. Arms: (A) the biconditional est_tokens > ceiling <=> over_ceiling="1", swept across a band that provably contains BOTH states, with non-vacuity asserted on each half; (B) no --max-tokens => no ceiling attribute and no verdict; (C) est_tokens re-derived exactly from the delivered bytes at both rates, at every point, which is what catches an emitted-but-uncharged disclosure; (D) the legend defines the attribute against the ceiling actually on that root; (E) --token-budget alone and beside --max-tokens; (F) determinism; (G) five mutation controls, each re-running the SAME judge over a deliberately corrupted real document. Presence tests read the root element through an XML parser, never a text grep — the legend DEFINES over_ceiling= (verbs_for.h:719). test/shapingflagcheck.sh (A) re-pinned 20 -> 21 --max-tokens read sites: the new site is a DISCLOSURE of a budget --for --detail=N already honored, so kShapingVerbs' honorsMaxTokens column is unchanged. Gate count 567 -> 568 in all 8 published sites. test/printf_parity.manifest: the `help` hash re-pinned, 3f0237b0... -> 9b6add07..., because this commit edits --help on purpose. printffmtparitycheck is a byte-parity fence over 12 labels and `help` is one of them; its FAIL text ("any file whose conversion moved these bytes must be reverted") is written for the printf -> std::print conversion case, where the whole claim is that the bytes must NOT move. There is no conversion here — the two new --help blocks are the change, so the bytes moved deliberately and the pin is what needs updating, not the prose. Reviewed rather than rubber-stamped: the manifest diff is EXACTLY ONE LINE, `help` STDOUT; the other 11 labels are byte-identical and help's own STDERR hash is unchanged (still e3b0c442..., the empty-string digest). The baseline it moves FROM is main's, re-read on this rebase tip rather than carried from the branch's pre-rebase value — a hash computed against an older main would pin bytes no binary in this history produces. Pre-flighted against the WIDENED fence lane/stdprint-conversion brings (40 labels, purely additive — it rewrites none of the 12): 39 pass, 1 fail, and the one is `help`. So this change moves exactly one label out of forty, and the one-line invariant survives that landing whichever order the two lanes take. --test-gate does not name printffmtparitycheck for a help edit: it routes by call edges, and script-to-binary is not one — it discloses that as script_gates_unmodelled= rather than implying the list is complete. If you touch printUsage, run that gate by hand. Local, on this rebase tip after a --clean-first rebuild (five landings under this branch, and an incremental build across a branch switch can produce a binary that exists at no single commit — CLAUDE.md documents that failure at length): formaxtokenscheck, printffmtparity, manifest, shapingflag, docscommands, deck, deckclaim, readmedrift, readmeexample, fordisclosure, w3fixbudget and legendcoverage all green; determinism + golden + xmllint clean; --quality-delta exit 0 (its one gating row, short-horizon-churn churn=self on runForLens, acked with its reason). Every pinned number here was RE-DERIVED on this tip — the gate count from regression.sh's own loop, the read-site count from the gate's own grep expression, the parity hash from the rebuilt binary — none carried forward from the pre-rebase branch. docs/COMMANDS.md is regenerated from that same binary. Closes #61 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
…— disclose it `ripwire . --for=Q --detail=30 --max-tokens=300` printed `max_tokens="300"` on the <ctx> root and delivered `est_tokens="2640"` — 8.8x the ceiling it named — with no over_ceiling= anywhere. Reported by YogevKr as #61, reproduced verbatim on 2026-09-09, the third of three honesty-class issues from that reporter. The flag was never inert: it shapes the body count, and `<bodies capped="1">` disclosed THAT cut honestly right beside the silence. What --max-tokens does not do is bound the document — verbs_for.h turns it into detailBodyBudget, a budget over the BODIES alone, so the header, signatures, legend and symbol table are never charged against it. METHODOLOGY §9 #6 states the defect in one sentence: "a ceiling attribute names the ceiling actually applied." DISCLOSURE, NOT ENFORCEMENT — and the argument, because this is a §9 decision. §9 #2: "when a ceiling would cut something above the cliff, compress first, move prose into attributes second, and if it still does not fit, exceed the ceiling with over_ceiling="1" rather than drop the row that would have terminated the search." Thirty small functions totalling ~3.4K tokens, complete, ARE the terminating answer. A rung that trimmed them to fit 300 tokens would make the tool worse and would still have been perfectly honest, so no rung was added. The ladder was applied first and came up empty on this shape: the reporter measured --legend=compact at 8,761 B / est 2,974, still far past 300. What was missing is the VERDICT the default map has computed since §F5 (main.cpp, maxTokensFit.isOverCeiling) — the "as the default map does" the reporter's own Expected behavior cites. THE OPEN QUESTION, decided explicitly: --max-tokens is NOT converged onto the whole bundle. The tool already has a flag that means "bound the document" (--token-budget, the reporter's verified workaround: est 902 at --token-budget=1000) and one that means "shape the map" (--max-tokens). Convergence is allowed under one-step-smart-defaults and new-tool-no-compat-debt, but it is a DEFAULT change whose effect is to CUT rows, which is the direction §9 #1 says the data does not support — the budget flag "trims a ranking from the tail and cannot know which row would have ended the search". The two meanings stay, and both are now documented on --max-tokens and --detail in --help. WHAT CHANGED - src/verbs_for.h forLensOverCeiling: the XML dialect's over-ceiling predicate, a free function beside its JSON twin (forLensJsonOverCeiling) for the same reason that one is — runForLens is one of the largest bodies in the file and this is a contract of its own. Same rule, same unit, same attribute budget_tokens already answers to (packtask.h F2): over_ceiling="1" whenever est_tokens exceeds a ceiling the root states. Reused, not re-derived. The label is decided INSIDE the existing est_tokens fixpoint, so its own 17 bytes and its legend clause are charged — a disclosure that made est_tokens wrong is the one place that error matters most. - src/serialize.h: the legend sentence for a max_tokens-keyed verdict, beside the budget_tokens one, plus overCeilingLegendFor so no surface picks the wording by hand. Keyed on which ceilings the ROOT CARRIES, not on which one fired, so the choice cannot be made stale by the fixpoint it rides inside. A budget-only document is byte-identical to before. - src/cli.h / docs/COMMANDS.md (regenerated): --max-tokens and --detail=N now state what the flag bounds, what it does not, and which flag bounds the document. MEASURED, this repo's fixture of 30 tiny TS functions: --max-tokens=300 goes 9,647 -> 9,714 B (the 67-byte disclosure) and now reads `max_tokens="300" est_tokens="3589" over_ceiling="1"`; --max-tokens=8000 fits and is byte-identical at 14,968 B with no attribute. GATE FIRST (non-negotiable #1). test/formaxtokenscheck.sh, written before the code and red on the pre-fix binary at 6 of 9 band points plus the named reproduction. Arms: (A) the biconditional est_tokens > ceiling <=> over_ceiling="1", swept across a band that provably contains BOTH states, with non-vacuity asserted on each half; (B) no --max-tokens => no ceiling attribute and no verdict; (C) est_tokens re-derived exactly from the delivered bytes at both rates, at every point, which is what catches an emitted-but-uncharged disclosure; (D) the legend defines the attribute against the ceiling actually on that root; (E) --token-budget alone and beside --max-tokens; (F) determinism; (G) five mutation controls, each re-running the SAME judge over a deliberately corrupted real document. Presence tests read the root element through an XML parser, never a text grep — the legend DEFINES over_ceiling= (verbs_for.h:719). test/shapingflagcheck.sh (A) re-pinned 20 -> 21 --max-tokens read sites: the new site is a DISCLOSURE of a budget --for --detail=N already honored, so kShapingVerbs' honorsMaxTokens column is unchanged. Gate count 568 -> 569 in all 8 published sites, DERIVED from regression.sh's own loop on this tip. Worth recording how that number was nearly wrong: the previous revision of this branch published 568, and the lane that landed underneath it (#85, tgrep) had itself bumped 567 -> 568. So README.md, docs/EVALS.md and the deck did NOT conflict on rebase — the two lanes had written IDENTICAL text — and git auto-merged them to a tree publishing 568 while this branch's own loop names 569. That is the silent-merge failure this repo keeps re-learning, and the only thing that catches it is re-deriving from the loop rather than trusting a clean merge. A gate-count bump that merges CLEAN onto a main which has landed a gate since you branched is the failure, not the success; regression.sh conflicted loudly and the three prose sites did not. test/printf_parity.manifest: the `help` hash re-pinned, 72b76cbc... -> d9b77634..., because this commit edits --help on purpose. printffmtparitycheck is a byte-parity fence over 12 labels and `help` is one of them; its FAIL text ("any file whose conversion moved these bytes must be reverted") is written for the printf -> std::print conversion case, where the whole claim is that the bytes must NOT move. There is no conversion here — the two new --help blocks are the change, so the bytes moved deliberately and the pin is what needs updating, not the prose. Reviewed rather than rubber-stamped: the manifest diff is EXACTLY ONE LINE, `help` STDOUT; the other 11 labels are byte-identical and help's own STDERR hash is unchanged (still e3b0c442..., the empty-string digest). The baseline it moves FROM is main's, re-read on this rebase tip rather than carried from the branch's pre-rebase value — a hash computed against an older main would pin bytes no binary in this history produces. Pre-flighted against the WIDENED fence lane/stdprint-conversion brings (40 labels, purely additive — it rewrites none of the 12): 39 pass, 1 fail, and the one is `help`. So this change moves exactly one label out of forty, and the one-line invariant survives that landing whichever order the two lanes take. --test-gate does not name printffmtparitycheck for a help edit: it routes by call edges, and script-to-binary is not one — it discloses that as script_gates_unmodelled= rather than implying the list is complete. If you touch printUsage, run that gate by hand. Local, on this rebase tip after a --clean-first rebuild (five landings under this branch, and an incremental build across a branch switch can produce a binary that exists at no single commit — CLAUDE.md documents that failure at length): formaxtokenscheck, printffmtparity, manifest, shapingflag, docscommands, deck, deckclaim, readmedrift, readmeexample, fordisclosure, w3fixbudget and legendcoverage all green; determinism + golden + xmllint clean; --quality-delta exit 0 (its one gating row, short-horizon-churn churn=self on runForLens, acked with its reason). Every pinned number here was RE-DERIVED on this tip — the gate count from regression.sh's own loop, the read-site count from the gate's own grep expression, the parity hash from the rebuilt binary — none carried forward from the pre-rebase branch. docs/COMMANDS.md is regenerated from that same binary. Closes #61 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
…— disclose it `ripwire . --for=Q --detail=30 --max-tokens=300` printed `max_tokens="300"` on the <ctx> root and delivered `est_tokens="2640"` — 8.8x the ceiling it named — with no over_ceiling= anywhere. Reported by YogevKr as #61, reproduced verbatim on 2026-09-09, the third of three honesty-class issues from that reporter. The flag was never inert: it shapes the body count, and `<bodies capped="1">` disclosed THAT cut honestly right beside the silence. What --max-tokens does not do is bound the document — verbs_for.h turns it into detailBodyBudget, a budget over the BODIES alone, so the header, signatures, legend and symbol table are never charged against it. METHODOLOGY §9 #6 states the defect in one sentence: "a ceiling attribute names the ceiling actually applied." DISCLOSURE, NOT ENFORCEMENT — and the argument, because this is a §9 decision. §9 #2: "when a ceiling would cut something above the cliff, compress first, move prose into attributes second, and if it still does not fit, exceed the ceiling with over_ceiling="1" rather than drop the row that would have terminated the search." Thirty small functions totalling ~3.4K tokens, complete, ARE the terminating answer. A rung that trimmed them to fit 300 tokens would make the tool worse and would still have been perfectly honest, so no rung was added. The ladder was applied first and came up empty on this shape: the reporter measured --legend=compact at 8,761 B / est 2,974, still far past 300. What was missing is the VERDICT the default map has computed since §F5 (main.cpp, maxTokensFit.isOverCeiling) — the "as the default map does" the reporter's own Expected behavior cites. THE OPEN QUESTION, decided explicitly: --max-tokens is NOT converged onto the whole bundle. The tool already has a flag that means "bound the document" (--token-budget, the reporter's verified workaround: est 902 at --token-budget=1000) and one that means "shape the map" (--max-tokens). Convergence is allowed under one-step-smart-defaults and new-tool-no-compat-debt, but it is a DEFAULT change whose effect is to CUT rows, which is the direction §9 #1 says the data does not support — the budget flag "trims a ranking from the tail and cannot know which row would have ended the search". The two meanings stay, and both are now documented on --max-tokens and --detail in --help. WHAT CHANGED - src/verbs_for.h forLensOverCeiling: the XML dialect's over-ceiling predicate, a free function beside its JSON twin (forLensJsonOverCeiling) for the same reason that one is — runForLens is one of the largest bodies in the file and this is a contract of its own. Same rule, same unit, same attribute budget_tokens already answers to (packtask.h F2): over_ceiling="1" whenever est_tokens exceeds a ceiling the root states. Reused, not re-derived. The label is decided INSIDE the existing est_tokens fixpoint, so its own 17 bytes and its legend clause are charged — a disclosure that made est_tokens wrong is the one place that error matters most. - src/serialize.h: the legend sentence for a max_tokens-keyed verdict, beside the budget_tokens one, plus overCeilingLegendFor so no surface picks the wording by hand. Keyed on which ceilings the ROOT CARRIES, not on which one fired, so the choice cannot be made stale by the fixpoint it rides inside. A budget-only document is byte-identical to before. - src/cli.h / docs/COMMANDS.md (regenerated): --max-tokens and --detail=N now state what the flag bounds, what it does not, and which flag bounds the document. MEASURED, this repo's fixture of 30 tiny TS functions: --max-tokens=300 goes 9,647 -> 9,714 B (the 67-byte disclosure) and now reads `max_tokens="300" est_tokens="3589" over_ceiling="1"`; --max-tokens=8000 fits and is byte-identical at 14,968 B with no attribute. GATE FIRST (non-negotiable #1). test/formaxtokenscheck.sh, written before the code and red on the pre-fix binary at 6 of 9 band points plus the named reproduction. Arms: (A) the biconditional est_tokens > ceiling <=> over_ceiling="1", swept across a band that provably contains BOTH states, with non-vacuity asserted on each half; (B) no --max-tokens => no ceiling attribute and no verdict; (C) est_tokens re-derived exactly from the delivered bytes at both rates, at every point, which is what catches an emitted-but-uncharged disclosure; (D) the legend defines the attribute against the ceiling actually on that root; (E) --token-budget alone and beside --max-tokens; (F) determinism; (G) five mutation controls, each re-running the SAME judge over a deliberately corrupted real document. Presence tests read the root element through an XML parser, never a text grep — the legend DEFINES over_ceiling= (verbs_for.h:719). test/shapingflagcheck.sh (A) re-pinned 20 -> 21 --max-tokens read sites: the new site is a DISCLOSURE of a budget --for --detail=N already honored, so kShapingVerbs' honorsMaxTokens column is unchanged. Gate count 568 -> 569 in all 8 published sites, DERIVED from regression.sh's own loop on this tip. Worth recording how that number was nearly wrong: the previous revision of this branch published 568, and the lane that landed underneath it (#85, tgrep) had itself bumped 567 -> 568. So README.md, docs/EVALS.md and the deck did NOT conflict on rebase — the two lanes had written IDENTICAL text — and git auto-merged them to a tree publishing 568 while this branch's own loop names 569. That is the silent-merge failure this repo keeps re-learning, and the only thing that catches it is re-deriving from the loop rather than trusting a clean merge. A gate-count bump that merges CLEAN onto a main which has landed a gate since you branched is the failure, not the success; regression.sh conflicted loudly and the three prose sites did not. test/printf_parity.manifest: the `help` hash re-pinned, 72b76cbc... -> d9b77634..., because this commit edits --help on purpose. printffmtparitycheck is a byte-parity fence over 12 labels and `help` is one of them; its FAIL text ("any file whose conversion moved these bytes must be reverted") is written for the printf -> std::print conversion case, where the whole claim is that the bytes must NOT move. There is no conversion here — the two new --help blocks are the change, so the bytes moved deliberately and the pin is what needs updating, not the prose. Reviewed rather than rubber-stamped: the manifest diff is EXACTLY ONE LINE, `help` STDOUT; the other 11 labels are byte-identical and help's own STDERR hash is unchanged (still e3b0c442..., the empty-string digest). The baseline it moves FROM is main's, re-read on this rebase tip rather than carried from the branch's pre-rebase value — a hash computed against an older main would pin bytes no binary in this history produces. Pre-flighted against the WIDENED fence lane/stdprint-conversion brings (40 labels, purely additive — it rewrites none of the 12): 39 pass, 1 fail, and the one is `help`. So this change moves exactly one label out of forty, and the one-line invariant survives that landing whichever order the two lanes take. --test-gate does not name printffmtparitycheck for a help edit: it routes by call edges, and script-to-binary is not one — it discloses that as script_gates_unmodelled= rather than implying the list is complete. If you touch printUsage, run that gate by hand. Local, on this rebase tip after a --clean-first rebuild (five landings under this branch, and an incremental build across a branch switch can produce a binary that exists at no single commit — CLAUDE.md documents that failure at length): formaxtokenscheck, printffmtparity, manifest, shapingflag, docscommands, deck, deckclaim, readmedrift, readmeexample, fordisclosure, w3fixbudget and legendcoverage all green; determinism + golden + xmllint clean; --quality-delta exit 0 (its one gating row, short-horizon-churn churn=self on runForLens, acked with its reason). Every pinned number here was RE-DERIVED on this tip — the gate count from regression.sh's own loop, the read-site count from the gate's own grep expression, the parity hash from the rebuilt binary — none carried forward from the pre-rebase branch. docs/COMMANDS.md is regenerated from that same binary. Closes #61 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 10, 2026
…edup was already dead buildGraph/2b was the largest phase on rails after the loop-hoist round (61.9 ms, 38.7 ms of it std::sort over 3,916 closures averaging 1,020 ids). Converted to a thresholded radix: rails buildGraph/2b 65.2 -> 35.0 ms median (-46%), n=7 interleaved, reps do not overlap rails buildGraph 223.0 -> 186.7 ms (-16%) rails --callers=main -7.8% / -8.5% median over two independent n=21 A/Bs rails default map -7.6% / -7.3% median go / django / rust-analyzer / private C++ / this repo: within +-1.7%, no regression The recorded kRadixThreshold of 2048 in sortutil.h does NOT transfer: it guards 12-byte Edge records and a score gather, while this is one 4-byte item with a direct key whose 12-bit id range lets the no-op pass skip collapse it to two passes. Re-measured for this shape the crossover is 64 (narrow keys) to 128 (full 32-bit); the new radixSortIdsAscending takes 128 and its comment says why the two numbers must not be unified. Unthresholded, the same conversion regresses four of six corpora by 3-8x -- the threshold is what makes it safe. The std::unique is removed, not replaced by a VERIFY (release still evaluates a VERIFY expression, which would reintroduce the O(n) scan). It removed 0 elements in 24,216 calls across six corpora at 0.99 ms on rails, and is dead BY CONSTRUCTION: w is appended in the same branch that stamps seenEpoch[w], so a file reaches trans[s] at most once per source. The sibling ancestorsReach has always relied on that same stamp with no dedup. The SORT stays -- discovery order is deterministic but is not id order, so the sort is what makes trans a pure function of adj (non-negotiable #2). Gate first: test/includeprecisecheck.sh gains a sorted+unique postcondition sweep, a diamond fixture, a cycle fixture, and a 400-node scrambled synthetic graph checked against an independent mark-sweep oracle. That last arm is load-bearing -- with the sort deleted entirely every fixture-based arm still passed, because the fixture's closures are under ten elements and already ascending. Both mutation controls recorded in bench/PROFILE.md. Byte-identical on six corpora x three verbs (18 comparisons, 18 distinct output sizes). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
xCatG
added a commit
to xCatG/ripwire
that referenced
this pull request
Sep 11, 2026
…check.sh Adds .kt support (kParserVer 85) via fwcd/tree-sitter-kotlin, pinned past v0.3.8 for its scanner segfault fix. Zero ASan/UBSan/LSan findings across 501 real .kt files (8 local Android/JVM repos) despite that history. Extraction covers classes/objects/companion objects/functions/calls/imports, inheritance clauses (captureBases recognizes `delegation_specifier`), and complexity scoring (isDecisionType/cc_isNestingControl recognize `when_entry`/ `when_expression`/`do_while_statement`/Kotlin's own `catch_block` spelling — `lang == Lang::Kotlin` checked first so the other 21 languages short-circuit on a byte compare rather than a strcmp; `catch_block` is a different, unrelated node in the vendored Swift/Elixir grammars, verified against all 22 vendored parsers before scoping it). Canonical ids are scope-qualified (kotlinEnclosingScopeOf) and parameter counts are real (countParams counts `parameter` children specifically for Kotlin's parameter list — a naive "every named child" count over-counts, since a parameter's own modifiers and default value are its SIBLINGS in this grammar, not nested inside it; verified against a real parse before writing the fix). Import edges are captured (Kotlin joins Java's DepDialect, since a mixed module's imports genuinely cross the language boundary). Kotlin's function_body/class_body are positional children, not fields — the generic field-based body lookup silently returns null for every Kotlin definition, which is not cosmetic: graph.h's decl/def collapse reads a null body as "bodyless forward declaration" and deletes that candidate from resolution. Kotlin rides ingest_sidecap.h's existing ObjC positional-body fallback (widened, `class_body`/`enum_class_body` added) rather than a new mechanism — the same fallback ObjC has needed since ITS grammar exposes no `body:` field either. `enum class`'s body nests under `enum_class_body`, a DIFFERENT positional child than a plain class's `class_body`; verified via a standalone tree-sitter probe against the vendored grammar before fixing. Getting this right is what makes a same-name Kotlin/Java collision correctly surface as `ambiguous=` instead of silently resolving to whichever side happens to have a working body lookup. The JVM bridge (graph.h langCompatible) resolves Kotlin<->Java calls in both directions cleanly on real code (Nanidroid: unresolved 1425->545, ambiguous 1364->1496 — the rise is real collisions surfacing honestly via the fix above, not a regression). Disclosed, not fixed (a real new feature, not a bug): Kotlin navigation-expression receivers (`A.f()`) aren't recognized by ingest_binds.h's classifier, so an explicit receiver that would narrow a same-name collision between two Kotlin classes currently doesn't. Fixed the highest-severity language-dispatch finding: verbs_navigate.h's kExtSurfaceLangSlots was hand-sized off Lang::Elixir+1, so an unrecognized language's external-surface references silently clamped into slot 0 (Lang::Cpp). Every array in src/ hand-sized off a last-enumerator literal (verbs_navigate.h, verbs_report.h, lintrules.h, clones.h, htmlexport.h, tsprobe.cpp) now routes through model.h's kLangCount instead, so this class of bug can't recur on the next language append. Also deferred and disclosed rather than silently dropped: .kts (Gradle DSL needs its own trailing-lambda parse-quality probe), --slice support, and essential-complexity (ev=) scoring (Kotlin's when/elvis/!! vocabulary isn't measured against isDecisionType's containment claim yet). test/kotlincheck.sh: a real fixture (Kotlin<->Java bidirectional calls, a constructed same-name collision pair including an `enum class`/plain-class pair, cross-file calls, imports), every number pinned from an actual run, mutation arms proving each assertion non-tautological — including that a crash on the mutated fixture can't silently read as a passed assertion (checks binary exit status before trusting absent output). Verified: kotlincheck.sh ALL PASS on build/ripwire and a --clean-first asan/ripwire; zero ASan/UBSan/LSan findings across 501 real .kt files (8 local repos) plus the fixture; manifestcheck.sh and every gate whose golden value this change legitimately moved (kCatalogLangs, the doctor grammar count 22->23, README's --callers example, the vendored-scanner serialize() safety classification, the dependency-capable-set ledger) re-verified clean. Independently re-tested afterward on 5 larger real-world Kotlin corpora not part of the gated suite (android/nowinandroid, android/compose-samples, android/architecture-samples, ktorio/ktor, signalapp/Signal-Android — ~9,500 additional .kt files, up to 6,131 files/64,899 symbols in one repo): zero crashes, zero ASan/UBSan/LSan findings, deterministic, well-formed on every one. The grammar's one identified parsing gap — `@Composable () -> Unit`-shaped annotated function-type parameters — affects ~22.8% of Composable-heavy files on the largest sample measured, localized to a few bytes each with the enclosing declaration intact (the same shape as this repo's existing Metal precedent). Also validated against a hand-written adversarial corpus (41 files: merge conflict markers, mid-edit fragments, ObjC/Java/Kotlin interop, unicode identifiers/emoji/RTL, deep nesting, syntax edge cases, deliberately broken input including invalid-UTF-8 binary garbage) — zero crashes, zero ASan findings, byte-identical determinism. That adversarial pass surfaced one genuine disclosure gap, unrelated to Kotlin: measureFileHealth (ingest_crawl.h) validated the leading sample for well-formed UTF-8 nowhere in the pipeline, so a file with invalid UTF-8 but no tree-sitter ERROR/MISSING node reported no degraded-parse signal at all — `--skipped`'s disclosure contract silently had a hole. Fixed by reusing jsonesc::utf8SeqLen (no new UTF-8 validator) to scan the same sample already walked for whitespace, recording bad-sequence byte positions, and counting only the ones NOT already inside a recorded ERROR span into err/err_ratio — first-cut version double-counted (badUtf8 added unconditionally, before cross-checking ERROR spans), pushing err_ratio above its documented <=1.000 ceiling on `\xff\xff\xff@#$%^&*(`-shaped input; caught by two independent reviews (a backgrounded codex second opinion and a separately user-launched Opus /code-review), both of which also caught the missing kParserVer bump (84->85, ingest_cache.h + its quality.h mirror) this behavior change needed. parsehealthcheck.sh gained two arms (bad-UTF-8-flagged, good-UTF-8-negative- control) verifying both the fix and the absence of a false positive. Two independent correctness reviews ran against the original port before it landed (/code-review --fix on the fable model, and a separate codex second opinion), then a four-angle /simplify pass (reuse, simplification, efficiency, altitude); this follow-up round added a second codex pass plus a separately user-launched Opus /code-review, both run against the adversarial-corpus fixes — every fix from all reviews re-verified against 8+ corpora and re-gated clean before landing. Follow-up review round: a four-angle /simplify pass (reuse, simplification, efficiency, altitude) plus an independent /code-review high --fix pass, both run against this diff before it lands. /simplify eliminated measureFileHealth's errorSpans vector and its O(n*m) rescan in favor of a std::lower_bound dedup over sorted bad-UTF-8 positions, consolidated repeated lang==Kotlin tests in isDecisionType/cc_isNestingControl, and merged langCompatible's two return branches. The independent review then found two genuine, pre-existing test-coverage gaps the port had shipped without: captureBases's Kotlin delegation_specifier handling (both the wrapped-call `Shape()` and bare-interface `Labeled` shapes) and countParams for multi-parameter Kotlin functions had zero fixture coverage — no prior fixture exercised a base-class clause or more than one parameter, so neither fix had anything pinning it. Both are now covered (kotlincheck.sh §9/§10, with non-tautological mutation arms 7e/7f). The review also reverted the langCompatible merge back to an early-return shape (langCompatible runs per candidate on a hot resolution path; the merged form paid for the JVM-bridge comparisons even on the common C-family- only case) and refined measureFileHealth's dedup to never hold a transiently over-counted value mid-walk. Re-verified: kotlincheck.sh and parsehealthcheck.sh ALL PASS (including the new coverage arms and a bad-UTF-8-inside-an-ERROR-span dedup test), ASan clean, full gate suite clean (gates=572 pass=565 skip=4 fail=3, all three failures pre-existing/environmental and untouched by this diff). Rebased onto origin/main a second time (186 commits: upstream independently landed Dart as language redhat-et#21 while this branch was in flight). Kotlin now appends AFTER Dart in Lang (index 22, kLangCount anchored on Lang::Kotlin). kParserVer collided a second time: upstream independently claimed 86 (Ruby receiver), 87 (markdown scanner counter saturation) and 88 (Dart) while this branch still held 86/87 from the first rebase — an exact collision with this branch's own prior use of those numbers. Resolved the same way as the first collision: re-bumped to the next free values (89, 90) rather than keeping either side's number, full history preserved in the ingest_cache.h comment. Six gate failures surfaced after the merge, each investigated individually rather than assumed to be either all-expected or all-flaky: g1configcheck.sh and dartcheck.sh/elixircheck.sh each carried their own stale hardcoded doctor-probe/fuzz-target/seed counts (each language's own gate only bumps its own copy, a drift-prone hand-maintained pattern — fixed by updating all three plus a second stale fuzz-executable-name set in g1configcheck.sh that was masked locally by a Clang-only SKIP but would have failed CI's Clang legs). ingest_names.h's kotlinEnclosingScopeOf had three literal std::strcmp calls never converted to the kindIs() house style introduced by upstream's own per-node strcmp-to-kindIs refactor, since this file predated that refactor and wasn't part of the original conflict set — converted, fourth (runtime-variable) strcmp call correctly left unconverted. ingest_crawl.h's kLangTable auto-merged cleanly (both .dart and .kt rows present) but its array extent literal silently stayed at 47 — bumped to 48 after an exact row count. limitstablecheck.sh needed docs/LIMITS.md regenerated via its own generator script. docs/COMMANDS.md, its newest showcase capture, and every <!-- gatecount -->-marked reference were regenerated/re-derived rather than hand-edited, per upstream's newly introduced generated-artifact pattern for these documents. mcpattrparitycheck.sh and (in the final pre-push run) fillordercheck.sh each failed once under -j 6 parallel contention and passed cleanly on repeated standalone re-runs — confirmed transient, not code issues. Re-verified after all of the above: full gate suite clean (gates=602 pass=593-598 skip=4 fail=4, only the 4 pre-existing/environmental failures: w3fixlegendcheck.sh's root-spelling-dependent --dead-code path check, showcasecapturecheck.sh's pre-existing --stray-content=lane caption- generation bug, optremarkscheck.sh's Clang-only PGO arms, and nongitqmetricscheck.sh's non-git-corpus environmental check); ASan/UBSan clean on both src and test/kotlinfix under a --clean-first rebuild. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016aPNTmY4XV8bHEkuhdtZnE PR redhat-et#126 review round (CodeRabbit automated review, 2026-09-10): three findings, all fixed. 1. Bodyless Kotlin class/interface definitions were silently deleted from the graph whenever a same-named Java definition existed elsewhere. Kotlin has no forward-declaration syntax for types — `interface Taggable` with no braces at all is the type's sole, complete definition, unlike a bodyless Kotlin FUNCTION (an interface member signature) or a C header prototype. The existing ingest_sidecap.h positional-body fallback (added for §5/§8's already-bodied collision fixtures) has no body child to find for a truly bodyless declaration, so bodyByte stayed 0 and graph.h's decl/def collapse (`hasBody`, used identically in buildGraph's §1e collapse and declToDefFollowThrough) read it as a droppable forward decl. Fixed by widening both `hasBody` copies with a narrow, GATED clause: `sym.lang == Lang::Kotlin && sym.kind == SymKind::Class`. New fixture (kotlincheck.sh §11): a bodyless `interface Taggable` colliding with a same-named Java class, with a mutation arm proving the assertion is not a tautology. 2. `third_party/deps/kotlin/src/scanner.c`'s `stack_push` called `abort()` outright when nested interpolated strings drove the delimiter stack to `TREE_SITTER_SERIALIZATION_BUFFER_SIZE` — a DELIBERATE process termination, not UB, so no sanitizer flagged it and it was invisible to every ASan/fuzz sweep this session ran (the fuzz seeds never happened to nest ~512 string opens deep). Violates this repo's own contract: a bad file must degrade, never crash the process. Fixed as a proper vendor patch (third_party/patches/kotlin/001-stack-push-no-abort.patch, following the yaml/001 and markdown/001 precedent for this exact defect shape): `stack_push` now returns `bool`, and `scan_string_start` propagates a `false` through the standard tree-sitter "no external token matched" recovery path (an ERROR node) instead of aborting. No kParserVer change — this only changes behavior on previously-ABORTING input, never on anything that used to parse and cache successfully. 3. `scan_string_content`'s escaped-dollar-at-string-end handling treated the FIRST quote after an escaped `\$` as the complete closing delimiter even inside a TRIPLE-quoted string, whose delimiter is three quotes — `"""a\$"""` lost its first closing quote and the remainder mis-tokenized. Fixed as a vendor patch (kotlin/002-triple-dollar-escape.patch): defers to the existing triple-quote-aware end_char handling a few lines down (the same is_triple deferral the plain-backslash case two branches below already uses). Real parse-output change on real input, so kParserVer bumped 90 -> 91 (ingest_cache.h + its quality.h mirror). Both vendor patches gained live tripwires in vendorpatchcheck.sh (new arm J): a generated 700-deep nested-interpolation fixture (the ASan-flavour crash tripwire for redhat-et#2) and a committed tripledollar.kt fixture (the plain-build exit-0-wrong-answer tripwire for redhat-et#3, checked via degraded_parse=0 and that the symbol declared right after the tricky string still extracts). Re-verified: kotlincheck.sh and vendorpatchcheck.sh ALL PASS on both build/ripwire and a --clean-first asan/ripwire; zero ASan/UBSan/LSan findings re-confirmed across all 8 local real-world Kotlin corpora (501 files) on the fixed binary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lessons folded from 27 repositories and 27 papers into one deterministic executable, from a labeled survey of 225 tools across 26 categories — every row carries the lesson taken and the exact flag or file where it lives.
docs/LINEAGE.md: classic papers (6), modern research (21), the tool field in two honest tiers (folded 27 / surveyed 225 — surveyed ≠ borrowed-from, stated twice), plus the exclusion log: candidates dropped for lacking a traceable lesson, named rather than hidden.readmedriftcheck.sharm E (5 sub-arms): the advertised counts are derived from the tables mechanically and the suite fails if the sentence and the ledger ever disagree — red-first proven.🤖 Generated with Claude Code