merge fix/expand-estimator: guard the phantom-map estimator, harden the gate, scope the callees legend (verifier #2 #11)

Author: joyful-ii-V-I

.ripwire_quality_acks

@@ -53,9 +53,12 @@ tool refuses.
 [What it misses, and what to run next →](#what-it-misses-and-what-to-run-next)
 
 **New (2026-08-15), each measured, gated, and re-derived on every CI run:**
-- **`--expand` answers "show me this function" in one call, −57.6% tokens** — an exact-name ask now
+- **`--expand` answers "show me this function" in one call, −47.3% tokens** — an exact-name ask now
   skips the ranked-map preamble by default and the body arrives with its file's sibling symbols and
   imports inline (`sibs=`/`inc=`), so the follow-up "what else is in this file?" call never happens.
+  (Re-derived 2026-08-15 fix-expand round: `--expand=emitGrepReport` — 47,252 B classic 200-row-map
+  bundle vs 24,911 B today's exact-name default, on this repo; the figure moves as the corpus grows
+  and is not itself CI-gated, so re-measure before citing it in a future round.)
 - **`--grep` groups, deduplicates, and speaks boolean** — per-file grouping, identical-line collapse
   ("this exact guard appears at 6 call sites" is the finding), and `--and=`/`--not=`/`--grep-scope=`:
   a two-term ask returns the *complete* answer at **−78%** of the single-term dump. Default view now
@@ -801,8 +804,8 @@ $ ripwire . --callers=rankGraphTeleport
 <s t="fn" n="runEval" p="./src/eval.h:168"/>
 <s t="fn" n="rankGraph" p="./src/graph.h:2057"/>
 <s t="fn" n="anchoredLexicalRank" p="./src/graph.h:2393"/>
-<s t="fn" n="churnRankedGraph" p="./src/main.cpp:11489"/>
-<s t="fn" n="runDefaultMap" p="./src/main.cpp:11604"/>
+<s t="fn" n="churnRankedGraph" p="./src/main.cpp:11527"/>
+<s t="fn" n="runDefaultMap" p="./src/main.cpp:11642"/>
 <s t="fn" n="getIndex" p="./src/mcpindex.h:946"/>
 </callers>
 ```

README.md

@@ -127,9 +127,25 @@ inline constexpr const char* kCallHierarchyLegendOpen =
     "lists the symbols that CALL of=; the callees form lists the symbols of= itself calls. of= is the selector "
     "you passed, defs= how many DEFINITIONS that name resolved to (the rows UNION every def's neighbours), and "
     "count= the number of DISTINCT neighbour symbols (a floor, per counts_floor=), which the rows window with limit= and offset=. "
-    "When emitted by callees, bodyless_defs= (when present) counts how many of the defs= are declarations with no body (header-only or forward-declared); "
-    "zero callees may mean no body to read callees from rather than truly no dependencies. "
     "A neighbour that is an indexed function-like #define is a macro row (t=\"macro\", role=\"macro\" on the XML row): "
     "the edge crosses a macro expansion, not a plain call — rows carry no role= otherwise. ";
 
+// V1 fix (verifier finding 3, 2026-08-15): bodyless_defs= is a CALLEES-only attribute — main.cpp's emitter
+// gates it behind `!wantCallers`, so a --callers document can never carry it. It used to sit inside
+// kCallHierarchyLegendOpen above, which both forms print, so every --callers call paid ~235 B for a clause
+// it could never need. Appended only on the callees form (see the call site in main.cpp), so the clause
+// still appears verbatim wherever the attribute CAN appear — legendcoveragecheck's callees-side coverage is
+// unaffected; only the callers-side dead weight is gone.
+inline constexpr const char* kCallHierarchyLegendCalleesOnly =
+    "When emitted by callees, bodyless_defs= (when present) counts how many of the defs= are declarations with no body (header-only or forward-declared); "
+    "zero callees may mean no body to read callees from rather than truly no dependencies. ";
+
+// The composed opener, one call for the caller — keeps the wantCallers/callees branch out of
+// runCallHierarchy (already this file's largest dispatcher) rather than adding a ternary at the call site.
+inline std::string callHierarchyLegendOpen( bool wantCallers )
+{
+    return wantCallers ? std::string( kCallHierarchyLegendOpen )
+                        : std::string( kCallHierarchyLegendOpen ) + kCallHierarchyLegendCalleesOnly;
+}
+
 } // namespace rw

src/graphlegend.h

@@ -5849,12 +5849,17 @@ std::optional<int> runCallHierarchy( const MainDispatch& d )
 
         // §H4 §3.4: the FIRST legend these two verbs have ever shipped (0 bytes before — which is why every
         // one of their root attributes sits in test/legendcoverage_baseline.txt), and the floor marker that
-        // is the round's honest half. ONE string for both forms, printed BEFORE the format branches so the
+        // is the round's honest half. ONE opener for both forms, printed BEFORE the format branches so the
         // columnar and default shapes carry the identical disclosure. JSON has no comment-node analogue, so
         // there the marker travels as the counts_floor key on the root object instead.
+        // V1 fix (verifier finding 3): bodyless_defs= is callees-only (main.cpp gates the attribute itself
+        // behind !wantCallers a few lines up), so its defining sentence rides along only on the callees
+        // form — a --callers call no longer pays for vocabulary it can never emit. The wantCallers branch
+        // lives in rw::callHierarchyLegendOpen (graphlegend.h), not here, so it does not add to this
+        // already-large dispatcher's own complexity.
         if( !cfg.json )
         {
-            std::printf( "%s%s-->", rw::kCallHierarchyLegendOpen, rw::graphCountDisclosure().c_str() );
+            std::printf( "%s%s-->", rw::callHierarchyLegendOpen( wantCallers ).c_str(), rw::graphCountDisclosure().c_str() );
         }
 
         // --format=columnar (RESEARCH lever 1): the same page window, re-encoded as a path-table + parallel
@@ -12220,7 +12225,14 @@ int runDefaultMap( const MainDispatch& d )
         // + the pre-rendered <bodies> + "</ctx>". Rendering-and-measuring beats arithmetic here: the map's
         // est_tokens digits depend on the payload charge, and a probe that prices a shape it then fails to
         // build is the exact climbCeilingLadder failure mode this file already documents.
-        const std::size_t bundleBytes = ( sizeof( "<ctx>" ) - 1 ) + measureEmittedMapBytes( mapTopK, payloadTokens )
+        // V1 fix (verifier finding 1, 2026-08-15): mapTopK==0 means UNLIMITED inside serialize(), not "no
+        // map" — an unguarded call here priced the whole-repo map (~1MB on this tree) that the emission
+        // path below never prints (mapTopK==0 skips straight to the topK>0 branch's `else`, see the two
+        // guarded siblings at the ceiling verdict and the topK>0 emission gate). Same guard here: a map
+        // that will not be emitted must not be charged, exactly like every other measureEmittedMapBytes
+        // call site in this function.
+        const std::size_t bundleBytes = ( sizeof( "<ctx>" ) - 1 )
+                                      + ( mapTopK > 0 ? measureEmittedMapBytes( mapTopK, payloadTokens ) : 0 )
                                       + bodiesSection.xml.size() + ( sizeof( "</ctx>" ) - 1 );
         wholeFile = rw::renderWholeFiles( ing, expandNodes, redactPtr, d.notesPtr, cfg.compress );   // D2: shaped candidate
         ExpandServeChoice choice = chooseExpandServe( bundleBytes, wholeFile, cfg.packBudgetBytes );

src/main.cpp

@@ -47,6 +47,13 @@ cd "$TMP"
 fileBytes="$( wc -c <"$TMP/fix/small.c" | tr -d ' ' )"
 
 # ── (1) small-file symbol: the whole file is the cheapest complete answer ─────────────────────────────
+# V1 fix (verifier finding 1, 2026-08-15): smallProbe is an EXACT-NAME match, so its own default bundle
+# is now the LEAN (topk_default="0") body — not map+body — per the same V1 default arm (2) already
+# accounts for on bigProbe007. That makes small.c's fixture genuinely byte-minimal a REQUIREMENT, not
+# cosmetic: with the estimator's phantom-map bug fixed, a lean body beats any file bytes it does not
+# have to pay a descriptive header comment for, so small.c carries NO leading comment (unlike big.c,
+# whose bulk swamps a header either way) — a comment here was previously masking the correct comparison
+# by inflating the file side just enough to keep mode="whole-file" for the wrong reason.
 "$BIN" fix --expand=smallProbe --no-cache >"$TMP/small.xml" 2>/dev/null
 grep -q 'mode="whole-file"' "$TMP/small.xml" \
     && ok "(1) small-file --expand serves mode=\"whole-file\"" \

test/expandmodecheck.sh

@@ -1,5 +1,3 @@
-// expandmodefix/small.c — the SMALL-file case for test/expandmodecheck.sh: the whole file is byte-cheaper
-// than the default --expand bundle (map + body), so auto mode-selection must serve mode="whole-file".
 int smallProbe( int value )
 {
     return value * 2 + 1;

test/expandmodefix/small.c

@@ -112,9 +112,56 @@ else
     ok "(E) bundle mode ships no ranked map either"
 fi
 
+# ── (G) V1 fix regression guard (verifier finding 1, 2026-08-15) — the shape (B) and (E) both
+#        structurally could not exercise. (B) forces bundle mode via --pack-budget-bytes=10, so the
+#        estimator's own value never gets compared against a real whole-file candidate. (E)'s
+#        bigProbe007 fixture is tiny and hand-built, so even a wildly wrong bundle estimate happens not
+#        to flip the mode there. Neither arm runs at the DEFAULT budget on a REAL, non-trivial repo — the
+#        one place the defect actually manifested: measureEmittedMapBytes(mapTopK, …) was called
+#        UNGUARDED for the bundle-size estimate, and mapTopK==0 means UNLIMITED inside serialize() (not
+#        "no map"), so the estimator priced a ~1 MB whole-repo map that the emitter never intended to
+#        print. chooseExpandServe then compared that phantom bundle against the real whole-file candidate
+#        and picked mode="whole-file" — serving the ENTIRE 48 KB src/darkflags.h in place of the ~1 KB
+#        body the exact-name default promises. RED on the pre-fix binary
+#        (reason="file 48222B < bundle 1054283B", mode="whole-file"); GREEN once the estimator is
+#        guarded exactly like its two siblings at the ceiling verdict and the topK>0 emission gate
+#        (`mapTopK > 0 ? measureEmittedMapBytes(...) : 0`).
+"$BIN" "$ROOT" --expand=endsWithView --no-cache >"$TMP/real_default.xml" 2>"$TMP/real_default.err"
+"$BIN" "$ROOT" --expand=endsWithView --no-cache --top-k=0 >"$TMP/real_tk0.xml" 2>/dev/null
+realTk0Bytes=$( wc -c < "$TMP/real_tk0.xml" | tr -d ' ' )
+
+if grep -q 'mode="whole-file"' "$TMP/real_default.xml"; then
+    no "(G) default --expand=endsWithView on the real repo wrongly served whole-file: $( grep -oE '<ctx[^>]*>' "$TMP/real_default.xml" )"
+else
+    ok "(G) default --expand=endsWithView on the real repo correctly stays in bundle mode"
+fi
+
+# (G-a) the SERVED BODY (everything but the <ctx ...> opening tag's own mode=/reason= decoration, which
+#       composing with M6 is the documented, (E)-gated contract) must be byte-identical to explicit
+#       --top-k=0's — proving the estimator and the emitter now agree on what mapTopK==0 means.
+sed 's/<ctx[^>]*>//' "$TMP/real_default.xml" > "$TMP/real_default_body.xml"
+sed 's/<ctx[^>]*>//' "$TMP/real_tk0.xml"      > "$TMP/real_tk0_body.xml"
+diff -q "$TMP/real_default_body.xml" "$TMP/real_tk0_body.xml" >/dev/null \
+    && ok "(G-a) default's served body is byte-identical to explicit --top-k=0's" \
+    || no "(G-a) default's served body diverges from explicit --top-k=0's — the estimator or the emitter disagree on what mapTopK==0 means"
+
+# (G-b) when topk_default="0" is in effect and a reason= fires, the bundle byte count it PRICES must be
+#       the REAL served size (== the --top-k=0 byte count), never a phantom map-inclusive estimate. This
+#       is the precise, load-bearing number the V1 defect corrupted.
+pricedBundle=$( grep -oE 'reason="bundle [0-9]+B' "$TMP/real_default.xml" | grep -oE '[0-9]+' )
+if [ -n "$pricedBundle" ]; then
+    if [ "$pricedBundle" = "$realTk0Bytes" ]; then
+        ok "(G-b) reason= prices the bundle at exactly the --top-k=0 byte count (${pricedBundle}B) — no phantom map"
+    else
+        no "(G-b) reason= priced the bundle at ${pricedBundle}B but --top-k=0 actually serves ${realTk0Bytes}B — the reason= string still prices a map that mapTopK==0 will never emit"
+    fi
+else
+    no "(G-b) no reason=\"bundle NNNB ...\" clause found on the real-repo default root — unexpected mode, see (G) above: $( grep -oE '<ctx[^>]*>' "$TMP/real_default.xml" )"
+fi
+
 # ── (F) well-formedness + determinism ─────────────────────────────────────────────────────────────────
 if command -v xmllint >/dev/null 2>&1; then
-    for f in uniq dup5 dup tk5 tk0 big; do
+    for f in uniq dup5 dup tk5 tk0 big real_default real_tk0; do
         xmllint --noout "$TMP/$f.xml" 2>/dev/null && ok "(F) $f.xml well-formed" || no "(F) $f.xml fails xmllint"
     done
 else

test/expandtopk0check.sh

@@ -107,10 +107,16 @@ grep -q 'srcmut_sigchange' "$TMP/err" && no "the constant nonsense suggestion (s
 # ── (d) plain name and canonical-id forms stay byte-identical to pre-fix (only file:name changed) ───────
 #    NoteIndex::empty's canonical id ("path::scope::name") — a scoped method, so it actually carries "::"
 #    (a scope-less free function's canonical id degrades to its bare name and can't test this branch).
-#    The id is looked up live via --expand (it embeds the corpus path exactly as invoked — "." vs an
-#    absolute ROOT produce different id= strings — so it must never be hardcoded).
-CANON_ID="$( "$BIN" "$ROOT" --expand='src/notes.h:empty' --no-cache 2>/dev/null | grep -o 'id="[^"]*NoteIndex::empty"' | head -1 | sed 's/^id="//;s/"$//' )"
-[ -n "$CANON_ID" ] || { no "could not look up NoteIndex::empty's canonical id via --expand"; CANON_ID="./src/notes.h::NoteIndex::empty"; }
+#    The id is looked up live via --outline, not --expand (it embeds the corpus path exactly as invoked —
+#    "." vs an absolute ROOT produce different id= strings — so it must never be hardcoded). V1 fix
+#    (verifier finding 1, 2026-08-15): --expand's own exact-name default now correctly picks whichever of
+#    lean-bundle/whole-file is genuinely cheaper (the bug this fix closed made whole-file win here for the
+#    WRONG reason — a phantom map size — which is what let a bare --expand double as an id= lookup before).
+#    Bundle mode's <b> body tag carries no id= at all, so --expand is no longer a mode-independent way to
+#    fetch a canonical id; --outline always rides the classic 200-row map (no V1 lean default applies to
+#    it) and its <s> rows carry id= unconditionally, so it is the stable lookup path here.
+CANON_ID="$( "$BIN" "$ROOT" --outline='src/notes.h:empty' --no-cache 2>/dev/null | grep -o 'id="[^"]*NoteIndex::empty"' | head -1 | sed 's/^id="//;s/"$//' )"
+[ -n "$CANON_ID" ] || { no "could not look up NoteIndex::empty's canonical id via --outline"; CANON_ID="./src/notes.h::NoteIndex::empty"; }
 BARE_A="$( uses_elem 'buildGraph' )"
 BARE_B="$( uses_elem 'buildGraph' )"
 [ "$BARE_A" = "$BARE_B" ] && ok "bare-name form is stable/reproducible: $BARE_A" || no "bare-name form not reproducible"