Linux portability: compile, runtime, sanitizer, leak, and gate-semantics fixes from first real CI contact - #1
Merged
Merged
Conversation
…nction 'from_chars'
macos-14's libc++ declares the FLOATING-POINT std::from_chars overloads `= delete`, so the
one FP call in the tree (isUniversalOrAllowlistedNumber's magic-number filter) is a hard
compile error on both the release and the asan leg of the first public CI run. Integer
from_chars is present everywhere and is untouched — a tree-wide sweep found exactly one FP
site.
New src/charconvcompat.h provides rw::parseFloating, with the same (ptr, ec) contract as
std::from_chars( …, chars_format::general ):
* Detection is a requires-expression, NOT __cpp_lib_to_chars. That macro covers to_chars
and from_chars TOGETHER, so libc++ leaves it undefined while shipping a working FP
from_chars — on this repo's own dev machine (Apple Clang 21) the macro is UNDEFINED and
the std call compiles, so keying off it would have moved the dev machine onto the
fallback. The requires-expression asks the only question that matters and is correct for
both `= delete` (libc++) and never-declared.
* The strtod/strtof fallback is compiled on EVERY platform, so the dev build proves it
still type-checks and the macos-14 leg proves it still behaves.
* It handles the three places strtod's grammar is WIDER than from_chars(general) — leading
whitespace, a leading '+', and hex floats — plus strtod's ERANGE-on-subnormal, which
from_chars reports as a plain success.
Verified: a 64-vector differential probe (signs, exponents, inf/nan spellings, hex, leading
'+'/whitespace, empty, trailing garbage, overflow/underflow/subnormal boundaries) run over
BOTH float and double under the production -O2 -ffast-math -fno-finite-math-only flags —
128/128 identical ptr, ec and (on success) value versus std::from_chars.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…: No such file or directory
kqueue/kevent is a BSD interface; <sys/event.h> simply does not exist on Linux, so both
ubuntu-24.04 legs of the first public CI run died at this include. Two separate Darwin-isms
in this header are fixed:
* <sys/event.h> and every kevent body are now behind RIPWIRE_HAS_KQUEUE (Darwin + the four
BSDs). No inotify implementation: FsWatcher ALREADY specifies the "kqueue unavailable"
degrade — kq stays -1, healthy stays false, drainHadEvent() reports "assume changed" and
getIndex() runs the FULL stat/mtime dir sweep on every request. A platform without
kqueue takes that identical path, so the MCP staleness CONTRACT is untouched: a stale
index is still detected on request, by the per-file mtime+size loop that runs regardless
of the watcher. Only a redundant-work elision is lost. inotify is named in the header as
the future upgrade — it is new code with its own event-semantics bug surface, and the
poll fallback is already correct.
* struct stat's sub-second mtime field is spelled st_mtimespec on Darwin/BSD and st_mtim
on Linux, and NEITHER name exists on the other platform. mtimeOf/statOf both used the
Darwin spelling — a second, independent Linux compile error hidden behind the include.
New mcpdetail::mtimeNsOf carries the same three-way ladder ingest.cpp's statSizeMtime
already uses, whole-second last resort included.
The guard is written `#ifndef RIPWIRE_HAS_KQUEUE` on purpose, so -DRIPWIRE_HAS_KQUEUE=0
compiles the Linux path on a Mac. Verified that way, not by inspection: a full
-DRIPWIRE_HAS_KQUEUE=0 build is clean, and mcpwatchercheck / mcpstalecheck / mcpreloadcheck
/ mcpcontractcheck are ALL PASS against that binary — including the watcher gate's
scenario B, whose subject is precisely "degraded watcher, adds still detected".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pthread_main_np' not declared
Both are Darwin-only pthread extensions and neither is declared on Linux, so ThreadData's
constructor failed to compile on both ubuntu legs of the first public CI run. Replaced with
three shims in prof::detail, each documented with what its platform's answer is actually
worth:
* threadIdNumeric() — Darwin: pthread_threadid_np (unchanged). Linux: syscall(SYS_gettid),
which is the same kernel task id gdb/htop/perf show, so a report row stays matchable
against a tracer. Elsewhere: hash of std::this_thread::get_id().
* isInitialThread() — Darwin: pthread_main_np (unchanged). Linux: getpid() == gettid(),
which is the EXACT definition of the initial thread, not an approximation. Elsewhere: a
first-caller latch (registration happens on a thread's first PROFILE_SCOPE and main()
runs before any worker spawns).
* copyThreadName() — pthread_getname_np is *_np too, but unlike the other two it exists
with this exact signature on Darwin AND glibc/musl; only the residual platforms lose the
name, and the report already prints "unnamed" for that.
All three feed the PROFILE REPORT only (tid= column, the [main] tag, the thread name) —
nothing here can reach a ripwire output byte, so determinism is not in play.
Verified on the dev Mac: -DRIPWIRE_PROFILE=ON build is clean and its report over
test/fixture still shows 7 distinct tids with EXACTLY ONE tagged [main] — i.e. the Darwin
branch is behaviourally unchanged by the refactor. The generic last-resort branch was
compiled and run in isolation (distinct ids per thread, initial-thread latch true on main
and false on a worker, empty name). Only real CI can prove the Linux branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e=' option: 'integer'
`integer` is a CLANG-ONLY UBSan group (the defined-but-suspicious conversion checks, not UB).
GCC has no equivalent and rejects the entire -fsanitize= option, so the ubuntu asan leg of
the first public CI run never got past configure — ubuntu's default cc is gcc.
(a) CMakeLists.txt — the G1 check list is built as a list and filtered by compiler id. Under
GCC, `integer` is removed AND the three -fno-sanitize= exemptions plus every
-fsanitize-ignorelist= are skipped, because each of those names a member/feature of the
same Clang-only group and would fail identically one level down. address / undefined /
float-divide-by-zero / float-cast-overflow are all GCC-supported and stay. A configure-
time STATUS message names what was dropped, why, and what remains, so nobody reads a
green gcc asan run as full-G1 evidence.
(b) .github/workflows/ci.yml — the Linux asan job now configures with CC=clang CXX=clang++
(installed explicitly rather than assumed preinstalled), so PUBLIC CI keeps the COMPLETE
G1 stack on Linux and the two matrix legs' gates mean the same thing. The GCC-filtered
path is for a contributor building asan with gcc: degrade honestly, don't fail to
configure. The RELEASE job's compiler is deliberately untouched — default gcc on Linux
is a feature, and it found three real portability bugs on its first run.
CONTRIBUTING.md's G1 paragraph stated the stack unconditionally; it now says build G1 with
Clang and what the GCC path costs.
Verified: the exact filter block, extracted verbatim into a standalone CMake probe and driven
with each compiler id — AppleClang and Clang produce the byte-identical pre-change flag string
with ignorelists ON; GNU produces
`-fsanitize=address,undefined,float-divide-by-zero,float-cast-overflow` with ignorelists OFF.
The real -DRIPWIRE_ASAN=ON build on this Mac configures and builds clean (AppleClang branch,
unchanged). ci.yml re-parses as valid YAML with both configure steps present. Only real CI can
prove gcc accepts the filtered list and that clang-18 handles the full stack on ubuntu.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…has no link on Apple) The sweep behind the four CI errors turned up one more thing worth fixing rather than reporting: the parallel ingest and the MCP qsnap-prefetch worker are std::thread, and nothing in the build ever asked for a threading library. On Apple platforms the pthread runtime lives inside libSystem so this is invisible, which is why a Mac-only tree never noticed. On Linux it depends on the libc: merged into glibc from 2.34 (so ubuntu-24.04 happens to link anyway), a separate -lpthread before that, and -pthread also changes CODEGEN rather than only the link line. find_package(Threads) + Threads::Threads asks CMake for whatever the platform actually needs; on macOS it resolves to nothing, so the dev build is byte-unaffected (verified: clean reconfigure + build, "Found Threads: TRUE"). Everything else the sweep looked at was already correct and is left alone — see the lane report for the full list (profilePmc.h's kperf is already __APPLE__-guarded WITH inert stubs, main.cpp's _NSGetExecutablePath is guarded with a /proc/self/exe branch, ingest.cpp already carries the three-way st_mtimespec/st_mtim ladder, mcpserver.h's socket includes are complete for glibc, and there are no BSD-only APIs, no C++23 features GCC 13 lacks, and no missing libc++-transitive includes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… one literal flag string g1configcheck.sh grepped CMakeLists.txt for the literal `-fsanitize=address,undefined,integer,float-divide-by-zero,float-cast-overflow`. K4 replaced that with a five-member list, a compiler-id filter and a list(JOIN), so the gate went red on a tree whose G1 set is unchanged for Clang. The replacement asserts MORE than the string did, because a filtered list has failure modes a literal does not: the complete set is declared; the -fsanitize= string is joined from that same list (so no second hand-maintained literal can drift away from it); the only subtraction is exactly `integer` and only under CMAKE_CXX_COMPILER_ID GNU (a REMOVE_ITEM count of 1, so a later lane cannot quietly drop `undefined` too); and the Clang-only -fno-sanitize= exemptions and -fsanitize-ignorelist= ride the same finding. ALL PASS (19 checks). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--quality-delta of the whole lane against its base (d6f6e15) reported one GATING row and one duplication row. Both were real and both are now gone (gating="0", and the only remaining rows are the unavoidable minor api-surface entries for the new helpers themselves): * duplication — charconvcompat.h's isInfiniteBits was a hand-written double/float pair with two hand-typed hex patterns (39 duplicated tokens). Now one constrained template whose sentinel is bit_cast from std::numeric_limits<T>::infinity(), so neither width's bit pattern can be mistyped and the exponent mask is derived, not written out. * verbosity (GATING) — FsWatcher grew 80 → 94 lines, almost all of it a comment inside arm() restating the fallback that the header-level note above the includes already explains in full. The two platform seams are now one line each (80 → 88, sev=minor). Re-verified after the rewrite, not assumed: the 64-vector differential probe is still 128/128 identical to std::from_chars across float and double under -O2 -ffast-math -fno-finite-math-only, and the -DRIPWIRE_HAS_KQUEUE=0 build plus mcpwatchercheck / mcpstalecheck / mcpreloadcheck against it are still ALL PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…c → SIGABRT (exit 134)
fopen() on a directory FAILS on macOS and SUCCEEDS on Linux/glibc, so readFile()'s
fseek/ftell/resize ran against a directory handle and died on the resize. The Ubuntu
probe reproduced it standalone through cachefuzzcheck's directory_at_cache_path arm,
which has always been green here because the macOS fopen refused first.
shapeOfPath() (ingest.cpp) is the tri-state the cache seams actually need: Absent is
the ordinary silent cold-start miss, RegularFile is the only usable shape, anything
else is disclosed once and self-heals into a full reparse — the same path a checksum
mismatch takes. Checked BEFORE the open, never after, because a FIFO at the path would
block inside fopen("rb") and no post-open fstat can undo that. No platform #ifdef: the
S_ISREG test is correct on macOS too, which is why cachefuzzcheck stays green here.
Both halves: loadCache refuses to read one, saveCache refuses to publish over one
(ahead of the serialize, so a directory no longer costs a whole wasted pass before
rename(tmp,dir) fails EISDIR at the bottom). quality.h's readQSnapBlob gets the same
guard — same shape, same directory gate arm in cachefuzzcheck part 2, and ifstream
opens a directory on libstdc++ exactly like fopen does.
Proof on this Mac: `--cache=<dir>` and `--cache=/dev/null` both exit 0, both emit
output byte-identical to a `--no-cache` cold parse, both disclose the shape once per
site. cachefuzzcheck ALL PASS (dev + ASan tables).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…non-kqueue platforms K2's fallback emitted DEGRADED_PATH_ALERT( "no kqueue on this platform" ) from FsWatcher::arm on every arm() call of every Linux build. A degrade alert marks an UNEXPECTED fallback — something that normally works did not, this run. On a build with no kqueue at all there is no fast path to fall back FROM: the stat-sweep is the only path the binary has, taken forever, by design. The alert was therefore a line nobody could act on, and it reddened the stderr-clean gates that correctly read an alert as a signal. So the !RIPWIRE_HAS_KQUEUE branch is now silent, and the RUNTIME kqueue() failure on a kqueue platform — the fast path exists and did not come up — keeps its alert unchanged. The freshness CONTRACT is identical either way (unhealthy → getIndex() always sweeps), which is exactly why the compiled-out branch has nothing to report. Rationale moved to this file's kqueue preamble, where the platform split is already explained. Proof: a `-DRIPWIRE_HAS_KQUEUE=0` build driven through initialize + a tools/call now produces an EMPTY stderr. mcpwatchercheck / mcpreloadcheck / mcpstalecheck / mcpverbscheck / qualitystalecheck are ALL PASS against BOTH that build and the normal kqueue build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…time/size/mode helpers
The idiom `stat -f FMT "$1" || stat -c FMT "$1"` does not fall back. On GNU coreutils
`-f` means FILESYSTEM status and takes NO format argument, so FMT is parsed as a second
FILE. Measured here against coreutils 9.11 (via /opt/homebrew/opt/coreutils):
$ stat -f %i /tmp/l3probe/f1
File: "/tmp/l3probe/f1"
ID: ... Namelen: ? Type: apfs
Blocks: Total: ... Free: ...
Inodes: Total: ... Free: ...
stat: cannot read file system information for '%i': No such file or directory
exit 1
Six lines of filesystem block on STDOUT, exit 1 — so the `||` arm does run and appends
the right number UNDERNEATH the junk. Consequences by helper: a string compare
(`[ "$PERM" = "700" ]`) can never hold; a numeric compare (`[ "$f_mtime" -gt … ]`) dies
with "integer expression expected"; and qsnapprefetchcheck's `stat -f '%i %m' || echo
MISSING` reports MISSING forever, i.e. a gate that passes by comparing nothing to
nothing.
Fixed uniformly: detect the flavour ONCE with `stat --version` and define a single
named reader per script (inode_of / mtime_of / mtime_ns / size_of / mode_of /
apparentsize / inode_mtime / file_mode). Gates stay self-contained — no shared sourced
file, because this repo has no such convention. The probe listed 8 affected scripts;
grepping tree-wide found 12 (g1freshcheck, statgatecheck, portablecachecheck and
prcontextcheck carried the same trap in mtime/size/mode variants), and no raw
`stat -f … || stat -c …` remains anywhere under test/, scripts/ or .github/.
All 12 ALL PASS on this Mac (the BSD arm is the one this host executes; the GNU arm is
proved by the probe above and lands with the next remote Linux run).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ee actions/checkout@v4 sites actions/checkout@v4 defaults to a --depth 1 clone, which leaves ONE commit of history in the tree. The churn / co-change / ownership / hotspot gates (churnjoincheck and friends) mine `git log` for real, so on a shallow checkout they do not error — they measure zero and fail, or pass while measuring nothing. The probe proved the reddening directly. One line per checkout step, with the reason named at the site. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bstdc++), rejected on macOS (libc++)
Took the FIX path, not the document-the-divergence path, because the tractable check is
small and provably cannot over-reject.
ECMAScript's IdentityEscape forbids `\<letter>` for any letter that is not a recognised
escape. libc++ enforces it; libstdc++ does not, and silently reads `\Q` as the literal
letter Q. That is the worse half of the split: the lenient side does not error, it
answers a DIFFERENT question and hands the result back as a measurement.
The accepted set was MEASURED with a probe against this host's libc++, not inferred from
the grammar: `\b \B \d \D \s \S \w \W \f \n \r \t \v` stand alone, and `\c \x \u` are
accepted with a tail the engine still validates. Everything else after a backslash —
digits (back-references), `$`, `_`, punctuation, non-ASCII bytes — libc++ already
accepts, so it is left entirely to the engine. nonPortableRegexEscape() therefore
rejects EXACTLY the set libc++ already rejected and nothing more: no pattern that
searches on macOS today stops searching, and Linux stops misreading the Perl-isms.
It runs first inside regexCompileError(), the one chokepoint --regex/--no-prefilter
already refuse through, and grepCollect's belt-and-braces reject now calls that same
function instead of its own try/catch, so a library/MCP caller cannot get a pattern the
CLI would refuse.
test/regexcheck.sh needed NO change: its `\Q\E` case already asserted exit 1 + stderr +
no hits element, and that expectation now holds on both platforms rather than only one.
Verified here: `\Q\E`, `\A`, `\p{L}` refuse at exit 1; `\d+`, `\w\s`, `\\Q`, `\$`,
`\cA`, `\x41`, `(?:foo)` all still search at exit 0. regexcheck / regexrefusecheck /
regexbombcheck / grepcheck ALL PASS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s churn="<none>" on Linux
ROOT CAUSE: the gate fixture, not the join. No source change was warranted and none was
made.
Ruled out first, in source: every git invocation in gitmine.h / gitoracle.h that reads
PATHS already carries `-c core.quotepath=false` (the one that does not is
`rev-parse --show-toplevel`, deliberately, and it prints a directory verbatim), so the
octal-escaping hypothesis was already closed. The join itself is a byte-exact hash
lookup on a derived key; hasCombiningMark() only COUNTS decomposed names, it never
refuses them.
What actually happens: §8b's fixture bodies were `int nfdFn( int a ) { return a + 1; }`
— cognitive complexity ZERO. --hotspots ranks on churn × ccx and emits no <f> row for a
zero-complexity file no matter how many commits touch it, so churn_of() read "" and the
arm reported churn="<none>". On macOS this stayed invisible because git composes the
name and the arm takes the DISCLOSURE branch, which never calls churn_of; on Linux git
records NFD verbatim, the JOIN branch runs, and it asserted on a row its own fixture
could not produce. The gate's two branches are selected by the host, so each platform
only ever exercised one of them.
RED-FIRST, on this Mac: forcing `core.precomposeunicode=false` on §8b's repo reproduces
the probe's line verbatim —
FAIL G2: the NFD file reports churn="<none>" on a platform whose git spells it identically
Fix: mk_nfd_repo() writes mkfn-shaped bodies (one `if`, so the file can be ranked at
all), and a new §8c builds the same fixture with core.precomposeunicode=false — a Linux
checkout's native behaviour — so the JOIN half now runs unconditionally on EVERY host
instead of only on Linux. Post-fix, that forced arm reports churn="3" with no
disclosure: the join binds NFD bytes byte-exactly and always did.
churnjoincheck ALL PASS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One honest section, only what the Ubuntu probe (and this lane's own measurements) established: gcc needs ~3 GB of RAM per parallel job for src/main.cpp at -O2 where clang fits in 2 GB, and the failure mode is the OOM killer rather than a diagnostic; the suite shells out to xmllint / ripgrep / bc / jq / curl / python3 / git; a shallow clone and a root shell both make gates fail or prove nothing; XDG_CACHE_HOME pointing at a directory that does not exist silently disables caching (the ladder's mkdir is not recursive); GNU stat is not BSD stat and the `-f … || -c …` fallback does not fall back (L3); and the missing kqueue is the designed path, reproducible on a Mac with -DRIPWIRE_HAS_KQUEUE=0 (L2). deckcheck_allowlist.txt gains git's own `--depth`, quoted in that note — the allowlist's category (b), a flag belonging to a different tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The real-Linux branch smoke (Ubuntu 24.04) reported:
FAIL missing an AGG row
from test/recallevalcheck.sh #2, while every metric underneath actually cleared its
floor. Linux numbers from that same run:
recall lenient_r5=85.7 (floor 85) mrr 0.651 (floor 0.60) pollution 8.6 (ceiling 16)
ranking lenient_r5=75.0 (floor 70) mrr 0.676 (floor 0.55) pollution 0.0 (ceiling 5)
— byte-for-byte the numbers this Mac produces, so the eval itself never differed.
Cause: recallevalcheck.sh:76-77 wrote the tab-separated machine rows as
grep -E '^AGG\trecall\t'
BSD/TRE grep (macOS) expands \t to a tab inside an ERE; GNU grep does not — it
reads '\t' as an escaped 't', i.e. a literal 't', so '^AGG' followed by a tab never
matched on Linux and both REC and RNK came back empty. Every downstream assertion
then read empty strings.
Fix: real tabs via the $'…' form, which both shells expand before grep ever sees the
pattern. Line 122's CLASS row already used exactly this idiom and was unaffected on
Linux — it is now the idiom everywhere in the file, with the reason recorded inline.
Swept test/ bench/ scripts/ .github/ docs/ for the same trap in grep/egrep and in sed
patterns: these two lines were the only occurrences (the one other '\t' hit is prose
inside bench/recalleval/run_recalleval.py's module docstring).
Red-first is not possible for the Linux half of this from a Mac — BSD grep passes
either spelling — so the evidence is the smoke output cited above. macOS gate re-run
after the change: ALL PASS (16 arms), numbers unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gned-integer-overflow
The first real Linux G1 run (Ubuntu 24.04, clang 18 + libstdc++, the complete
-fsanitize=address,undefined,integer,float-divide-by-zero,float-cast-overflow stack)
aborted immediately:
string_view.tcc:124:25: runtime error: unsigned integer overflow: 0 - 1
reached from rw::lowerExtensionOf (src/ingest.cpp:160) via std::string_view::rfind,
and again at string_view.tcc:109 — the find twin — partway through cachefuzzcheck's
qsnap ASan sweep, taking that gate down with it.
The wrap is libstdc++'s own and is deliberate: its find/rfind scan loops are written
`for (++__size; __size-- > 0;)`, so the counter wraps past zero on exactly the
iteration that ends the loop. Unsigned wrap is defined behaviour; `integer` is the
Clang-only "defined but suspicious" group, not a UB group. Nothing in ripwire can
avoid it short of not calling find/rfind, and G1 is -fno-sanitize-recover=all, so a
single wrap is a hard stop rather than a report. libc++ writes those loops without a
wrap, which is the whole reason macOS never saw this in months of local G1 runs.
Fix: a fourth generated -fsanitize-ignorelist, applied to the C++ targets only
(the grammars are C and have no C++ standard library in them), scoped as narrowly as
the mechanism allows — this one header, this one check:
[unsigned-integer-overflow]
src:*/bits/string_view.tcc
Every other integer and conversion check stays on for our own code, in the same
translation units. It sits inside the existing RIPWIRE_HAS_CLANG_INTEGER_SANITIZER
guard for the same reason K4 put the others there: both the check name and
-fsanitize-ignorelist= belong to the Clang-only group GCC rejects outright.
Proof the mechanism does what the comment claims, on this Mac's own clang (an
ignorelist entry naming a HEADER, with -fno-sanitize-recover=all in force): a 6-line
.tcc with the identical `for (++n; n-- > 0;)` shape aborts with the identical
diagnostic — "runtime error: unsigned integer overflow: 0 - 1" — at exit 134
unguarded, and exits 0 with the src: entry added. Nothing else changed.
macOS proof that the entry is inert but well-formed (the list is on the clang path
macOS also takes): asan reconfigured and fully rebuilt clean, and
LSAN_OPTIONS=suppressions=lsan_suppressions.txt ./asan/ripwire . >/dev/null
exits 0 with no sanitizer output.
Only the Linux re-run can prove the abort itself is gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…c++ (real product bug)
REAL PRODUCT BUG, not a gate artefact. The first real Linux run (Ubuntu 24.04,
clang 18 + libstdc++):
$ ripwire test/regexbombfix --regex='(a+)+b' --no-prefilter --no-cache
… CPU-bound, still running at 560 s, killed by the harness
On this Mac the same command finishes instantly, and regexbombcheck has been green
for months on that basis. The reason is in the gate's own header, stated as a fact
about the toolchain rather than as an assumption: Apple libc++ has a complexity
budget and abandons a pathological match with regex_error(error_complexity), which
grepScanText's catch turns into a skipped file — the A4-F10 "degrade, don't die"
path. libstdc++ has NO such budget. It never throws, so that catch is never reached
and the process just backtracks exponentially. On Linux a user's pathological --regex
does not degrade; it hangs the tool, with no output and no way to tell it apart from
a slow scan.
FIX — the L5 shape, one verdict decided from the pattern TEXT before either engine
sees it. A structural pre-compile guard at the single chokepoint,
regexCompileError() in src/search.h, running ahead of the compile probe for the same
reason L5's escape screen does: the answer must be a pure function of the pattern,
not of whose backtracker is linked in, and not of what happens to be in the corpus.
Refusal is the right outcome rather than a silent skip — a skipped file reads as a
measurement, an exit-1 refusal that names the construct cannot.
CAUGHT — an unbounded quantifier ('*', '+', '{n,}') applied to a group that already
repeats without bound anywhere inside it, at any depth. That is the whole exponential
family reachable by inspection, and each shape is an arm of the gate:
(a+)+b the classic, and the one Linux hung on
(a*)*b nullable-inner twin
(a+)*b star-flavoured outer
(a{2,})+b {n,} is as unbounded as +
((a+))+b inner repetition one group deeper than the outer quantifier
((a)+)+b the repetition is ON the inner group, not on an atom inside it
(a+|b)+c alternation is no defence when a branch repeats without bound
NOT CAUGHT, deliberately — over-refusal is silent and costs users working patterns,
so precision is asserted as hard as the refusals are, one gate arm each: '?' and
'{n,m}' are BOUNDED and never drive the blowup ((a?)+b, (a{1,3})+b); a group with no
repetition inside is safe however it is quantified ((abc)+, (a|b)+, (a)+b); '+' in a
character class or behind a backslash is a literal ([a+]+b); an unquantified group is
safe whatever it holds ((a+)b, (a+)(b)+); top-level repetition is fine (a+b). All nine
already passed against the PRE-fix binary, so they are real controls, not tautologies.
KNOWN GAP, documented in the source rather than papered over: overlapping alternation
like (a|a)+b is a genuine bomb whose branches only overlap semantically, invisible to
a structural scan. grepScanText's mid-match try/catch is therefore KEPT as
belt-and-braces, and the gate probes (a|a)+b as an INFO arm — it will report the gap
on a libstdc++ host rather than assert something a Mac cannot see.
Scope: --regex only, the one place a user's own pattern meets an unbounded corpus.
--arch's path-rule regexes use the same engine but come from a committed rules file.
The refusal names the construct, the family it belongs to, why it cannot simply be
answered ("libc++ abandons the match in under a second, libstdc++ never gives up at
all (measured: still running after 560 s)"), and two workarounds — collapse the two
repetitions ('(a+)+' is the language of 'a+'), or bound the outer one ('(\s*\w+){1,20}').
GATE CONTRACT CHANGED, red-first. test/regexbombcheck.sh asserted the old "exit 0,
XML well-formed, other files still match" degrade contract, which a compile-time
refusal cannot satisfy and should not. Rewritten to the new unified contract and run
against the PRE-fix binary first: 30 FAIL (every bomb scanned at exit 0, no refusal,
no workaround, verdict corpus-dependent) with all 9 precision arms already green.
After the fix: ALL PASS, 44 arms. Every bomb run is wall-clock capped in portable
shell (no timeout(1) on stock macOS) so the Linux hang reddens the gate instead of
stalling the suite.
Verified here: regexbombcheck / regexcheck / regexrefusecheck / grepcheck /
grepscancheck / grepcontextcheck / grepseamcheck / jsonrefusallegendcheck ALL PASS;
det-gate ×2 byte-identical; xmllint clean; ripwirepubliccheck + docscommandscheck ALL
PASS (no --help surface moved, so COMMANDS.md and the capture are untouched);
formatcheck ALL PASS; --quality-delta gating=0.
Only a Linux re-run can prove the hang itself is gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…stead of banning src: outright g1configcheck.sh went red on M1, correctly and by design: its dependency-policy arm is a whitelist of the EXACT sanitizer exemptions the build is allowed to carry, and it banned any `src:`-scoped rule outright, because a file-scoped rule is the easy way to smuggle a whole directory out of the sanitizer. M1 adds the first legitimate one, so the ratchet has to move deliberately — which is what this commit is. The ban is kept in spirit and tightened in practice. `src:` may now appear EXACTLY once, and that once must be `src:*/bits/string_view.tcc`; `[unsigned-integer-overflow]` may open exactly two ignorelists (bash's fun:scan and this one). A second file rule, or a different path in the first, reds the gate. The `fun:*` wildcard ban is untouched. While writing it the arm's own mutation control caught a flaw in the FIRST version of this change, which is worth recording because it applies to every counter in this block: an ignorelist here is one CMake string holding several `\n`-joined entries, so `grep -c` — which counts matching LINES — reads a second entry planted on an existing line as zero new entries. A planted `src:*/bits/basic_string.tcc` passed. The three counters this arm now depends on are taken by OCCURRENCE via `grep -o | wc -l`. Mutation-proved both ways after the fix, against a copy of CMakeLists.txt: * second src: rule on the SAME line -> FAIL (src:-scoped=2 of which string_view.tcc=1) * audited path broadened to src:*/bits/ -> FAIL (src:-scoped=1 of which string_view.tcc=0) * restored -> ALL PASS Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…erns are invalid 'is not a valid regular expression:' -> 'refused, nothing was scanned:' — regexCompileError() also refuses VALID ECMAScript (L5 non-portable escapes, M2 catastrophic-backtracking family); the reason string names which case fired. Fix authored+gated by the spun-off session; quality-delta debt was pre-commit-only (short-horizon-churn vs HEAD vanishes at commit; the api-surface sym=add quirk is pre-existing and ledgered upstream). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…elist (basic_string.h/.tcc)
The Linux re-smoke (Ubuntu 24.04, clang 18 + libstdc++, full G1, -fno-sanitize-recover=all) died past
M1's string_view.tcc entry on two more headers carrying the identical intentional-wrap idiom:
bits/basic_string.h:490 _S_compare returns `__n1 - __n2` computed in size_type — wraps whenever the
left operand is shorter. Reached from a plain std::string operator<= inside a
sort comparator, i.e. from ordinary ripwire code.
bits/basic_string.tcc:689 the basic_string twin of the string_view find/rfind loop, same
`for (++__size; __size-- > 0;)` shape.
With both appended to the existing [unsigned-integer-overflow] section the smoke proved the Linux ASan
self-run goes exit-0 / empty-stderr and cachefuzzcheck's 13 UBSan arms go green. Library-internal,
intentional, not UB, and inert on libc++ (macOS never saw them) — but the entries must still parse there.
g1configcheck.sh: M1b asserted EXACTLY ONE `src:` rule; the audit is widened to the exact THREE-entry set
— each path counted by OCCURRENCE (not by line, per M1b's own lesson that a smuggled entry rides an
existing line) plus the total pinned at 3. Mutation control re-proved in both directions on this tree:
a planted fourth entry appended to an existing line -> FAIL (src:-scoped=4); the basic_string.h entry
removed -> FAIL (src:-scoped=2, basic_string.h=0); the path swapped for stl_vector.h -> FAIL
(src:-scoped=3, basic_string.h=0). Restored -> ALL PASS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…objects
Real product leak, found by real LeakSanitizer on the Linux re-smoke and structurally invisible to Apple
clang (LSan does not run there — CMakeLists sets detect_leaks=0 on Darwin because the arm64 runtime
rejects it):
Direct leak of 672 B in 3 objects: ts_malloc_default -> ts_query_new (query.c:2995)
-> rw::(anonymous)::compileQueryStandalone src/ingest.cpp:631
-> ingest thread lambda src/ingest.cpp:4203
firing cachefuzzcheck arms [asan:devnull_cache_path] and [asan:directory_at_cache_path]
OWNERSHIP STORY. compileQueryStandalone compiles one TSQuery per distinct grammar on a background thread;
ingest() installs the results into the process-global compiledQueryCache() single-threaded after the join,
and the parse pool only ever BORROWS the raw pointer. The install loop deletes a DISPLACED query on an
in-process re-ingest (A652/A4-F16), so growth in a long-lived MCP server was already bounded — but nothing
ever freed the queries finally RESIDENT in the map. The map's own destructor drops the pointers at exit and
the blocks go unreachable, which is precisely the direct leak above.
FIX. CompiledQueryCache — a struct whose destructor frees every query it still holds, the same shape as
this file's existing ParserGuard/TreeGuard convention, applied at the CONTAINER because the container is
the owner (per-entry move-only guards would have to survive the map's rehash-and-move for no benefit).
compiledQueryCache() keeps its signature, so no call site changes. Not an LSan suppression:
lsan_suppressions.txt exists for tree-sitter's INTERNED, never-freed data, and these are ordinary per-run
allocations with a well-defined lifetime.
NO DOUBLE FREE. Keys are deduplicated BY GRAMMAR before compiling, so no two entries can alias one
TSQuery; a displaced entry is deleted at the moment it is overwritten and never left in the map; the
destructor runs at teardown, single-threaded, after every parse pool has joined.
MACOS PROOF (LSan cannot see the leak here, so the proof is ownership + no regression). Temporarily
instrumented the destructor with a free counter and ran the ASan build:
--cache=/dev/null (full-reparse route) -> dtor freed 1 TSQuery, exit 0, no sanitizer report
--cache=<a directory> (full-reparse route) -> dtor freed 2 TSQuery, exit 0, no sanitizer report
cold run, valid cache -> dtor freed 2 TSQuery, exit 0
warm run, same cache -> dtor freed 0 (nothing compiled — which is why an already
warm ASan self-run never showed the leak)
--quality-delta (TWO ingests in one process, i.e. the displacement path) -> dtor freed 14, ASan clean
Instrumentation removed before commit. cachefuzzcheck ALL PASS; ASan self-run exit 0, empty stderr.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…er also hang libstdc++
M2 screened `(a+)+b` and its unbounded-inner family structurally, and left `(a?)+b` / `(a{1,3})+b` as
"must still scan" CONTROLS in regexbombcheck.sh on the reasoning that '?' and '{m,n}' are bounded and so
cannot drive the blowup. That reasoning was libc++ behaviour written down as a law.
The Linux re-smoke ran this gate's own control set on Ubuntu 24.04 / clang 18 / libstdc++ and BOTH
controls HUNG — killed on the harness's wall-clock cap, on the same fixture and in the same way as the
(a+)+b bomb they existed to contrast with. Bounded is not unambiguous: '(a?)+' splits a run of 'a' in as
many ways as '(a+)+' does because the inner may also match EMPTY, and '(a{1,3})+' because the inner's
width varies. Cross-platform-identical behaviour admits one verdict per pattern, so the screen is WIDENED
rather than the controls relabelled per platform.
NEW REFUSED FAMILY: an unbounded quantifier ('*', '+', '{n,}') applied to a group containing ANY
quantifier anywhere inside it, at any depth — bounded ones included. Adds (X?)+, (X{m,n})+, ((X)?)+ and
(X{n})+ to the (X+)+ / (X*)* / (X+)* / (X{n,})+ / ((X+))+ / ((X)+)+ / (X+|Y)+ set. Exact '{n}' is in on
purpose: a fixed-count inner is unambiguous only when what it repeats is fixed-width, and '((ab|c){2})+d'
is a real bomb that reading the quantifier alone cannot tell apart from '(a{3})+b'. Implementation is one
flag per open group widened from "repeats unboundedly inside" to "carries any quantifier inside"; the
refusal condition still requires the OUTER quantifier to be unbounded, which is what keeps every
bounded-outer and unquantified-outer form legal.
'(?:' / '(?=' / '(?!' now step over the group-modifier '?' (regexGroupModifierLength). Under the widened
flag it would otherwise read as a quantifier and refuse every '(?:abc)+' — a pure over-refusal the old
narrow flag never risked. '(?:a+)+b' is still refused, and is now an explicit bomb arm.
Refusal text: family list extended, the "bounded is no defence" reason stated, and the measured evidence
now cites the re-smoke's two hangs alongside the 560 s (a+)+b figure. The old workaround sentence offered
'([a-z]\.?)+' as a collapse target — which the widened screen REFUSES, so the message would have proposed
an illegal pattern. Replaced with '(a?)+' -> 'a*' and '(a{1,3})+' -> 'a+', both accurate and both directly
about the newly refused class; the bounded-OUTER workaround '(\s*\w+){1,20}' is unchanged and is now a
gate control so it can never become self-contradictory again. The message moved into
catastrophicRegexMessage() so the scan loop reads as the small state machine it is — with that and the
modifier helper, --quality-delta reports no complexity or verbosity regression for this change.
RED-FIRST: the four flipped/added bomb arms were added to regexbombcheck.sh FIRST and run against the
pre-fix binary — 15 FAILs ((a?)+b, (a{1,3})+b, ((a)?)+b, (a{3})+b each "exit 0 (expected 1)" + "printed a
hits= element — a scan happened"). The three new precision controls pass on both binaries. Post-fix:
ALL PASS on both the dev and the ASan build. Every bomb arm stays wall-clock-capped at 20 s.
FULL CONTROL-SET DISPOSITION (all still scan at exit 0): a+b, (a)+b, (abc)+, (a|b)+, [a+]+b, (a+)b,
(a+)(b)+, plus new (?:abc)+, (\s*\w+){1,20}, (a?)b. Swept every --regex pattern in test/, docs/, README,
src/, skills/, scripts/ and bench/ (36 patterns) against the new binary: the only refusals are the
pre-existing intended ones ((a+)+b, \Q\E, and the three malformed patterns). (a|a)+b remains the
documented static-invisible gap and is left as the INFO arm.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… in-gate fixture repo
CI CONSEQUENCE: on the published fresh-history repo — i.e. every clone but the author's,
and therefore every CI leg — this gate exited 1 unconditionally. Its two NEGATIVE arms were
pinned to named pairs out of the PRIVATE dev repo's git history (ingest.cpp<->model.h,
main.cpp<->notes.h) which the exported history does not contain, so both printed
"pair not found even with --pack-top-n=1000 (fixture drifted — cannot assert)" and failed.
A gate that can only pass on one laptop is not a gate.
Shape (a), not an allowlist and not a deleted assertion: mkCochangeFixture() now builds a
throwaway git repo with a scripted co-change history containing, by construction, one pair
of every kind the surprise predicate has to separate — direct #include, transitive 2-hop,
reverse, cross-directory bare-name (-I) include, two uncoupled dependency-capable pairs, and
a dependency-INCAPABLE .sh side. One co-change wave touches every file, so every pair has
identical together=/deg= support and the only variable between arms is the #include
predicate itself. It needs no history but its own: identical verdict on a fresh clone, a
shallow CI checkout, and the author's machine.
Gate-health items this closes:
- the POSITIVE control was picked from live top-30 output, so it could silently stop being
a control. The fixture's positive pairs are uncoupled by construction, and one of them
(alpha.cpp<->beta.h) is the same .cpp/.h shape as the negatives minus the include line,
so a single edited fixture line flips it.
- the §P9.1 -I-residue arm was vacuous on this corpus ("nothing to assert"). It now
hard-asserts against bench/probe.cpp <-> src/leaf.h.
Mutation-tested, 5/5 mutants killed: break the include chain (2 arms red), give the positive
pair a real include path (1 red), break the bare-name include (1 red), remove the
dep-incapable side (2 red), drop the waves below the 3-commit support floor (9 red).
The live corpus is still swept — §A9.3's uncapped no-leak sweep wants a big real corpus — but
every live arm is now presence-guarded: absent row -> SKIP naming the missing precondition,
present row -> the same hard assert as before. Two of the three live arms are demonstrably
present on this checkout and still assert.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ms onto an in-gate fixture
CI CONSEQUENCE: 7 of this gate's ~303 arms red on any fresh clone, so every CI leg failed.
Neither red was about paging:
--mentions=main page_verb's (C) seam check needs >= 6 rows and has_more= needs >= 4. The
published repo has exactly 3 docs naming `main`, so 2 arms red on a fact
about how many READMEs mention a symbol.
--stray-content a fresh clone has one branch and nothing stray -> ZERO rows, 5 arms red.
Worse, CI checks out SHALLOW, and this verb's own header documents that a
shallow clone makes EVERY ref unanalysable (v="unknown") — so no real repo
could have asserted here either.
Shape (a), not a precondition-skip: the paging contract (limit windows, offset advances,
has_more terminates) is a property of the CODE, not of the corpus, so the honest fix is to
give those two arms a corpus that supplies rows by construction rather than to stop asserting.
mkPagingFixture() builds one throwaway tree — 8 markdown docs naming one symbol, plus 8
branches each authoring lines HEAD does not have — carrying its own full history, so it
asserts identically on a fresh clone, a shallow CI checkout, and the author's machine. The
14 assertions per verb are unchanged; only the corpus supplying the rows moves.
run() gains a PAGE_CORPUS override defaulting to $ROOT, so the other ~290 arms are untouched
and byte-for-byte unaffected. Both verbs still meet the LIVE corpus in section (K)'s
honoring-set loop.
Red-first proven in both directions: shrink the fixture to 3 docs -> exactly the 2 mentions
arms red again; shrink it to 1 stray branch -> exactly the 5 stray-content arms red again.
Present rows -> hard assert; absent rows -> the same failures the old gate emitted.
Suite effect: 296 PASS / 7 FAIL -> 303 PASS / 0 FAIL, exit 1 -> exit 0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d reason, they no longer fail the leg CI CONSEQUENCE: CI runs the whole suite twice per leg — once against the Release binary and once against the plain build-debugalerts binary. estchargecheck (#14) and qualitystalecheck (arms 7 and 8c) assert a DEGRADED_PATH_ALERT, which Release compiles out via NDEBUG, and both took the FAIL branch when they could not see it. So the Release half of every leg was unconditionally red and could never be trusted — the 2026-07-27 trap inverted: a leg that always fails proves exactly as little as one that always passes. The two-flavour design already says which leg owns this duty ("if you add a degrade path, it is the PLAIN run that proves it"), so under NDEBUG these arms must SKIP with the missing precondition named, not fail. What they must NEVER do is pass silently. The skip is gated on TWO independent readings, because either alone can lie: 1. an unrelated, already-gated degrade path (--since=notadate -> "[math degraded]"). If it is silent too, alerts are unobservable globally rather than this seam having broken. 2. --version's build-type token — versioncheck's source of truth, set by CMakeLists from CMAKE_BUILD_TYPE. Release/RelWithDebInfo/MinSizeRel define NDEBUG; nothing else does. Only when both agree is the skip honest. Alert observable but this seam silent -> FAIL (regression). Both silent on a dev/asan flavour -> FAIL (the binary contradicts its own version string). Red-first proven in both directions, 4 mutants: - Release + flavour forced non-NDEBUG -> both gates FAIL, not skip (the skip really is gated on the flavour reading, not unconditional) - plain + alerts forced unobservable -> qualitystale FAILs naming build type "dev" - plain + the #14 seam forced silent -> estcharge FAILs on a build that CAN see alerts - unmutated: plain asserts (estcharge 123 PASS / 0 SKIP-degrade, qualitystale 37 PASS), Release skips (estcharge 112 PASS + 1 named SKIP, qualitystale 31 PASS + 2 named SKIPs); all four runs exit 0. Swept the rest of the suite for the same class empirically — a full 317-gate parallel run against the Release binary — and these two are the only degrade-alert gates that red under NDEBUG. Also: the O1/O2 fixture repos set a git user.email, and ripwirepubliccheck arm 5b allows only the synthetic domains x.com/t.com/test.com/example.com/example.invalid. Moved both fixtures onto example.invalid (caught by the same Release sweep). ci.yml's comment updated to record the new Release-leg semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r data; owned-object entry points unsuppressed leak:ts_query_* was broader than the file's stated purpose: it silenced the pre-N2 compiledQueryCache() leak in every gate that loads this file, which is why that real product leak could only surface in cachefuzzcheck's suppression-free ASan arms. leak:ts_parser_* / leak:ts_subtree_* had the same masking shape — ParserGuard/TreeGuard own every parser and tree, so a leak through those entry points is a product bug that must stay visible. Only the grammar/static entry points (ts_language_*, tree_sitter_*) match the file's charter and remain, as insurance. Verified on the Linux leg (Ubuntu 24.04 / clang 18.1.3 — the only G1 runtime with LeakSanitizer): with NO suppression file loaded at all, the flagless repo self-run, the fixture self-run, and the five CI heavy-verb gates are leak-clean; non-vacuity proven by a deliberate-leak probe (caught, exit 1) and LSAN_OPTIONS=verbosity=1 showing the tracer engage inside the real binary. With the trimmed file: the documented gate (exit 0, empty stderr), cppqualcheck, the det-gate 2-run diff (byte-identical), recallbufcheck and mcpreadloopcheck all green; macOS self-run green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…content instead of pinning HEAD~1
CI CONSEQUENCE: the Release CI leg went red on this gate mid-session, with three rows of
"est_tokens=<none> over the N budget with NO budget-floor-exceeded — silently over its own
ceiling". Nothing was wrong with --pr-context: a concurrent lane's tip commit touched only
lsan_suppressions.txt, a file --pr-context does not count, so `--pr-context=HEAD~1` returned
files="0" and a bundle with no est_tokens for the arm to read. Any PR whose last commit is
docs-only reproduces it — this is the same live-history class as O1/O2, and the same one this
file already documents ("after a run of doc-only commits HEAD~1's diff was ONE file"), which
is why (8f) was sent to a fixture repo.
These arms assert a content-INDEPENDENT property (the verb's own fit contract at every
budget), so they never needed a fixed base — only a base with something in it. Walk back to
the newest ancestor whose changed set is non-empty, bounded at HEAD~12; if nothing in that
window changes a counted file, SKIP naming that, and note that (8f)'s fixture repo still
carries the budget-BINDS half. With a non-empty base every assertion is byte-for-byte the one
it replaced.
Caught only because the full sequential suite was re-run against a Release binary after the
tree moved; the earlier parallel run predated the offending commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe PR adds Linux, compiler, and filesystem portability, safer cache and regex handling, portable floating-point parsing, owned query resources, centralized hashing, and deterministic test coverage. ChangesPortability, runtime safety, and validation
Estimated code review effort: 4 (Complex) | ~60 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Around line 71-73: Update all three actions/checkout@v4 steps identified by
their fetch-depth: 0 configuration to set persist-credentials: false, preserving
full-history checkout behavior and changing no other workflow logic.
In `@src/ingest.cpp`:
- Around line 559-575: Replace the path-only shape validation in src/ingest.cpp
lines 559-575 with a descriptor-backed cache reader that opens once using
O_NONBLOCK, fstats the descriptor, requires S_ISREG, and reads through that same
descriptor. Update the cache-loading site in src/ingest.cpp lines 1153-1158 and
the corresponding site in src/quality.h lines 1116-1121 to use this reader,
removing validation followed by path-based reopening; preserve absent-file
misses and degraded handling for non-regular paths.
In `@src/search.h`:
- Line 1008: Update grepCollect to expose the regex refusal from
regexCompileError instead of returning only raw and isBudgetReached. Add or
propagate a refusal state/error string through grepCollect’s result, ensure the
regex && regexCompileError(pat) rejection populates it before returning, and
update direct callers such as grepHits to distinguish refusal from a true
negative.
In `@test/estchargecheck.sh`:
- Around line 38-40: Guard the directory change in estchargecheck.sh by making
the cd "$ROOT" operation fail immediately when ROOT is missing or inaccessible.
Preserve the existing BIN validation and ensure no subsequent flavour probe or
measurement runs from the wrong working directory.
In `@test/qualitystalecheck.sh`:
- Around line 235-238: Update the NDEBUG skip conditions in arm 7 at
test/qualitystalecheck.sh:235-238 and arm 8c at
test/qualitystalecheck.sh:309-312 to require both alerts_observable equals 0 and
ndebug_flavour equals 1. Preserve the existing skip reasons and failure behavior
when either reading indicates the alert path is observable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3e21c4f2-9278-448f-9084-d3c8e346045e
📒 Files selected for processing (33)
.github/workflows/ci.ymlCMakeLists.txtCONTRIBUTING.mdlsan_suppressions.txtsrc/charconvcompat.hsrc/infra/profileScope.hsrc/ingest.cppsrc/main.cppsrc/mcpindex.hsrc/quality.hsrc/search.htest/cachehashcheck.shtest/cachesplitcheck.shtest/churnjoincheck.shtest/clonecachecheck.shtest/cochangesurprisecheck.shtest/deckcheck_allowlist.txttest/estchargecheck.shtest/evictioncheck.shtest/floormarkcheck.shtest/g1configcheck.shtest/g1freshcheck.shtest/headsnapcachecheck.shtest/mcpeditmodecheck.shtest/pagingsweepcheck.shtest/portablecachecheck.shtest/prcontextcheck.shtest/qsnapcachecheck.shtest/qsnapprefetchcheck.shtest/qualitystalecheck.shtest/recallevalcheck.shtest/regexbombcheck.shtest/statgatecheck.sh
Comment on lines
+559
to
+575
| inline PathShape shapeOfPath( const std::string& path ) noexcept | ||
| { | ||
| struct stat st; | ||
| const bool isStatable = ::stat( path.c_str(), &st ) == 0; | ||
| return !isStatable ? PathShape::Absent : ( S_ISREG( st.st_mode ) ? PathShape::RegularFile : PathShape::Other ); | ||
| } | ||
|
|
||
| // The READ seam's use of it, named so loadCache reads as one decision instead of three lines of shape | ||
| // analysis. Absent stays silent (the ordinary cold-start miss); an odd shape is disclosed here, once, from | ||
| // the one site that knows the read is what got refused. | ||
| inline bool isReadableCacheBlob( const std::string& path ) noexcept | ||
| { | ||
| const PathShape shape = shapeOfPath( path ); | ||
| if( shape == PathShape::Other ) | ||
| DEGRADED_PATH_ALERT( "ingest: cache path is not a regular file (directory/device/fifo) — cache treated as corrupt (full reparse)" ); | ||
| return shape == PathShape::RegularFile; | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Make cache-file validation atomic with the read.
stat() and the later path-based open are separate operations. A local process can replace a verified regular file with a FIFO between them. fopen() or std::ifstream can then block instead of degrading to a cache miss.
Open the path once with O_NONBLOCK, call fstat() on that descriptor, require S_ISREG, and read through that same descriptor.
src/ingest.cpp#L559-L575: replace the path-onlystat()helper with descriptor-backed validation.src/ingest.cpp#L1153-L1158: use the descriptor-backed cache reader instead of validating then reopeningpath.src/quality.h#L1116-L1121: use the same descriptor-backed validation and read path.
📍 Affects 2 files
src/ingest.cpp#L559-L575(this comment)src/ingest.cpp#L1153-L1158src/quality.h#L1116-L1121
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/ingest.cpp` around lines 559 - 575, Replace the path-only shape
validation in src/ingest.cpp lines 559-575 with a descriptor-backed cache reader
that opens once using O_NONBLOCK, fstats the descriptor, requires S_ISREG, and
reads through that same descriptor. Update the cache-loading site in
src/ingest.cpp lines 1153-1158 and the corresponding site in src/quality.h lines
1116-1121 to use this reader, removing validation followed by path-based
reopening; preserve absent-file misses and degraded handling for non-regular
paths.
| // called (regexCompileError above). Kept as a belt-and-braces early REJECT for library/MCP callers that | ||
| // did not ask; each worker compiles its own copy so no std::regex object is shared. | ||
| if( regex ) { try { const std::regex probe( pat, std::regex::ECMAScript | std::regex::optimize ); (void)probe; } catch( ... ) { return {}; } } | ||
| if( regex && regexCompileError( pat ) ) return {}; // L5: the SAME verdict the CLI seam uses, incl. the platform-divergent escapes |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n --type=cpp -C3 '\bgrepCollect\s*\('
rg -n --type=cpp -C3 'struct[[:space:]]+GrepCollection'Repository: redhat-et/ripwire
Length of output: 155
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Tracked files matching search headers:"
git ls-files | rg '(^|/)search\.h$|search\.cpp$|search\.cc$|$' | head -100
echo
echo "grepCollect references:"
rg -n -C3 '\bgrepCollect\s*\(' . || true
echo
echo "GrepCollection references:"
rg -n -C3 'GrepCollection' . || true
echo
echo "regexCompileError references:"
rg -n -C3 'regexCompileError' . || true
echo
echo "DEGRADED_PATH_ALERT references:"
rg -n -C3 'DEGRADED_PATH_ALERT' . || trueRepository: redhat-et/ripwire
Length of output: 3474
🏁 Script executed:
#!/bin/bash
set -u
echo "== locate search.h =="
git ls-files | rg '(^|/)src/search\.h$|(^|/)search\.h$'
echo
echo "== locate grepCollect/GrepCollection/regexCompileError/DEGRADED_PATH_ALERT (all tracked files) =="
python3 - <<'PY'
import subprocess
files = subprocess.check_output(['git', 'ls-files'], text=True).splitlines()
symbols = ['grepCollect', 'GrepCollection', 'regexCompileError', 'DEGRADED_PATH_ALERT', 'emitGrepReport']
for sym in symbols:
print(f"\n--- {sym} ---")
for f in files:
try:
txt = open(f, errors='ignore').read()
except Exception:
continue
idx = -1
found = 0
while True:
idx = txt.find(sym, idx+1)
if idx < 0:
break
found += 1
if found <= 20:
line = txt[:idx].count('\n') + 1
lines = txt.splitlines()
lo, hi = max(1,line-3), min(len(lines), line+3)
print(f"{f}:{line}:")
for n in range(lo, hi+1):
print(f"{n:4}: {lines[n-1]}")
if found == 0:
print("(none)")
PY
echo
echo "== search around src/src/search.h if present =="
if [ -f src/search.h ]; then
wc -l src/search.h
sed -n '960,1025p' src/search.h | nl -ba -v960
fiRepository: redhat-et/ripwire
Length of output: 50382
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== src/search.h: 934-1020 =="
sed -n '934,1020p' src/search.h | nl -ba -v934
echo
echo "== src/main.cpp: 6577-6635 =="
sed -n '6577,6635p' src/main.cpp | nl -ba -v6577
echo
echo "== direct grepCollect textual context around source files only =="
python3 - <<'PY'
import re, subprocess
files = subprocess.check_output(['git','ls-files'], text=True).splitlines()
files = [f for f in files if f.endswith(('.h','.cpp','.cc))]
cands = []
for f in files:
txt = open(f, errors='ignore').read()
for idx in re.finditer(r'\bgrepCollect\s*\(', txt):
line = txt[:idx.start()].count('\n')+1
cands.append((f,line,txt.splitlines()[line-1].strip()))
for f,line,s in cands:
if s.startswith('grepCollect') or s.startswith('const GrepCollection'):
print(f'{f}:{line}: {s}')
PYRepository: redhat-et/ripwire
Length of output: 224
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== src/search.h: 934-1020 =="
sed -n '934,1020p' src/search.h | awk '{printf "%d: %s\n", NR+933, $0}'
echo
echo "== src/main.cpp: 6577-6635 =="
sed -n '6577,6635p' src/main.cpp | awk '{printf "%d: %s\n", NR+6576, $0}'
echo
echo "== direct grepCollect textual context around source files only =="
python3 - <<'PY'
import re, subprocess
files = subprocess.check_output(['git','ls-files'], text=True).splitlines()
files = [f for f in files if f.endswith(('.h','.cpp','.cc'))]
cands = []
for f in files:
txt = open(f, errors='ignore').read()
for idx in re.finditer(r'\bgrepCollect\s*\(', txt):
line = txt[:idx.start()].count('\n')+1
cands.append((f,line,txt.splitlines()[line-1].strip()))
for f,line,s in cands:
print(f'{f}:{line}: {s}')
PYRepository: redhat-et/ripwire
Length of output: 12709
Make regex refusals observable from grepCollect.
grepCollect discards regexCompileError() and only returns raw plus isBudgetReached, so direct callers such as grepHits() cannot tell a refused pattern from a true negative. Propagate the error string or add a refusal state before the belt-and-braces reject.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/search.h` at line 1008, Update grepCollect to expose the regex refusal
from regexCompileError instead of returning only raw and isBudgetReached. Add or
propagate a refusal state/error string through grepCollect’s result, ensure the
regex && regexCompileError(pat) rejection populates it before returning, and
update direct callers such as grepHits to distinguish refusal from a true
negative.
Comment on lines
38
to
40
| [ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first"; exit 2; } | ||
| cd "$ROOT" | ||
| echo "estchargecheck: BIN=$BIN" |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Guard the cd.
If $ROOT does not exist, the script continues in the current directory. The flavour probe then runs against a wrong or missing test/fixture, and every later arm measures the wrong tree.
🛠️ Proposed fix
-cd "$ROOT"
+cd "$ROOT" || { echo "cannot cd to $ROOT"; exit 2; }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| [ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first"; exit 2; } | |
| cd "$ROOT" | |
| echo "estchargecheck: BIN=$BIN" | |
| [ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first"; exit 2; } | |
| cd "$ROOT" || { echo "cannot cd to $ROOT"; exit 2; } | |
| echo "estchargecheck: BIN=$BIN" |
🧰 Tools
🪛 Shellcheck (0.11.0)
[warning] 39-39: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test/estchargecheck.sh` around lines 38 - 40, Guard the directory change in
estchargecheck.sh by making the cd "$ROOT" operation fail immediately when ROOT
is missing or inaccessible. Preserve the existing BIN validation and ensure no
subsequent flavour probe or measurement runs from the wrong working directory.
Source: Linters/SAST tools
Comment on lines
+235
to
+238
| elif [ "$ndebug_flavour" -eq 1 ]; then | ||
| skip "arm 7's degrade assertions (one [math degraded] alert naming the surviving sidecar and the git-HEAD fallback) — $DEGRADE_SKIP_WHY" | ||
| else | ||
| no "this binary compiles DEGRADED_PATH_ALERT out (Release/NDEBUG): arm 7's degrade assertion CANNOT be made — run the PLAIN build" | ||
| no "arm 7: no DEGRADED_PATH_ALERT is observable, yet --version reports build type \"$BUILD_FLAVOUR\", which does NOT define NDEBUG — the alert seam regressed on a flavour that should be able to see it" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
The NDEBUG skip drops the alerts_observable reading in both arms. The probe at lines 188-196 produces two readings, and the header at lines 182-184 requires both to agree before a skip is honest. Both skip branches test ndebug_flavour alone, so a Release build that can observe the unrelated degrade path but lost its own alert reports SKIP instead of FAIL, and $DEGRADE_SKIP_WHY then prints a false claim.
test/qualitystalecheck.sh#L235-L238: change the condition to[ "$alerts_observable" -eq 0 ] && [ "$ndebug_flavour" -eq 1 ], matchingtest/estchargecheck.shline 689.test/qualitystalecheck.sh#L309-L312: apply the same two-reading condition to arm 8c.
📍 Affects 1 file
test/qualitystalecheck.sh#L235-L238(this comment)test/qualitystalecheck.sh#L309-L312
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test/qualitystalecheck.sh` around lines 235 - 238, Update the NDEBUG skip
conditions in arm 7 at test/qualitystalecheck.sh:235-238 and arm 8c at
test/qualitystalecheck.sh:309-312 to require both alerts_observable equals 0 and
ndebug_flavour equals 1. Preserve the existing skip reasons and failure behavior
when either reading indicates the alert path is observable.
…FAIL text, not just its name PR #1's first CI round (run 30732976779) produced eight bare lines of the form "FAIL absorb gate (X.sh failed)" across macos-14 and ubuntu-24.04 and nothing else: the loop runs every gate with `>/dev/null 2>&1`, so the gate's own FAIL row, its compile log, its diff and any sanitizer report were all discarded. Every one of the eight had to be re-derived by hand from the source. That is the single most expensive property of the current CI. On failure the loop now re-runs ONLY the gate that failed with output captured, and echoes a 25-line window prefixed with the gate name, plus the exact rerun command. Cost: one extra run of the few gates that already failed. Exit semantics are unchanged — `no` still sets fail=1. WINDOW, not head: a gate's failing arm is usually not in its first 25 lines (lintrulescheck emits ~35 PASS rows before its later arms), so a plain head would show 25 PASSes and hide the failure. The window starts at the first line matching the repo's own marker (` FAIL …` / `FAILURES ABOVE`, anchored and case-SENSITIVE so a PASS row whose prose contains "failure" cannot hijack it), falling back to shapes a gate that never reached its own reporting prints (error:/fatal/Sanitizer/…), and finally to the TAIL for a gate that died silently. Proved live, both branches, by breaking two gates at the front of the list in a real run: FAIL absorb gate (archcheck.sh failed, rc=1) [archcheck] FAIL deliberate late-arm failure — THIS is the line the window must start at [archcheck] expected 'x', got 'y' [archcheck] (window of 25 from line 31 of 32; rerun: RIPWIRE_BIN=… bash test/archcheck.sh) FAIL absorb gate (lintcheck.sh failed, rc=1) [lintcheck] (no failure-shaped line — last 25 of 0 shown; rerun: …) i.e. it skipped 30 decoy PASS rows to land on the FAIL, and named the silent-death case honestly. It then caught a real one unaided (gateexitcheck, fixed in a later commit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 6, 2026
… to the top hit Harvest-B card C4, reproduced first on the operator's own 157-document agent-memory directory: `--recall --top-k=6 --max-tokens=5000` returned `total=157 shown=1 truncated=1`, and the one document emitted was the round's own prompt file. On a synthetic corpus of one long on-topic document plus five 150-byte on-topic ones, `--max-tokens` of 2000, 5000 and 8000 ALL returned `shown=1`: 750 bytes of matching prose dropped from a 20,000-byte budget, and tripling the budget bought more of document #1 and never a second document. The count of documents an agent got back was decided by the SIZE OF THE TOP HIT, not by the budget it asked for. Cause: buildRecall's budget loop was greedy first-fit — walk the ranking, give each document all the room left, `break` the moment one had to be truncated. A loop that decides one document's slice from `payload.size()` alone cannot know five more are waiting behind it, so the three phases are now separate: LOAD every candidate, ALLOCATE, then EMIT. The allocation is water-filling (recallServedPrefix + waterFillRecallShares): serve the longest rank PREFIX that can be given a readable slice each (kRecallShareFloorBytes, 900 B ~ 350 tokens — codesight's measured wiki-article size, the mechanism this card was harvested from), then divide the rest equally, letting every document that needs LESS than its share take only what it needs and hand the surplus back. Five 150-byte notes therefore cost 750 bytes, not five equal shares: spreading never wastes budget on documents that are already whole. Ranking still decides WHO; the budget decides HOW MANY. The opposite failure — dividing by --top-k so a single-hit query gets one stub — is what the prefix rule prevents, and a budget too small for even one readable slice still serves the top document down to kRecallMinBodyBytes, exactly as the old loop did. DISCLOSURE (H9, a ceiling applied is a ceiling named): the header gains `share_bytes=N`, the per-document ceiling, present only when it BOUND a document — so a run that fits its budget is byte-identical, and the --token-budget refusal branch strips it for the same reason it already strips over_ceiling= (both describe the artifact it withheld). MEASURED, same command, same budget: 157-doc memory dir 1 -> 6 documents in FEWER bytes (10,536 -> 9,302); this repo's PLAN_*/DESIGN_* corpus 1 -> 6. GATE FIRST: test/recallbudgetcheck.sh §8 (12 arms), written and shown RED on the pre-fix binary — 8.1 shown=1 of 6 matched, 8.2 zero of five small documents emitted, 8.5 no share_bytes= disclosure, 8.7b MCP door identical, 8.9 shown=1/1/1 across 2K/5K/8K. Family is DERIVED, not hard-coded: arm 8.7 greps src/ for the recallFor call sites and fails if a third front door appears that the gate does not exercise. Arms 8.3/8.4/8.5b/ 8.6/8.8 were green pre-fix and pin the opposite failures (ceiling still bounds, no over-spread on a single hit, silence when nothing was bound, inert at a non-binding budget, determinism). §8 lives in the existing recall budget gate rather than a new file: manifestcheck.sh cross-checks the gate count against docs/EVALS.md, which another thread owns this round. Also here, because --quality-delta named them: emitRecallBudgeted's two attribute strips folded into withoutHeaderAttr (complexity 14 -> below its baseline), formatRecallHeader's four copies of one optional-attribute ternary folded into optionalRecallAttr (verbosity 70 -> 68). --quality-delta gating=0; the two short-horizon-churn rows on this file are acked with a reason (recall.h's hot lines are what the fix has to edit). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dazKind
pushed a commit
to dazKind/ripwire
that referenced
this pull request
Sep 7, 2026
…RED (k1,b), not just defaults RED first (CLAUDE.md non-negotiable redhat-et#1), against the code as it stands on this commit: (1) FAIL RIPWIRE_BM25_K1=8 RIPWIRE_BM25_B=0.1 produced byte-identical output to the default — env override is not wired in k1/b are duplicated `constexpr double k1 = 1.5, b = 0.75;` declarations (src/lexical.h ~721 and ~1040), and the pruned branch's MaxScore early-termination bound (cap1, ~785) derives its cap from its own local copy. Nothing today lets k1/b be configured, so nothing today can prove the bound stays a genuine upper bound once they are. This gate pre-registers that proof for the A4 unification about to land. Manually verified during authoring (not automated -- both states require hand-editing src/lexical.h, which this commit does not touch): wiring RIPWIRE_BM25_K1/RIPWIRE_BM25_B into a shared Bm25Params/resolveBm25Params(), then deliberately making the bound (bm25ImpactBound) ignore the resolved params and keep reading the unconfigured default -- exactly the "changed k1/b without updating the bound" defect this gate exists to catch -- produced a measurable divergence between pruned and RIPWIRE_NO_PRUNE=1 (exhaustive) output at several configured (k1,b) corners (e.g. k1=6.0,b=0.05): real candidates were being wrongly discarded. Routing the bound through the same resolved params restored byte-identical parity across the full clamp range tried (0.1..10.0 x 0.0..1.0, nine points). Check (5) in the gate is that same parity sweep, kept permanently. Also covers: the env value clamps to its documented range rather than passing through raw (2), malformed env input degrades instead of crashing (3), the pruning-active premise so the sweep isn't vacuous (4), determinism at a configured (k1,b) (6), and xml well-formedness (7). Added to test/regression.sh's absorbed-gate loop in this commit (test/manifestcheck.sh requires it); docs/EVALS.md's three gate-count citations move 541 -> 542 alongside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dazKind
pushed a commit
to dazKind/ripwire
that referenced
this pull request
Sep 7, 2026
…e the named file's own decl/def partner TWO defects the round-C head-to-head found in --situ, both about a relationship the tool already knows and does not say. F2 — the composed co-change zero conflated two different facts. --cochange refuses loudly on an empty window (commits="0" window="18mo@HEAD", exit 1) and --rank-by=churn stamps "(no churn evidence)", but every surface that COMPOSES co-change rendered a bare (0) — the CLI report adding "(none, or no git history)", which means neither "none found" nor "none exists". Non-negotiable redhat-et#3 forbids exactly that. The window and the commit count now travel ON SituationFacts, so the four composing surfaces cannot disclose it four ways or three of them forget it: * --situ [3] — window="…" commits="…" on the header, and the empty case now says which of the two zeros it is * situational_awareness (MCP) — cochange_window / cochange_commits beside `forgotten`, the surface where an empty array reads most like an answer * --handoff <heuristic> — cochange_window= / cochange_commits= (appended AFTER n=/candidates=/capped=; `<heuristic n="…"` is a shape gates read positionally) * --pr-context <cochange> — had commits= and no window to read it against F3 — TAKEN FROM COCOINDEX. `--situ=db/wal_manager.cc` on RocksDB spent 3,104 B and never named the gold `db/wal_manager.h` anywhere, while an embedding search returned it as result redhat-et#1 in 55 bytes; same shape on table/get_context.cc; S2 recall 2/14. Not a retrieval failure: section [1] ranks by dependent-symbol count, which surfaces the biggest test files and can NEVER surface the header, because a header does not depend on the source that implements it. It is a different relationship, and one ripwire already holds. declDefPartners() — another file defining symbols under the SAME (scope, name) identity. Stated that way it is not a C++ special case: it covers a .h/.cc pair, an ObjC .h/.m, a C# partial class, a .pyi stub and a .d.ts, without reading one file extension or comparing one basename. The guard is a MAJORITY test rather than a number pulled from the air — 2*shared >= min(symbols(a), symbols(b)) — so two large files sharing one common free-function name are refused, and the row publishes `shared` so the reader audits it. No embedding mode was added and none is coming (G3). The blast-radius counts are unchanged: the partner is printed above them as its own labelled fact, never merged into them. Measured after: --situ=db/wal_manager.cc -> db/wal_manager.h (10 shared) as row one; table/get_context.cc -> table/get_context.h (16); db/version_set.cc -> db/version_set.h (138). Gates: sincewindowcheck arm 4 now 18 assertions, all GREEN, including a NEGATIVE CONTROL that two 13-symbol files sharing one name are refused (the majority test is live, not decorative) and an arm that a git repo with no commits stamps window="18mo" WITHOUT @Head rather than claiming an anchor it does not have. deckcheck: six foreign tool flags quoted in lane C2's head-to-head write-up (gortex --dataset/--detach/ --entry-point/--kind, uv --python, rg --sort) allowlisted — that gate was red on this lane's base. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dazKind
pushed a commit
to dazKind/ripwire
that referenced
this pull request
Sep 7, 2026
…e competitor ideas landed
Six lanes. The comparison existed to find where ripwire LOSES and take the ideas;
the scoreboard was never the deliverable, and it is the part that did not survive.
MEASURED, AND NOT PUBLISHED AS A CLAIM. Against a random-rank placebo on 30
questions over rocksdb @ 0e2801ac -- a corpus none of the arms' authors wrote --
ripwire won 6 (tokens-to-correct-answer; 9-10-11 on plain recall, a worse cut)
against a pre-registered majority of 16. Per the registration's own stop
condition, no ranking claim is published from this round. The cause is the useful
part: on "what changed recently" a random path list at ripwire's own matched
31.7 KB budget names 21 of 30 gold files to ripwire's 2, because 31.7 KB of paths
covers 76% of the corpus. Without the placebo arm this round publishes a recall
number that is mostly budget.
IDEAS TAKEN FROM THE ARMS THAT BEAT US.
gortex -- mine the repository's OWN history, not the last N months of wall
clock. It mined 9,854 co-change edges where --cochange returned zero
and exited 1 claiming "git unavailable / no history", false on both
counts. Now --cochange returns 1,246 pairs and --hotspots 674 ranked
on the same pin. Two miners were already correct and their comments
already argued for anchoring: the argument had been won twice and
never propagated.
cocoindex -- when the question names a file, that file's own decl/def partner is
usually the answer. It returned db/wal_manager.h as result redhat-et#1 in
55 B where --situ missed it entirely in 3,104 B. Folded without its
architecture: no embeddings, G3 intact.
spotbugs/graphify -- confidence as an axis separate from severity, landed as
prov="split" and rendered dashed rather than faded, because a faded
solid line reads as a distant edge rather than an uncertain one.
AFTER THE FIXES, ON THE FROZEN QUESTIONS: completed 6/30 -> 9/30, gold named
26 -> 29, S2 recall 2/14 -> 5/14. Still short of 16. An improved tool that still
fails its own stop condition, published as one.
FOUND BY PROBING RATHER THAN BY THE CARDS: precise=3 on ripwire's own map with no
--scip anywhere; --dead-code returning count="0" on src/ where an existing
--graph-query composition returns 489; two of this round's own decline reasons
wrong; and ripwire's published retrieval numbers excluded by the same known-item
rule that excludes gortex's.
LINEAGE moves 38 -> 41 folded and 239 -> 237 surveyed: two promotions, one new
row, and one staged promotion DECLINED because it would have pointed at the same
lesson and the same artifact as an existing row, inflating a published count.
Battery 556 gates, 554 pass, 2 environmental skips, 0 fail, on a frozen tree with
the binary rebuilt to match HEAD. Determinism x2, xmllint, formatcheck,
ASan/UBSan/LSan clean on 13 paths, --quality-delta regressions=0 gating=0 acked=0.
The gate loop is untouched and manifestcheck still derives 542.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
… — lane 3 The ranking was computed and then discarded. --recall scored a document's markdown sections, sorted them by relevance, re-sorted them into document order, and let the byte budget prefix-cut the result — so on any document larger than its share the answer was unreachable at EVERY ceiling. Measured on a 616 KB docs/COMMANDS.md, the --field-affinity section at line 3570 of 4755 was ranked #1, selected, and served at none of 1500/4000/12000/40000 tokens; what arrived was the table of contents, because the table of contents is at the front of the file. Sections are now own-prose units that tile, EMIT spends the allowance in rank order whole-units-at-a-time, and the disclosure reports what was actually served ([sections: S of R selected (N in doc) ... dropped_by_budget=D], with lines= naming exactly the ranges present in the body). The ancestor rule is the part that took three tries. Neither proposed fix worked: a df<=S/2 term margin is measurably inert (ancestors survive on genuine own-prose terms), and "keep only if the own-prose score is positive" is inert by proof, since BM25's idf is positive for every n<=S. Both are ADMISSION rules and the defect was RANKING: those units were correctly admitted, they just outranked the answer on evidence held by subsections nested inside them. The rank key now scales a unit's score by its own-prose evidence over the strongest own-prose evidence in its subtree. Measured, no metric regressing: answer reachability 5/14 -> 9/14 on a frozen 1.88 MB corpus; natural-language-first 43 -> 79 at 1600 tokens; query-term coverage +58/+56/+40 at 4000/8000/16000. Per-document budget monotonicity holds over 1,380 ceilings, 0 shrinks; cross-document it does NOT, and every surface that claimed otherwise is now qualified rather than quietly wrong. Also in this landing: the gate written before the fix (recallpassagecheck, whose two expected-red arms an independently built fix turned green); two existing gates that could not see the defects they were written for; the directory-as-knowledge-base pattern documented with its two real conditions (dump to .md, keep ## headings) after an earlier draft overclaimed which extensions count as documents; skill routing for the dumped-output moment at zero description budget; the context-mode survey recorded in LINEAGE with the three pillars declined and why; and the capture scrub — 21 internal planning headings that reached public main because scrub() hid the document's NAME and published its HEADINGS, fixed at the root with a shared leak predicate and swept by docscommandscheck arm (E) and ripwirepubliccheck arm 2b. CI 34301707466: 26/26 green, including the Release/gcc leg whose doctorcheck fixture no-op reddened the previous run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
…rminism arm — name the writer, fix it, watch for the next CI run 34298150602 (macOS plain, shard 2/2): tokenbudgetcheck's `--for` determinism arm got est_tokens 3949 then 3947. That is at="<sha>" vs at="<sha>+dirty" — six bytes, two tokens at 2.5 B/tok — and the dirty bit is `git status --porcelain`, which src/gitstamp.h runs from ANY crawl root inside the checkout. Issue #71 blamed test/prbudgetcheck.sh's appends to "$ROOT/src/mod4.cpp"; that gate rebinds ROOT to its own mktemp fixture at :48, has never written outside it, and src/mod4.cpp does not exist. The writer was test/gateexitcheck.sh: its arm (A) copied fixtures to test/gateexitfix/.gateprobe.*.sh, untracked, for ~20 ms each — observed live under a 50 Hz git-status watcher — three worker slots from tokenbudgetcheck in that shard's run order. - test/pargates.py: the shared-tree tripwire, sibling of the shared-binary one. Baseline `git status --porcelain --untracked-files=all` before the run, sample it every 0.25 s (PARGATES_DIRT_POLL_SEC) with --no-optional-locks, and report every NEW line with the gates in flight when it was seen; a hit fails the run. It is a sampler and says so ("a floor, not a total"); a non-git root reports tree_writes=unwatched and is never turned red. - test/pargatescheck.sh: three functional arms on the real script — WRITER (seen red: rc!=0, path and gate named, although the gate itself passed), CONTROL (same gate, mktemp: clean), UNWATCHED (non-git corpus: disarmed, rc 0) — plus two static pins. - test/gateexitcheck.sh: (A)'s probe copies go to a mktemp dir (the fixtures never read $0); (E)'s must sit beside the real gate, so .gitignore now names `.gateprobe.*` and new arm (F) proves the hiding on the checkout's own .gitignore text seeded into a scratch repo, with an un-ignored twin as the mutation control, plus git check-ignore in situ and a per-probe status check in (E). Seen red with the ignore line removed. - test/tokenbudgetcheck.sh: #1, #14 and #15's pair crawl a private copy of src/ under mktemp, outside every repository, so no concurrent gate can dirty it. Assertion set unchanged. - test/prbudgetcheck.sh: the fixture root is spelled $FIX instead of rebinding ROOT, so the grep that produced the misdiagnosis finds nothing. Behaviour identical, 33 PASS. - CONTRIBUTING.md: the rule, where a gate author meets it. Not fixed here, disclosed: a dozen other gates run a stamped verb twice on the real checkout (adaptivecheck, cochangesurprisecheck, docanchorcheck, forcalibfactscheck, forlenscheck, forrankordercheck, packtaskcheck, attrvocabcheck, …). With the writer gone and the tripwire armed they are safe; the tripwire, not a per-gate copy, is the class fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
…— disclose it `ripwire . --for=Q --detail=30 --max-tokens=300` printed `max_tokens="300"` on the <ctx> root and delivered `est_tokens="2640"` — 8.8x the ceiling it named — with no over_ceiling= anywhere. Reported by YogevKr as #61, reproduced verbatim on 2026-09-09, the third of three honesty-class issues from that reporter. The flag was never inert: it shapes the body count, and `<bodies capped="1">` disclosed THAT cut honestly right beside the silence. What --max-tokens does not do is bound the document — verbs_for.h turns it into detailBodyBudget, a budget over the BODIES alone, so the header, signatures, legend and symbol table are never charged against it. METHODOLOGY §9 #6 states the defect in one sentence: "a ceiling attribute names the ceiling actually applied." DISCLOSURE, NOT ENFORCEMENT — and the argument, because this is a §9 decision. §9 #2: "when a ceiling would cut something above the cliff, compress first, move prose into attributes second, and if it still does not fit, exceed the ceiling with over_ceiling="1" rather than drop the row that would have terminated the search." Thirty small functions totalling ~3.4K tokens, complete, ARE the terminating answer. A rung that trimmed them to fit 300 tokens would make the tool worse and would still have been perfectly honest, so no rung was added. The ladder was applied first and came up empty on this shape: the reporter measured --legend=compact at 8,761 B / est 2,974, still far past 300. What was missing is the VERDICT the default map has computed since §F5 (main.cpp, maxTokensFit.isOverCeiling) — the "as the default map does" the reporter's own Expected behavior cites. THE OPEN QUESTION, decided explicitly: --max-tokens is NOT converged onto the whole bundle. The tool already has a flag that means "bound the document" (--token-budget, the reporter's verified workaround: est 902 at --token-budget=1000) and one that means "shape the map" (--max-tokens). Convergence is allowed under one-step-smart-defaults and new-tool-no-compat-debt, but it is a DEFAULT change whose effect is to CUT rows, which is the direction §9 #1 says the data does not support — the budget flag "trims a ranking from the tail and cannot know which row would have ended the search". The two meanings stay, and both are now documented on --max-tokens and --detail in --help. WHAT CHANGED - src/verbs_for.h forLensOverCeiling: the XML dialect's over-ceiling predicate, a free function beside its JSON twin (forLensJsonOverCeiling) for the same reason that one is — runForLens is one of the largest bodies in the file and this is a contract of its own. Same rule, same unit, same attribute budget_tokens already answers to (packtask.h F2): over_ceiling="1" whenever est_tokens exceeds a ceiling the root states. Reused, not re-derived. The label is decided INSIDE the existing est_tokens fixpoint, so its own 17 bytes and its legend clause are charged — a disclosure that made est_tokens wrong is the one place that error matters most. - src/serialize.h: the legend sentence for a max_tokens-keyed verdict, beside the budget_tokens one, plus overCeilingLegendFor so no surface picks the wording by hand. Keyed on which ceilings the ROOT CARRIES, not on which one fired, so the choice cannot be made stale by the fixpoint it rides inside. A budget-only document is byte-identical to before. - src/cli.h / docs/COMMANDS.md (regenerated): --max-tokens and --detail=N now state what the flag bounds, what it does not, and which flag bounds the document. MEASURED, this repo's fixture of 30 tiny TS functions: --max-tokens=300 goes 9,647 -> 9,714 B (the 67-byte disclosure) and now reads `max_tokens="300" est_tokens="3589" over_ceiling="1"`; --max-tokens=8000 fits and is byte-identical at 14,968 B with no attribute. GATE FIRST (non-negotiable #1). test/formaxtokenscheck.sh, written before the code and red on the pre-fix binary at 6 of 9 band points plus the named reproduction. Arms: (A) the biconditional est_tokens > ceiling <=> over_ceiling="1", swept across a band that provably contains BOTH states, with non-vacuity asserted on each half; (B) no --max-tokens => no ceiling attribute and no verdict; (C) est_tokens re-derived exactly from the delivered bytes at both rates, at every point, which is what catches an emitted-but-uncharged disclosure; (D) the legend defines the attribute against the ceiling actually on that root; (E) --token-budget alone and beside --max-tokens; (F) determinism; (G) five mutation controls, each re-running the SAME judge over a deliberately corrupted real document. Presence tests read the root element through an XML parser, never a text grep — the legend DEFINES over_ceiling= (verbs_for.h:719). test/shapingflagcheck.sh (A) re-pinned 20 -> 21 --max-tokens read sites: the new site is a DISCLOSURE of a budget --for --detail=N already honored, so kShapingVerbs' honorsMaxTokens column is unchanged. Gate count 567 -> 568 in all 8 published sites. test/printf_parity.manifest: the `help` hash re-pinned, 3f0237b0... -> 9b6add07..., because this commit edits --help on purpose. printffmtparitycheck is a byte-parity fence over 12 labels and `help` is one of them; its FAIL text ("any file whose conversion moved these bytes must be reverted") is written for the printf -> std::print conversion case, where the whole claim is that the bytes must NOT move. There is no conversion here — the two new --help blocks are the change, so the bytes moved deliberately and the pin is what needs updating, not the prose. Reviewed rather than rubber-stamped: the manifest diff is EXACTLY ONE LINE, `help` STDOUT; the other 11 labels are byte-identical and help's own STDERR hash is unchanged (still e3b0c442..., the empty-string digest). The baseline it moves FROM is main's, re-read on this rebase tip rather than carried from the branch's pre-rebase value — a hash computed against an older main would pin bytes no binary in this history produces. Pre-flighted against the WIDENED fence lane/stdprint-conversion brings (40 labels, purely additive — it rewrites none of the 12): 39 pass, 1 fail, and the one is `help`. So this change moves exactly one label out of forty, and the one-line invariant survives that landing whichever order the two lanes take. --test-gate does not name printffmtparitycheck for a help edit: it routes by call edges, and script-to-binary is not one — it discloses that as script_gates_unmodelled= rather than implying the list is complete. If you touch printUsage, run that gate by hand. Local, on this rebase tip after a --clean-first rebuild (five landings under this branch, and an incremental build across a branch switch can produce a binary that exists at no single commit — CLAUDE.md documents that failure at length): formaxtokenscheck, printffmtparity, manifest, shapingflag, docscommands, deck, deckclaim, readmedrift, readmeexample, fordisclosure, w3fixbudget and legendcoverage all green; determinism + golden + xmllint clean; --quality-delta exit 0 (its one gating row, short-horizon-churn churn=self on runForLens, acked with its reason). Every pinned number here was RE-DERIVED on this tip — the gate count from regression.sh's own loop, the read-site count from the gate's own grep expression, the parity hash from the rebuilt binary — none carried forward from the pre-rebase branch. docs/COMMANDS.md is regenerated from that same binary. Closes #61 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
…— disclose it `ripwire . --for=Q --detail=30 --max-tokens=300` printed `max_tokens="300"` on the <ctx> root and delivered `est_tokens="2640"` — 8.8x the ceiling it named — with no over_ceiling= anywhere. Reported by YogevKr as #61, reproduced verbatim on 2026-09-09, the third of three honesty-class issues from that reporter. The flag was never inert: it shapes the body count, and `<bodies capped="1">` disclosed THAT cut honestly right beside the silence. What --max-tokens does not do is bound the document — verbs_for.h turns it into detailBodyBudget, a budget over the BODIES alone, so the header, signatures, legend and symbol table are never charged against it. METHODOLOGY §9 #6 states the defect in one sentence: "a ceiling attribute names the ceiling actually applied." DISCLOSURE, NOT ENFORCEMENT — and the argument, because this is a §9 decision. §9 #2: "when a ceiling would cut something above the cliff, compress first, move prose into attributes second, and if it still does not fit, exceed the ceiling with over_ceiling="1" rather than drop the row that would have terminated the search." Thirty small functions totalling ~3.4K tokens, complete, ARE the terminating answer. A rung that trimmed them to fit 300 tokens would make the tool worse and would still have been perfectly honest, so no rung was added. The ladder was applied first and came up empty on this shape: the reporter measured --legend=compact at 8,761 B / est 2,974, still far past 300. What was missing is the VERDICT the default map has computed since §F5 (main.cpp, maxTokensFit.isOverCeiling) — the "as the default map does" the reporter's own Expected behavior cites. THE OPEN QUESTION, decided explicitly: --max-tokens is NOT converged onto the whole bundle. The tool already has a flag that means "bound the document" (--token-budget, the reporter's verified workaround: est 902 at --token-budget=1000) and one that means "shape the map" (--max-tokens). Convergence is allowed under one-step-smart-defaults and new-tool-no-compat-debt, but it is a DEFAULT change whose effect is to CUT rows, which is the direction §9 #1 says the data does not support — the budget flag "trims a ranking from the tail and cannot know which row would have ended the search". The two meanings stay, and both are now documented on --max-tokens and --detail in --help. WHAT CHANGED - src/verbs_for.h forLensOverCeiling: the XML dialect's over-ceiling predicate, a free function beside its JSON twin (forLensJsonOverCeiling) for the same reason that one is — runForLens is one of the largest bodies in the file and this is a contract of its own. Same rule, same unit, same attribute budget_tokens already answers to (packtask.h F2): over_ceiling="1" whenever est_tokens exceeds a ceiling the root states. Reused, not re-derived. The label is decided INSIDE the existing est_tokens fixpoint, so its own 17 bytes and its legend clause are charged — a disclosure that made est_tokens wrong is the one place that error matters most. - src/serialize.h: the legend sentence for a max_tokens-keyed verdict, beside the budget_tokens one, plus overCeilingLegendFor so no surface picks the wording by hand. Keyed on which ceilings the ROOT CARRIES, not on which one fired, so the choice cannot be made stale by the fixpoint it rides inside. A budget-only document is byte-identical to before. - src/cli.h / docs/COMMANDS.md (regenerated): --max-tokens and --detail=N now state what the flag bounds, what it does not, and which flag bounds the document. MEASURED, this repo's fixture of 30 tiny TS functions: --max-tokens=300 goes 9,647 -> 9,714 B (the 67-byte disclosure) and now reads `max_tokens="300" est_tokens="3589" over_ceiling="1"`; --max-tokens=8000 fits and is byte-identical at 14,968 B with no attribute. GATE FIRST (non-negotiable #1). test/formaxtokenscheck.sh, written before the code and red on the pre-fix binary at 6 of 9 band points plus the named reproduction. Arms: (A) the biconditional est_tokens > ceiling <=> over_ceiling="1", swept across a band that provably contains BOTH states, with non-vacuity asserted on each half; (B) no --max-tokens => no ceiling attribute and no verdict; (C) est_tokens re-derived exactly from the delivered bytes at both rates, at every point, which is what catches an emitted-but-uncharged disclosure; (D) the legend defines the attribute against the ceiling actually on that root; (E) --token-budget alone and beside --max-tokens; (F) determinism; (G) five mutation controls, each re-running the SAME judge over a deliberately corrupted real document. Presence tests read the root element through an XML parser, never a text grep — the legend DEFINES over_ceiling= (verbs_for.h:719). test/shapingflagcheck.sh (A) re-pinned 20 -> 21 --max-tokens read sites: the new site is a DISCLOSURE of a budget --for --detail=N already honored, so kShapingVerbs' honorsMaxTokens column is unchanged. Gate count 568 -> 569 in all 8 published sites, DERIVED from regression.sh's own loop on this tip. Worth recording how that number was nearly wrong: the previous revision of this branch published 568, and the lane that landed underneath it (#85, tgrep) had itself bumped 567 -> 568. So README.md, docs/EVALS.md and the deck did NOT conflict on rebase — the two lanes had written IDENTICAL text — and git auto-merged them to a tree publishing 568 while this branch's own loop names 569. That is the silent-merge failure this repo keeps re-learning, and the only thing that catches it is re-deriving from the loop rather than trusting a clean merge. A gate-count bump that merges CLEAN onto a main which has landed a gate since you branched is the failure, not the success; regression.sh conflicted loudly and the three prose sites did not. test/printf_parity.manifest: the `help` hash re-pinned, 72b76cbc... -> d9b77634..., because this commit edits --help on purpose. printffmtparitycheck is a byte-parity fence over 12 labels and `help` is one of them; its FAIL text ("any file whose conversion moved these bytes must be reverted") is written for the printf -> std::print conversion case, where the whole claim is that the bytes must NOT move. There is no conversion here — the two new --help blocks are the change, so the bytes moved deliberately and the pin is what needs updating, not the prose. Reviewed rather than rubber-stamped: the manifest diff is EXACTLY ONE LINE, `help` STDOUT; the other 11 labels are byte-identical and help's own STDERR hash is unchanged (still e3b0c442..., the empty-string digest). The baseline it moves FROM is main's, re-read on this rebase tip rather than carried from the branch's pre-rebase value — a hash computed against an older main would pin bytes no binary in this history produces. Pre-flighted against the WIDENED fence lane/stdprint-conversion brings (40 labels, purely additive — it rewrites none of the 12): 39 pass, 1 fail, and the one is `help`. So this change moves exactly one label out of forty, and the one-line invariant survives that landing whichever order the two lanes take. --test-gate does not name printffmtparitycheck for a help edit: it routes by call edges, and script-to-binary is not one — it discloses that as script_gates_unmodelled= rather than implying the list is complete. If you touch printUsage, run that gate by hand. Local, on this rebase tip after a --clean-first rebuild (five landings under this branch, and an incremental build across a branch switch can produce a binary that exists at no single commit — CLAUDE.md documents that failure at length): formaxtokenscheck, printffmtparity, manifest, shapingflag, docscommands, deck, deckclaim, readmedrift, readmeexample, fordisclosure, w3fixbudget and legendcoverage all green; determinism + golden + xmllint clean; --quality-delta exit 0 (its one gating row, short-horizon-churn churn=self on runForLens, acked with its reason). Every pinned number here was RE-DERIVED on this tip — the gate count from regression.sh's own loop, the read-site count from the gate's own grep expression, the parity hash from the rebuilt binary — none carried forward from the pre-rebase branch. docs/COMMANDS.md is regenerated from that same binary. Closes #61 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 9, 2026
…— disclose it `ripwire . --for=Q --detail=30 --max-tokens=300` printed `max_tokens="300"` on the <ctx> root and delivered `est_tokens="2640"` — 8.8x the ceiling it named — with no over_ceiling= anywhere. Reported by YogevKr as #61, reproduced verbatim on 2026-09-09, the third of three honesty-class issues from that reporter. The flag was never inert: it shapes the body count, and `<bodies capped="1">` disclosed THAT cut honestly right beside the silence. What --max-tokens does not do is bound the document — verbs_for.h turns it into detailBodyBudget, a budget over the BODIES alone, so the header, signatures, legend and symbol table are never charged against it. METHODOLOGY §9 #6 states the defect in one sentence: "a ceiling attribute names the ceiling actually applied." DISCLOSURE, NOT ENFORCEMENT — and the argument, because this is a §9 decision. §9 #2: "when a ceiling would cut something above the cliff, compress first, move prose into attributes second, and if it still does not fit, exceed the ceiling with over_ceiling="1" rather than drop the row that would have terminated the search." Thirty small functions totalling ~3.4K tokens, complete, ARE the terminating answer. A rung that trimmed them to fit 300 tokens would make the tool worse and would still have been perfectly honest, so no rung was added. The ladder was applied first and came up empty on this shape: the reporter measured --legend=compact at 8,761 B / est 2,974, still far past 300. What was missing is the VERDICT the default map has computed since §F5 (main.cpp, maxTokensFit.isOverCeiling) — the "as the default map does" the reporter's own Expected behavior cites. THE OPEN QUESTION, decided explicitly: --max-tokens is NOT converged onto the whole bundle. The tool already has a flag that means "bound the document" (--token-budget, the reporter's verified workaround: est 902 at --token-budget=1000) and one that means "shape the map" (--max-tokens). Convergence is allowed under one-step-smart-defaults and new-tool-no-compat-debt, but it is a DEFAULT change whose effect is to CUT rows, which is the direction §9 #1 says the data does not support — the budget flag "trims a ranking from the tail and cannot know which row would have ended the search". The two meanings stay, and both are now documented on --max-tokens and --detail in --help. WHAT CHANGED - src/verbs_for.h forLensOverCeiling: the XML dialect's over-ceiling predicate, a free function beside its JSON twin (forLensJsonOverCeiling) for the same reason that one is — runForLens is one of the largest bodies in the file and this is a contract of its own. Same rule, same unit, same attribute budget_tokens already answers to (packtask.h F2): over_ceiling="1" whenever est_tokens exceeds a ceiling the root states. Reused, not re-derived. The label is decided INSIDE the existing est_tokens fixpoint, so its own 17 bytes and its legend clause are charged — a disclosure that made est_tokens wrong is the one place that error matters most. - src/serialize.h: the legend sentence for a max_tokens-keyed verdict, beside the budget_tokens one, plus overCeilingLegendFor so no surface picks the wording by hand. Keyed on which ceilings the ROOT CARRIES, not on which one fired, so the choice cannot be made stale by the fixpoint it rides inside. A budget-only document is byte-identical to before. - src/cli.h / docs/COMMANDS.md (regenerated): --max-tokens and --detail=N now state what the flag bounds, what it does not, and which flag bounds the document. MEASURED, this repo's fixture of 30 tiny TS functions: --max-tokens=300 goes 9,647 -> 9,714 B (the 67-byte disclosure) and now reads `max_tokens="300" est_tokens="3589" over_ceiling="1"`; --max-tokens=8000 fits and is byte-identical at 14,968 B with no attribute. GATE FIRST (non-negotiable #1). test/formaxtokenscheck.sh, written before the code and red on the pre-fix binary at 6 of 9 band points plus the named reproduction. Arms: (A) the biconditional est_tokens > ceiling <=> over_ceiling="1", swept across a band that provably contains BOTH states, with non-vacuity asserted on each half; (B) no --max-tokens => no ceiling attribute and no verdict; (C) est_tokens re-derived exactly from the delivered bytes at both rates, at every point, which is what catches an emitted-but-uncharged disclosure; (D) the legend defines the attribute against the ceiling actually on that root; (E) --token-budget alone and beside --max-tokens; (F) determinism; (G) five mutation controls, each re-running the SAME judge over a deliberately corrupted real document. Presence tests read the root element through an XML parser, never a text grep — the legend DEFINES over_ceiling= (verbs_for.h:719). test/shapingflagcheck.sh (A) re-pinned 20 -> 21 --max-tokens read sites: the new site is a DISCLOSURE of a budget --for --detail=N already honored, so kShapingVerbs' honorsMaxTokens column is unchanged. Gate count 568 -> 569 in all 8 published sites, DERIVED from regression.sh's own loop on this tip. Worth recording how that number was nearly wrong: the previous revision of this branch published 568, and the lane that landed underneath it (#85, tgrep) had itself bumped 567 -> 568. So README.md, docs/EVALS.md and the deck did NOT conflict on rebase — the two lanes had written IDENTICAL text — and git auto-merged them to a tree publishing 568 while this branch's own loop names 569. That is the silent-merge failure this repo keeps re-learning, and the only thing that catches it is re-deriving from the loop rather than trusting a clean merge. A gate-count bump that merges CLEAN onto a main which has landed a gate since you branched is the failure, not the success; regression.sh conflicted loudly and the three prose sites did not. test/printf_parity.manifest: the `help` hash re-pinned, 72b76cbc... -> d9b77634..., because this commit edits --help on purpose. printffmtparitycheck is a byte-parity fence over 12 labels and `help` is one of them; its FAIL text ("any file whose conversion moved these bytes must be reverted") is written for the printf -> std::print conversion case, where the whole claim is that the bytes must NOT move. There is no conversion here — the two new --help blocks are the change, so the bytes moved deliberately and the pin is what needs updating, not the prose. Reviewed rather than rubber-stamped: the manifest diff is EXACTLY ONE LINE, `help` STDOUT; the other 11 labels are byte-identical and help's own STDERR hash is unchanged (still e3b0c442..., the empty-string digest). The baseline it moves FROM is main's, re-read on this rebase tip rather than carried from the branch's pre-rebase value — a hash computed against an older main would pin bytes no binary in this history produces. Pre-flighted against the WIDENED fence lane/stdprint-conversion brings (40 labels, purely additive — it rewrites none of the 12): 39 pass, 1 fail, and the one is `help`. So this change moves exactly one label out of forty, and the one-line invariant survives that landing whichever order the two lanes take. --test-gate does not name printffmtparitycheck for a help edit: it routes by call edges, and script-to-binary is not one — it discloses that as script_gates_unmodelled= rather than implying the list is complete. If you touch printUsage, run that gate by hand. Local, on this rebase tip after a --clean-first rebuild (five landings under this branch, and an incremental build across a branch switch can produce a binary that exists at no single commit — CLAUDE.md documents that failure at length): formaxtokenscheck, printffmtparity, manifest, shapingflag, docscommands, deck, deckclaim, readmedrift, readmeexample, fordisclosure, w3fixbudget and legendcoverage all green; determinism + golden + xmllint clean; --quality-delta exit 0 (its one gating row, short-horizon-churn churn=self on runForLens, acked with its reason). Every pinned number here was RE-DERIVED on this tip — the gate count from regression.sh's own loop, the read-site count from the gate's own grep expression, the parity hash from the rebuilt binary — none carried forward from the pre-rebase branch. docs/COMMANDS.md is regenerated from that same binary. Closes #61 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 10, 2026
…osure one call away `ripwire --help` printed 185,540 B / ~46,000 tokens in one flat document: every flag's purpose AND every caveat that flag has ever earned, interleaved. On a tool whose pitch is token efficiency that was the most expensive thing it could print. The fix is a SPLIT, not a cull. Nothing was deleted; 133 opening lines were relocated. --help one plain-speech line per flag ~4,473 tokens (was ~46,385) --help=--FLAG that flag's whole entry, every disclosure intact --help=SECTION one family at full detail --help=all the entire former catalog WHAT DECIDED THE DESIGN. Not length — reachability. test/legendcostcheck.sh records that an outside evaluation (callstack/agent-device #2400) measured ripwire spending ~10% MORE tokens than grep-and-read, traced it to the per-call legend, and that the fix (--legend=compact) was ALREADY documented in --help: "buried four lines into a schema description, so nobody extracted it." That sentence sits at line 1465 of 1597. An evaluator holding the exact question did not find the answer they were standing on. The defect was not that the text was long; it was that a fact inside it had no address. So the flags got addresses. Measured here first: 156 entries, median 6 lines, mean 10, max 46; the top 30 carried 53% of the bytes and 61 opening lines did not stand alone, so a "keep line 1, drop the rest" cull would have shipped 61 broken fragments. Every rewritten line pushes its old text down one row rather than replacing it, so tier 2 gained the summaries (+7.4%) and lost nothing. ALSO: a missing <dir> no longer answers with the catalog. It printed all 185 KB to stderr — ~46,000 tokens for the likeliest first-run typo — while an unknown flag cost 32 bytes. Now 362 B that names the four things you probably meant. MACHINE CONSUMERS READ TIER 2. 52 invocations across 41 gates plus docs/docs_commands_build.py now call --help=all: an assertion about DOCUMENTED CONTENT must not pass or fail on where a sentence sits. clicheck keeps arm (e) on plain --help because that arm tests what a user types. GATE FIRST (non-negotiable #1): test/helpbudgetcheck.sh was written and observed RED before any of this. It holds tier 1 to a 7,000-token ceiling AND asserts tier 2 still carries every row — opposite failure modes, so fixing one by breaking the other cannot pass. Arm (I) is deliberately narrower than its name and says so: it proves a summary is not truncated (open bracket, dangling punctuation, over-width), not that it reads well. An earlier draft flagged five complete sentences ending in prepositions and would have made the prose worse to satisfy the checker. RE-PINS, both legitimate and both checked before re-pinning: - test/printffmtparitycheck.sh: the red set was confirmed to be exactly {help} (11 PASS, 1 FAIL, d9b77634 -> f066b2b8) before UPDATE_GOLDEN. The corpus gains `help_all` and `help_one` so both tiers and one addressed entry are fenced; pinning only one would let the other move silently. - docs/COMMANDS.md regenerated from a --clean-first build, never hand-edited. Gate count 570 -> 571 across its seven spellings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 10, 2026
…ourth is refuted with numbers METHODOLOGY §9 #3 says never cut silently and #6 says a ceiling attribute names the ceiling ACTUALLY applied. Three caps did neither. Each fix is disclosed only when the cap actually bit — the pr_converged shape (src/prconverge.h): silence means untruncated, presence means truncated. G4 pays nothing otherwise. 1. kGrepMatchedLineMaxBytes (512 B, src/search.h) cut the matched line of EVERY --grep and --verify hit with no ellipsis and no attribute, on the only content those answers carry: a 512 B source line and a truncated 50 KB minified line printed byte-identical payloads. The row now carries line_bytes="N", the WHOLE line's byte length. The NUMBER rather than a bare capped="1" because it is what decides the reader's next move — the row already carries the deterministic follow-up (p=/l=, the root's next=), so what was missing is whether following it is worth a read. NO ellipsis here, deliberately, and this is where it differs from (2): a grep payload is raw file bytes by contract — the boolean --and/--not filter reads the same line and the --at= follow-up is expected to reproduce it — so the fact goes in an attribute (§9 #4) rather than into the bytes. lineBytes joins grepGroupByFile's fold key: two long lines can share a 512 B prefix and differ in true length, and folding those would print one line_bytes= for sites it does not describe; untruncated rows carry 0, so the key is byte-identical wherever the cap did not fire. The MCP grep verb is the third caller of grepEnrich and serves NO matched text at all, so it has no cut to disclose — asserted rather than assumed, arm A5. 2. cleanSig's kMaxSig (240 B, src/serialize.h) hard-broke every emitted signature — --pack-signatures, --for's <sigs>, <calls> callee rows, --lego — mid-token, with no marker. It now goes through truncateUtf8WithEllipsis, the tool's ONE truncator, exactly as the three other signature cuts (kForTailSigBytes, kForCapTailSigBytes, packtask.h's tail sig) already do. In-band here because a signature is already a RENDERING, not raw bytes: the body is stripped and whitespace runs collapse, so matching its three siblings is what consistency means. The visible prefix is unchanged at 240 B; only the "…" is new. The loop collects one byte past the cap so the shared truncator can do its own codepoint back-off, and a separate flag carries the fact because the trailing-space trim can pull a cut string back under the cap and a cut that trims back under is still a cut. 3. A DEFAULT --for enforced kForPayloadBudgetBytes (7500 B) on every run and named no ceiling: budget_tokens= rode only an EXPLICIT --token-budget, so a trimmed default bundle disclosed THAT it was cut (<sigs shown= total= capped="1">) while the number that cut it appeared nowhere. Compare --pack-task, whose default lands on its root as budget_tokens="6000" — same class of ceiling, two honesty outcomes. The <ctx> root now carries budget_bytes="7500", and the JSON dialect ",budget_bytes":7500. THE UNIT IS BYTES, and that is the decision the finding asked for. What the default applies IS a byte constant — bundleBudget is kForPayloadBudgetBytes verbatim and the ladder compares rendered bytes against it. A token spelling would have to divide by a rate, and the two rates in play disagree on purpose: est_tokens prices at kBytesPerTokenDefault (2.50) while a ceiling is SIZED at the conservative kMinBytesPerToken. Any token number printed here would be one no ladder ever applied, which is the exact failure §9 #6 names. DEFAULT REGIME ONLY: the explicit regime already names the caller's own ceiling, so exactly one of the two rides a trimmed bundle — never both, never neither (gate arms C8/C10). It rides the ROOT and is spliced AFTER the sigs render, with dropped_positive=/bundle=: the first cut of this fix put it on the <sigs> open tag with the ladder's own per-run share as the value, which is more precise and wrong here — forbudgetmonotoncheck pins the sig section byte-identical between the default ceiling and any explicit ceiling above it, and a per-run number breaks that identity while every served row stays the same. The late splice leaves the ladder's input untouched, so <sigs> is byte-for-byte what it was. The clause defining the attribute is spliced on the same condition (kForOverCeilingLegend's precedent) so an untrimmed bundle pays for neither. packSignatures gains cappedOut, the ladder verdict its JSON twin has always returned, so the caller reads a boolean instead of re-parsing rendered bytes. THE MCP `for` VERB CARRIES IT TOO (§P8: one element name, one attribute order, both surfaces). That dialect is budgeted by default the same way (mcpverbs.h forBudgetBytes) and had the same silence; fixing the CLI alone would have replaced one honesty gap with a disagreement between two surfaces an agent can reach for the same answer. Same two conditions, same splice point, same sentence — arm C14 pins the sentence byte-identical across the two. 4. kSliceRdMaxIter (64, src/slice.h) is REFUTED, not fixed: it is unreachable, so it cuts nothing and has nothing to disclose. Structurally, the reaching-definition lattice has no cross-slot flow — a use reads its own slot, a def assigns it a singleton (SliceRdWalker::unit) — so each slot's loop transfer is X -> const ∪ (X if a path passes through), whose ascending chain from `entry` stabilises on the SECOND round regardless of program shape. Measured: instrumenting the converged break and running --slice over 3,026 symbols of this tree's own src/ gives 4,528 loop fixpoints, max iter = 1 (508 at 0, 4,020 at 1, none higher); an adversarial C fixture (while > for > do-while nesting around a 40-case fallthrough switch with continue/break) gives max iter = 1, and an adversarial Python one (nested loops with try/except/finally and continue) the same. The bound fires at iter >= 63. Its comment already says it "only guards a broken lattice", and that is what it is: an XML attribute that can provably never appear is ungateable by construction (non-negotiable #1) and would be decoration. DEGRADED_PATH_ALERT stays the right instrument for it (guardrail #4). GATE FIRST (non-negotiable #1): test/capdisclosurecheck.sh was written and shown RED before any src/ edit. Every arm asserts three things, because a disclosure gate that only greps for its own attribute proves nothing: CROSSING (the fixture's emitted payload really is shorter than the source, or the element really is capped), DISCLOSURE, and SILENCE (the same verb on an uncrossed fixture carries nothing). MUTATION CONTROL, run against a binary built from the parent commit: every DISCLOSURE arm fails (A2, A4, B2, B4, C2, C5, C7) while every CROSSING and SILENCE arm still passes. Gate count 573 -> 574, derived from THIS tip's own `for _g in` loop after the rebase (main landed optremarkshotcheck while this lane was green; the union of the two loops is the number), across all eight published sites. Only test/regression.sh conflicted — README/EVALS/deck auto-merged clean at the stale 573, which is the failure mode, so all three were reset from origin/main and re-bumped from the merged loop. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 10, 2026
…ourth is refuted with numbers METHODOLOGY §9 #3 says never cut silently and #6 says a ceiling attribute names the ceiling ACTUALLY applied. Three caps did neither. Each fix is disclosed only when the cap actually bit — the pr_converged shape (src/prconverge.h): silence means untruncated, presence means truncated. G4 pays nothing otherwise. 1. kGrepMatchedLineMaxBytes (512 B, src/search.h) cut the matched line of EVERY --grep and --verify hit with no ellipsis and no attribute, on the only content those answers carry: a 512 B source line and a truncated 50 KB minified line printed byte-identical payloads. The row now carries line_bytes="N", the WHOLE line's byte length. The NUMBER rather than a bare capped="1" because it is what decides the reader's next move — the row already carries the deterministic follow-up (p=/l=, the root's next=), so what was missing is whether following it is worth a read. NO ellipsis here, deliberately, and this is where it differs from (2): a grep payload is raw file bytes by contract — the boolean --and/--not filter reads the same line and the --at= follow-up is expected to reproduce it — so the fact goes in an attribute (§9 #4) rather than into the bytes. lineBytes joins grepGroupByFile's fold key: two long lines can share a 512 B prefix and differ in true length, and folding those would print one line_bytes= for sites it does not describe; untruncated rows carry 0, so the key is byte-identical wherever the cap did not fire. The MCP grep verb is the third caller of grepEnrich and serves NO matched text at all, so it has no cut to disclose — asserted rather than assumed, arm A5. 2. cleanSig's kMaxSig (240 B, src/serialize.h) hard-broke every emitted signature — --pack-signatures, --for's <sigs>, <calls> callee rows, --lego — mid-token, with no marker. It now goes through truncateUtf8WithEllipsis, the tool's ONE truncator, exactly as the three other signature cuts (kForTailSigBytes, kForCapTailSigBytes, packtask.h's tail sig) already do. In-band here because a signature is already a RENDERING, not raw bytes: the body is stripped and whitespace runs collapse, so matching its three siblings is what consistency means. The visible prefix is unchanged at 240 B; only the "…" is new. The loop collects one byte past the cap so the shared truncator can do its own codepoint back-off, and a separate flag carries the fact because the trailing-space trim can pull a cut string back under the cap and a cut that trims back under is still a cut. 3. A DEFAULT --for enforced kForPayloadBudgetBytes (7500 B) on every run and named no ceiling: budget_tokens= rode only an EXPLICIT --token-budget, so a trimmed default bundle disclosed THAT it was cut (<sigs shown= total= capped="1">) while the number that cut it appeared nowhere. Compare --pack-task, whose default lands on its root as budget_tokens="6000" — same class of ceiling, two honesty outcomes. The <ctx> root now carries budget_bytes="7500", and the JSON dialect ",budget_bytes":7500. THE UNIT IS BYTES, and that is the decision the finding asked for. What the default applies IS a byte constant — bundleBudget is kForPayloadBudgetBytes verbatim and the ladder compares rendered bytes against it. A token spelling would have to divide by a rate, and the two rates in play disagree on purpose: est_tokens prices at kBytesPerTokenDefault (2.50) while a ceiling is SIZED at the conservative kMinBytesPerToken. Any token number printed here would be one no ladder ever applied, which is the exact failure §9 #6 names. DEFAULT REGIME ONLY: the explicit regime already names the caller's own ceiling, so exactly one of the two rides a trimmed bundle — never both, never neither (gate arms C8/C10). It rides the ROOT and is spliced AFTER the sigs render, with dropped_positive=/bundle=: the first cut of this fix put it on the <sigs> open tag with the ladder's own per-run share as the value, which is more precise and wrong here — forbudgetmonotoncheck pins the sig section byte-identical between the default ceiling and any explicit ceiling above it, and a per-run number breaks that identity while every served row stays the same. The late splice leaves the ladder's input untouched, so <sigs> is byte-for-byte what it was. The clause defining the attribute is spliced on the same condition (kForOverCeilingLegend's precedent) so an untrimmed bundle pays for neither. packSignatures gains cappedOut, the ladder verdict its JSON twin has always returned, so the caller reads a boolean instead of re-parsing rendered bytes. THE MCP `for` VERB CARRIES IT TOO (§P8: one element name, one attribute order, both surfaces). That dialect is budgeted by default the same way (mcpverbs.h forBudgetBytes) and had the same silence; fixing the CLI alone would have replaced one honesty gap with a disagreement between two surfaces an agent can reach for the same answer. Same two conditions, same splice point, same sentence — arm C14 pins the sentence byte-identical across the two. 4. kSliceRdMaxIter (64, src/slice.h) is REFUTED, not fixed: it is unreachable, so it cuts nothing and has nothing to disclose. Structurally, the reaching-definition lattice has no cross-slot flow — a use reads its own slot, a def assigns it a singleton (SliceRdWalker::unit) — so each slot's loop transfer is X -> const ∪ (X if a path passes through), whose ascending chain from `entry` stabilises on the SECOND round regardless of program shape. Measured: instrumenting the converged break and running --slice over 3,026 symbols of this tree's own src/ gives 4,528 loop fixpoints, max iter = 1 (508 at 0, 4,020 at 1, none higher); an adversarial C fixture (while > for > do-while nesting around a 40-case fallthrough switch with continue/break) gives max iter = 1, and an adversarial Python one (nested loops with try/except/finally and continue) the same. The bound fires at iter >= 63. Its comment already says it "only guards a broken lattice", and that is what it is: an XML attribute that can provably never appear is ungateable by construction (non-negotiable #1) and would be decoration. DEGRADED_PATH_ALERT stays the right instrument for it (guardrail #4). GATE FIRST (non-negotiable #1): test/capdisclosurecheck.sh was written and shown RED before any src/ edit. Every arm asserts three things, because a disclosure gate that only greps for its own attribute proves nothing: CROSSING (the fixture's emitted payload really is shorter than the source, or the element really is capped), DISCLOSURE, and SILENCE (the same verb on an uncrossed fixture carries nothing). MUTATION CONTROL, run against a binary built from the parent commit: every DISCLOSURE arm fails (A2, A4, B2, B4, C2, C5, C7) while every CROSSING and SILENCE arm still passes. RE-PIN: test/docdemotegolden_for.xml 5505 -> 5517 B and test/docdemotegolden_noroute.xml 9556 -> 9568 B, +12 B each = FOUR three-byte U+2026 markers from (2), plus the est_tokens recount. Verified before re-pinning: with est_tokens= normalised and the four new ellipses deleted, live and previous goldens are byte-identical on both fixtures — no ranking, demotion, route or budget byte moved, and neither root is capped, so (3) is absent from both. That fixture is where (2) shows up at all: a markdown section's "signature" is prose, so four of those rows were over the 240 B cap and had been cut invisibly for the life of the golden. The cause is recorded in the gate header beside the five earlier re-pins. test/printf_parity.manifest needed NO re-pin: none of its 14 pinned verbs reaches a changed emitter. Gate count 574 -> 575, derived from THIS tip's own `for _g in` loop after the LAST of three rebases (main landed optremarkshotcheck and then callsrankordercheck while this lane was in flight; the union of the two loops is the number, every time). Only test/regression.sh ever conflicted — README/EVALS/deck auto-merged clean at the stale number on all three, which is exactly the failure mode, so all three files were reset from origin/main and re-bumped from the merged loop each time. The last rebase crossed a lane that changed serialize.h and verbs_for.h too (the <calls> rank-order fix), so it was followed by a --clean-first rebuild before anything was re-measured. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 10, 2026
…at can move CodeRabbit on #113: the /daily badge is rendered live (served as SVG with a 4-hour cache; today it reads "#1 · C++ · Repository Of The Day", with no date), so an alt text pinning "#1" and "2026-09-07" can drift from what a sighted reader sees. The alt now says what the image is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lennix1337
pushed a commit
to lennix1337/ripwire
that referenced
this pull request
Sep 11, 2026
…line stops being a second heading Vertical space at the top of the README is at a premium. Two changes, both on lines a first-time visitor sees without scrolling: - The Trendshift "redhat-et#1 C++ Repository of the Day" badge (2026-09-07) sits beside the paddle-out wave on one centered line. Both are inline images in one paragraph, with no floats: on a narrow screen they stack, centered, and the <details> below never wraps around them. GitHub's renderer keeps align="middle" and the badge's width/height (checked via POST /markdown). - The tagline under the H1 goes from `##` to bold: one heading block and its underline less, and the H1 no longer shares the top with a second large heading. An H1 is already the largest text GitHub renders (inline style and <font size> are stripped), so this is the lever that makes it read bigger. No link in the repo targets the removed anchor. The wave's alt text now carries both of its lines. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 11, 2026
andriytyurnikov
pushed a commit
to andriytyurnikov/ripwire
that referenced
this pull request
Sep 11, 2026
…Node is 30% of it Records what a `sample` of the FIXED binary on a cold llvm-project map says about the walks this lane did NOT convert, so the next lane starts from a number instead of a judgment call. `ts_node_child_iterator_next` is still the redhat-et#1 busy leaf at 14.26% (127 453 busy samples, a 12 s window of a 46 s run at load 36 — a window share, not a whole-run share). Attributing each of its samples to the nearest non-tree-sitter caller: captureTagsFacts 7 304, bindsVisitNode 5 614, qualifierOf 2 629, enclosingScopeOf 1 040, cc_isCountableLocalDecl 676, cc_walk 531. bindsVisitNode is the one class-2 site this lane deliberately left indexed (it needs the INDEX for `ts_node_field_name_for_child`), and it is 30% of that leaf's cost — so the semantic change its fix needs, the cursor's O(1) `ts_tree_cursor_current_field_name`, has a measured reason to be worth its own gate. Comment-only; no arm, no count, no binary change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
to andriytyurnikov/ripwire
that referenced
this pull request
Sep 12, 2026
…cs/QUALITY_DELTA_CATALOG.md The "What --quality-delta catches" placeholder (slide 29) is filled from the catalog (commit 553dbadb on docs/quality-delta-catalog-2026-09-11). Every example commit is on main 766913d; the catalog itself has not landed. Four cards, varied in kind and in what the finding led to: - redhat-et#1 new-clone-of-reused-helper: vendoredPathPrefixes re-spelled registeredMacroNames, a helper with three callers, while the round that built the per-kind dials was under way; both now call mergeBuiltinsWithConfig. - redhat-et#6 nesting (and complexity 141 -> 178): the inline per-root scan became std::any_of in a named function. - redhat-et#7 dead-code: the predicates a rewrite orphaned were deleted, not acked. The card also names the one the kind missed, namesFileNotKept, whose only caller was itself dead (IP-8). - redhat-et#3 duplication, Type-3: the shared digit-run locator became headerFieldDigits, and the card says the detector still gates a 63-token residue, which the catalog classes as a false positive (IP-4). Each card's speaker notes cite the catalog section, the base -> before -> after commits, the verbatim row(s), and a one-line reproduce command copied from the catalog's Reproduce block. All five catalog examples considered for the slide (redhat-et#1, redhat-et#3, redhat-et#6, redhat-et#7, redhat-et#8) were re-run on 2026-09-11 with a built_from=766913d02 build, and every row, gating count and exit code matched the catalog. Why four and not six: the render shows a two-column pane holds 10 lines of 39 columns at 8 pt (a 40-column line wraps). With 5 or 6 entries each pane drops to about 3 lines, too few for a before and an after. So redhat-et#5 and redhat-et#8 are left out, and redhat-et#3 carries the detector-limits card. Layout changes in qdExamples, each forced by the render: - The kind box is sized to its text. new-clone-of-reused-helper is 2.38 in at 11 pt and wrapped out of the old fixed 2.2 in box. - Card padding and pane gap are tighter, and two-column code is 8 pt (was 8.5), so a pane fits 10 x 39 instead of about 9 x 36. - Entries may carry an optional notes array, which is validated and printed under that card's line in the notes. - The kicker for a filled slide no longer reads as a placeholder. The placeholder path is unchanged. Snippets are the catalog's excerpts, trimmed to fit. "…" marks elided code, long lines are wrapped, and a comment line stands for code the catalog says was elided or deleted. The notes say so. Gates on this tree: deckcheck ALL PASS (0 bad values, 0 stale), deckclaimcheck ALL PASS (179 long flags, 34 slides), readmedriftcheck ALL PASS. All three pass with the built_from=766913d02 build, and again with the scratchpad main_096e3544 build. The pptx and PDF are regenerated (pptxgenjs 4.0.1, LibreOffice impress_pdf_Export, 34 pages). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joyful-ii-V-I
added a commit
that referenced
this pull request
Sep 13, 2026
…llowance, and the route= code shipped with no reading anywhere Two defects on the same header, both found by CI at 76f5747. 1. RUNG ZERO FIRED ON THE ALLOWANCE, NOT ON THE CEILING. Under an explicit --token-budget, --for prices its header against the delivered-byte allowance (budget x 2.36 x 1.15) and, when the document does not fit, first drops the three explanatory legend clauses whose loss costs no fact (confidence, tail, the sc= rule). kCeilingFirstEntryTolerance exists for the RESIDUAL a lens cannot trim -- a first signature is not divisible -- but it also gated that first, free drop, so a document 1..15% over its budget that still carried all three clauses shipped over_ceiling="1" with every one of them riding. On CI: fornotesbudgetcheck's 1640 rung measured est_tokens=1755 (+8%) and forrootlegendcheck's arm2 800 rung 831. The free drop is now tried against the number the ROOT PROMISES (budget x 2.36) and only what remains after it is judged by the tolerance; same fit test otherwise. The 1640 rung now reads 1402 with its seven rows intact and the dropped-clause note present, the 800 rung 787. No tolerance moved and no ceiling moved: the rung that costs no unique information simply runs first. estchargecheck's late-label sweep control is re-anchored on that measurement. The control exists to prove the sweep CROSSES the band the defect lives in (a root saying over_ceiling="1" while the document still fits the allowance); with the free drop running earlier that residual band sits at 780..810 on this corpus, not inside 1200..1500, so the arm swept 700..3300 step 10 on the new binary and starts at 760. Left at 1200 it would have been inert, which the gate's own comment forbids. 2. route= WAS A CODE WITH NO READING. Row 6 turned route= from prose into a code, and the reading was then trimmed to the sc= rule alone to buy ceiling headroom -- which left an agent holding route="subtoken+body:declined" with nothing in the document to read it by. legendcoveragecheck (A) named it: two NEW undefined first-screen attributes (for-auto | ctx@route, for-budgeted | ctx@route). The fix is the legend, not the baseline: test/legendcoverage_baseline.txt is a ratchet its own header says may only be edited DOWNWARD, and an upward edit there would have recorded this lane's own regression as accepted debt. The reading is now two clauses, one spelling shared by both dialects (graphlegend.h kForIdRouteLegend 29 B, kForRouteCodeLegend 54 B; verbs_for.h's kForCompactLegendRoute is now an alias of the second, so a code cannot acquire two readings). sc= rides every answer, because every scoped row carries sc=. The route= code rides only the answers whose root carries route= -- forRouteAttrPresent, read off the BUILT root open rather than re-derived, on the CLI lens and the MCP for twin alike. Both stay ceiling-droppable and both stay exempt from the signature-trim charge, with the emitted bytes (not the constant's) subtracted from the sig ledger in each dialect. The dropped-clause note names sc=/route= again. The fuller reading of each code -- what :broad and :declined(word;carriers,defs) weigh, where the anchors are -- still lives once in the help text's no-route entry. Measured. The 83 B of reading is absorbed by every ceiling rung (forbudgetmonotoncheck #1/#5, fornotesbudgetcheck, forrootlegendcheck, compactlegendcheck, fillordercheck all ALL PASS, which is what defect 1 bought). It crosses forrankordercheck's 4% ratchet on the two SMALLEST frozen-fixture bundles only, where 83 B is 4.7%: ffifix/'geometry area of a shape' 2050 -> 2148 (+4.78%), hostilefix/'call a native function from python' 2074 -> 2172 (+4.73%). Those two bases follow the output under the gate's own q5 precedent (a TOOL change crossing 4% on a git-less frozen copy, so the whole delta is the tool's); the other seven fixtures and all ten repository queries stay at their registered bases inside the ratchet. Goldens: docdemotegolden_for.xml 5,383 -> 5,437 B (est_tokens 2153 -> 2175), ONE identified change -- the route clause, which this conceptual query's root carries. Verified before re-pinning: with that clause and est_tokens= normalised out, live and previous goldens are byte-identical. docdemotegolden_noroute.xml is the control and did not move: no route=, no clause. Red/green: legendcoveragecheck (A) reds with 2 new lines against the previous build and ALL PASS here; fornotesbudgetcheck/forrootlegendcheck/estchargecheck red on CI at 76f5747, ALL PASS here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kapoorsunny
pushed a commit
to kapoorsunny/ripwire
that referenced
this pull request
Sep 14, 2026
… beaten The Trendshift badge pointed at the DAILY C++ feed. The repository took redhat-et#1 C++ for the WEEK, so the badge was understating its own result: a reader saw the narrower ranking while the wider one was the true one. Two substitutions on one line, and nothing else. The image source moves from `repositories/217924/daily` to `.../weekly`, and the alt text from "Repository of the Day" to "Repository of the Week" so the accessible name states the same fact the image does. The repository id, the three utm parameters and the dimensions are untouched, as is the house markup style: descriptive alt text, no `target=` or `rel=`, and `>` rather than `/>`. No gate reads the badge (checked: nothing in test/ matches the image URL or the centred-paragraph wrapper), so this carries no gate change and no pin to move. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
andriytyurnikov
pushed a commit
to andriytyurnikov/ripwire
that referenced
this pull request
Sep 14, 2026
…d=; it now prints the scope alone as sc=
THE DEFECT. A scoped map row printed id="path::scope::name" under an <f p="path"> wrapper that had
just printed the same path, and a lens <d> row printed it beside its own p=. On this repository's
flagless map that was 137 of 137 scoped rows — 11.2% of the document on the 2026-09-12 re-measure.
route= carried 107 B of prose ("routed: subtoken+body BM25 (--for's default) — no strong name hit,
multi-word conceptual query") on every --for answer, and a compact bundle printed one <c n= l=/> row
per same-named callee.
THE FIX. The row carries sc= (the enclosing scope), the one segment nothing else on the page holds;
the legend states the composition (the full id is p::sc::n, p= from the row or its <f>), and every
selector keeps accepting the composed path::scope::name — the resolver was never changed. route= is
a code (name-exact(X) | subtoken+body[:broad|:declined(word;carriers,defs)]) whose reading lives in
the legend; the anchors: evidence clause is unchanged. Same-named callees of one calls block merge
into <c n="pick" l="70,69"/>; shown= still counts callees and the legend says so. The sc=/route=
reading is ONE shared clause (graphlegend.h kForIdRouteLegend) on the CLI lens and the MCP twin,
ceiling-droppable with the confidence clause (the dropped note names it) and exempt from the
signature-trim charge on both surfaces — charged, it dropped one ranked row on the MCP twin
(mcpforparitycheck (2), the 2026-09-04 regression shape) and re-trimmed the explicit-ceiling sigs
below the default's (forbudgetmonotoncheck redhat-et#1/redhat-et#5). The --json twins print "sc".
MEASURED (wc -c, this commit's build against the pre-change build of the same tree):
flagless map, this repository 26,402 -> 22,354 B (-4,048 B, -15.3%; the same 185 rows)
test/cppqualfix 2,935 -> 2,781 B (-5.2%) test/nestedqualfix 2,045 -> 1,937 B (-5.3%)
test/accessshapefix 1,953 -> 1,962 B (+9 B: four scoped rows do not pay for the longer reading)
--for, three tasks on this tree: the bundle is byte-shaped, so the row savings served rows, not bytes —
25 -> 28, 21 -> 24 and 3 -> 3 signature rows at 10,042 -> 10,145, 10,256 -> 10,316, 5,971 -> 6,022 B;
route= 111 -> 23 B per answer; the reading costs 259 B per default-dialect answer.
MCP for, the same three tasks: 8,752 -> 9,056, 8,882 -> 9,149, 1,965 -> 2,173 B with 26 -> 30, 23 -> 27, 1 -> 1 rows.
GATE, RED FIRST. test/scroundtripcheck.sh (new, listed in test/regression.sh; the gate count is
regenerated to 613): against the pre-change binary 13 of 17 arms FAIL (no sc= row, the composed set is
empty, no <d> composes, --json carries "id"); against this build ALL PASS — the multiset of p::sc::n
composed from the rows equals the id= multiset the old binary printed on cppqualfix (14) and
nestedqualfix (11), every composed id resolves through --expand to bodies of that path and name, a
mutated scope (ZZnoScope) serves no body, the old spelling still resolves on --expand and --callers,
and --json mirrors the attribute.
PINS MOVED, with the measured number: compactlegendcheck map 810 -> 920 (908), map-diff 800 -> 910
(901), pack-signatures 680 -> 780 (775), metrics 720 -> 820 (814), query 630 -> 730 (723), pack-task
820 -> 980 (974), pack-top-n 660 -> 770 (761) — the one new whole-document sc= reading; the ten-verb
loop 4,900 -> 5,000 (4,946); fillordercheck est_tokens 884 -> 894; printf_parity.manifest re-pinned
for exactly {flagless, expand, around, pack_signatures, pack_task, help_all}; five goldens regenerated
(test/golden.xml, anchorfix/golden_for.xml, routefix/golden_for.xml, docdemotegolden_for.xml,
docdemotegolden_noroute.xml — the row shape and the reading, nothing else in the diff); the showcase
caption re-derived by its own recount (89.5/81.8/84.3 -> 91.8/85.2/85.5, both copies).
mcpforparitycheck (6) re-authored onto an identifier query: "parse tree" routes subtoken+body:broad,
the same ranker no_route forces, so the served sets could only differ by what the route prose
displaced from the byte-shaped sigs — an artefact, not the argument. Thirty-one gates that asserted
the id= shape now assert n=/sc= or compose the id; every honesty assertion is kept. Three
legendcoverage baseline lines closed (for/for-auto/for-budgeted ctx@route); quality-delta gating=0
after acking this lane's own short-horizon churn; ASan clean on every touched emit path; determinism
and xmllint hold on the map, --for (both dialects) and --pack-task.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
joyful-ii-V-I
pushed a commit
that referenced
this pull request
Sep 19, 2026
…ffer lane/expand-ambiguous-bodies added the --expand ride-along note= attribute, formatted into main.cpp's char noteBuf[220] (runDefaultMap). fixedbufsweep.sh re-derives its whole fixed-buffer population from source every run (CLAUDE.md non-negotiable #1: a plausible-looking classification is not a checked one), so the new call site failed as UNCLASSIFIED, and the pinned S6 enumeration (calls/mentions/sites/rows) had drifted by one each (CI run 35418635836). Added a TABLE row: noteBuf's ONE interpoland is mapTopK, an int (11 digits worst case) — 121 B of fixed literal + 132 B total against 219 usable + NUL, 87 B of margin, no string interpolation and nothing user-supplied, so "safe" by the same all-numeric reasoning as this file's fileOpen/bundleOpen rows. Re-derived EXPECTED (225->226 calls/sites, 333->334 mentions, 98->99 rows) from the gate's own INFO line rather than by arithmetic, per the file's own header rule, and appended a dated round entry to the pin's history comment. caught-by: CI (run 35418635836, fixedbufsweep.sh, S1/S6) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tree had only ever compiled on one macOS/AppleClang machine. The first real CI run failed at Build on every leg; three cloud-Linux verification rounds and six fix lanes later, every layer is proven on real Linux (Ubuntu 24.04, gcc 13.3 + clang 18, libstdc++).
What's in here (24 commits)
Compile layer — kqueue guarded with a designed stat-sweep fallback (no alert: on non-kqueue platforms it is the normal path), Darwin-only pthread calls shimmed (
gettid),st_mtimespec→portable, FPfrom_charsfallback for older libc++ (requires-probe, 128/128 differential vectors),Threads::Threadslinked explicitly, sanitizer list made compiler-aware (full G1 stack preserved in CI via clang).Runtime layer (found by running the full 317-gate suite on Linux) —
--cache=<dir>crash fixed (fopen-on-directory succeeds on Linux; S_ISREG guard + degrade), GNU-stat trap fixed across 12 gate scripts,fetch-depth: 0(churn gates need history), portable regex-escape validation, NFD churn-join gate fixture corrected.The two finds that justify the whole exercise:
std::regexnever throwserror_complexity, so catastrophic-backtracking patterns spun forever where libc++ failed fast. A structural pre-compile screen now refuses the bomb families identically on every platform, with named workarounds.Gate semantics — the two corpus-shape gates are re-anchored onto deterministic in-gate fixture repos (they now assert everywhere, including CI), and the two NDEBUG-observability gates skip-with-named-reason on Release (proven by the plain leg, which CI runs second for exactly this reason).
Evidence
regression.shexit 0 on both flavours (plain + Release), parallel suite 315/317 with an empty expected-red set (1 documented timing flake, passes standalone ×3), determinism byte-identical, xmllint clean, quality-delta 0.🤖 Generated with Claude Code