Repository navigation
feat(rpc): shared ownership, leases and persistent bounded text roots for the compact consumer (S03b-1a, part of #4272) - #4378
Merged
Conversation
… for the compact consumer (S03b-1a, part of #4272)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #4272. S03b-1a: shared ownership, operation leases and persistent bounded text roots for the compact current-state consumer (R3.C6.4 substrate), behind the OFF-by-default compact-updates flag. Base: main.
Rulings: RULING-26 (S03b design; Q3 bounded reads: explicit size and count limits, typed over-limit errors, no silent truncation) and RULING-26a (S03b-1 split into 1a/1b; this PR is 1a). Closes RULING-26a conditions 1-3; condition 4 and structured values/index pages are S03b-1b.
S03b-1a: shared ownership, operation leases and persistent text roots
Part of #4272. RULING-26a limits this increment to the text/storage ownership substrate. No public capability is enabled; S03b-2 owns the complete reducer and S03b-3 owns lane integration. S07b owns the real old-consumer and replay-removal gate.
Local head:
91f22e70ae3d1f53065f59c1e6599c8e1c213747.Base:
ca8f30ae153a83f666fafac664bf29874f74a3cb.Branch:
feat/4272-s03b1-storage-substrate. No push. Working tree clean.Author/committer: YeonGyu-Kim code.yeon.gyu@gmail.com.
Design
Use immutable, counted binary text roots over the existing bounded UTF-16 leaves. Appending replaces the logarithmic right path; freezing retains the root rather than copying a leaf vector. Only ownership-category transitions propagate to children, so an additional cut on a live root is constant work. Existing mutable packed tails remain bounded by one leaf.
Commit dependencies are one-to-many, scoped to the actual index or text-node owner. Duplicate references count for ownership but not twice for commit readiness. Operation leases retain content until the operation unwinds; release/retirement invalidates results rather than deleting files under an in-flight read.
Reject a wrapper that first materializes the old whole value and slices it. Range reads must validate unit and segment limits before reading any leaf. The retained whole-view compatibility reader is not a bounded collection API.
S03a paths deliberately left unbounded for S03b-1b
Anchors below are the input S03a revision, before 1a edits. The tests are required 1b tests, not tests claimed to ship here.
compact-content-index.ts:34-41,87-105entries()materialization. Direct frozen-index page operations still need their own leases; the real minimal consumer holds its cut through its operation in 1a.4272-persistent-index-pages.test.ts: freeze a large collection with bounded changed-path work; traverse every row with <=128 descriptors and <=64 KiB per page; assert peak resident pages; hold a page read across release/compaction without early reclamation.compact-content-index.ts:118-1724272-bounded-index-compaction.test.ts: hold reads/writes over a large collection, mutate selected keys, assert bounded resident pages and no stale override loss.compact-content-index.ts:182-211pageOfmap and base page list, filters rows and builds leaf/key arrays.4272-persistent-index-pages.test.ts: large-base single-key publication allocates only a bounded changed path, preserving frozen versions.compact-content-index.ts:213-2184272-bounded-index-pages.test.ts: enforce encoded byte/count ceilings before decode and inspect peak resident page buffers during traversal.compact-index-pages.ts:14-294272-bounded-index-pages.test.ts: RED for a single >64 KiB row and oversized key; reference-valued encoding stays within both limits, without silent truncation.compact-session-state.ts:49-65,81-82,110-1324272-bounded-current-components.test.ts: huge block/message collections and structured values survive persistent component roots, bounded descriptor/value pages, and a peak-resident-pages assertion.compact-session-state.ts:136-155,253-2774272-bounded-current-components.test.ts: one-block change/freeze in a large message does not enumerate unrelated blocks; recovery uses paged components, with whole legacy assembly charged separately.1b also owns spilling structural navigation/index metadata, oversized-key references and the rest of structured-value migration. 1a's immutable text navigation nodes are resident and explicitly counted, not claimed to provide a constant-memory large-collection implementation.
Contract closure
RULING-26 Q3 is closed only for text ranges and ownership; its structured value/page half belongs to 1b. RULING-26a conditions 1-3 are this increment's scope. Condition 4 belongs to 1b. RULING-26 Q1/Q2/Q4 remain with the seeded reducer and lane/cut integration; this change makes no claim to close them.
Evidence
RED -> GREEN
baseRows: 0, overrideSlots: 2instead of2, 0; exit 1.released, no double decrement.CompactStorageReadError/ENOENT; exit 1.retired.size_limitbefore I/O.count_limitbefore I/O.CompactSessionStatetext paths; root pin does not walk unchanged leaves; failed spill retains the complete root and publishes no partial index.The final focused command passed 12 files / 55 tests, exit 0:
bun run --cwd packages/coding-agent test test/suite/regressions/4272- test/suite/rpc-compact-content- test/suite/rpc-structured-sink.test.ts test/suite/no-sync-in-session-path.test.tsThis includes the unchanged legacy byte-identity writer fixture and the 1 MiB / 1,024-update current-content fixture. A blanket retirement guard initially broke S03a's post-retirement removal test; the guard now permits teardown removals, and the existing test was not altered.
Gates
nice -n 10 bun run buildnode_modules/.bin/tsc --noEmitbun run checknode scripts/lint-ratchet.mjsafter stagingoxlint --deny-warnings <changed files>oxfmt --check <changed files>git diff --cached --checkrpc-drive.mjs --self-testnode scripts/check-pr-changelog.mjs --base origin/main --labels no-changelogafter commitchanges/unreleased/s03b1.md: a fragment is named <pr-number>.md. The brief explicitly requires this temporary filename. Rename it once the PR number exists; do not treat the pre-commit no-diff PASS as coverage.LSP requests timed out; compiler checks passed. No remote/SSH steps were needed. No review or merge claim is made.
Parity and X07
Old S03a and the new consumer received the same append/replace/spill/pending/failed/remove schedule. Five frozen views were byte-identical: 131,809 UTF-8 bytes, SHA-256
a92cccae1ef3a4c959ab68d75a87ae80d77298f5ce39c6ca1e47ae963d29f701, both exit 0.At the committed head, the zero-peer 1,048,576-code-unit / 1,024-update fixture measured:
Text-node accounting uses an explicit fixed 192-byte descriptor/edge charge (5,952 bytes for 31 nodes), not an RSS claim. S07 replay, S12 worker transfer and whole legacy view assembly remain separate costs, not claimed eliminated.
Size
390 added pure production LOC; raw production diff +434/-73, net +361. Every changed source/test file is <=250 pure LOC.
packages/coding-agent/src/modes/rpc/compact-commit-dependencies.tssrc/modes/rpc/compact-content-index.tssrc/modes/rpc/compact-content-store.tssrc/modes/rpc/compact-read-lease.tssrc/modes/rpc/compact-session-state.tssrc/modes/rpc/compact-spool-storage.tssrc/modes/rpc/compact-text-roots.tssrc/modes/rpc/compact-text-view.tstest/suite/regressions/4272-bounded-text-roots.test.tstest/suite/regressions/4272-shared-text-ownership.test.tstest/suite/regressions/4272-text-operation-leases.test.tsThe fragment
changes/unreleased/s03b1.mdis 25 documentation lines.Assumptions and remaining decisions
Persistent means immutable live-host versions, not crash-restart durability. Text navigation metadata remains resident and counted; structured/index metadata spill and direct index page operation migration belong to 1b. No new whole-collection enumeration or copy was introduced. The only remaining handoff action for this increment is assigning the PR number and renaming its fragment. No unresolved 1a product-contract question was found.
Architectural review: responsibilities are separated into reference readiness, cut leases, spool I/O, text trees and compatibility assembly; acceptance stays synchronous; bounds are checked before I/O; async readers retain ownership; no new dependency or fallback layer was added; every new export has a production caller; the real consumer exercises the new tree and range paths.
Local receipts are in
senpi-s03b1/local-ignore/qa-evidence/20261026-s03b1a/. The standalone parity and X07 drivers are in this run directory.SHA-256: