Skip to content

feat(rpc): shared ownership, leases and persistent bounded text roots for the compact consumer (S03b-1a, part of #4272) - #4378

Merged
code-yeongyu merged 2 commits into
mainfrom
feat/4272-s03b1-storage-substrate
Oct 26, 2026
Merged

code-yeongyu merged 2 commits into
mainfrom
feat/4272-s03b1-storage-substrate

Conversation

@code-yeongyu

Copy link
Copy Markdown
Owner

Part of #4272. S03b-1a: shared ownership, operation leases and persistent bounded text roots for the compact current-state consumer (R3.C6.4 substrate), behind the OFF-by-default compact-updates flag. Base: main.

Rulings: RULING-26 (S03b design; Q3 bounded reads: explicit size and count limits, typed over-limit errors, no silent truncation) and RULING-26a (S03b-1 split into 1a/1b; this PR is 1a). Closes RULING-26a conditions 1-3; condition 4 and structured values/index pages are S03b-1b.

S03b-1a: shared ownership, operation leases and persistent text roots

Part of #4272. RULING-26a limits this increment to the text/storage ownership substrate. No public capability is enabled; S03b-2 owns the complete reducer and S03b-3 owns lane integration. S07b owns the real old-consumer and replay-removal gate.

Local head: 91f22e70ae3d1f53065f59c1e6599c8e1c213747.
Base: ca8f30ae153a83f666fafac664bf29874f74a3cb.
Branch: feat/4272-s03b1-storage-substrate. No push. Working tree clean.
Author/committer: YeonGyu-Kim code.yeon.gyu@gmail.com.

Design

Use immutable, counted binary text roots over the existing bounded UTF-16 leaves. Appending replaces the logarithmic right path; freezing retains the root rather than copying a leaf vector. Only ownership-category transitions propagate to children, so an additional cut on a live root is constant work. Existing mutable packed tails remain bounded by one leaf.

Commit dependencies are one-to-many, scoped to the actual index or text-node owner. Duplicate references count for ownership but not twice for commit readiness. Operation leases retain content until the operation unwinds; release/retirement invalidates results rather than deleting files under an in-flight read.

Reject a wrapper that first materializes the old whole value and slices it. Range reads must validate unit and segment limits before reading any leaf. The retained whole-view compatibility reader is not a bounded collection API.

S03a paths deliberately left unbounded for S03b-1b

Anchors below are the input S03a revision, before 1a edits. The tests are required 1b tests, not tests claimed to ship here.

S03a path Remaining unbounded behavior Covering 1b regression
compact-content-index.ts:34-41,87-105 Resident base arrays/key map, full override copy and page pin walk on freeze, complete entries() materialization. Direct frozen-index page operations still need their own leases; the real minimal consumer holds its cut through its operation in 1a. 4272-persistent-index-pages.test.ts: freeze a large collection with bounded changed-path work; traverse every row with <=128 descriptors and <=64 KiB per page; assert peak resident pages; hold a page read across release/compaction without early reclamation.
compact-content-index.ts:118-172 Compaction selects every eligible override, gathers touched pages and accumulates all merged rows/written pages. 4272-bounded-index-compaction.test.ts: hold reads/writes over a large collection, mutate selected keys, assert bounded resident pages and no stale override loss.
compact-content-index.ts:182-211 Publication copies the entire pageOf map and base page list, filters rows and builds leaf/key arrays. 4272-persistent-index-pages.test.ts: large-base single-key publication allocates only a bounded changed path, preserving frozen versions.
compact-content-index.ts:213-218 A page read decodes the whole page and filters through a complete key set. 4272-bounded-index-pages.test.ts: enforce encoded byte/count ceilings before decode and inspect peak resident page buffers during traversal.
compact-index-pages.ts:14-29 Packing sizes whole rows, accepts one oversized row, and returns all packed pages together. 4272-bounded-index-pages.test.ts: RED for a single >64 KiB row and oversized key; reference-valued encoding stays within both limits, without silent truncation.
compact-session-state.ts:49-65,81-82,110-132 Structured settled rows embed block arrays; active freeze/read copies collections and whole-view recovery accumulates all messages. 4272-bounded-current-components.test.ts: huge block/message collections and structured values survive persistent component roots, bounded descriptor/value pages, and a peak-resident-pages assertion.
compact-session-state.ts:136-155,253-277 Status walks every active header; sealing sorts/materializes all block descriptors; the compatibility view assembles complete text/messages. Leaf-vector copies and parallel whole-leaf reads are 1a's responsibility, but the collection copies remain. 4272-bounded-current-components.test.ts: one-block change/freeze in a large message does not enumerate unrelated blocks; recovery uses paged components, with whole legacy assembly charged separately.

1b also owns spilling structural navigation/index metadata, oversized-key references and the rest of structured-value migration. 1a's immutable text navigation nodes are resident and explicitly counted, not claimed to provide a constant-memory large-collection implementation.

Contract closure

RULING-26 Q3 is closed only for text ranges and ownership; its structured value/page half belongs to 1b. RULING-26a conditions 1-3 are this increment's scope. Condition 4 belongs to 1b. RULING-26 Q1/Q2/Q4 remain with the seeded reducer and lane/cut integration; this change makes no claim to close them.

Evidence

RED -> GREEN

Regression RED evidence Final result
Two subscribers, two keys and duplicate references share one leaf S03a: baseRows: 0, overrideSlots: 2 instead of 2, 0; exit 1. GREEN; both indexes compact, either owner survives release, final release reclaims once.
Released consumer cut and repeated release S03a returned the released view; exit 1. GREEN; typed released, no double decrement.
Held cut read across replacement/release S03a lost the file and returned CompactStorageReadError/ENOENT; exit 1. GREEN; file survives the operation and the stale result is refused.
Held leaf read across retirement Same predecessor ENOENT failure; exit 1. GREEN; retirement joins the operation and returns typed retired.
Size boundary Removing the size check returned 32,769 units; exit 1. GREEN; 32,768 succeeds, excess is typed size_limit before I/O.
Count boundary Removing count enforcement returned a two-leaf surrogate pair under a one-leaf limit; exit 1. GREEN; two segments succeed, one-segment excess is typed count_limit before I/O.
Persistent root pin/path sharing, immutable cuts, spill failure Preservation/representation tests, not claimed as predecessor REDs for nonexistent APIs. GREEN through real CompactSessionState text paths; root pin does not walk unchanged leaves; failed spill retains the complete root and publishes no partial index.

The final focused command passed 12 files / 55 tests, exit 0:

bun run --cwd packages/coding-agent test test/suite/regressions/4272- test/suite/rpc-compact-content- test/suite/rpc-structured-sink.test.ts test/suite/no-sync-in-session-path.test.ts

This includes the unchanged legacy byte-identity writer fixture and the 1 MiB / 1,024-update current-content fixture. A blanket retirement guard initially broke S03a's post-retirement removal test; the guard now permits teardown removals, and the existing test was not altered.

Gates

Command Exit / result
nice -n 10 bun run build 0; full workspace build, coding-agent bundle built.
node_modules/.bin/tsc --noEmit 0.
bun run check 0; full lint/format/ratchet, dependency/import/entry-graph/lock checks, both TypeScript projects and browser smoke.
node scripts/lint-ratchet.mjs after staging 0; every count matches baseline.
oxlint --deny-warnings <changed files> 0.
oxfmt --check <changed files> 0.
git diff --cached --check 0.
RPC CLI rpc-drive.mjs --self-test 0; 4/4 checks, isolated/offline, real auth unchanged.
node scripts/check-pr-changelog.mjs --base origin/main --labels no-changelog after commit 1, expected filename-only blocker: changes/unreleased/s03b1.md: a fragment is named <pr-number>.md. The brief explicitly requires this temporary filename. Rename it once the PR number exists; do not treat the pre-commit no-diff PASS as coverage.

LSP requests timed out; compiler checks passed. No remote/SSH steps were needed. No review or merge claim is made.

Parity and X07

Old S03a and the new consumer received the same append/replace/spill/pending/failed/remove schedule. Five frozen views were byte-identical: 131,809 UTF-8 bytes, SHA-256 a92cccae1ef3a4c959ab68d75a87ae80d77298f5ce39c6ca1e47ae963d29f701, both exit 0.

At the committed head, the zero-peer 1,048,576-code-unit / 1,024-update fixture measured:

Barrier Resident payload Pending copy Spilled live Pinned-only Reclaim debt Text nodes
First write held 2,097,152 65,536 0 0 0 31
Spill quiescent 131,072 0 1,966,080 0 0 31
Current removed, cut retained 131,072 0 1,966,080 2,097,152 0 31
Cut released, unlink held 0 0 0 0 1,966,080 0
Reclamation quiescent 0 0 0 0 0 0

Text-node accounting uses an explicit fixed 192-byte descriptor/edge charge (5,952 bytes for 31 nodes), not an RSS claim. S07 replay, S12 worker transfer and whole legacy view assembly remain separate costs, not claimed eliminated.

Size

390 added pure production LOC; raw production diff +434/-73, net +361. Every changed source/test file is <=250 pure LOC.

File under packages/coding-agent/ Pure LOC
src/modes/rpc/compact-commit-dependencies.ts 46
src/modes/rpc/compact-content-index.ts 227
src/modes/rpc/compact-content-store.ts 161
src/modes/rpc/compact-read-lease.ts 35
src/modes/rpc/compact-session-state.ts 248
src/modes/rpc/compact-spool-storage.ts 177
src/modes/rpc/compact-text-roots.ts 141
src/modes/rpc/compact-text-view.ts 35
test/suite/regressions/4272-bounded-text-roots.test.ts 155
test/suite/regressions/4272-shared-text-ownership.test.ts 60
test/suite/regressions/4272-text-operation-leases.test.ts 98

The fragment changes/unreleased/s03b1.md is 25 documentation lines.

Assumptions and remaining decisions

Persistent means immutable live-host versions, not crash-restart durability. Text navigation metadata remains resident and counted; structured/index metadata spill and direct index page operation migration belong to 1b. No new whole-collection enumeration or copy was introduced. The only remaining handoff action for this increment is assigning the PR number and renaming its fragment. No unresolved 1a product-contract question was found.

Architectural review: responsibilities are separated into reference readiness, cut leases, spool I/O, text trees and compatibility assembly; acceptance stays synchronous; bounds are checked before I/O; async readers retain ownership; no new dependency or fallback layer was added; every new export has a production caller; the real consumer exercises the new tree and range paths.

Local receipts are in senpi-s03b1/local-ignore/qa-evidence/20261026-s03b1a/. The standalone parity and X07 drivers are in this run directory.

SHA-256:

1c09323a90ee3c046a35daaa144cc9a4d931d290ee6bfda21cfb1ff0df8cc439  receipts.md
3939b9e09151bb9fea1a540b28c788a86721d5a49d496d9bc8d6673f595725fb  static.txt
b46bbd371ffe95201577dabf455ef7338f9e7ade899a6e1c888af15c845ff15e  red-ownership.txt
694951418b4f33f2b01d5f5f7d769e21a76e6254a301fcce3cbdece033d79dc9  red-leases.txt
14f3c09473c5bbed1e45719f9154b99ab43fd6c2c1172f2b0d64b4f8908cbc7c  parity.txt
f316d8c3e23e4a1f5439a6e3f8cb290e11a5a1e7b89b3441aa8555d99563675a  x07.json

@code-yeongyu code-yeongyu added the no-changelog No changelog required label Oct 26, 2026
@code-yeongyu
code-yeongyu merged commit 9c60d89 into main Oct 26, 2026
44 checks passed
@code-yeongyu
code-yeongyu deleted the feat/4272-s03b1-storage-substrate branch October 26, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog No changelog required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant