Skip to content

feat(rpc): ordered host acceptance lane, worker frame ordinals and bootstrap before admission (part of #4272) - #4333

Merged
code-yeongyu merged 9 commits into
mainfrom
feat/4272-s04a-ordered-lane
Oct 27, 2026
Merged

code-yeongyu merged 9 commits into
mainfrom
feat/4272-s04a-ordered-lane

Conversation

@code-yeongyu

@code-yeongyu code-yeongyu commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Part of #4272.

Summary

S04a of contract clause R3.C1.1 (ruling 10 split): one ordered acceptance lane per routing session. With the engine-socket compact-updates flag on (<brand>_ENGINE_SOCKET_COMPACT_UPDATES, explicit-true, OFF by default), every in-process record and every worker output frame enters the lane before SessionEventWriter.enqueue. The lane is host-owned. It mints a sessionEpoch per AgentSession incarnation and an accepted sequence, and it holds a minimal transcript root containing only accepted appends. Worker frames carry a per-incarnation ordinal; the host checks the ordinal, applies the frame, and only then grants credit. Bootstrap is accepted before the session opens. Off, nothing is constructed and every byte, frame and timing is unchanged.

Ruling 12 places the messageId/turnId mint here: the producer (AgentSession) mints messageId, and the lane mints turnId and captures messageId at acceptance.

What changed

  • session-acceptance-lane.ts (new): bootstrap, accept, freeze, release, retire.
    • Recovery kinds take a sequence: agent, turn, message and tool-execution records, entry_appended and session_abort. Responses and every other record pass through in FIFO order with no sequence.
    • A change of outer correlation (clientMessageId/clientTurnId/clientMessages) is its own transition, sequenced just before its record, so the flushed turn prefix is sequenced in order with its content (Q6). The end of a correlation is a transition too: a correlatable record with no client identity (a trigger turn, an extension steer) after a correlation stood is sequenced as "no correlation".
    • Accepted appends are kept as their entry ids only: no content copy, no serialization on the host loop, nothing that aliases a manager entry. Frozen roots share the append-only id array and are read up to count.
  • core/session-recovery-identity.ts (new) and agent-session.ts: opaque randomUUID() message ids.
    • An id is minted at message_start. A message_end with no start mints at that end; an update with no start gets none.
    • Ids live in a WeakMap side table, so no event object gains a property. The two copies on the way to the lane (ClientMessageEvents, tagSessionRecord) forward the carrier.
    • Worker frames carry it as recovery.
  • In-process: session-registry-open.ts bootstraps the entry's lane before entry.state = "open", so router admission and its reply come later. session-binding.ts accepts each record before writer.enqueue; session_replaced bootstraps the next epoch. The isolate's own registry (workerIsolate) opens no lane, so the host owns exactly one.
  • Worker (session-worker*.ts):
    • commit { acceptance } asks the isolate for a bootstrap on ready (accepted before commit resolves) and for ordered output frames (ordinal, recovery, plus bootstrap on session_replaced).
    • The host checks the ordinal before any mutation. A gap, duplicate or reorder denies credit and fails the incarnation with session_worker_frame_gap / session_worker_frame_duplicate.
    • A valid frame runs the existing reconcile, is applied to the lane, then goes to writer.enqueue; credit follows the existing backpressure wait.
    • Failure retires the lane.
  • Q1 freeze with its real caller: an event-resync shed freezes the attached sessions' roots before the drain await. The cut releases them on takeSnapshot, on actor close (peer drop), on drain failure, and on session seal (lane retire). Resync bytes, reasons and order are unchanged (byte test).

RED / GREEN (contract test ids)

RED ran at base production code: src/ stashed, new tests present. GREEN ran at this head. Logs: s04a-red.log, s04a-green.log, s04a-green-worker.log.

Item Test RED at base GREEN
T5/X03, Q2: worker acceptance held while its fork is on disk; cut stays in the old epoch; grants follow acceptance; one bootstrap mints a new epoch 4272-worker-acceptance-cut "T5/X03 holds worker acceptance…" fail: no lane on the worker client pass
Worker bootstrap before admission; worker frames carry the producer's id and the lane captured it before the next credit (ruling 12 condition) same test fail pass
Q4 gap / duplicate / reorder: credit denied, typed failure, sequence unchanged, worker exits (no hang) 4272-worker-frame-order (see the contract map below) fail pass
Writer refusal: no credit, no partial root 4272-worker-frame-order "Q4 writer refusal…" fail pass
T5 in-process order, Q6 correlation sequenced before its content, responses unsequenced, ids minted, legacy bytes equal to the reference path 4272-inprocess-acceptance "T5 sequences in-process records…" fail (correlation … expected null) pass
Bootstrap before in-process admission (flag on), no lane (flag off) "bootstrap precedes in-process admission" flag on fails, flag off passes pass
Ruling 12: mint at start, at a start-less terminal, turnId for a turn with no turn_start, nothing invented "mints opaque message and turn ids…" passes at base: new modules only, no wiring pass
Q1: root released on takeSnapshot, drain failure, seal (close and abort), peer drop; zero live roots 4272-resync-root-release-paths ×5 (see the contract map below) fail: registerAcceptance missing pass
Q1 byte test: resync bytes identical with and without a lane 4272-resync-cut-release "keeps the resync bytes" fail pass
Flag off: no lane, no ordinal, no carrier "flag off: the worker client owns no lane…" pass pass

RED summary at base: 3 files, 12 failed / 3 passed (exit 1). GREEN: the three S04a files plus 33 neighbouring RPC suites (bindings, writer, fanout, resync, structured sink incl. its byte test, worker routing/reservations/isolation/output/rebind, socket host, multi-session) at the formatted tree: 36 files, 198 tests, exit 0 (s04a-neighbours.log). The worker file alone: 6/6, exit 0 (s04a-green-worker.log). The earlier s04a-green.log is the first GREEN attempt, before three test-side fixes: prompt dispatched inside its connection, held reconcile selected by the session_replaced frame, transcript count read from disk.

Contract test map (ruling 10)

One named test per case. Paths are under packages/coding-agent/test/suite/regressions/. Worker cases use the real SessionWorkerClient, WorkerSessionRegistry and router through 4272-worker-acceptance-support.ts. Every wait there is bounded, so a hang fails the test instead of stalling it.

Contract case file:line Test name
Q1 takeSnapshot 4272-resync-root-release-paths.test.ts:14 Q1 takeSnapshot: the delivered resync releases the shed's frozen root, zero live roots
Q1 drain failure 4272-resync-root-release-paths.test.ts:21 Q1 drain failure: a failing peer transport releases the shed's frozen root, zero live roots
Q1 seal (lane retire) by close 4272-resync-root-release-paths.test.ts:27 Q1 seal by close_session: the lane retires and releases the shed's frozen root, zero live roots
Q1 seal (lane retire) by abort 4272-resync-root-release-paths.test.ts:33 Q1 seal by abort: an error seal retires the lane and releases the shed's frozen root, zero live roots
Q1 peer drop (actor close) 4272-resync-root-release-paths.test.ts:39 Q1 peer drop: closing the peer's actor releases the shed's frozen root, zero live roots
Q1 byte test 4272-resync-cut-release.test.ts:6 Q1 keeps the resync bytes, reasons and order the peer received without a lane
Q2 replace 4272-epoch-per-replacement.test.ts:114 Q2 replace: new_session mints a new sessionEpoch and its sequences are never compared with the old epoch
Q2 fork 4272-epoch-per-replacement.test.ts:118 Q2 fork: fork mints a new sessionEpoch and its sequences are never compared with the old epoch
Q2 switch 4272-epoch-per-replacement.test.ts:122 Q2 switch: switch_session mints a new sessionEpoch and its sequences are never compared with the old epoch
Q2 cut held at reconcile stays in the old epoch 4272-worker-acceptance-cut.test.ts:20 T5/X03 holds worker acceptance while its fork is on disk: the cut keeps the old epoch until credit
Q4 gap 4272-worker-frame-order.test.ts:50 Q4 gap: a skipped worker frame ordinal is denied credit and fails the incarnation with session_worker_frame_gap
Q4 duplicate 4272-worker-frame-order.test.ts:59 Q4 duplicate: a repeated worker frame ordinal is denied credit and fails the incarnation with session_worker_frame_duplicate
Q4 reorder 4272-worker-frame-order.test.ts:70 Q4 reorder: frames arriving out of order are both denied credit and fail the incarnation with session_worker_frame_gap
Q4 writer refusal 4272-worker-frame-order.test.ts:79 Q4 writer refusal: an in-order frame the writer refuses gets no credit, retires the lane and publishes no root past it
Q6 in-process 4272-inprocess-acceptance.test.ts:39 T5 sequences in-process records in producer order, correlation before its content, with legacy bytes unchanged
Q6 across the worker boundary 4272-worker-correlation-order.test.ts:11 Q6 worker: a correlation transition carried in worker frames is sequenced just before its content
Q6 end of a correlation 4272-worker-correlation-order.test.ts:77 Q6 worker: a trigger turn after a client turn sequences the end of the correlation, so the cut attributes it to no client message
Host transcript retention bounded 4272-inprocess-acceptance.test.ts:192 keeps host retention per accepted append bounded: entry ids only, no stringify on the host loop
Bootstrap before admission, in-process 4272-bootstrap-before-admission.test.ts:70 bootstrap before admission, in-process: accepted while the entry is opening, before router admission and the open reply
Bootstrap before admission, worker 4272-bootstrap-before-admission.test.ts:95 bootstrap before admission, worker: accepted from 'ready' inside commit, before the entry opens, admission and the open reply

What each asserts:

  • Q1: the shed froze exactly one root, and live roots are 0 after that path. Both seal tests also assert the lane is retired; takeSnapshot and peer drop leave it live.
  • Q2: the replacement bootstraps a new UUID epoch at sequence 0 with an empty transcript root. Refs accepted before the client receives session_replaced all carry the old epoch, and every later ref carries the new one, with no interleaving. The new epoch's numbering restarts at 1, below the old epoch's last sequence, so sequences order only within their epoch.
  • Q4: the credit is conflict, the registry gets exactly one typed failure, the lane is retired, the sequence and epoch are unchanged, and the worker exits within the bound. A reorder reaches the host as the later frame first, so its typed kind is session_worker_frame_gap. The late frame is refused as well, with no second failure.
  • Q6 worker: one ordinal per physical frame, contiguous. The correlated turn_start frame advances the sequence by exactly 2: the correlation transition at n-1, its record at n. The next frame takes n+1, and every later frame keeps the same correlation transition and takes at most one sequence.
  • Bootstrap: the order is bootstrap (entry opening), then commit resolves (worker; entry still opening), then router admission (entry open), then the open_session reply. All four see one epoch, at sequence 0 through admission.

The new tests cover existing behaviour and needed no production change. Mutation checks, each run against the file it targets with the source restored afterwards:

Mutation Result
Worker ignores the frame's bootstrap on session_replaced Q2 ×3 fail
takeSnapshot stops releasing its cut Q1 takeSnapshot fails
Correlation transition takes no sequence Q6 worker fails
In-process bootstrap moved after entry.state = "open" in-process bootstrap test fails

Logs: s04a-gaps-green.log, s04a-gaps-mut-M{1,2,3,4}.log.

Review fold (review at 73348f2: FIX_FIRST, 2 MEDIUM + LOW-1)

Fold commit 05e56d1d5b. Each fix had its RED first (s04a-fold-red.log, production code at fc3d965cba), then GREEN (s04a-fold-green.log).

  • MEDIUM-1, the end of a client correlation was never recorded. Fix: session-acceptance-lane.ts:162-163. A record with neither clientMessageId nor clientTurnId still returned early unless a correlation stood and the record is a correlated kind. correlatedRecord (client-message-events.ts:10) is the one predicate for those kinds; ClientMessageEvents.output (:95) uses it too, so the lane and the emitter cannot drift. The end takes its own sequence just before the first uncorrelated record, and later uncorrelated records add no transition. Test 4272-worker-correlation-order.test.ts:77, "Q6 worker: a trigger turn after a client turn sequences the end of the correlation…": a real worker gets prompt { clientMessageId: "c-1" } and then send_custom_message { triggerTurn: true }. RED at the parent: the cut attributes the trigger turn to no client message: expected 'c-1' to be undefined. GREEN: pass.
  • MEDIUM-2, the host kept an unbounded serialized copy of the transcript. Fix: session-acceptance-lane.ts:182-187 keeps each accepted entry's id (TranscriptRoot.entryIds) and drops the JSON.stringify(entry). No serialized copy is needed for correctness until S04b, because nothing reads content from the root in S04a. That fits S04-design's persisted base {fileIdentity, count, idFingerprint}: S04b resolves content through its base and references, and ids cannot be mutated by mirror trimming. So I removed the copy and added no byte cap. Test 4272-inprocess-acceptance.test.ts:192, "keeps host retention per accepted append bounded…": 64 appends of 256 KiB entries. It asserts zero JSON.stringify calls during accept, ids in order up to count, no alias after the entries are mutated, and a root under 64×16+128 bytes. RED at the parent: no serialization per append on the host loop: expected 64 to be +0. GREEN: pass.
  • LOW-1, the writer-refusal test passed for an unrelated reason. Test rewritten at 4272-worker-frame-order.test.ts:79. The session stays open and the real writer refuses only the injected frame. The test asserts the lane accepted the frame before the refusal (sequence +1, not retired), credit conflict, the typed failure, the lane retired, zero live roots (a root held across the frame is released) and no acceptance afterwards. Mutation proof, with the source restored afterwards:
    • M-B, refusal ignored (!writer.enqueue(...) && false): the new test fails with a refused frame gets no credit: expected 1 to be 2 (s04a-fold-low1-mutB-new.log).
    • M-C, the lane accepts only after writer.enqueue succeeds (breaks the accept-then-refusal order): the new test fails with the lane accepted the frame before the writer refused it: expected { sequence: 1, retired: false } to deeply equal { sequence: 2, retired: false }. The old parent test still passes under M-C, which is the LOW-1 defect (s04a-fold-low1-mutC-{new,old}.log).
    • M-A, fail() no longer retires the lane: both tests still pass, because callbacks.failure also retires the lane through the registry's forgetSession. Retirement has two routes, so M-A is not a break (s04a-fold-low1-mutA-{new,old}.log).
  • Not fixed (noted): LOW-2, the single open assistant id slot in session-recovery-identity.ts:43: no current path sends a direct assistant message_start while a stream is open. LOW-3, the flag accepting 1 as well as true: a wording choice for the contract owner.

Moved to S04b, and why

  • The SessionManager transition journal, verified persisted bases ({fileIdentity, count, idFingerprint}) and Q3's typed mismatch failure belong to S04b. So do branch/reset/replacement roots and the worker fork bootstrap carrying the branched root. In S04a a replacement bootstrap carries only the new leaf and an empty appended list.
  • Q5 (a single pre-bind transition over 16 MiB marks the transcript component unavailable with a typed reason): S04a has no pre-bind transition path. The pre-ready journal frames come with S04b's manager journal, so Q5 lands there.
  • The held-reconcile RED for C1.1's last sentence is here in its S04a form: epoch, sequence and root are held, then a new epoch follows. The assertion that the cut contains no fork entry ID or file identity needs S04b's branched root.
  • runtimeItemKey (ruling 12 condition 3) belongs to the S05/S06 row model; S04a adds no row.

Production size

Production diff against origin/main after the review fold: 17 files, +405 / -27 (the fold added +25 / -9 net in session-acceptance-lane.ts and client-message-events.ts). Before the fold: +380 / -18. 312 added lines are code, the rest are comments. The new files are session-acceptance-lane.ts (147 pure LOC), session-recovery-identity.ts (30) and engine-socket-flag.ts (5). Tests: 10 files, +1053 (the conformance-gap commit added +504 / -114, no production lines). This is within the ~340-390 estimate and under the 600 stop.

Gates

At head 4055ecb: bun run build exit 0; tsc --noEmit exit 0 (re-run after the final type-export edit); node scripts/lint-ratchet.mjs exit 0 (every count matches the baseline); oxfmt --check on changed files exit 0; bun run check exit 0. Logs s04a-{build,tsc,tsc-final,ratchet,oxfmt,check}.log. No senpi-qa CLI run: flag off means no user-reachable change.

At head 05e56d1 (review fold): bun run build exit 0; tsc --noEmit exit 0; oxfmt --check and oxlint --deny-warnings on the 5 changed source/test files exit 0; git add then node scripts/lint-ratchet.mjs exit 0 (every count matches the baseline); check-pr-changelog.mjs --base origin/main --labels no-changelog exit 0 (the PR carries the label). All eight 4272-* files plus session-record-stream-projection, rpc-client-message-identity and rpc-client-message-ledger: 11 files, 43 tests, exit 0. Logs s04a-fold-{build,tsc,oxfmt,oxlint,ratchet,changelog-label,red,green}.log.

At head fc3d965 (conformance-gap tests, no production change): bun run build exit 0; tsc --noEmit exit 0; git add then node scripts/lint-ratchet.mjs exit 0 (every count matches the baseline); oxfmt --check on the 9 changed files exit 0; all eight 4272-* test files, 22 tests, exit 0. Logs s04a-gaps-{build,tsc,ratchet,oxfmt,green}.log.

Notes for review

  • No flag existed in merged S01 code (S01 shipped no flag). This PR adds the reader core/engine-socket-flag.ts with the env name S02b uses (ENGINE_SOCKET_COMPACT_UPDATES). S02b's structuredLaneEnabled() should import it when it rebases (LOW conflict).
  • SessionAcceptanceLane.liveRoots has no production reader. It is the observable for the Q1 leak test; retired is read by the fanout.
  • session-event-writer.ts (1080 lines) and agent-session.ts are already far over the 250-line file ceiling. This PR adds 6 and 8 lines there and puts the logic in new files.

…otstrap before admission (part of #4272)

S04a of R3.C1.1 behind the engine-socket compact-updates flag (off by default): one acceptance lane per routing session minting sessionEpoch and accepted sequence before SessionEventWriter.enqueue; worker frames checked by ordinal, applied, then credited; bootstrap accepted before open; correlation transitions sequenced; producer-minted messageId and lane-minted turnId (ruling 12); event-resync shed freezes and releases roots on every path.
@code-yeongyu code-yeongyu added the no-changelog No changelog required label Oct 26, 2026
@code-yeongyu
code-yeongyu marked this pull request as ready for review October 26, 2026 05:07
@code-yeongyu code-yeongyu added the review-queued Queued for batched Opus review label Oct 26, 2026
One named test per contract case, split by behaviour: Q1 root release on takeSnapshot, drain failure, seal by close and by abort, and peer drop; Q2 a new sessionEpoch on new_session, fork and switch_session, with refs never interleaved across epochs; Q4 gap, duplicate and reorder as separate tests asserting the typed failure, denied credit and a bounded worker exit; Q6 correlation sequenced with its content across worker frames; bootstrap accepted before the entry opens, router admission and the open reply in both runtimes. The Q1 and Q4 it.each blocks move into the new files. No production change.
…pted entry ids (part of #4272)

Review fold for #4333. A correlatable record without client identity after a
correlation stood (a trigger turn, an extension steer) is now sequenced as the
end of that correlation, so a cut attributes it to no client message. The host
lane keeps each accepted transcript append as its entry id only, with no
serialized copy and no stringify on the host loop. The writer-refusal test now
exercises accept-then-refusal with the session still open.
@code-yeongyu code-yeongyu removed the review-queued Queued for batched Opus review label Oct 26, 2026
code-yeongyu and others added 5 commits October 26, 2026 17:29
…t and drop stringify from correlation compare (part of #4272)

Co-authored-by: sisyphus-dev-ai <sisyphus-dev-ai@users.noreply.github.com>
…d-lane

# Conflicts:
#	packages/coding-agent/src/modes/rpc/session-worker-client.ts
…d-lane

# Conflicts:
#	packages/coding-agent/src/core/agent-session.ts
…d-lane

# Conflicts:
#	packages/coding-agent/src/core/agent-session.ts
@code-yeongyu
code-yeongyu merged commit 40cd139 into main Oct 27, 2026
44 checks passed
@code-yeongyu
code-yeongyu deleted the feat/4272-s04a-ordered-lane branch October 27, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog No changelog required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant