Promise.all + ctx.task emits duplicate effects with collided journal sequence numbers

[rogelsm]

R2 · SDK bug report — Promise.all + ctx.task journal collision

Title

Promise.all + multiple ctx.task emissions cause duplicate effects with collided journal sequence numbers

Summary

When a process function calls ctx.task from inside multiple Promise.all branches simultaneously, the babysitter SDK assigns the same stepId (and same journal sequence number) to two different effects. The journal ends up with two entries at the same seq; the state cache cannot reconcile; iterations stall or emit ghost effects.

Reproduction

Minimal process file:

import { defineTask } from '@a5c-ai/babysitter-sdk';

export const taskA = defineTask('parallel-a', () => ({ kind: 'agent', title: 'A' }));
export const taskB = defineTask('parallel-b', () => ({ kind: 'agent', title: 'B' }));
export const taskC = defineTask('parallel-c', () => ({ kind: 'agent', title: 'C' }));

export async function process(_inputs, ctx) {
  const [a, b, c] = await Promise.all([
    ctx.task(taskA, {}),
    ctx.task(taskB, {}),
    ctx.task(taskC, {}),
  ]);
  return { a, b, c };
}

Run with babysitter run:create ... && babysitter run:iterate .... Examine the journal — at least two events at the same seq.

Concrete evidence (from a real run)

In cookbook run 01KT797KHQ6RFVMJ55TPBQ1K26 (V2 lane epic, 2026-06-03):

• Journal seq 4 had TWO entries: 000004.01KT798JNFRF4GCVFBBN0531TC.json (Lane B mockup task) and 000004.01KT798JNFRF4GCVFBBN0531TD.json (was Lane A's impl task)
• babysitter run:iterate returned the error Journal sequence gap detected at 000004.01KT798JNFRF4GCVFBBN0531TD.json (expected 5, got 4)
• Recovery required manually deleting one of the duplicate-seq files

The issue is that step IDs (S000003, S000004, etc.) appear to be assigned by Promise.all interleaving order, but the journal sequence numbers (000001, 000002, ...) collide because emission order is non-deterministic at sub-ms granularity.

Suggested fix direction

StepIds should derive from the invocation key (process file path + task name + cumulative invocation count for that task), not from monotonic emission order. Same for journal sequence numbers — they should advance once per persisted event, with deterministic ordering even when emission is concurrent.

OR: Document that Promise.all + ctx.task is unsupported, and recommend sequential for...of loops.

Environment

• @a5c-ai/babysitter-sdk: (current version on cookbook, 2026-06-03)
• Node 24
• macOS / Claude Code harness

[a5c-ai]

Triage Summary

Raw Promise.all([ctx.task(...)]) still has an SDK orchestration semantics gap: multiple effects can be journaled, but the iterate result reports only the first pending action. The originally reported duplicate journal sequence number did not reproduce on the current installed SDK.

Critical Evaluation

The issue is valid, but the reporter's root-cause analysis needs correction. Current code already serializes journal appends through the per-run append queue in packages/sdk/src/storage/journal.ts, and ctx.task now derives stable invocation keys in packages/sdk/src/runtime/intrinsics/task.ts. A disposable repro matching the issue produced unique journal seqs for all three effects.

The remaining problem is still worth fixing or documenting: Promise.all rejects with the first EffectRequestedError, so run:iterate returns one nextAction even though the other ctx.task calls have already appended EFFECT_REQUESTED entries. run:status then sees all pending effects, which can look like ghost effects or a stalled run to a host.

Root Cause Analysis

Affected areas:

• packages/sdk/src/runtime/intrinsics/task.ts - emits EFFECT_REQUESTED and computes stable invocation keys.
• packages/sdk/src/storage/journal.ts - serializes append sequence allocation and write.
• packages/sdk/src/runtime/orchestrateHelpers.ts - maps a single EffectRequestedError to a single nextAction; only ParallelPendingError returns a batch.
• packages/sdk/src/runtime/intrinsics/parallel.ts - supported batching path for concurrent effects.
• packages/sdk/src/runtime/__tests__/orchestrateIteration.integration.test.ts and packages/sdk/src/storage/__tests__/storage.test.ts - related coverage exists, but not a direct raw Promise.all([ctx.task(...)]) regression.

Local repro with babysitter 5.1.1-staging.a2902438a36f:

• First iterate wrote three EFFECT_REQUESTED entries for parallel-a, parallel-b, and parallel-c with unique seqs 2, 3, and 4.
• The iterate response returned only one nextAction.
• run:status reported pendingByKind.node = 3.

Affected Components

• Package: sdk
• Type: bug
• Effort: effort:medium

Recommended Approach

Add an end-to-end regression for raw Promise.all([ctx.task(...)]) before changing runtime behavior. The test should assert journal sequence uniqueness, pending effect count, and what run:iterate returns.

Then choose one explicit contract:

• Support raw Promise.all(ctx.task) by surfacing all newly requested effects consistently, ideally through the same batch semantics used by ParallelPendingError / ctx.parallel.all.
• Or reject/document raw Promise.all(ctx.task) as unsupported and require ctx.parallel.all([() => ctx.task(...)]) for concurrent effect emission.

Do not weaken EffectIndex journal sequence validation; that would hide real corruption.

Related Issues/PRs

• Journal sequence collision when --non-interactive auto-approves multiple breakpoints; spurious RUN_FAILED follows #191 / PR Plan: Journal sequence collision fix #194: related journal sequence collision for non-interactive breakpoint auto-approval; this appears to be the source of the current append queue fix.
• [runtime] Duplicate RUN_COMPLETED event emitted on post-completion state rebuild #181 / PR Plan: prevent duplicate RUN_COMPLETED emissions #193: duplicate RUN_COMPLETED on post-completion rebuild; related journal correctness area, but different root cause.
• Add 'babysitter run:halt' command for sealing divergent runs #880: run:halt recovery feature request, motivated partly by divergent runs after sequence/collision bugs.
• PR feat: ConcurrentEffects within-harness dispatch #705: ConcurrentEffects within-harness dispatch; relevant to the concurrency contract but not a direct fix for raw Promise.all(ctx.task).

Priority

priority:medium. This affects SDK orchestration correctness and host behavior, but the original duplicate-sequence symptom did not reproduce on the current installed SDK and ctx.parallel.all is an available intended concurrency path.

Risk Analysis

Risk level: risk:medium.

• Changing raw Promise.all(ctx.task) handling can alter replay determinism and pending-effect ordering.
  • Mitigation: write the failing end-to-end regression first and assert deterministic journal order plus returned nextActions behavior.
• Relaxing journal validation would mask real corruption.
  • Mitigation: keep EffectIndex strict; fix collection/scheduling semantics instead.
• Supporting raw Promise.all separately could create a second concurrency model beside ctx.parallel.all and ConcurrentEffects.
  • Mitigation: route support through existing parallel/batch semantics, or explicitly document/reject the raw pattern.

[a5c-ai]

Triage Summary

• One-line: Valid SDK bug/guardrail gap around raw Promise.all([ctx.task(...)]) parallel task emission and journal sequencing.

Critical Evaluation

• Reporter's analysis needs correction: the current source already has two mitigations the report suggests: stable task-key based invocation identity and queued journal append sequencing. The exact reported pattern is still worth treating as a valid regression/coverage gap because the issue includes real-run evidence from June 3 and there is no exact integration test for raw Promise.all([ctx.task(...)]).
• Suggested fix needs refinement: do not start by changing step-id or journal sequence semantics. First add an integration regression test for the exact minimal process in the issue. If it fails, fix the uncovered runtime path. If it passes in current source, investigate SDK/package/deployment drift between the cookbook run and staging/global SDK.

Root Cause Analysis

• Affected files:
  • packages/sdk/src/runtime/intrinsics/task.ts
  • packages/sdk/src/storage/journal.ts
  • packages/sdk/src/runtime/intrinsics/parallel.ts
  • packages/sdk/src/runtime/replay/effectIndex.ts
• Current code evidence:
  • task.ts derives the primary invocation key from task id, label, argument-shape hash, and per-base count, while retaining legacy step-id lookup for replay compatibility.
  • journal.ts wraps appendEvent in a per-run append queue before computing max(seq)+1 and writing the journal file.
• Existing tests cover related behavior but not this exact edge:
  • concurrent appendEvent writes produce unique sequential journal files;
  • Promise.all breakpoint behavior is covered;
  • ctx.parallel.all recovery from already-created task promises is covered.
• Missing guardrail: an integration test using the exact issue process:
  await Promise.all([ctx.task(taskA, {}), ctx.task(taskB, {}), ctx.task(taskC, {})]).

Affected Components

• Packages: sdk
• Labels applied: bug, sdk, effort:medium, priority:high, risk:high

Recommended Approach

1. Add an integration regression test under packages/sdk/src/runtime/__tests__/ using the minimal process from the issue.
2. Assert that first iteration returns a coherent pending state and the journal has contiguous, unique sequence numbers and filenames.
3. If the test fails, fix the runtime/journal path without relaxing sequence validation.
4. If the test passes, reproduce against the packaged CLI/SDK version used by cookbook and resolve deployment/version drift.
5. Clarify docs/API guidance: prefer ctx.parallel.all([() => ctx.task(...)]) for supported parallel task dispatch, or add a runtime guard for raw Promise.all([ctx.task(...)]) if unsupported.

Related Issues/PRs

• Related but not duplicate: Journal sequence collision when --non-interactive auto-approves multiple breakpoints; spurious RUN_FAILED follows #191 / Plan: Journal sequence collision fix #194 handled an older journal sequence collision path.
• Related historical commits: e4a1ef8f0 (ctx.task replay drift keys) and a18bb5a10 (journal append sequence serialization).

Priority

• priority:high: if real in current packaged SDK, this corrupts/stalls core orchestration runs and can require manual recovery.
• Not marked critical because current source has related mitigations and the exact raw Promise.all task pattern still needs a focused regression test.

Risk Analysis

• risk:high: replay identity and journal sequencing are central SDK contracts.
• Mitigations:
  • preserve legacy replay compatibility when changing invocation keys;
  • keep strict journal sequence validation;
  • test both source and packaged CLI behavior;
  • add multi-process append stress coverage if the in-process repro passes but cookbook still fails.