From 41816dda72740f96357359b8ca2979e5f2ce8c3b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Titsworth-Morin?= Date: Mon, 24 Aug 2026 22:38:29 +0000 Subject: [PATCH 1/4] task: add taskrunner handoff mismatch bookkeeping --- ...taskrunner-handoff-mismatch-bookkeeping.md | 94 +++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md diff --git a/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md new file mode 100644 index 000000000..c7bb2f221 --- /dev/null +++ b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md @@ -0,0 +1,94 @@ +# TaskRunner handoff mismatch bookkeeping + +SAM task: `01M0TY33ERGTBJ5ZRDZHYHS0QJ` · Output branch: `sam/fix-remaining-tasksession-lifecycle-yhs0qj` + +## Problem + +`recoverStuckTasks()` still persists high-volume warnings that say: + +> TaskRunner DO completed but task still in 'in_progress' — possible D1 update failure + +Current production evidence shows this is usually not a D1 terminal-state failure. The +TaskRunner DO sets `completed=true` after successful orchestration handoff while the D1 task +row intentionally remains active until the agent/user lifecycle reaches an explicit terminal +state. Persisting that normal tuple as a warning leaves healthy or restorable task/session +workbook state displayed as a possible failure. + +## Production evidence + +Read-only production queries on 2026-08-24 after PR #1896 production deploy +(`2026-08-24T11:33:05Z`, merge `03595efd4`) found: + +- `0` new tasks failed with `workspace_deleted`. +- `0` `ceiling_supersession_query_failed` rows. +- `276` `do_task_status_mismatch` warnings across `24` task IDs. +- All sampled post-deploy mismatch rows had TaskRunner `completed=true`, + `doCurrentStep='running'`, and a non-terminal task whose liveness was one of: + - `workspace_deleted_snapshot_resumable`: `197` rows / `17` tasks + - `workspace_deleted_superseded_by_live_wake`: `54` rows / `5` tasks + - `task_acp_session_live`: `24` rows / `13` tasks + - `workspace_sleeping_resumable`: `1` row / `1` task + +This matches draft idea `01KT90PKF6167SXZ9YZY0R26MM`'s 2026-08-24 update: the original +TaskRunner/D1 drift finding is now mostly misleading telemetry from normal handoff, not a +demonstrated D1 write defect. + +## Research findings + +- PR #1896 / merge `03595efd4` is on current `main`; the dominant superseded-predecessor + false-failure path is already fixed and must not be reimplemented. +- `apps/api/src/scheduled/stuck-tasks.ts` already uses task-scoped liveness to reconcile + genuinely dead TaskRunner/D1 drift, and existing tests cover missing/error TaskRunner + status with dead runtime convergence. +- The remaining noisy branch is the `doStatus?.completed` diagnostic path in + `stuck-tasks.ts`: it persists `do_task_status_mismatch` for every completed TaskRunner + while D1 remains active, even when `currentStep='running'` represents successful + orchestration handoff and liveness is live/resumable/superseded. +- The existing 30-minute dedupe intentionally repeats these warnings forever for preserved + candidates. A genuine pre-handoff completed-DO inconsistency should be durable diagnostic + evidence, but one row per task is enough; repeating it does not help convergence. + +## Implementation checklist + +- [ ] Classify TaskRunner `completed=true` + `currentStep='running'` + active D1 task as normal + handoff, not a persisted D1 mismatch warning. +- [ ] Keep conclusive dead-runtime reconciliation unchanged. +- [ ] Preserve a durable warning for genuine completed-DO/pre-handoff active-task + inconsistencies. +- [ ] Change mismatch dedupe from a 30-minute repeat window to one durable diagnostic per task. +- [ ] Add deterministic regression tests for: + - [ ] live normal handoff (`task_acp_session_live`) emits no persisted mismatch warning; + - [ ] restorable/superseded preserved normal handoff emits no persisted mismatch warning; + - [ ] genuine pre-handoff completed-DO inconsistency emits one durable fact-based diagnostic; + - [ ] an existing diagnostic suppresses repeats. +- [ ] Update source-contract tests that currently encode the old repeated warning behavior. +- [ ] Run focused tests and required quality checks. +- [ ] Perform required specialist reviews. +- [ ] Create/push a focused PR; do not deploy to staging and do not merge. + +## Acceptance criteria + +- Normal TaskRunner handoff no longer creates `do_task_status_mismatch` rows for live, + resumable, or live-superseded work. +- A genuinely inconsistent completed TaskRunner before handoff still produces a bounded, + accurately named warning with diagnostic context. +- The sweep does not repeat the same mismatch warning every 30 minutes. +- No status terminalization semantics change, and the canonical idleness rule remains intact: + child tasks/durable waits do not keep parent sessions awake. + +## References + +- SAM idea `01M0SG7ZEE1XARK4QDG7V6HDPN` +- SAM idea `01KT90PKF6167SXZ9YZY0R26MM` +- PR #1896: https://github.com/raphaeltm/simple-agent-manager/pull/1896 +- `.claude/rules/47-control-loop-io-budget.md` +- `.claude/rules/58-terminal-verdicts-must-match-the-resumer.md` +- `.claude/rules/62-tests-must-observe-the-real-trigger.md` +- `.claude/rules/66-ownership-handoff-must-record-the-supersession.md` + +## Workflow exception + +The `/do` workflow normally commits the task file to `main` before implementation. This +repository auto-deploys production from `main`, and this task explicitly says not to merge or +enter the shared staging/deployment wave yet. The task file therefore lives on the output +branch and will be reviewed with the code PR. From 7fb05e03195cd7c4bcf280dc662001e5e1c69fda Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Titsworth-Morin?= Date: Mon, 24 Aug 2026 22:44:19 +0000 Subject: [PATCH 2/4] fix(api): suppress normal taskrunner handoff warnings --- apps/api/src/scheduled/stuck-tasks.ts | 124 ++++--- .../tests/unit/recovery-resilience.test.ts | 15 +- apps/api/tests/unit/stuck-tasks.test.ts | 308 ++++++++++++++++++ ...taskrunner-handoff-mismatch-bookkeeping.md | 29 +- 4 files changed, 417 insertions(+), 59 deletions(-) diff --git a/apps/api/src/scheduled/stuck-tasks.ts b/apps/api/src/scheduled/stuck-tasks.ts index 549621a31..9fd5495c6 100644 --- a/apps/api/src/scheduled/stuck-tasks.ts +++ b/apps/api/src/scheduled/stuck-tasks.ts @@ -83,6 +83,8 @@ import { const SUPERSEDED_TERMINATION_MESSAGE = 'Superseded by a later session wake; the conversation continued in a replacement ' + 'task and has since ended.'; +const TASK_RUNNER_NORMAL_HANDOFF_STEP = 'running'; +const TASK_RUNNER_MISMATCH_RECOVERY_TYPE = 'do_task_status_mismatch'; function parseMs(value: string | undefined, fallback: number): number { if (!value) return fallback; @@ -109,6 +111,22 @@ function describeStep(step: string | null): string { return STEP_DESCRIPTIONS[step] ?? step; } +function isNormalCompletedTaskRunnerHandoff( + task: StuckTaskCandidate, + doStatus: NonNullable +): boolean { + return task.status === 'in_progress' && doStatus.currentStep === TASK_RUNNER_NORMAL_HANDOFF_STEP; +} + +function taskRunnerMismatchKind( + task: StuckTaskCandidate, + doStatus: NonNullable +): 'completed_handoff_missing_in_d1' | 'completed_before_running_handoff' { + return doStatus.currentStep === TASK_RUNNER_NORMAL_HANDOFF_STEP && task.status !== 'in_progress' + ? 'completed_handoff_missing_in_d1' + : 'completed_before_running_handoff'; +} + export interface StuckTaskResult { failedQueued: number; failedDelegated: number; @@ -1223,50 +1241,72 @@ export async function recoverStuckTasks(env: Env): Promise { }); } - if (doStatus?.completed) { - // Reconcile only with conclusive dead-runtime evidence. Live or unknown - // task-scoped runtime state remains active and is logged for investigation. - // Deduplicate the persisted mismatch signal independently of reconciliation. - log.warn('stuck_task.do_completed_but_task_active', { - taskId: task.id, - taskStatus: task.status, - doCurrentStep: doStatus.currentStep, - doRetryCount: doStatus.retryCount, - }); - - // Deduplicate: only persist if no recent mismatch record exists for this task - const recentMismatch = await env.OBSERVABILITY_DATABASE.prepare( - `SELECT id FROM platform_errors - WHERE task_id = ? AND context LIKE ? AND timestamp > ? - LIMIT 1` - ) - .bind(task.id, '%do_task_status_mismatch%', Date.now() - 30 * 60 * 1000) - .first(); - - if (!recentMismatch) { - await persistError( - env.OBSERVABILITY_DATABASE, - { - source: 'api', - level: 'warn', - message: `TaskRunner DO completed but task still in '${task.status}' — possible D1 update failure`, - context: { - recoveryType: 'do_task_status_mismatch', + if (doStatus?.completed && !isStuck) { + if (isNormalCompletedTaskRunnerHandoff(task, doStatus)) { + // `transitionToInProgress` deliberately stores TaskRunner + // `completed=true` at the successful handoff boundary while the D1 task + // remains active until an explicit agent/user terminal path. This is + // normal lifecycle bookkeeping, not D1 drift; production evidence on + // 2026-08-24 showed these rows were live, restorable, or live-superseded. + log.info('stuck_task.do_completed_handoff_active', { + taskId: task.id, + taskStatus: task.status, + executionStep: task.execution_step, + doCurrentStep: doStatus.currentStep, + doRetryCount: doStatus.retryCount, + livenessReason: liveness?.reason ?? null, + action: 'observed_normal_handoff', + }); + } else { + const mismatchKind = taskRunnerMismatchKind(task, doStatus); + log.warn('stuck_task.do_completed_active_state_mismatch', { + taskId: task.id, + taskStatus: task.status, + executionStep: task.execution_step, + doCurrentStep: doStatus.currentStep, + doRetryCount: doStatus.retryCount, + mismatchKind, + }); + + // One durable diagnostic per task is enough. Repeating the same + // preserved candidate every 30 minutes caused the production noise that + // hid the real state: normal handoff and resumable/superseded sessions. + const existingMismatch = await env.OBSERVABILITY_DATABASE.prepare( + `SELECT id FROM platform_errors + WHERE task_id = ? AND context LIKE ? + LIMIT 1` + ) + .bind(task.id, `%${TASK_RUNNER_MISMATCH_RECOVERY_TYPE}%`) + .first(); + + if (!existingMismatch) { + await persistError( + env.OBSERVABILITY_DATABASE, + { + source: 'api', + level: 'warn', + message: + `TaskRunner DO reports completed at '${doStatus.currentStep}' while ` + + `task remains '${task.status}' — active state mismatch before normal handoff convergence`, + context: { + recoveryType: TASK_RUNNER_MISMATCH_RECOVERY_TYPE, + mismatchKind, + taskId: task.id, + taskStatus: task.status, + executionStep: task.execution_step, + doCurrentStep: doStatus.currentStep, + doRetryCount: doStatus.retryCount, + timeForCheck, + taskRunnerProbeOutcome: doProbe.outcome, + livenessReason: liveness?.reason ?? null, + }, + userId: task.user_id, taskId: task.id, - taskStatus: task.status, - executionStep: task.execution_step, - doCurrentStep: doStatus.currentStep, - doRetryCount: doStatus.retryCount, - timeForCheck, - taskRunnerProbeOutcome: doProbe.outcome, - livenessReason: liveness?.reason ?? null, + sessionId: task.chat_session_id, }, - userId: task.user_id, - taskId: task.id, - sessionId: task.chat_session_id, - }, - env - ); + env + ); + } } } } diff --git a/apps/api/tests/unit/recovery-resilience.test.ts b/apps/api/tests/unit/recovery-resilience.test.ts index c84608451..8aef704b2 100644 --- a/apps/api/tests/unit/recovery-resilience.test.ts +++ b/apps/api/tests/unit/recovery-resilience.test.ts @@ -167,19 +167,22 @@ describe('stuck-tasks DO health checks (TDF-7)', () => { expect(healthCheckSection).toContain('timeForCheck'); }); - it('detects DO-completed-but-task-active mismatch', () => { - expect(stuckTasksSource).toContain('stuck_task.do_completed_but_task_active'); + it('detects completed-DO active-state mismatches while recognizing normal handoff', () => { + expect(stuckTasksSource).toContain('stuck_task.do_completed_handoff_active'); + expect(stuckTasksSource).toContain('stuck_task.do_completed_active_state_mismatch'); expect(stuckTasksSource).toContain('doStatus?.completed'); }); it('records DO mismatch in OBSERVABILITY_DATABASE', () => { - expect(stuckTasksSource).toContain("recoveryType: 'do_task_status_mismatch'"); + expect(stuckTasksSource).toContain('TASK_RUNNER_MISMATCH_RECOVERY_TYPE'); + expect(stuckTasksSource).toContain("'do_task_status_mismatch'"); }); - it('deduplicates DO mismatch records (30 min window)', () => { - expect(stuckTasksSource).toContain('recentMismatch'); + it('deduplicates DO mismatch records once per task', () => { + expect(stuckTasksSource).toContain('existingMismatch'); expect(stuckTasksSource).toContain('do_task_status_mismatch'); - expect(stuckTasksSource).toContain('30 * 60 * 1000'); + expect(stuckTasksSource).toContain('One durable diagnostic per task is enough'); + expect(stuckTasksSource).not.toContain('30 * 60 * 1000'); }); it('tracks doHealthChecked count in result', () => { diff --git a/apps/api/tests/unit/stuck-tasks.test.ts b/apps/api/tests/unit/stuck-tasks.test.ts index c035b2581..bbe417acf 100644 --- a/apps/api/tests/unit/stuck-tasks.test.ts +++ b/apps/api/tests/unit/stuck-tasks.test.ts @@ -143,6 +143,16 @@ function createMockEnv( } as unknown as Env; } +function persistedRecoveryTypes(): unknown[] { + return vi + .mocked(persistError) + .mock.calls.map(([, payload]) => payload.context?.recoveryType); +} + +function expectNoPersistedTaskRunnerMismatch(): void { + expect(persistedRecoveryTypes()).not.toContain('do_task_status_mismatch'); +} + describe('recoverStuckTasks', () => { beforeEach(() => { vi.clearAllMocks(); @@ -1054,6 +1064,185 @@ describe('recoverStuckTasks', () => { }); describe('prompt dead-runtime reconciliation', () => { + function completedTaskRunnerStatus(currentStep = 'running') { + return { + completed: true, + currentStep, + retryCount: 0, + lastStepAt: Date.now() - 10 * 60 * 1000, + }; + } + + function activeHandoffResponses(taskId: string) { + const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000).toISOString(); + const recentHeartbeat = new Date(Date.now() - 30 * 1000).toISOString(); + return new Map([ + [ + "status IN ('queued', 'delegated', 'in_progress')", + { + results: [ + { + id: taskId, + project_id: 'proj-1', + user_id: 'user-1', + status: 'in_progress', + execution_step: 'running', + updated_at: threeHoursAgo, + started_at: threeHoursAgo, + workspace_id: 'ws-1', + auto_provisioned_node_id: 'node-1', + chat_session_id: 'chat-1', + }, + ], + }, + ], + [ + 'w.chat_session_id', + { + results: [ + { + workspace_status: 'running', + chat_session_id: 'chat-1', + node_id: 'node-1', + node_status: 'running', + health_status: 'healthy', + last_heartbeat_at: recentHeartbeat, + node_runtime: 'vm', + }, + ], + }, + ], + ]); + } + + function deletedSnapshotHandoffResponses(taskId: string) { + const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000).toISOString(); + const sleepingAt = new Date(Date.now() - 30 * 60 * 1000).toISOString(); + const expiresAt = new Date(Date.now() + 7 * 24 * 60 * 60 * 1000).toISOString(); + return new Map([ + [ + "status IN ('queued', 'delegated', 'in_progress')", + { + results: [ + { + id: taskId, + project_id: 'proj-1', + user_id: 'user-1', + status: 'in_progress', + execution_step: 'awaiting_followup', + updated_at: threeHoursAgo, + started_at: threeHoursAgo, + workspace_id: 'ws-1', + auto_provisioned_node_id: 'node-1', + chat_session_id: 'chat-1', + }, + ], + }, + ], + [ + 'w.chat_session_id', + { + results: [ + { + workspace_status: 'deleted', + chat_session_id: 'chat-1', + node_id: 'node-1', + node_status: 'deleted', + health_status: 'stale', + last_heartbeat_at: threeHoursAgo, + node_runtime: 'vm', + }, + ], + }, + ], + [ + 'FROM session_snapshots', + { + results: [ + { + chat_session_id: 'chat-1', + project_id: 'proj-1', + workspace_id: 'ws-1', + sleeping_at: sleepingAt, + sleep_status: 'sleeping', + expires_at: expiresAt, + status: 'available', + degradation: 'none', + recovery_attempts: 0, + }, + ], + }, + ], + ]); + } + + function liveSupersededHandoffResponses(taskId: string) { + const threeHoursAgo = new Date(Date.now() - 3 * 60 * 60 * 1000).toISOString(); + return new Map([ + [ + "status IN ('queued', 'delegated', 'in_progress')", + { + results: [ + { + id: taskId, + project_id: 'proj-1', + user_id: 'user-1', + status: 'in_progress', + execution_step: 'running', + updated_at: threeHoursAgo, + started_at: threeHoursAgo, + workspace_id: 'ws-1', + auto_provisioned_node_id: 'node-1', + chat_session_id: null, + }, + ], + }, + ], + [ + 'w.chat_session_id', + { + results: [ + { + workspace_status: 'deleted', + chat_session_id: null, + node_id: 'node-1', + node_status: 'deleted', + health_status: 'stale', + last_heartbeat_at: threeHoursAgo, + node_runtime: 'vm', + }, + ], + }, + ], + ['owner.status NOT IN', { results: [{ found: 1 }] }], + ]); + } + + function queuedCompletedMismatchResponses(taskId: string) { + const sixMinutesAgo = new Date(Date.now() - 6 * 60 * 1000).toISOString(); + return new Map([ + [ + "status IN ('queued', 'delegated', 'in_progress')", + { + results: [ + { + id: taskId, + project_id: 'proj-1', + user_id: 'user-1', + status: 'queued', + execution_step: 'agent_session', + updated_at: sixMinutesAgo, + started_at: null, + workspace_id: null, + auto_provisioned_node_id: null, + chat_session_id: 'chat-queued', + }, + ], + }, + ], + ]); + } + function resumableRuntimeResponses(workspaceStatus: 'sleeping' | 'recovery') { const tenMinutesAgo = new Date(Date.now() - 10 * 60 * 1000).toISOString(); return new Map([ @@ -1222,6 +1411,125 @@ describe('recoverStuckTasks', () => { }); }); + it('does not persist a D1 mismatch warning for a live completed TaskRunner handoff', async () => { + const env = createMockEnv( + activeHandoffResponses('task-live-handoff'), + { TASK_DO_MISMATCH_GRACE_MS: '60000' }, + completedTaskRunnerStatus() + ); + + const result = await recoverStuckTasks(env); + + expect(result.failedInProgress).toBe(0); + expect(result.deadRuntimeReconciled).toBe(0); + expect(result.doHealthChecked).toBe(1); + expectNoPersistedTaskRunnerMismatch(); + expect(env.OBSERVABILITY_DATABASE.prepare).not.toHaveBeenCalled(); + }); + + it('does not persist a D1 mismatch warning for a restorable completed handoff', async () => { + const env = createMockEnv( + deletedSnapshotHandoffResponses('task-restorable-handoff'), + { TASK_DO_MISMATCH_GRACE_MS: '60000' }, + completedTaskRunnerStatus() + ); + + const result = await recoverStuckTasks(env); + + expect(result.failedInProgress).toBe(0); + expect(result.deadRuntimeReconciled).toBe(0); + expect(result.doHealthChecked).toBe(1); + expectNoPersistedTaskRunnerMismatch(); + expect(env.OBSERVABILITY_DATABASE.prepare).not.toHaveBeenCalled(); + }); + + it('does not persist a D1 mismatch warning for a live-superseded completed handoff', async () => { + const env = createMockEnv( + liveSupersededHandoffResponses('task-live-superseded-handoff'), + { TASK_DO_MISMATCH_GRACE_MS: '60000' }, + completedTaskRunnerStatus() + ); + + const result = await recoverStuckTasks(env); + + expect(result.failedInProgress).toBe(0); + expect(result.deadRuntimeReconciled).toBe(0); + expect(result.doHealthChecked).toBe(1); + expectNoPersistedTaskRunnerMismatch(); + expect(env.OBSERVABILITY_DATABASE.prepare).not.toHaveBeenCalled(); + }); + + it('still reconciles a conclusively dead runtime after a completed TaskRunner handoff', async () => { + const env = createMockEnv( + deadRuntimeResponses('task-dead-completed-handoff'), + { TASK_DO_MISMATCH_GRACE_MS: '60000' }, + completedTaskRunnerStatus() + ); + + const result = await recoverStuckTasks(env); + + expect(result.failedInProgress).toBe(1); + expect(result.deadRuntimeReconciled).toBe(1); + expect(result.doHealthChecked).toBe(1); + expectNoPersistedTaskRunnerMismatch(); + expect(syncTriggerExecutionMock).toHaveBeenCalledWith( + env.DATABASE, + 'task-dead-completed-handoff', + 'failed', + expect.stringContaining('workspace_deleted') + ); + }); + + it('persists one diagnostic for a completed TaskRunner before normal handoff convergence', async () => { + const env = createMockEnv( + queuedCompletedMismatchResponses('task-queued-do-completed'), + { TASK_DO_MISMATCH_GRACE_MS: '60000' }, + completedTaskRunnerStatus('agent_session') + ); + + const result = await recoverStuckTasks(env); + + expect(result.failedQueued).toBe(0); + expect(result.doHealthChecked).toBe(1); + expect(persistError).toHaveBeenCalledWith( + env.OBSERVABILITY_DATABASE, + expect.objectContaining({ + source: 'api', + level: 'warn', + message: expect.stringContaining( + "TaskRunner DO reports completed at 'agent_session' while task remains 'queued'" + ), + context: expect.objectContaining({ + recoveryType: 'do_task_status_mismatch', + mismatchKind: 'completed_before_running_handoff', + taskId: 'task-queued-do-completed', + taskStatus: 'queued', + doCurrentStep: 'agent_session', + livenessReason: null, + }), + }), + env + ); + }); + + it('does not repeat an existing completed-DO active-task diagnostic', async () => { + const env = createMockEnv( + queuedCompletedMismatchResponses('task-queued-do-completed-once'), + { TASK_DO_MISMATCH_GRACE_MS: '60000' }, + completedTaskRunnerStatus('agent_session') + ); + env.OBSERVABILITY_DATABASE = { + prepare: vi.fn().mockReturnValue(mockPreparedStatement([{ id: 'existing-warning' }])), + } as unknown as D1Database; + + const result = await recoverStuckTasks(env); + + expect(result.failedQueued).toBe(0); + expect(result.doHealthChecked).toBe(1); + expectNoPersistedTaskRunnerMismatch(); + expect(env.OBSERVABILITY_DATABASE.prepare).toHaveBeenCalledTimes(1); + }); + it('reports the exact read-only reconciliation decision for admins', async () => { const tenMinutesAgo = new Date(Date.now() - 10 * 60 * 1000).toISOString(); const responses = deadRuntimeResponses('task-admin-diagnostics'); diff --git a/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md index c7bb2f221..f4a66f22e 100644 --- a/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md +++ b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md @@ -50,19 +50,19 @@ demonstrated D1 write defect. ## Implementation checklist -- [ ] Classify TaskRunner `completed=true` + `currentStep='running'` + active D1 task as normal +- [x] Classify TaskRunner `completed=true` + `currentStep='running'` + active D1 task as normal handoff, not a persisted D1 mismatch warning. -- [ ] Keep conclusive dead-runtime reconciliation unchanged. -- [ ] Preserve a durable warning for genuine completed-DO/pre-handoff active-task +- [x] Keep conclusive dead-runtime reconciliation unchanged. +- [x] Preserve a durable warning for genuine completed-DO/pre-handoff active-task inconsistencies. -- [ ] Change mismatch dedupe from a 30-minute repeat window to one durable diagnostic per task. -- [ ] Add deterministic regression tests for: - - [ ] live normal handoff (`task_acp_session_live`) emits no persisted mismatch warning; - - [ ] restorable/superseded preserved normal handoff emits no persisted mismatch warning; - - [ ] genuine pre-handoff completed-DO inconsistency emits one durable fact-based diagnostic; - - [ ] an existing diagnostic suppresses repeats. -- [ ] Update source-contract tests that currently encode the old repeated warning behavior. -- [ ] Run focused tests and required quality checks. +- [x] Change mismatch dedupe from a 30-minute repeat window to one durable diagnostic per task. +- [x] Add deterministic regression tests for: + - [x] live normal handoff (`task_acp_session_live`) emits no persisted mismatch warning; + - [x] restorable/superseded preserved normal handoff emits no persisted mismatch warning; + - [x] genuine pre-handoff completed-DO inconsistency emits one durable fact-based diagnostic; + - [x] an existing diagnostic suppresses repeats. +- [x] Update source-contract tests that currently encode the old repeated warning behavior. +- [x] Run focused tests and required quality checks. - [ ] Perform required specialist reviews. - [ ] Create/push a focused PR; do not deploy to staging and do not merge. @@ -92,3 +92,10 @@ The `/do` workflow normally commits the task file to `main` before implementatio repository auto-deploys production from `main`, and this task explicitly says not to merge or enter the shared staging/deployment wave yet. The task file therefore lives on the output branch and will be reviewed with the code PR. + +## Validation evidence + +- `pnpm --filter @simple-agent-manager/api test -- tests/unit/stuck-tasks.test.ts` — 52 passed. +- `pnpm --filter @simple-agent-manager/api test -- tests/unit/recovery-resilience.test.ts` — 69 passed. +- `pnpm --filter @simple-agent-manager/api typecheck` — passed. +- `pnpm --filter @simple-agent-manager/api lint` — passed. From 6dbedff4e8980edd746018f11f867277ea3ccc0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Titsworth-Morin?= Date: Mon, 24 Aug 2026 22:57:55 +0000 Subject: [PATCH 3/4] chore(task): record lifecycle review evidence --- apps/api/src/scheduled/stuck-tasks.ts | 3 ++- ...taskrunner-handoff-mismatch-bookkeeping.md | 23 ++++++++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/apps/api/src/scheduled/stuck-tasks.ts b/apps/api/src/scheduled/stuck-tasks.ts index 9fd5495c6..93cb9a2d3 100644 --- a/apps/api/src/scheduled/stuck-tasks.ts +++ b/apps/api/src/scheduled/stuck-tasks.ts @@ -34,6 +34,7 @@ import { DEFAULT_TASK_RUN_MAX_EXECUTION_MS, DEFAULT_TASK_STUCK_DELEGATED_TIMEOUT_MS, DEFAULT_TASK_STUCK_QUEUED_TIMEOUT_MS, + type TaskExecutionStep, } from '@simple-agent-manager/shared'; import { drizzle } from 'drizzle-orm/d1'; @@ -83,7 +84,7 @@ import { const SUPERSEDED_TERMINATION_MESSAGE = 'Superseded by a later session wake; the conversation continued in a replacement ' + 'task and has since ended.'; -const TASK_RUNNER_NORMAL_HANDOFF_STEP = 'running'; +const TASK_RUNNER_NORMAL_HANDOFF_STEP: TaskExecutionStep = 'running'; const TASK_RUNNER_MISMATCH_RECOVERY_TYPE = 'do_task_status_mismatch'; function parseMs(value: string | undefined, fallback: number): number { diff --git a/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md index f4a66f22e..a1b07b8bc 100644 --- a/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md +++ b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md @@ -63,7 +63,7 @@ demonstrated D1 write defect. - [x] an existing diagnostic suppresses repeats. - [x] Update source-contract tests that currently encode the old repeated warning behavior. - [x] Run focused tests and required quality checks. -- [ ] Perform required specialist reviews. +- [x] Perform required specialist reviews. - [ ] Create/push a focused PR; do not deploy to staging and do not merge. ## Acceptance criteria @@ -99,3 +99,24 @@ branch and will be reviewed with the code PR. - `pnpm --filter @simple-agent-manager/api test -- tests/unit/recovery-resilience.test.ts` — 69 passed. - `pnpm --filter @simple-agent-manager/api typecheck` — passed. - `pnpm --filter @simple-agent-manager/api lint` — passed. +- `pnpm typecheck` — 19 tasks passed. +- `pnpm lint` — 13 tasks passed; warning-only pre-existing a11y/hooks findings outside this diff. +- `pnpm test` — 604 test files / 8,230 tests passed; 21 tasks passed. +- `pnpm build` — 9 tasks passed; warning-only existing Turbo output warning for API build outputs. +- `git diff --check origin/main...HEAD` — passed. + +## Specialist review evidence + +- `test-engineer` — PASS. New deterministic regression tests call `recoverStuckTasks()` + through the scheduled-sweep entry point and model the relevant D1, TaskRunner DO, + ProjectData/ACP, snapshot, and supersession boundaries with concrete rows. +- `cloudflare-specialist` — PASS. No migration or binding changes. The hot production + path no longer writes to `OBSERVABILITY_DATABASE`; the remaining diagnostic lookup + is task-scoped and uses the existing `idx_platform_errors_task_id` index before + applying the JSON-context substring filter. +- `constitution-validator` — PASS. No URLs, timeouts, limits, secrets, or + deployment-specific identifiers were added. The new handoff sentinel is typed as + shared `TaskExecutionStep` protocol vocabulary. +- `task-completion-validator` — PASS. Research findings and acceptance criteria map to + the implemented diff and tests; UI/backend and multi-resource checks are not + applicable for this scheduled API bookkeeping change. From b502276b39d0add823015e2ecee7d0cd4d55545e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Titsworth-Morin?= Date: Mon, 24 Aug 2026 22:59:55 +0000 Subject: [PATCH 4/4] chore(task): record lifecycle PR --- .../2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md index a1b07b8bc..d889988fd 100644 --- a/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md +++ b/tasks/active/2026-08-24-taskrunner-handoff-mismatch-bookkeeping.md @@ -64,7 +64,7 @@ demonstrated D1 write defect. - [x] Update source-contract tests that currently encode the old repeated warning behavior. - [x] Run focused tests and required quality checks. - [x] Perform required specialist reviews. -- [ ] Create/push a focused PR; do not deploy to staging and do not merge. +- [x] Create/push a focused PR; do not deploy to staging and do not merge. ## Acceptance criteria @@ -81,6 +81,7 @@ demonstrated D1 write defect. - SAM idea `01M0SG7ZEE1XARK4QDG7V6HDPN` - SAM idea `01KT90PKF6167SXZ9YZY0R26MM` - PR #1896: https://github.com/raphaeltm/simple-agent-manager/pull/1896 +- PR #1899: https://github.com/raphaeltm/simple-agent-manager/pull/1899 - `.claude/rules/47-control-loop-io-budget.md` - `.claude/rules/58-terminal-verdicts-must-match-the-resumer.md` - `.claude/rules/62-tests-must-observe-the-real-trigger.md`