diff --git a/docs/plans/2026-08-12-spawn-robustness.md b/docs/plans/2026-08-12-spawn-robustness.md new file mode 100644 index 00000000..4ca2c507 --- /dev/null +++ b/docs/plans/2026-08-12-spawn-robustness.md @@ -0,0 +1,82 @@ +# Spawn Robustness Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Make every surface-creating spawn path recoverable and diagnosable when shell readiness, launcher submission, or post-creation work fails. + +**Architecture:** Share shell-prompt recognition between the MCP and app-server paths, and introduce one creation-failure scope that records identities as soon as cmux creates them and attaches those identities to any later error without changing the error's runtime type. Centralize readiness diagnostics for direct and `AgentLaunchError`-wrapped failures. Treat a launcher that returns to a shell as an explicit launch failure, verify a pending launcher command actually leaves the prompt after Return, and roll back a newly created worktree only after a failed launcher surface is closed. + +**Tech Stack:** TypeScript, Vitest, cmux client abstractions, MCP tool structured responses. + +--- + +### Task 1: Shared shell-prompt contract + +**Files:** +- Create: `src/shell-prompt.ts` +- Modify: `src/server.ts` +- Modify: `src/app-server-runtime.ts` +- Test: `tests/shell-prompt.test.ts` +- Test: `tests/app-server-runtime.test.ts` + +1. Write failing table tests for ready prompts ending in `$`, `%`, `#`, `>`, `❯`, `›`, and `»`, plus negative cases where text remains after the prompt terminator. +2. Run the focused tests and confirm `>`/Unicode prompts fail under the current matchers. +3. Add one shared matcher and replace both private implementations. +4. Run the focused tests green. + +### Task 2: Structural created-identity propagation + +**Files:** +- Create: `src/created-identity.ts` +- Modify: `src/server.ts` +- Test: `tests/created-identity.test.ts` +- Test: `tests/server.test.ts` +- Test: `tests/server-agent-tools.test.ts` + +1. Write failing unit tests proving an unclassified post-creation error receives the recorded identity, pre-creation errors receive none, prior batch identities accumulate, and error-supplied metadata cannot overwrite identity fields. +2. Write integration regressions that inject unknown post-creation failures in raw and managed creation paths. +3. Run focused tests and confirm identity is absent before implementation. +4. Add a creation scope that records identities and decorates any thrown error while preserving `instanceof` and `cause` behavior; make `err()` merge recorded identity last. +5. Record identities immediately after every current creation seam: `new_split`, `new_surface`, terminal `spawn`, managed `spawn`, `new_worktree_split`, and each `spawn_in_workspace` member. +6. Run focused tests green. + +### Task 3: Readiness and launcher diagnostics + +**Files:** +- Modify: `src/agent-engine.ts` +- Modify: `src/server.ts` +- Test: `tests/server.test.ts` +- Test: `tests/server-agent-tools.test.ts` + +1. Write failing regressions for an `AgentLaunchError` wrapping a readiness timeout and for a launcher returning to a shell with terminal error text. +2. Run focused tests and confirm `last_10_lines`/launcher text is lost or reduced to a generic timeout. +3. Set standard `Error.cause` on `AgentLaunchError`, recursively recover timeout diagnostics, and expose them through the shared error formatter. +4. Detect stable return-to-shell after launcher submission and return the captured terminal tail as diagnostics instead of waiting for a generic timeout. +5. Run focused tests green. + +### Task 4: Return verification and post-launch rollback + +**Files:** +- Modify: `src/server.ts` +- Test: `tests/server.test.ts` +- Test: `tests/server-agent-tools.test.ts` + +1. Write failing regressions where Return reports success but the launcher command remains at the shell prompt, and where a launcher failure after surface creation leaks a new worktree/branch/surface. +2. Run focused tests and confirm both failures. +3. After sending Return to a pending launcher command, require screen evidence that the pending command cleared or a supported CLI became ready. +4. On launch-phase `AgentLaunchError`, close the created surface, then roll back the newly created worktree and branch; preserve created identity and append cleanup diagnostics if either cleanup step fails. +5. Run focused tests green. + +### Task 5: Verification and worker handoff + +**Files:** +- Modify: `/Users/etanheyman/Gits/cmuxlayer/docs.local/plan/stability-v2/phase-7/findings.md` +- Modify: `/Users/etanheyman/Gits/cmuxlayer/docs.local/plan/stability-v2/collab.md` + +1. Write a PREDICTION block before suite execution with expected focused/full outcomes and likely failure boundaries. +2. Run focused suites, typecheck, build, full tests, and `git diff --check`; compare actuals against the prediction. +3. Run the daemon/runtime gate with a real cmuxlayer client because this lane changes MCP spawn behavior. +4. Review the final diff and run the bounded local CodeRabbit pre-commit review. +5. Commit with the live agent-identity trailer, push the assigned branch, and open a signed ready-for-review PR. +6. Append the collab log line and inbox-ping `cmuxlayerClaude-9c55eb04` with the PR URL. If the inbox is unarmed, append the PR URL as the final line of `phase-7/findings.md`. + diff --git a/src/agent-engine.ts b/src/agent-engine.ts index d444c6e4..ae26bf34 100644 --- a/src/agent-engine.ts +++ b/src/agent-engine.ts @@ -235,6 +235,7 @@ export interface SpawnAgentParams { * the surface, before launcher I/O or readiness polling can give the user time to move. */ on_surface_created?: (surface: { + agent_id: string; surface: string; workspace?: string; }) => void | Promise; @@ -261,8 +262,9 @@ export class AgentLaunchError extends Error { readonly surface_id: string, readonly workspace_id?: string, readonly launch_cause?: unknown, + readonly launch_phase: "focus" | "launch" = "launch", ) { - super(message); + super(message, launch_cause === undefined ? undefined : { cause: launch_cause }); this.name = "AgentLaunchError"; } } @@ -5389,6 +5391,7 @@ export class AgentEngine { } try { await spawnParams.on_surface_created?.({ + agent_id: agentId, surface: surface.surface, workspace: createdWorkspace, }); @@ -5408,6 +5411,7 @@ export class AgentEngine { surface.surface, createdWorkspace, surfaceFocusError, + "focus", ); } diff --git a/src/app-server-runtime.ts b/src/app-server-runtime.ts index 6c707f40..9e639a10 100644 --- a/src/app-server-runtime.ts +++ b/src/app-server-runtime.ts @@ -23,6 +23,7 @@ import type { InboxOpts } from "./inbox.js"; import { parseScreen } from "./screen-parser.js"; import { sanitizeTerminalInput } from "./sanitize.js"; import { matchReadyPattern } from "./pattern-registry.js"; +import { matchesShellPrompt } from "./shell-prompt.js"; import { assertMutationAllowed } from "./mode-policy.js"; import { findWorkspaceRefForRepo } from "./repo-workspace.js"; import { partitionPaneSurfacesByMembership } from "./pane-surfaces.js"; @@ -152,10 +153,6 @@ async function delay(ms: number): Promise { await new Promise((resolve) => setTimeout(resolve, ms)); } -function matchesShellPrompt(text: string): boolean { - return /(?:^|\n)[^\n]*[$%#]\s*$/.test(text); -} - function deriveRepoFromCwd(cwd: string): string { const repo = basename(cwd.trim()); if (!repo || repo === "." || repo === "/" || repo === "..") { diff --git a/src/created-identity.ts b/src/created-identity.ts new file mode 100644 index 00000000..0dcf618b --- /dev/null +++ b/src/created-identity.ts @@ -0,0 +1,64 @@ +const CREATED_IDENTITY = Symbol("cmuxlayer.created_identity"); + +type CreatedIdentityCarrier = { + [CREATED_IDENTITY]?: Record; +}; + +function asError(error: unknown): Error & CreatedIdentityCarrier { + if (error instanceof Error) { + return error as Error & CreatedIdentityCarrier; + } + return new Error(String(error), { cause: error }) as Error & + CreatedIdentityCarrier; +} + +export class CreatedIdentityScope { + private readonly identity: Record = {}; + + record(identity: Record): void { + for (const [key, value] of Object.entries(identity)) { + if (value !== undefined) { + this.identity[key] = value; + } + } + } + + append( + key: string, + identity: Record, + sameIdentity: (left: Record, right: Record) => boolean, + ): void { + const current = Array.isArray(this.identity[key]) + ? ([...(this.identity[key] as Record[])] as Record< + string, + unknown + >[]) + : []; + const index = current.findIndex((candidate) => + sameIdentity(candidate, identity), + ); + if (index >= 0) { + current[index] = { ...current[index], ...identity }; + } else { + current.push({ ...identity }); + } + this.identity[key] = current; + } + + attach(error: unknown): Error { + const target = asError(error); + const existing = target[CREATED_IDENTITY] ?? {}; + Object.defineProperty(target, CREATED_IDENTITY, { + configurable: true, + value: { ...existing, ...this.identity }, + }); + return target; + } +} + +export function createdIdentityFromError( + error: unknown, +): Record { + if (!(error instanceof Error)) return {}; + return { ...((error as Error & CreatedIdentityCarrier)[CREATED_IDENTITY] ?? {}) }; +} diff --git a/src/server.ts b/src/server.ts index 6cdaafd6..3320c2b3 100644 --- a/src/server.ts +++ b/src/server.ts @@ -118,6 +118,15 @@ import { isPickerOrMenuScreen, parseScreen, } from "./screen-parser.js"; +import { + launcherFailureFromShell, + matchShellPromptLine, + matchesShellPrompt, +} from "./shell-prompt.js"; +import { + CreatedIdentityScope, + createdIdentityFromError, +} from "./created-identity.js"; import { dispatch, ensureInboxFile, @@ -778,6 +787,16 @@ class BootPromptTimeoutError extends Error { } } +class LauncherReadinessError extends Error { + constructor( + message: string, + readonly last_10_lines: string[], + ) { + super(message); + this.name = "LauncherReadinessError"; + } +} + class BootPromptDeliveryError extends Error { constructor( message: string, @@ -966,6 +985,15 @@ function err(error: unknown, extra: Record = {}): ToolReturn { error.code === PLACEMENT_WORKSPACE_UNRESOLVED ? { error_code: PLACEMENT_WORKSPACE_UNRESOLVED } : {}; + const readinessTimeout = findErrorInChain( + error, + (candidate): candidate is BootPromptTimeoutError | LauncherReadinessError => + candidate instanceof BootPromptTimeoutError || + candidate instanceof LauncherReadinessError, + ); + const readinessExtra = readinessTimeout + ? { last_10_lines: readinessTimeout.last_10_lines } + : {}; const retryMeta = error && typeof error === "object" ? { @@ -993,7 +1021,9 @@ function err(error: unknown, extra: Record = {}): ToolReturn { ...deliverySafetyExtra, ...submitVerificationExtra, ...placementWorkspaceExtra, + ...readinessExtra, ...extra, + ...createdIdentityFromError(error), }; return { content: [{ type: "text", text: JSON.stringify(payload) }], @@ -1002,6 +1032,23 @@ function err(error: unknown, extra: Record = {}): ToolReturn { }; } +function findErrorInChain( + error: unknown, + predicate: (error: Error) => error is T, +): T | null { + const seen = new Set(); + let current = error; + while (current instanceof Error && !seen.has(current)) { + if (predicate(current)) return current; + seen.add(current); + current = + current instanceof AgentLaunchError && current.launch_cause !== undefined + ? current.launch_cause + : current.cause; + } + return null; +} + function requireValue( value: string | number | undefined, message: string, @@ -1721,36 +1768,6 @@ function isLauncherShellCommand(command: string): boolean { return /(?:^|\s)[\w.-]+(?:Claude|Codex|Cursor|Gemini)(?=\s|$)/.test(command); } -function matchShellPromptLine( - line: string, - opts?: { allowRootInput?: boolean }, -): { input: string } | null { - const normalized = line.trimEnd(); - const barePrompt = normalized.match(/^\s*([$%])(?:\s+(.*))?$/); - if (barePrompt) { - return { input: barePrompt[2] ?? "" }; - } - const rootPrompt = normalized.match(/^\s*#(?:\s+(.*))?$/); - if (rootPrompt && (!rootPrompt[1] || opts?.allowRootInput)) { - return { input: rootPrompt[1] ?? "" }; - } - - const prefixedPrompt = normalized.match( - /^\s*(?:(?:\S+@\S+)(?:\s+(?:~|\/)\S*)?|(?:\S+\s+)?(?:~|\/)\S*)(?:\s+\[[^\]]+\])?\s*[$%#](?:\s+(.*))?$/, - ); - return prefixedPrompt ? { input: prefixedPrompt[1] ?? "" } : null; -} - -function matchesShellPrompt(text: string): boolean { - const lines = normalizeTerminalText(text).split("\n"); - let end = lines.length; - while (end > 0 && !lines[end - 1]?.trim()) { - end -= 1; - } - const prompt = end > 0 ? matchShellPromptLine(lines[end - 1] ?? "") : null; - return prompt?.input.trim() === ""; -} - function shouldHandleCodexUpdateMenu( cli: CliType | undefined, text: string, @@ -2299,8 +2316,19 @@ export function screenShowsPendingShellInput( let activePromptIndex = -1; for (let index = end - 1; index >= 0; index -= 1) { const line = lines[index]?.trimEnd() ?? ""; - const prompt = matchShellPromptLine(line, promptOptions); + const strictPrompt = matchShellPromptLine(line, { + ...promptOptions, + strict: true, + }); + const prompt = strictPrompt ?? matchShellPromptLine(line, promptOptions); if (prompt) { + // The readiness matcher intentionally accepts any decorated $/%/# + // suffix. Only use that loose fallback as pending-input evidence for a + // launcher command; ordinary output such as "Building... 62%" is not a + // trustworthy prompt anchor. + if (!strictPrompt && !promptOptions.allowRootInput) { + return false; + } activePromptIndex = index; break; } @@ -4551,6 +4579,14 @@ export function createServer(opts?: CreateServerOptions): McpServer { const now = Date.now(); const updateState = parsed.cli_update_state; + const launcherFailure = launcherFailureFromShell(screen.text); + if (launcherFailure) { + throw new LauncherReadinessError( + `Launcher exited before reaching readiness on ${target.surface}: ${launcherFailure}`, + tailLines(lastText, 10), + ); + } + if (shouldHandleCodexUpdateMenu(opts.cli, screen.text)) { if (codexUpdateMenuAccepted) { const elapsedSinceAcceptMs = @@ -4684,6 +4720,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { } catch (error) { if ( error instanceof BootPromptTimeoutError || + error instanceof LauncherReadinessError || error instanceof BootPromptUpdateMenuBlockedError ) { throw error; @@ -4857,6 +4894,14 @@ export function createServer(opts?: CreateServerOptions): McpServer { const parsed = parseScreen(screen.text); const now = Date.now(); + const launcherFailure = launcherFailureFromShell(screen.text); + if (launcherFailure) { + throw new LauncherReadinessError( + `Launcher exited before reaching readiness on ${opts.surface}: ${launcherFailure}`, + tailLines(lastText, 10), + ); + } + if (parsed.cli_update_state === "updating") { updateWasSeen = true; updateStartedAt ??= now; @@ -4925,7 +4970,10 @@ export function createServer(opts?: CreateServerOptions): McpServer { return; } } catch (error) { - if (error instanceof BootPromptTimeoutError) { + if ( + error instanceof BootPromptTimeoutError || + error instanceof LauncherReadinessError + ) { throw error; } if (isSurfaceGoneReadFailure(error, opts.surface)) { @@ -5097,12 +5145,40 @@ export function createServer(opts?: CreateServerOptions): McpServer { } // The command can remain visible in shell history while the launcher is // already booting. Readiness detection is the authoritative launch check. - await waitForAgentLaunchReady({ - surface: opts.surface, - workspace: opts.workspace, - timeout_ms: opts.timeout_ms, - onUpdateShellRelaunch: relaunchOriginalCommand, - }); + try { + await waitForAgentLaunchReady({ + surface: opts.surface, + workspace: opts.workspace, + timeout_ms: opts.timeout_ms, + onUpdateShellRelaunch: relaunchOriginalCommand, + }); + } catch (readinessError) { + let pendingScreen; + try { + pendingScreen = await client.readScreen(opts.surface, { + workspace: opts.workspace, + lines: 80, + scrollback: false, + }); + } catch (readError) { + if (isSurfaceGoneReadFailure(readError, opts.surface)) { + throw new SurfaceGoneError(opts.surface, readError); + } + throw readinessError; + } + if ( + screenShowsPendingShellInput( + pendingScreen.text, + sanitizedCommand, + ) + ) { + throw new LauncherReadinessError( + `launcher command remained pending after Return on ${opts.surface}`, + tailLines(pendingScreen.text, 10), + ); + } + throw readinessError; + } } }, { @@ -6956,6 +7032,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { async (args) => { let result: CmuxNewSplitResult | undefined; let focusRestoreLease: FocusRestoreLease | null = null; + const creation = new CreatedIdentityScope(); try { const normalizedRole = normalizeToolAgentRole(args.role, "role"); const bootPromptPath = getBootPromptPath(args.boot_prompt_path); @@ -7106,6 +7183,11 @@ export function createServer(opts?: CreateServerOptions): McpServer { url: args.url, title: args.title, }); + creation.record({ + surface: result.surface, + workspace: result.workspace, + ...(result.surface_id ? { surface_id: result.surface_id } : {}), + }); assertSurfaceObserverEpochCurrent( rolePlacementObserverEpoch, "role-based new_split placement", @@ -7127,6 +7209,11 @@ export function createServer(opts?: CreateServerOptions): McpServer { url: args.url, title: args.title, }); + creation.record({ + surface: result.surface, + workspace: result.workspace, + ...(result.surface_id ? { surface_id: result.surface_id } : {}), + }); } if (args.focus === true) { focusRequestWarning = await focusCreatedSurface( @@ -7216,6 +7303,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { data, ); } catch (e) { + const caught = creation.attach(e); // Creation or boot delivery may fail after cmuxlayer selected a target // workspace. Return focus when the user has not moved since then. await restoreFocusAfterRender( @@ -7231,30 +7319,30 @@ export function createServer(opts?: CreateServerOptions): McpServer { ...(result.surface_id ? { surface_id: result.surface_id } : {}), } : {}; - if (e instanceof SurfaceGoneError) { - return err(e, surfaceGonePayload(e, createdIdentity)); + if (caught instanceof SurfaceGoneError) { + return err(caught, surfaceGonePayload(caught, createdIdentity)); } - if (e instanceof BootPromptTimeoutError) { - return err(e, { + if (caught instanceof BootPromptTimeoutError) { + return err(caught, { ...createdIdentity, - last_10_lines: e.last_10_lines, + last_10_lines: caught.last_10_lines, }); } - if (e instanceof BootPromptUpdateMenuBlockedError) { - return err(e, { + if (caught instanceof BootPromptUpdateMenuBlockedError) { + return err(caught, { ...createdIdentity, - error_code: e.error_code, - last_10_lines: e.last_10_lines, - recovery: e.recovery, + error_code: caught.error_code, + last_10_lines: caught.last_10_lines, + recovery: caught.recovery, }); } - if (e instanceof BootPromptDeliveryError) { - return err(e, { + if (caught instanceof BootPromptDeliveryError) { + return err(caught, { ...createdIdentity, - delivered_chars: e.delivered_chars, + delivered_chars: caught.delivered_chars, }); } - return err(e, createdIdentity); + return err(caught, createdIdentity); } }, ); @@ -7291,6 +7379,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { ANNOTATIONS.mutating, async (args) => { let result: CmuxNewSurfaceResult | undefined; + const creation = new CreatedIdentityScope(); try { const bootPromptPath = getBootPromptPath(args.boot_prompt_path); if (bootPromptPath) { @@ -7316,6 +7405,11 @@ export function createServer(opts?: CreateServerOptions): McpServer { type: args.type, url: args.url, }); + creation.record({ + surface: result.surface, + workspace: result.workspace, + ...(result.surface_id ? { surface_id: result.surface_id } : {}), + }); if (args.title) { await client.renameTab(result.surface, args.title, { workspace: result.workspace || targetWorkspace, @@ -7366,6 +7460,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { data, ); } catch (e) { + const caught = creation.attach(e); const createdIdentity = result ? { surface: result.surface, @@ -7373,30 +7468,30 @@ export function createServer(opts?: CreateServerOptions): McpServer { ...(result.surface_id ? { surface_id: result.surface_id } : {}), } : {}; - if (e instanceof SurfaceGoneError) { - return err(e, surfaceGonePayload(e, createdIdentity)); + if (caught instanceof SurfaceGoneError) { + return err(caught, surfaceGonePayload(caught, createdIdentity)); } - if (e instanceof BootPromptTimeoutError) { - return err(e, { + if (caught instanceof BootPromptTimeoutError) { + return err(caught, { ...createdIdentity, - last_10_lines: e.last_10_lines, + last_10_lines: caught.last_10_lines, }); } - if (e instanceof BootPromptUpdateMenuBlockedError) { - return err(e, { + if (caught instanceof BootPromptUpdateMenuBlockedError) { + return err(caught, { ...createdIdentity, - error_code: e.error_code, - last_10_lines: e.last_10_lines, - recovery: e.recovery, + error_code: caught.error_code, + last_10_lines: caught.last_10_lines, + recovery: caught.recovery, }); } - if (e instanceof BootPromptDeliveryError) { - return err(e, { + if (caught instanceof BootPromptDeliveryError) { + return err(caught, { ...createdIdentity, - delivered_chars: e.delivered_chars, + delivered_chars: caught.delivered_chars, }); } - return err(e, createdIdentity); + return err(caught, createdIdentity); } }, ); @@ -9915,6 +10010,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { }, ANNOTATIONS.mutating, async (args) => { + const creation = new CreatedIdentityScope(); try { if (args.type === "terminal") { if ( @@ -9962,6 +10058,10 @@ export function createServer(opts?: CreateServerOptions): McpServer { ...(placement.pane ? { pane: placement.pane } : {}), focus: args.focus, }); + creation.record({ + surface_id: created.surface, + workspace_id: created.workspace ?? workspace ?? null, + }); if (args.cwd) { await client.send( created.surface, @@ -10094,6 +10194,56 @@ export function createServer(opts?: CreateServerOptions): McpServer { args.worktree, args.mcp_profile as McpProfile | undefined, ); + const cleanupFailedLauncherArtifacts = async ( + error: Error, + agentId: string, + surface: string, + workspace?: string, + ): Promise => { + const record = engine.getAgentState(agentId); + const cleanupSurface = record?.surface_uuid?.trim() || surface; + try { + await client.closeSurface(cleanupSurface, { workspace }); + } catch (cleanupError) { + error.message = `${error.message}. Failed to close launcher surface ${cleanupSurface}: ${ + cleanupError instanceof Error + ? cleanupError.message + : String(cleanupError) + }`; + return false; + } + const current = engine.getAgentState(agentId); + if (current && !TERMINAL_AGENT_STATES.has(current.state)) { + try { + const failed = stateMgr.transition(agentId, "error", { + error: `Launcher surface closed after failed readiness: ${error.message}`, + }); + registry.set(agentId, failed); + } catch (stateError) { + error.message = `${error.message}. Failed to mark closed launcher agent ${agentId} terminal: ${ + stateError instanceof Error + ? stateError.message + : String(stateError) + }`; + } + } + if (worktree.prepared?.created && worktree.repoRoot) { + try { + await rollbackPreparedWorktree( + worktree.repoRoot, + worktree.prepared, + opts?.worktreeExec, + ); + } catch (rollbackError) { + error.message = `${error.message}. Worktree rollback also failed: ${ + rollbackError instanceof Error + ? rollbackError.message + : String(rollbackError) + }`; + } + } + return true; + }; let focusRestoreLease = await focusTargetBeforeSplit( spawnWorkspace, args.focus !== true, @@ -10124,6 +10274,11 @@ export function createServer(opts?: CreateServerOptions): McpServer { boot_prompt_timeout_ms: args.boot_prompt_timeout_ms, on_surface_created: async (created) => { surfaceCreated = true; + creation.record({ + agent_id: created.agent_id, + surface_id: created.surface, + workspace_id: created.workspace ?? spawnWorkspace ?? null, + }); focusRestoreLease = await capturePostCreationFocus( focusRestoreLease, created, @@ -10131,6 +10286,18 @@ export function createServer(opts?: CreateServerOptions): McpServer { }, }); } catch (e) { + if ( + e instanceof AgentLaunchError && + e.launch_phase === "launch" && + e.launch_cause instanceof LauncherReadinessError + ) { + await cleanupFailedLauncherArtifacts( + e, + e.agent_id, + e.surface_id, + e.workspace_id, + ); + } let rollbackError: unknown = null; if ( !surfaceCreated && @@ -10197,6 +10364,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { let bootPromptDelivery: Awaited> | undefined; + let launcherSurfaceClosed = false; try { { const deliveryWorkspace = spawnDeliveryWorkspace( @@ -10259,6 +10427,15 @@ export function createServer(opts?: CreateServerOptions): McpServer { } } } catch (e) { + creation.attach(e); + if (e instanceof LauncherReadinessError) { + launcherSurfaceClosed = await cleanupFailedLauncherArtifacts( + e, + result.agent_id, + result.surface_id, + spawnDeliveryWorkspace(result, spawnWorkspace), + ); + } const message = e instanceof Error ? e.message : String(e); const clearBootPromptPending = () => { const record = resolveSpawnRecord( @@ -10300,7 +10477,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { // timeout, restore immediately instead of starting a second wait. await restoreFocusAfterRender( focusRestoreLease, - result.surface_id, + launcherSurfaceClosed ? undefined : result.surface_id, spawnDeliveryWorkspace(result, spawnWorkspace), { waitForReady: false }, ); @@ -10419,44 +10596,47 @@ export function createServer(opts?: CreateServerOptions): McpServer { formatOk("spawn_agent", formattedData), ); } catch (e) { - if (e instanceof AgentLaunchError) { - if (e.launch_cause instanceof DeliverySafetyGateError) { - return err(e.launch_cause, { - agent_id: e.agent_id, - surface_id: e.surface_id, - workspace_id: e.workspace_id, - error_code: e.launch_cause.error_code, - submit_verified: e.launch_cause.submit_verified, - screen: e.launch_cause.screen, + const caught = creation.attach(e); + if (caught instanceof AgentLaunchError) { + if (caught.launch_cause instanceof DeliverySafetyGateError) { + creation.attach(caught.launch_cause); + return err(caught.launch_cause, { + agent_id: caught.agent_id, + surface_id: caught.surface_id, + workspace_id: caught.workspace_id, + error_code: caught.launch_cause.error_code, + submit_verified: caught.launch_cause.submit_verified, + screen: caught.launch_cause.screen, }); } - if (e.launch_cause instanceof SurfaceGoneError) { + if (caught.launch_cause instanceof SurfaceGoneError) { + creation.attach(caught.launch_cause); return err( - e.launch_cause, - surfaceGonePayload(e.launch_cause, { - agent_id: e.agent_id, - surface_id: e.surface_id, - workspace_id: e.workspace_id, + caught.launch_cause, + surfaceGonePayload(caught.launch_cause, { + agent_id: caught.agent_id, + surface_id: caught.surface_id, + workspace_id: caught.workspace_id, }), ); } - return err(e, { - agent_id: e.agent_id, - surface_id: e.surface_id, - workspace_id: e.workspace_id, + return err(caught, { + agent_id: caught.agent_id, + surface_id: caught.surface_id, + workspace_id: caught.workspace_id, }); } - if (e instanceof DeliverySafetyGateError) { - return err(e, { - error_code: e.error_code, - submit_verified: e.submit_verified, - screen: e.screen, + if (caught instanceof DeliverySafetyGateError) { + return err(caught, { + error_code: caught.error_code, + submit_verified: caught.submit_verified, + screen: caught.screen, }); } - if (e instanceof SurfaceGoneError) { - return err(e, surfaceGonePayload(e)); + if (caught instanceof SurfaceGoneError) { + return err(caught, surfaceGonePayload(caught)); } - return err(e); + return err(caught); } }, ); @@ -10506,6 +10686,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { }, ANNOTATIONS.mutating, async (args) => { + const creation = new CreatedIdentityScope(); let focusRestoreLease: FocusRestoreLease | null = null; let result: Awaited> | undefined; let mutationWorkspace: string | undefined; @@ -10561,6 +10742,11 @@ export function createServer(opts?: CreateServerOptions): McpServer { crash_recover: args.crash_recover, on_surface_created: async (created) => { surfaceCreated = true; + creation.record({ + agent_id: created.agent_id, + surface_id: created.surface, + workspace_id: created.workspace ?? mutationWorkspace ?? null, + }); focusRestoreLease = await capturePostCreationFocus( focusRestoreLease, created, @@ -10665,7 +10851,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { formatOk("new_worktree_split", formattedData), ); } catch (e) { - let caught: unknown = e; + let caught: unknown = creation.attach(e); if ( !result && !surfaceCreated && @@ -10694,6 +10880,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { } } } + caught = creation.attach(caught); await restoreFocusAfterRender( focusRestoreLease, result?.surface_id, @@ -10819,6 +11006,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { }, ANNOTATIONS.mutating, async (args) => { + const creation = new CreatedIdentityScope(); const originFocus = await currentFocusTarget(); let focusRestoreLease: FocusRestoreLease | null = null; let workspace: string | undefined; @@ -10889,6 +11077,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { if (!workspace) { throw new Error("create_workspace returned an empty workspace ref"); } + creation.record({ workspace, workspace_id: workspace }); focusRestoreLease = await focusTargetBeforeSplit( workspace, @@ -10914,6 +11103,18 @@ export function createServer(opts?: CreateServerOptions): McpServer { role: agent.role, auto_archive_on_done: false, on_surface_created: async (created) => { + const identity = { + agent_id: created.agent_id, + surface_id: created.surface, + workspace_id: created.workspace ?? workspace ?? null, + }; + creation.record(identity); + creation.append( + "agents", + identity, + (left, right) => + left.surface_id === right.surface_id, + ); focusRestoreLease = await capturePostCreationFocus( focusRestoreLease, created, @@ -10925,6 +11126,12 @@ export function createServer(opts?: CreateServerOptions): McpServer { surface_id: result.surface_id, workspace_id: result.workspace_id ?? workspace ?? null, }; + creation.record(activeSpawnIdentity); + creation.append( + "agents", + activeSpawnIdentity, + (left, right) => left.surface_id === right.surface_id, + ); createdAgentIdentities.push(activeSpawnIdentity); lastSurface = result.surface_id; const originalLaunchCommand = originalLaunchCommandsBySurface.get( @@ -11090,6 +11297,7 @@ export function createServer(opts?: CreateServerOptions): McpServer { }, ); } catch (e) { + const caught = creation.attach(e); await restoreFocusAfterRender( focusRestoreLease, lastSurface, @@ -11097,11 +11305,11 @@ export function createServer(opts?: CreateServerOptions): McpServer { { waitForReady: false }, ); const failedIdentity = - e instanceof AgentLaunchError + caught instanceof AgentLaunchError ? { - agent_id: e.agent_id, - surface_id: e.surface_id, - workspace_id: e.workspace_id ?? null, + agent_id: caught.agent_id, + surface_id: caught.surface_id, + workspace_id: caught.workspace_id ?? null, } : activeSpawnIdentity; const failureAgents = [...createdAgentIdentities]; @@ -11120,62 +11328,64 @@ export function createServer(opts?: CreateServerOptions): McpServer { ...(failedIdentity ?? {}), ...(failureAgents.length > 0 ? { agents: failureAgents } : {}), }; - if (e instanceof AgentLaunchError) { - if (e.launch_cause instanceof DeliverySafetyGateError) { - return err(e.launch_cause, { + if (caught instanceof AgentLaunchError) { + if (caught.launch_cause instanceof DeliverySafetyGateError) { + creation.attach(caught.launch_cause); + return err(caught.launch_cause, { ...failureIdentityPayload, - error_code: e.launch_cause.error_code, - submit_verified: e.launch_cause.submit_verified, - screen: e.launch_cause.screen, + error_code: caught.launch_cause.error_code, + submit_verified: caught.launch_cause.submit_verified, + screen: caught.launch_cause.screen, }); } - if (e.launch_cause instanceof SurfaceGoneError) { + if (caught.launch_cause instanceof SurfaceGoneError) { + creation.attach(caught.launch_cause); return err( - e.launch_cause, - surfaceGonePayload(e.launch_cause, failureIdentityPayload), + caught.launch_cause, + surfaceGonePayload(caught.launch_cause, failureIdentityPayload), ); } - return err(e, failureIdentityPayload); + return err(caught, failureIdentityPayload); } - if (e instanceof DeliverySafetyGateError) { - return err(e, { + if (caught instanceof DeliverySafetyGateError) { + return err(caught, { ...failureIdentityPayload, - error_code: e.error_code, - submit_verified: e.submit_verified, - screen: e.screen, + error_code: caught.error_code, + submit_verified: caught.submit_verified, + screen: caught.screen, }); } - if (e instanceof SubmitVerificationError) { - return err(e, { + if (caught instanceof SubmitVerificationError) { + return err(caught, { ...failureIdentityPayload, submit_verified: false, - retry_count: e.retry_count, + retry_count: caught.retry_count, }); } - if (e instanceof SurfaceGoneError) { - return err(e, surfaceGonePayload(e, failureIdentityPayload)); + if (caught instanceof SurfaceGoneError) { + return err(caught, surfaceGonePayload(caught, failureIdentityPayload)); } - if (e instanceof BootPromptTimeoutError) { - return err(e, { + if (caught instanceof BootPromptTimeoutError) { + return err(caught, { ...failureIdentityPayload, - last_10_lines: e.last_10_lines, + last_10_lines: caught.last_10_lines, }); } - if (e instanceof BootPromptUpdateMenuBlockedError) { - return err(e, { + if (caught instanceof BootPromptUpdateMenuBlockedError) { + return err(caught, { ...failureIdentityPayload, - error_code: e.error_code, - last_10_lines: e.last_10_lines, - recovery: e.recovery, + error_code: caught.error_code, + last_10_lines: caught.last_10_lines, + recovery: caught.recovery, }); } - if (e instanceof BootPromptDeliveryError) { - return err(e, { + if (caught instanceof BootPromptDeliveryError) { + return err(caught, { ...failureIdentityPayload, - delivered_chars: e.delivered_chars, + delivered_chars: caught.delivered_chars, }); } - return err(e, failureIdentityPayload); + return err(caught, failureIdentityPayload); } }, ); diff --git a/src/shell-prompt.ts b/src/shell-prompt.ts new file mode 100644 index 00000000..8fc26794 --- /dev/null +++ b/src/shell-prompt.ts @@ -0,0 +1,76 @@ +const SHELL_PROMPT_TERMINATOR = "[$%#>❯›»]"; + +export function matchShellPromptLine( + line: string, + opts?: { allowRootInput?: boolean; strict?: boolean }, +): { input: string } | null { + const normalized = line.trimEnd(); + const barePrompt = normalized.match( + new RegExp(`^\\s*(${SHELL_PROMPT_TERMINATOR})(?:\\s+(.*))?$`, "u"), + ); + if (barePrompt && barePrompt[1] !== "#") { + return { input: barePrompt[2] ?? "" }; + } + if ( + barePrompt?.[1] === "#" && + (!barePrompt[2] || opts?.allowRootInput) + ) { + return { input: barePrompt[2] ?? "" }; + } + + if (!opts?.strict) { + // Preserve the app-server's established readiness contract while still + // exposing text after the decorated terminator to pending-input checks. + const decoratedPrompt = normalized.match( + /^.+?[$%#](?:\s+(.*))?$/u, + ); + if (decoratedPrompt) { + return { input: decoratedPrompt[1] ?? "" }; + } + } + + const prefixedPrompt = normalized.match( + new RegExp( + `^\\s*(?:(?:\\S+@\\S+)(?:\\s+(?:~|\\/)\\S*)?|(?:.*\\s)?(?:~|\\/)\\S*)(?:\\s+\\[[^\\]]+\\])?\\s*${SHELL_PROMPT_TERMINATOR}(?:\\s+(.*))?$`, + "u", + ), + ); + return prefixedPrompt ? { input: prefixedPrompt[1] ?? "" } : null; +} + +export function matchesShellPrompt(text: string): boolean { + return matchesShellPromptWithOptions(text, false); +} + +export function matchesShellPromptStrict(text: string): boolean { + return matchesShellPromptWithOptions(text, true); +} + +function matchesShellPromptWithOptions(text: string, strict: boolean): boolean { + const lines = text.replace(/\r\n?/g, "\n").split("\n"); + let end = lines.length; + while (end > 0 && !lines[end - 1]?.trim()) { + end -= 1; + } + const prompt = + end > 0 + ? matchShellPromptLine(lines[end - 1] ?? "", { strict }) + : null; + return prompt?.input.trim() === ""; +} + +export function launcherFailureFromShell(text: string): string | null { + if (!matchesShellPromptStrict(text)) return null; + const lines = text + .replace(/\r\n?/g, "\n") + .split("\n") + .map((line) => line.trimEnd()); + while (lines.length > 0 && !lines.at(-1)?.trim()) lines.pop(); + if (lines.length < 2) return null; + const adjacentLine = lines.at(-2)?.trim() ?? ""; + return /(?:command not found|no such file(?: or directory)?|permission denied|traceback \(most recent call last\)|invalid (?:option|argument|model|effort))/i.test( + adjacentLine, + ) + ? adjacentLine + : null; +} diff --git a/tests/created-identity.test.ts b/tests/created-identity.test.ts new file mode 100644 index 00000000..78890b5a --- /dev/null +++ b/tests/created-identity.test.ts @@ -0,0 +1,65 @@ +import { describe, expect, it } from "vitest"; +import { + CreatedIdentityScope, + createdIdentityFromError, +} from "../src/created-identity.js"; + +class UnclassifiedPostCreationError extends Error {} + +describe("CreatedIdentityScope", () => { + it("attaches recorded identity without changing the error type or cause", () => { + const cause = new Error("socket stderr"); + const error = new UnclassifiedPostCreationError("later failure", { + cause, + }); + const scope = new CreatedIdentityScope(); + scope.record({ surface: "surface:7", workspace: "workspace:2" }); + + const attached = scope.attach(error); + + expect(attached).toBe(error); + expect(attached).toBeInstanceOf(UnclassifiedPostCreationError); + expect(attached.cause).toBe(cause); + expect(createdIdentityFromError(attached)).toEqual({ + surface: "surface:7", + workspace: "workspace:2", + }); + }); + + it("does not invent identity before creation", () => { + const scope = new CreatedIdentityScope(); + expect(createdIdentityFromError(scope.attach(new Error("preflight")))).toEqual( + {}, + ); + }); + + it("accumulates prior batch identities and updates the failing member", () => { + const scope = new CreatedIdentityScope(); + const sameSurface = ( + left: Record, + right: Record, + ) => left.surface_id === right.surface_id; + scope.append( + "agents", + { agent_id: "pending-a", surface_id: "surface:a" }, + sameSurface, + ); + scope.append( + "agents", + { agent_id: "agent-a", surface_id: "surface:a" }, + sameSurface, + ); + scope.append( + "agents", + { agent_id: "agent-b", surface_id: "surface:b" }, + sameSurface, + ); + + expect(createdIdentityFromError(scope.attach(new Error("batch")))).toEqual({ + agents: [ + { agent_id: "agent-a", surface_id: "surface:a" }, + { agent_id: "agent-b", surface_id: "surface:b" }, + ], + }); + }); +}); diff --git a/tests/fixtures/golem-dispatch-contract.zsh b/tests/fixtures/golem-dispatch-contract.zsh new file mode 100644 index 00000000..21cf27b6 --- /dev/null +++ b/tests/fixtures/golem-dispatch-contract.zsh @@ -0,0 +1,22 @@ +# Hermetic CI snapshot of the launcher clauses cmuxlayer's spawn schema relies on. +if [[ -n "${CLAUDE_MODEL:-}" ]]; then + _claude_model="$CLAUDE_MODEL" +else + _claude_model="claude-opus-5[1m]" +fi + +_golem_parse_codex_flags() { + _flag_codex_effort="xhigh" + case "$1" in + --effort) + case "$2" in + medium|high|xhigh|ultra) _flag_codex_effort="$2" ;; + esac + ;; + esac +} + +_golem_launch_cursor() { + _golem_refuse_agent_model_override "$@" + REPOGOLEM_ALLOW_MODEL="${REPOGOLEM_ALLOW_MODEL:-0}" +} diff --git a/tests/model-policy-drift.test.ts b/tests/model-policy-drift.test.ts index 632a623c..450083b6 100644 --- a/tests/model-policy-drift.test.ts +++ b/tests/model-policy-drift.test.ts @@ -1,6 +1,7 @@ import { existsSync, readFileSync } from "node:fs"; import { homedir } from "node:os"; import { join } from "node:path"; +import { fileURLToPath } from "node:url"; import { beforeAll, describe, expect, it } from "vitest"; import { CODEX_EFFORT_VALUES, @@ -98,16 +99,34 @@ describe("model-policy drift gate", () => { }); }); -const dispatchPath = join( +const installedDispatchPath = join( homedir(), ".config/ralphtools/golem-dispatch.zsh", ); -const launcherAbsent = !existsSync(dispatchPath); +const contractFixturePath = fileURLToPath( + new URL("./fixtures/golem-dispatch-contract.zsh", import.meta.url), +); +const dispatchPath = existsSync(installedDispatchPath) + ? installedDispatchPath + : contractFixturePath; + +describe("hermetic golem-dispatch contract", () => { + it("keeps the CI fallback parseable and aligned", () => { + const fixture = readFileSync(contractFixturePath, "utf8"); + expect(parseClaudeDefault(fixture)).toBe( + MODEL_POLICY_CONTRACT.cli.claude.defaultModel, + ); + expect(parseCodexEffortValues(fixture)).toEqual(CODEX_EFFORT_VALUES); + expect(parseCursorLauncher(fixture)).toContain( + "_golem_refuse_agent_model_override", + ); + expect(fixture).toContain(MODEL_OVERRIDE_ENV); + }); +}); -describe.skipIf(launcherAbsent)("model-policy parity with installed golem-dispatch", () => { - // Read lazily in beforeAll, not at describe-body collection time: a skipped - // suite (installed launcher absent, e.g. CI) still evaluates the describe - // body, so a top-level readFileSync would throw before the skip takes effect. +describe("model-policy parity with golem-dispatch contract", () => { + // Developer machines check the installed launcher. CI checks the committed + // launcher-contract snapshot so this suite never silently disappears. let dispatchText: string; beforeAll(() => { dispatchText = readFileSync(dispatchPath, "utf8"); diff --git a/tests/server-agent-tools.test.ts b/tests/server-agent-tools.test.ts index 03ca5d41..c6e52bb5 100644 --- a/tests/server-agent-tools.test.ts +++ b/tests/server-agent-tools.test.ts @@ -50,6 +50,7 @@ const hermeticSpawnStateDirs: string[] = []; let hermeticSpawnFixtureSequence = 0; const originalLauncherRegistryPath = process.env.CMUXLAYER_LAUNCHER_REGISTRY_PATH; +const originalAllowModel = process.env.REPOGOLEM_ALLOW_MODEL; afterEach(async () => { if (originalLauncherRegistryPath === undefined) { @@ -58,6 +59,11 @@ afterEach(async () => { process.env.CMUXLAYER_LAUNCHER_REGISTRY_PATH = originalLauncherRegistryPath; } + if (originalAllowModel === undefined) { + delete process.env.REPOGOLEM_ALLOW_MODEL; + } else { + process.env.REPOGOLEM_ALLOW_MODEL = originalAllowModel; + } await Promise.allSettled( serverContexts.map( (context) => context.lifecycleStartPromise ?? Promise.resolve(), @@ -93,6 +99,7 @@ const AGENT_TOOLS = [ function makeLifecycleExec(opts?: { closeKeepsSurface?: boolean; createdWorkspace?: string; + shellPrompt?: string; shellNeverReady?: boolean; surfaceUuid?: string; }): ExecFn { @@ -131,7 +138,7 @@ function makeLifecycleExec(opts?: { createdSurfaceCount === 1 ? "surface:new" : `surface:new-${createdSurfaceCount}`; - readyText = "$ "; + readyText = opts?.shellPrompt ?? "$ "; promptPending = false; } if (args.includes("close-surface") && !opts?.closeKeepsSurface) { @@ -1217,6 +1224,7 @@ describe("lean spawn tool responses", () => { }); it("spawn_agent rejects an unsupported model before creating a surface", async () => { + delete process.env.REPOGOLEM_ALLOW_MODEL; const mockExec = makeLifecycleExec(); const server = createLifecycleServer(mockExec); const spawn = (server as any)._registeredTools["spawn_agent"]; @@ -3737,6 +3745,302 @@ describe("agent lifecycle tool handlers", () => { expect(existsSync(worktreePath)).toBe(true); }); + it("spawn_agent closes a failed launcher surface before rolling back its new worktree", async () => { + vi.useFakeTimers(); + try { + const gitsDir = join(TEST_DIR, "Gits"); + const repoRoot = join(TEST_DIR, ".config", "ralph-launch-failure"); + const registryPath = join(TEST_DIR, "launchers-post-surface-rollback.zsh"); + const worktreePath = join(repoRoot, ".worktrees", "post-surface-failure"); + mkdirSync(repoRoot, { recursive: true }); + writeFileSync(registryPath, `repoGolem ralph "${repoRoot}"\n`); + vi.stubEnv("CMUXLAYER_LAUNCHER_REGISTRY_PATH", registryPath); + const worktreeExec = vi.fn().mockImplementation(async (_cmd, args) => { + if (args.includes("worktree") && args.includes("add")) { + mkdirSync(worktreePath, { recursive: true }); + } + if (args.includes("worktree") && args.includes("remove")) { + rmSync(worktreePath, { recursive: true, force: true }); + } + return { stdout: "", stderr: "" }; + }); + const baseExec = makeLifecycleExec({ surfaceUuid: "surface-uuid:new" }); + let launcherSent = false; + const exec = vi.fn().mockImplementation(async (cmd, args: string[]) => { + if ( + args.includes("send") && + /ralphCodex\b/.test(String(args.at(-1) ?? "")) + ) { + launcherSent = true; + return { stdout: "{}", stderr: "" }; + } + if (launcherSent && args.includes("read-screen")) { + return { + stdout: JSON.stringify({ + surface: "surface:new", + text: "zsh: command not found: ralphCodex\n$ ", + lines: 20, + scrollback_used: false, + }), + stderr: "", + }; + } + return baseExec(cmd, args); + }); + const server = createTrackedServer({ + exec, + stateDir: TEST_DIR, + sessionIdentityResolver: () => null, + worktreeHomeDir: gitsDir, + worktreeExec, + }); + const spawn = (server as any)._registeredTools["spawn_agent"]; + + const resultPromise = spawn.handler( + { + repo: "ralph", + cli: "codex", + role: "worker", + worktree: { + name: "post-surface-failure", + branch: "wt/post-surface-failure", + }, + boot_prompt_timeout_ms: 20, + }, + {} as any, + ); + await vi.advanceTimersByTimeAsync(1_000); + const parsed = parseToolResult(await resultPromise); + + expect(parsed.ok).toBe(false); + expect(parsed.error).toContain("zsh: command not found: ralphCodex"); + expect(parsed.surface_id).toBe("surface:new"); + expect(exec).toHaveBeenCalledWith( + "cmux", + expect.arrayContaining(["close-surface", "surface-uuid:new"]), + ); + expect(worktreeExec).toHaveBeenCalledWith( + "git", + expect.arrayContaining(["worktree", "remove", "--force", worktreePath]), + ); + expect(worktreeExec).toHaveBeenCalledWith( + "git", + expect.arrayContaining(["branch", "-D", "wt/post-surface-failure"]), + ); + const getState = (server as any)._registeredTools["get_agent_state"]; + const state = parseToolResult( + await getState.handler({ agent_id: parsed.agent_id }, {} as any), + ); + expect(state.state).toBe("error"); + expect(existsSync(worktreePath)).toBe(false); + } finally { + vi.useRealTimers(); + } + }); + + it("spawn_agent waits through an 800ms launcher first-paint delay without cleanup", async () => { + vi.useFakeTimers(); + try { + const baseExec = makeLifecycleExec(); + let launcherSentAt: number | null = null; + const exec = vi.fn().mockImplementation(async (cmd, args: string[]) => { + const text = String(args.at(-1) ?? ""); + if (args.includes("send") && text === "voicelayerCodex -s") { + launcherSentAt = Date.now(); + return { stdout: "{}", stderr: "" }; + } + if ( + launcherSentAt !== null && + args.includes("send-key") && + args.includes("return") + ) { + return { stdout: "{}", stderr: "" }; + } + if (launcherSentAt !== null && args.includes("read-screen")) { + const elapsed = Date.now() - launcherSentAt; + return { + stdout: JSON.stringify({ + surface: "surface:new", + text: + elapsed < 800 + ? "$ voicelayerCodex -s" + : "codex> ", + lines: 20, + scrollback_used: false, + }), + stderr: "", + }; + } + return baseExec(cmd, args); + }); + const server = createLifecycleServer(exec); + const spawn = (server as any)._registeredTools["spawn_agent"]; + + const resultPromise = spawn.handler( + { + repo: "voicelayer", + model: "codex", + cli: "codex", + boot_prompt_timeout_ms: 2_000, + }, + {} as any, + ); + await vi.advanceTimersByTimeAsync(0); + expect(launcherSentAt).not.toBeNull(); + await vi.advanceTimersByTimeAsync(3_000); + const parsed = parseToolResult(await resultPromise); + + expect(parsed.error).toBeUndefined(); + expect(parsed).toMatchObject({ ok: true }); + expect( + exec.mock.calls.some(([, args]) => args.includes("close-surface")), + ).toBe(false); + } finally { + vi.useRealTimers(); + } + }); + + it("spawn_agent ignores transient shell-rc errors above percentage boot progress", async () => { + vi.useFakeTimers(); + try { + const gitsDir = join(TEST_DIR, "Gits"); + const repoRoot = join(TEST_DIR, ".config", "ralph-progress"); + const registryPath = join(TEST_DIR, "launchers-progress.zsh"); + const worktreePath = join(repoRoot, ".worktrees", "boot-progress"); + mkdirSync(repoRoot, { recursive: true }); + writeFileSync(registryPath, `repoGolem ralph "${repoRoot}"\n`); + vi.stubEnv("CMUXLAYER_LAUNCHER_REGISTRY_PATH", registryPath); + const worktreeExec = vi.fn().mockImplementation(async (_cmd, args) => { + if (args.includes("worktree") && args.includes("add")) { + mkdirSync(worktreePath, { recursive: true }); + } + return { stdout: "", stderr: "" }; + }); + const baseExec = makeLifecycleExec(); + let launcherSentAt: number | null = null; + const exec = vi.fn().mockImplementation(async (cmd, args: string[]) => { + if ( + args.includes("send") && + /ralphCodex\b/.test(String(args.at(-1) ?? "")) + ) { + launcherSentAt = Date.now(); + return { stdout: "{}", stderr: "" }; + } + if (launcherSentAt !== null && args.includes("read-screen")) { + const elapsed = Date.now() - launcherSentAt; + return { + stdout: JSON.stringify({ + surface: "surface:new", + text: + elapsed < 500 + ? "zsh: command not found: pyenv\n⠋ Installing... 62%" + : "codex> ", + lines: 20, + scrollback_used: false, + }), + stderr: "", + }; + } + return baseExec(cmd, args); + }); + const server = createTrackedServer({ + exec, + stateDir: TEST_DIR, + sessionIdentityResolver: () => null, + worktreeHomeDir: gitsDir, + worktreeExec, + }); + const spawn = (server as any)._registeredTools["spawn_agent"]; + + const resultPromise = spawn.handler( + { + repo: "ralph", + cli: "codex", + role: "worker", + worktree: { + name: "boot-progress", + branch: "wt/boot-progress", + }, + boot_prompt_timeout_ms: 2_000, + }, + {} as any, + ); + await vi.advanceTimersByTimeAsync(0); + expect(launcherSentAt).not.toBeNull(); + await vi.advanceTimersByTimeAsync(3_000); + const parsed = parseToolResult(await resultPromise); + + expect(parsed).toMatchObject({ ok: true }); + expect( + exec.mock.calls.some(([, args]) => args.includes("close-surface")), + ).toBe(false); + expect(worktreeExec).not.toHaveBeenCalledWith( + "git", + expect.arrayContaining(["worktree", "remove"]), + ); + expect(existsSync(worktreePath)).toBe(true); + } finally { + vi.useRealTimers(); + } + }); + + it("spawn_agent keeps a generic launch-timeout surface and worktree recoverable", async () => { + vi.useFakeTimers(); + try { + const gitsDir = join(TEST_DIR, "Gits"); + const repoRoot = join(TEST_DIR, ".config", "ralph-timeout"); + const registryPath = join(TEST_DIR, "launchers-timeout.zsh"); + const worktreePath = join(repoRoot, ".worktrees", "launch-timeout"); + mkdirSync(repoRoot, { recursive: true }); + writeFileSync(registryPath, `repoGolem ralph "${repoRoot}"\n`); + vi.stubEnv("CMUXLAYER_LAUNCHER_REGISTRY_PATH", registryPath); + const worktreeExec = vi.fn().mockImplementation(async (_cmd, args) => { + if (args.includes("worktree") && args.includes("add")) { + mkdirSync(worktreePath, { recursive: true }); + } + return { stdout: "", stderr: "" }; + }); + const exec = makeLifecycleExec({ shellNeverReady: true }); + const server = createTrackedServer({ + exec, + stateDir: TEST_DIR, + sessionIdentityResolver: () => null, + worktreeHomeDir: gitsDir, + worktreeExec, + }); + const spawn = (server as any)._registeredTools["spawn_agent"]; + + const resultPromise = spawn.handler( + { + repo: "ralph", + cli: "codex", + role: "worker", + worktree: { + name: "launch-timeout", + branch: "wt/launch-timeout", + }, + boot_prompt_timeout_ms: 20, + }, + {} as any, + ); + await vi.advanceTimersByTimeAsync(1_000); + const parsed = parseToolResult(await resultPromise); + + expect(parsed.ok).toBe(false); + expect(parsed.error).toContain("waiting for shell readiness"); + expect( + exec.mock.calls.some(([, args]) => args.includes("close-surface")), + ).toBe(false); + expect(worktreeExec).not.toHaveBeenCalledWith( + "git", + expect.arrayContaining(["worktree", "remove"]), + ); + expect(existsSync(worktreePath)).toBe(true); + } finally { + vi.useRealTimers(); + } + }); + it("new_worktree_split rolls back a newly created worktree and branch when spawning fails", async () => { const gitsDir = join(TEST_DIR, "Gits"); const repoRoot = join(gitsDir, "cmuxlayer"); @@ -4510,6 +4814,63 @@ describe("agent lifecycle tool handlers", () => { ).toBe(true); }, 10_000); + it("spawn_agent fails with decorated-prompt pending evidence when Return never submits", async () => { + vi.useFakeTimers(); + try { + const baseExec = makeLifecycleExec(); + let launcherSent = false; + let launcherReturns = 0; + const exec = vi.fn().mockImplementation(async (cmd, args: string[]) => { + const text = String(args.at(-1) ?? ""); + if (args.includes("send") && text === "voicelayerCodex -s") { + launcherSent = true; + return { stdout: "{}", stderr: "" }; + } + if (launcherSent && args.includes("send-key") && args.includes("return")) { + launcherReturns += 1; + return { stdout: "{}", stderr: "" }; + } + if (launcherSent && args.includes("read-screen")) { + return { + stdout: JSON.stringify({ + surface: "surface:new", + text: "bash-5.2$ voicelayerCodex -s", + lines: 20, + scrollback_used: false, + }), + stderr: "", + }; + } + return baseExec(cmd, args); + }); + const server = createLifecycleServer(exec); + const spawn = (server as any)._registeredTools["spawn_agent"]; + + const resultPromise = spawn.handler( + { + repo: "voicelayer", + model: "codex", + cli: "codex", + boot_prompt_timeout_ms: 20, + }, + {} as any, + ); + await vi.advanceTimersByTimeAsync(1_000); + const parsed = parseToolResult(await resultPromise); + + expect(parsed.ok).toBe(false); + expect(parsed.error).toContain( + "launcher command remained pending after Return", + ); + expect(parsed.last_10_lines).toContain( + "bash-5.2$ voicelayerCodex -s", + ); + expect(launcherReturns).toBeGreaterThanOrEqual(1); + } finally { + vi.useRealTimers(); + } + }); + it("spawn_agent treats launch submit verification as advisory when readiness appears with shell history", async () => { const promptPath = join(TEST_DIR, "mandate.md"); writeFileSync(promptPath, "file prompt body", "utf8"); @@ -4751,6 +5112,58 @@ describe("agent lifecycle tool handlers", () => { } }); + it.each(["user in ~/repo > ", "❯ ", "› ", "» "])( + "spawn_agent launches from a ready %s shell prompt", + async (shellPrompt) => { + const server = createLifecycleServer(makeLifecycleExec({ shellPrompt })); + const spawn = (server as any)._registeredTools["spawn_agent"]; + + const result = await spawn.handler( + { + repo: "brainlayer", + model: "codex", + cli: "codex", + boot_prompt_timeout_ms: 20, + }, + {} as any, + ); + const parsed = parseToolResult(result); + + expect(parsed.ok).toBe(true); + expect(parsed.surface_id).toBe("surface:new"); + }, + ); + + it("spawn terminal preserves created identity when cwd delivery fails", async () => { + const baseExec = makeLifecycleExec(); + const exec = vi.fn().mockImplementation(async (cmd, args: string[]) => { + if ( + args.includes("send") && + String(args.at(-1) ?? "").startsWith("cd -- ") + ) { + throw new Error("deliberate terminal cwd failure"); + } + return baseExec(cmd, args); + }); + const server = createLifecycleServer(exec); + const spawn = (server as any)._registeredTools["spawn_agent"]; + + const result = await spawn.handler( + { + version: 1, + type: "terminal", + cwd: "/tmp/cmuxlayer-p7-terminal", + }, + {} as any, + ); + const parsed = parseToolResult(result); + + expect(parsed.ok).toBe(false); + expect(parsed.error).toContain("deliberate terminal cwd failure"); + expect(parsed.surface_id).toBe("surface:new"); + expect(parsed.workspace_id).toBe("workspace:1"); + }); + it("spawn_agent preserves the 10000ms shell-readiness default when no override is supplied", async () => { vi.useFakeTimers(); try { @@ -4888,6 +5301,9 @@ describe("agent lifecycle tool handlers", () => { ); expect(parsed.agent_id).toEqual(expect.any(String)); expect(parsed.surface_id).toBe("surface:new"); + expect(parsed.last_10_lines).toContain( + "agent launcher still starting", + ); } finally { vi.useRealTimers(); } @@ -5015,6 +5431,7 @@ describe("agent lifecycle tool handlers", () => { expect(parsed.agent_id).toEqual(expect.any(String)); expect(parsed.surface_id).toBe("surface:new"); expect(parsed.workspace_id).toBe("ws:1"); + expect(parsed.last_10_lines).toContain("terminal initializing"); } finally { vi.useRealTimers(); } diff --git a/tests/shell-prompt.test.ts b/tests/shell-prompt.test.ts new file mode 100644 index 00000000..05f7c709 --- /dev/null +++ b/tests/shell-prompt.test.ts @@ -0,0 +1,70 @@ +import { describe, expect, it } from "vitest"; +import { + launcherFailureFromShell, + matchShellPromptLine, + matchesShellPrompt, +} from "../src/shell-prompt.js"; + +describe("shell prompt recognition", () => { + it.each([ + "$ ", + "% ", + "# ", + "> ", + "❯ ", + "› ", + "» ", + "user in ~/code/cmuxlayer > ", + "etan@mac ~/Gits/cmuxlayer [main] ❯ ", + "➜ cmuxlayer git:(main) $ ", + "[etan@mac cmuxlayer]$ ", + "cmuxlayer (main) % ", + "bash-5.2$ ", + ])("accepts a ready %s prompt", (prompt) => { + expect(matchesShellPrompt(`old output\n${prompt}`)).toBe(true); + }); + + it.each([ + "> cmuxlayerCodex -s", + "❯ cmuxlayerClaude -s", + "user in ~/code/cmuxlayer > still pending", + "bash-5.2$ cmuxlayerCodex -s", + ])("rejects pending input at %s", (prompt) => { + expect(matchesShellPrompt(prompt)).toBe(false); + }); + + it("keeps root input fail-closed unless explicitly allowed", () => { + expect(matchShellPromptLine("# rm -rf example")).toBeNull(); + expect( + matchShellPromptLine("# echo safe", { allowRootInput: true }), + ).toEqual({ input: "echo safe" }); + }); + + it("recognizes only adjacent, specific launcher failure evidence", () => { + expect( + launcherFailureFromShell("zsh: command not found: cmuxlayerCodex\n$ "), + ).toBe("zsh: command not found: cmuxlayerCodex"); + expect(launcherFailureFromShell("build failed earlier\nsummary\n$ ")).toBeNull(); + expect(launcherFailureFromShell("error: cached warning\n$ ")).toBeNull(); + }); + + it.each([ + "⠋ Building bundle... 62%", + "Installing dependencies 45%", + "Context left: 12%", + "Total cost: $", + "issue #", + ])("does not treat a loose readiness suffix as launcher-exit evidence: %s", (line) => { + expect( + launcherFailureFromShell(`zsh: command not found: pyenv\n${line}`), + ).toBeNull(); + }); + + it.each([ + ["➜ cmuxlayer git:(main) $ ralphCodex -s", "ralphCodex -s"], + ["bash-5.2$ ralphCodex -s", "ralphCodex -s"], + ["[etan@mac cmuxlayer]$ ralphCodex -s", "ralphCodex -s"], + ])("captures pending input from decorated prompt %s", (line, input) => { + expect(matchShellPromptLine(line)).toEqual({ input }); + }); +});