Rescue tasks intermittently hang forever: stale shared broker reused without a health check; headless app-server inherits desktop MCP servers

[ahuimanu]

Rescue tasks intermittently hang forever; shared broker is reused without a health check, and headless app-server inherits interactive/desktop MCP servers

Environment

• Plugin: codex@openai-codex v1.0.6 (repo openai/codex-plugin-cc)
• codex-cli 0.144.4
• macOS (arm64), Claude Code
• Codex desktop app installed but not running (had crashed earlier; only orphan crashpad handlers present)

Summary

codex:codex-rescue tasks intermittently hang indefinitely with no output. The job log reaches Turn started and then goes completely silent — no item/*, no turn/completed. The foreground task process blocks until its internal wait elapses. It is intermittent: the same trivial prompt hangs on one run and completes in ~5 s on the next.

Reproduction

1. Run a trivial read-only rescue: node scripts/codex-companion.mjs task "Reply with exactly PONG. Do not read files or run commands."
2. Intermittently, it hangs. Job log:

   Starting Codex Task.
   Starting Codex task thread.
   Thread ready (<thread-id>).
   Turn started (<turn-id>).
   <no further lines; process blocks ~2 min until killed>
   

3. Clear the persisted broker and kill stray processes:

   rm -f ~/.claude/plugins/data/codex-openai-codex/state/<workspace>/broker.json
   pkill -f app-server-broker; pkill -f "codex app-server"
   

4. Re-run the identical command → completes in ~5.6 s (PONG).

What is NOT the cause (verified)

• codex app-server protocol is fine: a raw direct turn completes in ~4 s and emits turn/started / item/completed / turn/completed with the exact shapes the companion's state machine expects.
• Driving a turn through the broker socket by hand also completes in ~4 s.
• /codex:setup reports fully green (node, codex, ChatGPT auth).
• Broker code is byte-identical between plugin 1.0.1 and 1.0.6, so this is not a recent plugin regression.

Root cause 1 — shared broker reused on a shallow liveness check, with fragile teardown

ensureBrokerSession() (scripts/lib/broker-lifecycle.mjs) reuses an existing broker when isBrokerEndpointReady() succeeds — but that only probes whether the unix socket accepts a connection (150 ms), never whether the app-server behind it still responds. A broker whose backing app-server has wedged (stuck prior turn, slow MCP startup, drift after a codex CLI update underneath a long-lived process) still passes the probe, gets reused, and the next task connects and blocks forever.

Teardown only happens in the SessionEnd hook (scripts/session-lifecycle-hook.mjs). It does not run on crash / kill -9 / force-quit / hook timeout, so wedged brokers and their session dirs are orphaned (observed: leftover cxc-* dirs in $TMPDIR whose teardown never ran). Those orphans are exactly what root cause 1 then reuses. The broker being long-lived, detached, and shared across Claude Code sessions widens this window considerably.

Suggested fix 1

• Before reusing a broker, do a real bounded health round-trip (e.g. initialize or account/read with a short timeout), not just a socket-accept probe. On failure, tear down and respawn.
• Add a bounded timeout on turn/start → first notification inside captureTurn / withAppServer, and fall back to disableBroker direct mode on timeout (the existing shouldRetryDirect path only covers BROKER_BUSY / ENOENT / ECONNREFUSED, not a silent wedge).
• Make teardown more robust than SessionEnd alone — e.g. a stale-broker sweep on SessionStart, or a heartbeat/last-used timestamp with reap-on-connect.

Root cause 2 — headless rescue inherits interactive/desktop MCP servers

SpawnedCodexAppServerClient runs spawn("codex", ["app-server"]) with no config scoping, so the app-server reads the user's full ~/.codex/config.toml. That file is shared with the Codex desktop app, which registers automation MCP servers there — notably node_repl (browser/computer-use bridge) with startup_timeout_sec = 120. These start on every thread, including pure code-rescue turns that never use a browser. When the desktop bridge is unhealthy, a code task can stall on it — an observed hang was ~117 s, right at the 120 s node_repl timeout.

A code-review/rescue turn should not depend on desktop-app automation components being healthy.

Suggested fix 2

• Spawn the app-server for task/review with a scoped config that excludes interactive/desktop MCP servers (e.g. a dedicated profile or -c mcp_servers.<x>.enabled=false overrides), or otherwise opt out of MCP startup the rescue path doesn't need.
• At minimum, document the coupling and give a supported way to run the plugin's app-server without inheriting desktop-injected MCP servers.

Impact

Users experience the plugin as "randomly unreliable" — long hangs with no error — with no obvious recovery beyond manually deleting broker.json and killing processes. It worsens after codex CLI auto-updates and after desktop-app crashes, both of which raise the odds of a stale/wedged broker or an unhealthy inherited MCP server.