Agent lifecycle: heartbeat, pause/checkpoint, and withdrawal

[jeremymanning]

Description

src/agent/lifecycle.rs contains three TODO items for core agent lifecycle operations:

1. heartbeat() (line ~134): Send heartbeat to broker/coordinator, receive lease offers
2. pause() (line ~143): Checkpoint any active sandboxes, notify broker
3. withdraw() (lines ~176-177): Checkpoint and terminate all active sandboxes, notify broker/coordinator of withdrawal

These are critical for Principle III (Donor Sovereignty) — donors must be able to pause and withdraw at any time with clean state transitions.

Requirements

• Heartbeat: periodic signal to broker with node state, resource availability, active leases
• Pause: checkpoint all running sandboxes (SIGSTOP + snapshot), transition to Paused state, stop accepting new leases
• Withdraw: checkpoint all sandboxes, terminate them, wipe scoped working directory, revoke keypair, notify broker, leave no host state

Success Criteria

• Heartbeat sends periodic updates to broker and receives lease offers
• Pause checkpoints active work and transitions agent to Paused state
• Withdraw fully cleans up: no files, processes, scheduled tasks, or network state remain
• State transitions follow the AgentState machine (Enrolling→Idle→Working→Paused→Withdrawing)
• Integration test: agent joins, receives work, pauses, resumes, withdraws cleanly
• cargo test passes with zero regressions

Testing (Principle V)

• Run agent on real hardware, submit job, trigger pause, verify checkpoint saved
• Withdraw and verify zero host residue (scan for files, processes, network listeners)
• Test rapid pause/resume cycles under load
• Test withdraw during active job execution

[jeremymanning]

Partially addressed by spec 004 / PR #59 — not closing. heartbeat/pause/withdraw functions return structured payloads (HeartbeatPayload, PauseResult, WithdrawalReport). HOWEVER, no actual gossip transmit is wired — the payload is generated but not broadcast. The daemon.rs event loop does publish heartbeat payloads when it runs, but the standalone lifecycle functions don't.

Remaining work tracked here.

[jeremymanning]

Spec 005 progress — PR #61

STATUS: docstrings clarified. Full gossip wiring from standalone lifecycle functions still deferred.

Done:

• Rewrote src/agent/lifecycle.rs heartbeat docstring to reflect actual behavior: the daemon event loop (src/agent/daemon.rs) broadcasts heartbeat payloads over gossipsub at the heartbeat interval; the lifecycle heartbeat() function is the source the daemon consults.

Remaining:

• If the architectural intent is that AgentInstance::heartbeat ALSO publish directly (rather than just returning a payload), that wiring is not done. Pending a design decision in a future session: either remove the standalone function or wire it to publish. Currently it returns the payload and the daemon does the broadcasting separately.

See notes/session-2026-04-19-spec-005-kickoff.md.