What belongs in a real failure taxonomy for coding-agent runs?

[Keesan12]

Most agent runs do not fail in one generic way. The useful split we keep seeing is more like:

• budget exhaustion versus budget stop with a useful receipt
• verifier stalled versus verifier improved but blocked on approval
• tool success versus task success
• unsafe side effect blocked versus harmless retryable error
• context drift versus adapter or secret boundary mismatch

If you are building or operating coding agents, what buckets actually matter in practice?

I am trying to make MartinLoop more useful as an open-source control layer here, so I care more about the taxonomy being operationally honest than theoretically neat. If you have a better split, edge case, or field you would want captured in the stop receipt, I would really like to see it.

[ofekron]

Maintainer disclosure: I build Better Agent, a local workspace supervising native Claude, Codex, and Gemini coding-agent sessions.

The taxonomy that has stayed operationally useful for us is three-dimensional rather than one flat failure label:

1. Observed outcome

   • not_started
   • accepted_no_event
   • partial_output
   • runtime_stopped
   • runtime_failed
   • runtime_completed_evidence_missing
   • task_verified
   • task_reverted

2. Owning failure domain

   • dispatch/transport
   • provider/runtime
   • tool or permission
   • workspace/filesystem
   • context/instruction
   • orchestration/delegation
   • evidence/verification
   • budget/policy
   • recovery/reconciliation
   • external dependency

3. Recovery class

   • retry_safe: no side effect occurred
   • resume_safe: durable provider session and checkpoint exist
   • compensate_first: partial side effects must be inspected or reversed
   • approval_required: authority expired, changed, or was never granted
   • terminal: deterministic policy denial or unrecoverable corruption
   • uncertain: fail closed because side-effect outcome cannot be proven

Two distinctions catch many misleading “successes”:

• provider turn complete versus requested task verified;
• tool returned zero versus intended external state confirmed.

For a stop receipt I would capture run/attempt/provider-session IDs; last durable event and event-log range; requested scope; effective approval/policy versions; side effects observed; expected evidence and validation status; checks run/skipped; failure domain and trigger; retry/resume safety; next required authority; and whether recovery replay converged with the live state.

The edge case worth making first-class is accepted_no_event: the provider accepted or a process started, but no durable semantic event arrived. It is not the same as not_started, and blindly retrying can duplicate external work.

Another is recovered_outcome_uncertain: after restart, logs prove some actions occurred but cannot prove the terminal outcome. That should remain blocked/uncertain rather than being projected as failed or completed.

License note: Better Agent is source-available and free for non-commercial use; commercial use requires separate permission.