fix(reliability): a dead socket is a transport failure, not a tool bug - #262
Merged
Conversation
**Sentry's biggest tool-error bucket is not a tool error at all.**
`CLI-4N` (108 events), `CLI-4T` (44), `CLI-4A` (173) and their siblings
all report the same thing: `ToolExecutionError`, `tool=unknown`,
`cause_type=TypeError`. `CLI-4T`'s stack ends in
`undici … Socket.onHttpSocketClose` — a connection that died, filed as
"our tool broke".
`classifyFailure` ends in a catch-all `return "tool"`, and `toLlmFailure`
wraps whatever is left into `ToolExecutionError("unknown", …)`. Anything
that is not a typed `LlamaServerError` / `OpenAiHttpError` lands there —
which is every raw `undici` failure from a surface that does not wrap its
own errors: MCP streamable-http transports, embedding calls, vendor SDKs
carrying their own `fetch`. They arrive as a bare `TypeError: fetch
failed` with the real errno buried in `cause`.
That misfiling costs twice. The user is told `Turn failed [tool]` for
someone else's dead socket. And `shouldAdvance` refuses to advance on a
`tool` category — by definition our own bug, not the provider's — so a
genuine network failure never falls over to the next link in the chain,
which is exactly what the fallback chain exists to do.
`isNetworkError` walks the `cause` chain looking for connection-level
errnos (`ECONNREFUSED`, `ECONNRESET`, `ENOTFOUND`, `UND_ERR_*`, …) or
undici's stock socket messages (`fetch failed`, `terminated`, `socket
hang up`, `other side closed`). Those classify as `transport` and wrap as
`TransportError`, so the chain advances and the turn reads honestly.
Two deliberate exclusions:
- `EPIPE` / `EIO` are **not** in the set. Those are overwhelmingly stdio
— a closed host pipe, a vanished tty — and reading one as `transport`
would send the fallback chain hunting for a different provider over a
broken *local* stream.
- The check sits *after* the cancellation branch. An aborted request can
surface as `ECONNRESET`, and a user pressing Esc is not a fallover.
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Sentry shows
Three of the top tool-error issues in
cliare not tool errors:CLI-4Ncategory=tool,tool=unknown,cause_type=TypeError, zero stack framesCLI-4Tcategory=tool,tool=unknown,cause_type=TypeError, frames end inundici … Object.errorRequest ← Socket.onHttpSocketCloseCLI-4Acategory=tool,tool=unknown,cause_type=ErrorPlus
CLI-G,CLI-F,CLI-2K,CLI-4F,CLI-2S,CLI-Win the same shape — ~700 events filed as "our tool broke" for what the stack says is a connection that died.Root cause
classifyFailureends in a catch-allreturn "tool", andtoLlmFailurewraps the remainder intoToolExecutionError("unknown", …). Everything that is not a typedLlamaServerError/OpenAiHttpErrorlands there — i.e. every rawundicifailure from a surface that does not wrap its own errors (MCP streamable-http, embeddings, vendor SDKs with their ownfetch). Those arrive as a bareTypeError: fetch failedwith the real errno buried incause.The misfiling costs twice over:
Turn failed [tool]for someone else's dead socket;shouldAdvancerefuses to advance ontool(by definition our own bug, not the provider's), so a genuine network failure never falls over to the next provider — precisely what the fallback chain exists for.The change
src/llm/reliability/network-error.ts—isNetworkError/readNetworkErrorCodewalk thecausechain (bounded, cycle-safe) for connection-level errnos (ECONNREFUSED,ECONNRESET,ENOTFOUND,EAI_AGAIN,UND_ERR_*, …) or undici's stock socket messages (fetch failed,terminated,socket hang up,other side closed).classifyFailurereturnstransportfor those;toLlmFailurewraps them asTransportErrorcarrying the original ascause.Two deliberate boundaries:
EPIPE/EIOare excluded. Those are overwhelmingly stdio — a closed host pipe, a vanished tty — and reading one astransportwould send the chain hunting for a different provider over a broken local stream. (They are their own defect; separate PR.)ECONNRESET, and a user pressing Esc is not a fallover. Pinned by a test.Tests
network-error.test.ts(10 new), plusclassify-failure.test.tsandstep-executor.test.tscases proving afetch failedfrom the completion path surfaces asTransportErrorwhileTypeError: x.map is not a functionstill surfaces asToolExecutionError.npm run lintclean;npm test6330 passed / 1 failed — the failure issrc/sidecar/send-message-concurrency.test.ts, which fails identically on untouchedmainin this environment.