Skip to content

chat(): client-tool and approval waits skip onFinish and leave OpenTelemetry spans open #1028

Description

@JsonKim

TanStack AI version

0.42.0

Framework/Library version

Framework-agnostic - chat() directly in Node.js, without a UI framework

Describe the bug and the steps to reproduce it

Summary

When chat() pauses for client-side tool execution or tool approval, the
current server invocation ends by emitting an interrupt boundary. However, the
middleware terminal lifecycle is skipped when the tool phase is "wait".

As a result:

  • onFinish is not called for the completed server invocation.
  • otelMiddleware leaves root and iteration spans open.
  • Operation duration and token metrics are not finalized.
  • The same lifecycle is skipped when an unresolved client tool or approval is
    restored from the initial message history.

The overall generation is still waiting for a client result, but the current
server invocation has ended and should have exactly one terminal middleware
callback.

Steps to reproduce

  1. Define a client-side tool or a tool that requires approval.
  2. Add middleware with an onFinish callback, or configure otelMiddleware.
  3. Call chat() with an adapter that returns a tool call and finishes the
    iteration with finishReason: "tool_calls".
  4. Consume the stream until it emits a RUN_FINISHED interrupt for client-tool
    execution or approval.
  5. Inspect the middleware callbacks and telemetry state.

Actual behavior

  • The interrupt boundary is emitted.
  • onFinish is not called.
  • OpenTelemetry root and iteration spans remain unfinished.
  • gen_ai.client.operation.duration is not recorded.

For a pending interrupt restored from message history, a middleware error while
emitting the boundary can also produce RUN_ERROR and then incorrectly continue
into another provider call.

Expected behavior

A client-tool or approval wait should:

  • End the current middleware lifecycle exactly once with onFinish.
  • Finalize OpenTelemetry spans and metrics for that server invocation.
  • Continue using onAbort and onError for cancellation and failures.
  • Stop immediately if emitting a pending interrupt boundary fails.
  • Keep structured-output finalization suspended while waiting.
  • Resume and finalize structured output only in the later invocation that
    includes the client tool result or approval decision.

Root cause

The terminal middleware hook is skipped when toolPhase === "wait". Pending
client-tool and approval waits can also return before reaching the normal
terminal lifecycle block.

Middleware finalization and structured-output finalization need separate
semantics at this boundary: middleware should finish for the current invocation,
while structured output should remain pending for the overall generation.

A fix and regression coverage are available in #996.

Your Minimal, Reproducible Example - (Sandbox Highly Recommended)

https://stackblitz.com/~/github.com/JsonKim/tanstack-ai-client-tool-wait-repro

Screenshots or Videos (Optional)

No response

Do you intend to try to help solve this bug with your own PR?

Yes, I am also opening a PR that solves the problem along side this issue

Terms & Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    has-prAn open PR references this issuewaiting-on: maintainerThe ball is in the maintainers’ court

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions