Skip to content

endTurn tool-result short-circuit bypasses usage:updated/inference:completed telemetry #67

Description

@fgtb

Problem

Any turn that ends via a tool result with endTurn: true never fires usage:updated or inference:completed, so UsageTracker, CallLedger, and any TUI/webUI display driven by those events silently under-reports — or entirely misses — real usage for that turn. This isn't provider-specific; it reproduces on any agent that replies through a channel-style tool (e.g. connectome-host's channels module skip_reply/reply tools set endTurn: true on essentially every real turn).

Root cause

In framework.ts's driveStream, when a tool result has endTurn: true (shouldEndTurn), the framework takes a short-circuit exit: it calls stream.cancel() directly and emits inference:turn_ended — bypassing the stream's natural for await completion.

if (shouldEndTurn) {
  ...
  this.settleAgent(agent.name, { stopReason: 'turn_ended', speech: '' });
  this.emitTrace({ type: 'inference:turn_ended', agentName: agent.name });
}

usageTracker.onInferenceCompleted() — the only call site that emits usage:updated — lives exclusively inside the case 'complete': branch of driveStream's stream-event switch, reached only on a normal stream completion. The endTurn shortcut never reaches it, so for these turns:

  • usage:updated never fires (no cumulative usage tracking)
  • inference:completed never fires (no per-turn completion trace with tokenUsage)

Reproduction

Minimal repro via connectome-host (a downstream consumer), headless mode, single turn:

{"type":"text","content":"Say the word \"test\" and nothing else."}

Observed trace event sequence (types only):

lifecycle(ready), process:received, message:added, gate:decision, process:completed,
inference:started, inference:usage, inference:tool_calls_yielded, tool:started, tool:completed,
process:received, inference:turn_ended, process:completed, lifecycle(idle)

Note: inference:usage (a separate, per-round trace fired from Membrane's own stream.emit({type:'usage',...}), case 'usage': in driveStream) does fire and does carry tokenUsage.cacheCreation/tokenUsage.cacheRead correctly — so per-round cache/usage data is available on the wire, it's just never accumulated or surfaced through usage:updated/inference:completed when the turn ends via endTurn.

The triggering tool result in the repro:

{"type":"tool-result","callId":"toolu_...","result":{"success":true,"endTurn":true,"data":{"skipped":true,"note":"Turn ended; nothing sent."}}}

Impact

  • UsageTracker totals (and anything reading them, e.g. connectome-host's CallLedger/webUI cost display) undercount for any agent whose turns typically end via a channel/reply tool's endTurn — which in practice is most channel-routed agents' normal operation, not an edge case.
  • Cost/cache-monitoring dashboards built on usage:updated are unreliable for these agents even though the underlying per-round data (inference:usage) is correct and available.

Suggested fix direction

Call usageTracker.onInferenceCompleted() (and/or emit inference:completed) from the shouldEndTurn branch too, using the same per-round usage accumulation that already feeds inference:usage, rather than only from the natural-completion case 'complete': path.

Environment

  • @animalabs/agent-framework 0.6.8
  • Reproduced via connectome-host (anima-research/connectome-host), headless mode, provider: openrouter, model: anthropic/claude-sonnet-5 — but the bypass is provider-agnostic; it's triggered by the endTurn tool-result flag, not by any provider-specific behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions