Skip to content

Streamed model calls run in detached spawn: thread_id/task-locals lost, cancellation doesn't stop billed provider call #4460

Description

@senamakel

Part of the v0.58.7 → HEAD TinyAgents-migration audit (see parent issue).

Problem

ProviderModel moves every streamed provider call into a detached tokio::spawn producer (src/openhuman/tinyagents/model.rs:529) — which is virtually all interactive parent and subagent turns. Three consequences (all verified):

  1. Task-locals lost. thread_context::current_thread_id() is None inside the spawn → the managed backend's thread_id extension (compatible_request.rs:70-77, opted in at openhuman_backend.rs:124) is omitted on every streamed request. Backend-side thread attribution and prompt-cache grouping are silently broken; with_thread_id scopes (e.g. spawn_async_subagent.rs:486) are defeated.
  2. Route audit broken. record_resolved_provider_route writes to a task-local (resolved_route.rs:41-45) that no longer scopes the caller — channel audit events fall back to the requested route; alias/fallback resolution is invisible.
  3. Cancellation doesn't stop billing. On hard abort (AbortHandle), the spawned provider call is not dropped — it runs to completion in the background and is still billed. The legacy path called provider.chat inline (dropped on cancel).

Fix plan

  1. Capture the relevant task-local values (thread id, any scoped context) before spawning and re-establish them inside the spawned task (ThreadContext::scope(thread_id, async { provider.chat(...) })). Audit for other task-locals crossing this boundary (resolved-route slot → pass an explicit out-slot instead of a task-local, or scope it the same way).
  2. Tie the producer's lifetime to the consumer: hold the JoinHandle in an abort-on-drop guard so dropping the stream/turn future aborts the in-flight provider call. If the crate owns the consume side, wrap our future in the guard at the seam (run_turn_via_tinyagents_shared).
  3. Add a debug log when a provider call is aborted mid-flight so cancellation behavior is observable.

Acceptance criteria

  • Streamed requests to the managed backend carry the thread_id extension again (assert via mock backend in scripts/mock-api-* / e2e).
  • Channel audit events report the resolved route under streaming.
  • Hard-cancelling a turn aborts the in-flight provider request (mock server observes connection close, not completion).

Metadata

Metadata

Assignees

Labels

agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.bugpriority: highBreaks a specific featuresubtaskSubtask of a larger tracked effort.

Type

No type

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions