Skip to content

fix: prevent deferred inference requests from starving the event loop#66

Draft
antra-tess wants to merge 1 commit into
mainfrom
fix/busy-agent-scheduler-yield
Draft

fix: prevent deferred inference requests from starving the event loop#66
antra-tess wants to merge 1 commit into
mainfrom
fix/busy-agent-scheduler-yield

Conversation

@antra-tess

Copy link
Copy Markdown
Collaborator

Summary

  • always yield a real event-loop turn when inference requests are deferred behind a busy agent
  • preserve low-latency processing when queue traffic and background work coexist
  • throttle stale-request traces together with their once-per-minute warning
  • add regression coverage for the exact scheduler states observed during the Sol outage

Root cause

When a new wake arrived while an agent was streaming or waiting_for_tools, processInferenceRequests() requeued it. processNextEvent() treated the non-empty request queue as progress and skipped its normal backoff. If activeStreams was empty or stale at the same time, the loop had no macrotask yield at all.

That microtask-only polling loop pinned Bun at 100% CPU and starved the tool-result and HTTP I/O needed to make progress. Sol remained waiting_for_tools, while Fleet health probes timed out and reported the agent down.

Impact

Deferred wakes can no longer turn an otherwise recoverable tool wait into a self-sustaining event-loop wedge. Tool results, Web UI health checks, Discord traffic, and timers continue receiving event-loop time while the wake remains queued.

Validation

  • npm run build
  • focused scheduler, reconnect, and channel-routing tests
  • full agent-framework suite: 326 passed, 0 failed
  • deployed to Sol as a canary: Fleet returned UP, Web UI HTTP 200, zero pending requests, and normal CPU

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant