Skip to content

feat(auto): sticky route binding — plan the auto route once per session, not once per turn #6252

Description

@Hmbown

Problem

The auto model router re-resolves on every user message: plan_turn_route (crates/tui/src/turn_route_plan.rs:143) runs the classifier (:146-166) per turn, and [auto] cross_provider (#4411) allows hopping providers turn-to-turn.

Every provider/model change forces a declared prefix-cache re-pin (crates/tui/src/core/engine.rs:7341-7374) — a guaranteed full cache miss on the next turn. With cache reads priced far below fresh input on every major provider, a per-turn auto route that switches models can cost more than the cheap model saves. Auto routing must be session-sticky: route once, stay bound.

Change

  1. crates/tui/src/turn_route_plan.rs:143 — consult the session binding first. If bound and the binding still revalidates against ModelInventory (crates/tui/src/model_inventory.rs:77 — credential present, provider runnable, readiness), skip the classifier entirely and resolve the bound provider+model.
  2. crates/tui/src/session_manager.rs:935 — promote SavedSession.last_auto_route (today: write-only provenance, set at tui/ui/event_loop.rs:2584 / dispatch.rs:933) to a load-bearing binding read at plan time. Already Option with serde defaults, so migration-safe.
  3. Rebind only on: binding fails revalidation, the bound route errors at dispatch (client.rs:1421 / route/resolver.rs:218 re-resolution), or the user explicitly changes model/provider (already a declared re-pin).
  4. [auto] cross_provider becomes a bind-time scope, not a per-turn permission: it widens the candidate set when the binding is created, not afterwards.

Already true, keep it that way

Transport retries and stream-resume never re-plan or switch provider (client.rs:3570, turn_loop.rs:1786-1941) — failure handling stays on the bound route.

Acceptance

  • A session on model = "auto" makes at most one classifier call per binding; subsequent turns show routing_source: StickyBinding (feat(auto): receipts — record route bindings, rebinds, and sticky vs fresh routing source #6253).
  • No prefix-cache re-pin occurs between turns that stay on the binding (verify via PrefixStabilityManager drift attribution, crates/core/src/prefix_cache.rs).
  • Removing the bound provider credential and resuming the session triggers exactly one rebind, receipted.
  • Explicit user model change busts the binding deliberately and is recorded as such.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    • Status
      Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions