proposal(auto): learned cost/quality routing trained on receipts — exploration, replay labels, embedded artifact

[Hmbown]

Context

Depends on the sticky-binding and receipts issues (#6252, #6253). This is a proposal/tracking issue, not committed scope.

Once sticky auto bindings produce receipted outcomes (binding id, model, tokens, cost, TurnOutcomeStatus, rebind history), the ledger becomes training data for a learned cost/quality router: given a session's opening request, pick the cheapest candidate likely to succeed, escalate on failure. The open-source reference points evaluated for this:

• RouteLLM / LLMRouter (lm-sys/RouteLLM, ulab-uiuc/LLMRouter; Apache-2.0/MIT) — trained classifiers for cost/quality routing. The practical method for our 2-candidate shape (cheap vs escalation) is LLMRouter's racerrouter: a ~238K-param MLP over a query embedding with an explicit cost budget. Its checkpoint is a self-describing versioned dict; the forward pass is ~50 lines of Rust, no ML runtime.
• TensorZero (Apache-2.0, archived 2026-06) — Rust gateway-as-library; reference for config schema and fallback-chain design, not a dependency.

Open problems this issue tracks

1. Counterfactual labels. The ledger records only the candidate actually used; every published trainer needs per-candidate outcomes per prompt. Options: bounded exploration at session scope (a small fraction of new sessions bind to the alternate candidate — never per-turn, which would bust the prefix cache), plus offline replays of logged opening prompts against the other candidate for judge/outcome labels.
2. Text encoder. LLMRouter's classifiers sit on a 149M-param Longformer — too heavy to ship for a routing decision. Retrain on a small encoder exportable to ONNX (MiniLM/bge-small class) using LLMRouter's precomputed-embedding backend, keeping train-time (Python) and run-time (Rust) encoder consistency.
3. Artifact format + Rust scoring. Versioned checkpoint (candidate list, encoder id, weights, training summary); a tiny pure-Rust forward pass behind the same binding-creation path as scenario rules, so the learned router is one more bind-time decider with identical receipts.
4. Evaluation harness. Offline replay on ledger data must beat the scenario-rules baseline on cost-at-equal-quality before any live traffic sees it; exploration rate and escalation policy are config with conservative defaults.

Non-goals

• No per-turn learned routing (cache economics forbid it — see feat(auto): sticky route binding — plan the auto route once per session, not once per turn #6252).
• No second runtime or external router service; the routing decision stays a function call inside the Engine.
• No training on customer content; embeddings and labels derive from receipted metadata under the existing telemetry boundaries.

Acceptance (for the tracking issue itself)

A dated evaluation report: replay harness, baseline vs learned-router cost/quality on ledger-derived data, and a go/no-go recommendation for live exploration.