feat: Antigravity provider + configurable routing policy (difficulty bands, decision agent, fallback chains) - #2
Open
nuchareviews-beep wants to merge 24 commits into
Conversation
router.ts's isRoutableProvider() and benchmarks.ts's rankAutoModelOptions() each hardcoded their own separate provider allowlist. Adding "antigravity" to router.ts's copy made it eligible for thread creation, but rankAutoModelOptions kept using its own stale copy internally, so Antigravity candidates were silently excluded from every ranked (benchmarked) selection and could only ever be chosen via fallbackSelection -- which only runs when Codex, Claude Code, and Cursor are ALL simultaneously unavailable or quota-exhausted. Verified live against a running bb instance before this fix: two `bb autorouter route` calls (difficulty 2 and 92), plus one with an explicit "route this to agy" instruction and one naming a real agy model id directly, all still picked Codex every time. That's the gap this fix closes. Both files now read one shared ROUTABLE_PROVIDER_IDS from benchmarks.ts, so this class of two-lists-drift-apart bug can't recur. No fabricated benchmark score is introduced for Antigravity/ Gemini models -- rankAutoModelOptions correctly still returns no ranked option for them (no CursorBench entry exists), matching this repo's existing "does not invent scores for unmeasured models" policy. Added two tests: one confirming that policy still holds for Antigravity specifically, one confirming fallbackSelection actually returns a route on Antigravity when it's the only eligible candidate -- the real, narrow condition under which it gets picked in practice. All 30 tests pass, typecheck and build are clean.
Below a difficulty threshold (25/100), skip CursorBench-driven ranking and use a fixed priority instead: Antigravity (local agy, no per-token billing) first, then Codex, then Claude Code. Cursor keeps its normal benchmark-ranked path at every difficulty -- it's not part of this priority list. This is a real preference, not a fallback-of-last-resort: previously Antigravity only ever won when Codex, Claude Code, and Cursor were ALL simultaneously unavailable (see the prior commit). Simple tasks don't need a capability-matched model chosen from a benchmark curve built for harder work; they need the cheapest thing that can do them. An explicit model override (user-requested or from custom instructions) still takes priority over this -- it only applies to the automatic difficulty-based path. Verified live against a running bb instance: `bb autorouter route` at difficulty 0 and 3 now correctly picks Antigravity and produces a real response; difficulty 92 is unaffected and still picks Codex, so this doesn't regress normal-difficulty routing. 6 new tests (36 total, all passing) cover the priority order, quota-exhaustion fallthrough at each tier, the threshold boundary, and deferring to benchmark ranking when none of the three are eligible.
Replace the free-text "provider/model" input with a searchable provider/model list, following the same pattern as bb-plugin-prompt-enhancer's ModelSettingsSection: a listModels RPC backed by a KV-cached, stale-while-revalidate provider/model catalog, rendered as a cmdk-based Command picker. Settings storage is unchanged (decisionAgent stays a "provider/model" string, still readable via `bb autorouter config --decision-agent`); this only changes how it's set from the settings UI. Vendors components/ui/command.tsx adapted from prompt-enhancer's version, swapped to autorouter's own HugeIcons-based <Icon> instead of adding lucide-react as a second icon library.
The classifier's "automatic" mode used to hardcode a fixed provider priority (Cursor gpt-5.6-sol-medium -> Codex gpt-5.6-luna) directly in router.ts. Move it to a new automaticFallbackChain setting (ordered provider/model list, editable via the settings-page picker or `bb autorouter config --automatic-fallback`), so it's a preference users can reorder, extend, or clear rather than an opinion baked into the code. Default value reproduces the previous hardcoded order, so existing "automatic" behavior is unchanged unless a user edits it — verified via the existing router.test.ts suite (all 36 tests still pass unmodified, since they exercise resolveRoute through defaultAutorouterSettings). parseStoredSettings now merges stored settings with defaults before giving up, instead of discarding the whole object on schema mismatch — otherwise settings saved before this field existed (including this session's own decisionAgent override) would have silently reset to every default on first load after the update.
1. Per-difficulty model selection (settings.difficultyBands): replaces the
hardcoded "simple task" shortcut (SIMPLE_TASK_DIFFICULTY_MAX = 25,
SIMPLE_TASK_PROVIDER_PRIORITY = [antigravity, codex, claude-code]) with
a plain, editable list of {maxDifficulty, fallbackChain} bands, checked
low-to-high. Chain entries may be a bare provider (any/default model)
or an exact provider/model pin. Default settings reproduce the old
hardcoded band as ordinary data, not logic baked into router.ts. New
difficultyBandSelection() replaces simpleTaskSelection(); router.test.ts
updated to match (39 tests, was 36) plus new coverage for band ordering,
exact-model pins in a chain, and "no band covers this difficulty".
2. Decision section: merged the "Decision agent" picker and "Automatic
fallback chain" picker into one Command control per the same UX
pattern used for difficulty bands. In Automatic mode the single list
is the fallback-order editor (checking a model toggles its membership
in automaticFallbackChain); switching to a specific model pins the
classifier to it directly, and the fallback-order editor hides since
it's not in play. Same picker component now reused three ways
(Decision agent, and once per difficulty band) via a shared
DifficultyBandsSection component.
Verified: tsc clean, 39/39 tests pass, `bb plugin build` produces real
dist/ output.
Bands used to always mean "difficulty <= maxDifficulty" -- the comparator was implicit and hardcoded in difficultyBandSelection. Add a comparator field (one of <=, <, >=, >, ==) to each band, defaulting to "<=" so existing bands (including the shipped default) keep their exact prior meaning unless explicitly changed. This makes ">= 80" or "== 50" style bands expressible, not just "at or under N". - settings.ts: BAND_COMPARATORS, BandComparator, DEFAULT_BAND_COMPARATOR; difficultyBandSchema requires comparator (strict schema). - router.ts: new bandMatchesDifficulty() switches on the comparator; difficultyBandSelection() uses it instead of a hardcoded `<=`. - app.tsx: band editor gets a comparator <select> next to the threshold input; "Add band" seeds new bands with the default comparator. - server.ts: status output and --difficulty-bands JSON docs mention the field. Migration: parseStoredSettings now backfills a missing `comparator` on each stored band (defaulting to "<=") before re-validating, instead of letting one old-shape band fail validation and silently reset the whole settings object to every default -- the same class of bug fixed for top-level fields last commit, now handled one level deeper. Added settings.test.ts to cover this directly (it wasn't covered anywhere before), including the exact "field added after settings were stored" scenario that broke live earlier this session. Tests: 45 -> 50 (6 new bandMatchesDifficulty/comparator-aware-selection cases in router.test.ts, 5 new migration cases in settings.test.ts). tsc clean, bb plugin build produces real dist/ output.
Docs previously described a fixed, hardcoded "Antigravity priority for simple tasks" rule. Rewrite to document the actual current mechanism: settings.difficultyBands (comparator-aware, user-editable) and the merged Decision agent picker/fallback-order editor, with the old hardcoded behavior noted as just the shipped default value. Also adds a real screenshot of the rendered settings page (Decision agent picker + fallback order editor) for the PR description.
settings.difficultyBands now stores { minDifficulty, maxDifficulty,
fallbackChain } directly instead of { maxDifficulty, comparator,
fallbackChain }. Both bounds are inclusive and independently nullable
("unbounded" on that side), so a band expresses a genuine two-sided range
in one native shape -- e.g. 0-25 for one model, 26-75 for another, 76-100
for a third -- rather than five separate single-sided comparators that
had to be paired up by hand to express a range.
- settings.ts: difficultyBandSchema is now { minDifficulty, maxDifficulty
(both nullable ints, 0-100), fallbackChain }, with a refine() rejecting
min > max. migrateBand() converts both prior shapes -- the original
pre-comparator { maxDifficulty } (implicit "<=") and last commit's
{ maxDifficulty, comparator } -- into the equivalent range, so existing
stored settings (including this session's own live band) keep their
exact prior meaning. BAND_COMPARATORS/BandComparator/
DEFAULT_BAND_COMPARATOR are gone; nothing else referenced them outside
router.ts/app.tsx, both updated in this commit.
- router.ts: bandMatchesDifficulty() checks difficulty against
[minDifficulty, maxDifficulty] directly; band evaluation order sorts by
effective lower bound (null treated as -1) instead of maxDifficulty.
- app.tsx: the band editor's comparator <select> + single number input
become two number inputs ("min" / "max"), each blank = unbounded on
that side.
- server.ts: status output and --difficulty-bands JSON docs/example
updated to the range shape.
- README.md: rewrites the per-difficulty section to document ranges
instead of the comparator model it described in the last commit.
Tests: 53/53 (was 50) -- router.test.ts's comparator suite replaced with
range-equivalent coverage (unbounded-below, unbounded-above, two-sided,
exact-value, fully-unbounded, sort-by-effective-lower-bound, and an
explicit 0-25/26-75/76-100 three-band routing case), settings.test.ts
gets a migration case per prior shape variant. tsc clean, bb plugin build
produces real dist/ output.
Real crash: bb.sdk.system.usageLimits() has been observed to omit a provider's key entirely (rather than report a typed error status for it), which crashed the whole route with "Cannot read properties of undefined (reading 'status')" -- reproduced live via `bb autorouter route` right after independently confirming the same underlying bb-core quirk in bb-plugin-usage's own provider-limits fetch (unrelated plugin, same root cause -- not something this plugin can fix upstream). remainingQuota() now treats a missing entry the same as an in-band "error" status: assume available (1) rather than crash. Regression test added. Also repinned @get-bb/plugin-sdk to 0.4.21 via `bb plugin types .` (was still 0.4.8), matching the fix already applied to bb-plugin-omniroute-acp and bb-plugin-antigravity-acp this session. Tests: 54/54. tsc clean, bb plugin build produces real dist/ output.
routing, and escalation rules (schema only, not wired up yet) Shared baseline for the next feature epic, added first and in isolation so three parallel implementation threads can each build their own router.ts logic against a frozen contract without touching the same lines of the same file at the same time: - excludedModels: global exclude-list, filtered out of candidates before anything else runs. - allowedProviders: if non-empty, restricts every routing path to this provider set. This is the realistic, buildable form of "only route delegated work through OmniRoute" -- autorouter IS the thing that picks a model for new delegated threads in this stack, so constraining its own candidate pool is the actual enforcement point. Does not and cannot force Codex/Claude Code/Antigravity to stop using their own native tool loops for work they run directly -- that's a structural ceiling documented in AGENTS.md's Known Gaps, not something a setting can close. - taskTypeBands: checked before difficultyBands -- a task-type match (e.g. "vision" for image/video) is a stronger signal than a raw difficulty score for tasks that aren't hard, just a different kind of work. Seeded with a real default (vision -> antigravity, the only multimodal-capable provider in this stack) rather than left empty. - escalationRules: quota-triggered temporary escalation with a configurable response-count limit and an optional handoff note. Response count means routed *threads*, not turns within one thread -- autorouter only runs at thread-creation time. Backward-compat verified: old stored settings (pre-dating all four new fields) load correctly with the new fields backfilled to their defaults, nothing wiped -- this exact bug class hit twice already this session, so checked directly rather than assumed. tsc clean. Router logic, UI, and CLI flags land in follow-up commits.
…ches Three parallel implementations merged (feat/exclude-allowlist, feat/task-type-routing, feat/escalation-rules), each built in its own git worktree against the same frozen settings.ts contract so they couldn't collide while landing. One real, expected conflict: both task-type and escalation branches added their own type import to the same import block in router.ts/router.test.ts -- resolved by keeping both (not a logic conflict, just two additions to the same lines). 70/70 tests passing (54 baseline + 5 exclude/allowlist + 6 task-type + 5 escalation). tsc clean. None of resolveRoute, settings.ts, app.tsx, or server.ts were touched by any of the three branches -- final wiring of these standalone functions into resolveRoute, plus UI and CLI, is the next commit.
…resolveRoute Final integration of the three parallel worktree implementations merged in the previous commit: - filterExcludedAndDisallowed applied immediately after candidate discovery, so exclusions/allowlist govern the classifier's own model choice too, not just the final selection. - Escalation checked first, before classification -- an active window skips the classifier call entirely. If the escalated model isn't actually reachable, the window's response count is NOT consumed on that attempt, so it stays available for a request that can use it. - taskTypeBandSelection checked before difficultyBandSelection (task type is a stronger signal than a raw difficulty score for tasks that aren't hard, just different -- e.g. an image description). - ResolvedRoute gains an optional escalationNote; createRoutedThread prepends it as its own text part ahead of the user's prompt when an escalation is active, so the escalated model actually sees the handoff context instead of it being silently dropped. Also added CLI flags (--task-type-bands, --excluded-models, --allowed-providers, --escalation-rules) and status-output lines for all four settings -- these existed in settings.ts's schema but had no way to actually be set before this commit. 70/70 tests passing, tsc clean, bb plugin build produces real dist/ output. Live end-to-end verification (real bb autorouter route calls proving each feature actually changes routing behavior, not just that it compiles) is the next step before calling this done.
This reverts commit 1a9eab8.
This reverts commit 6f143c2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Supersedes #1. Includes everything from that PR (the
antigravityprovider becoming a real, working routable provider, plus the two-lists-drift-apart benchmark bug fix it caught and fixed) and a full configurable routing-policy system built on top of it — every routing decision that PR deliberately left as a fixed rule (to keep that PR focused) is now a plain, editable setting instead of logic baked intorouter.ts.What's in this PR
1. Antigravity as a real routable provider (from #1, unchanged)
antigravityprovider (bridges to the localagyCLI viabb-plugin-antigravity-acp) to Autorouter's routable-provider set.router.tsandbenchmarks.tseach hardcoded their own separate provider allowlist, so Antigravity was eligible for candidate discovery but silently excluded fromrankAutoModelOptions's ranking regardless. Both files now read one sharedROUTABLE_PROVIDER_IDS.fallbackSelectionwhen Codex/Claude Code/Cursor are all unavailable — unless a difficulty band below routes to it directly.2. Per-difficulty model selection: native two-sided ranges (new)
settings.difficultyBands: an ordered list of{ minDifficulty, maxDifficulty, fallbackChain }bands, checked low-to-high by effective lower bound. Both bounds are inclusive and independently nullable —minDifficulty: nullmeans unbounded below (matches down to 0),maxDifficulty: nullmeans unbounded above (matches up to 100), andminDifficulty === maxDifficultymatches an exact score. The first band whose range covers a task's difficulty score skips benchmark ranking entirely and routes straight through that band's own fallback chain.This is a genuine two-sided range, not a series of independent one-sided thresholds paired up by hand:
0–25for one model,26–75for another,76–100for a third, with no gaps or overlaps to manage. Chain entries can be a bare provider (any/default model) or an exactprovider/modelpin.This directly replaces what would otherwise have been a hardcoded rule (
SIMPLE_TASK_DIFFICULTY_MAX = 25/SIMPLE_TASK_PROVIDER_PRIORITY = [antigravity, codex, claude-code]) — that logic doesn't exist inrouter.tsat all; it's only the default value of this setting. Add, remove, reorder, or clear bands freely from the settings page or:bb autorouter config --difficulty-bands '[{"minDifficulty":null,"maxDifficulty":25,"fallbackChain":["antigravity","codex","claude-code"]},{"minDifficulty":26,"maxDifficulty":75,"fallbackChain":["codex/gpt-5.6-terra"]}]'(This went through one design iteration during review: the first version of this PR used a single threshold + comparator (
<=,<,>=,>,==) per band, which could express a one-sided cutoff but needed two bands hand-paired to express a real range. Replaced with native two-sided bounds, which subsumes every comparator case directly —<= Nis{null, N},>= Nis{N, null},== Nis{N, N}— while making an actual range a single band instead of an implicit pairing. Existing stored settings (including comparator-shaped bands saved during that iteration, and the original pre-comparator shape before that) migrate automatically — see Verification.)3. Decision agent: one merged control (new)
The model that rates each task's difficulty before routing is itself configurable, via a single searchable provider/model picker instead of a free-text box:
automaticFallbackChain, same shape and same picker pattern as the difficulty bands above. In this mode the picker doubles as the fallback-order editor — checking a model toggles its membership in the chain, shown right below as a reorderable list.If nothing in the fallback order is available, the classifier falls back to any launchable model, then whatever's first — a last-resort safety net, not a preference, so it isn't user-facing.
Screenshots
Decision agent picker + fallback-order editor, rendered live:
Per-difficulty model selection, rendered live — note this screenshot is from the single-comparator iteration (
<=dropdown + one threshold) described above, captured before the native-range rewrite in this update. The range UI replaces that dropdown with two bounded number inputs ("min" / "max", either blank for unbounded) in the same card layout; a refreshed screenshot will follow once available, but the underlying behavior shown here — ordered fallback chain, reorder/remove, add-band, add-model-to-chain — is otherwise unchanged by the rewrite:Verification
npx tsc --noEmit— cleannpx vitest run— 53/53 passing (28 original + 25 new across this PR: routable-provider regression coverage, band-range/exact-model-pin/two-sided-range cases, and settings-migration coverage for every prior stored shape)bb plugin build— produces realdist/outputbb autorouter routecall lands on the configured decision-agent model by tracing the hidden classifier thread's actualproviderId, not just trusting the setting saved.0–25/26–75/76+) correctly routes a difficulty-55 task to the middle band's pinned model (benchmarkScore: null, bypassing ranking) — not just the edges.settings.test.tsfor every shape.Scope
README.md,app.tsx,benchmarks.ts,router.ts,router.test.ts,server.ts,settings.ts,settings.test.ts,components/ui/command.tsx(vendored/adapted frombb-plugin-prompt-enhancer's picker component),docs/screenshots/.Update: model exclude-list, provider allowlist, task-type routing, and quota escalation (new since the above)
Four more settings shipped on top of everything above, same philosophy — plain, editable data, not logic baked into
router.ts:excludedModels— global exclude-list (bare provider or exactprovider/model), filtered out before anything else runs, including the classifier's own model choice.allowedProviders— if non-empty, restricts every routing path to that provider set. This is the realistic, buildable form of "only route through provider X": it constrains autorouter's own candidate pool (codex/claude-code/acp-cursor/antigravity). It does not make OmniRoute routable through autorouter — OmniRoute was deliberately never added to the routable-provider set; OmniRoute delegation happens via a separate mechanism (omniswarm_spawn), which already defaults to OmniRoute with its own fallback. Also does not and cannot stop Codex/Claude Code/Antigravity from using their own native tool loops for work they run directly — that's a structural ceiling, not something any setting closes.taskTypeBands— checked before difficulty bands: a task-type match (e.g.visionfor image/video/audio analysis) is a stronger signal than a raw difficulty score for tasks that aren't hard, just a different kind of work. Ships a real default (vision -> antigravity, the only multimodal-capable provider in this setup).escalationRules— quota-triggered temporary escalation. When a rule'sfromProviderreads quota-exhausted at routing time, the nextresponseLimitrouted threads (not turns within one thread — autorouter only runs at thread-creation time) go straight totoModel, with an optionalhandoffNoteprepended to the prompt, then the window closes and routing reverts automatically.Verification (all four, live — not just unit tests)
excludedModels: ["antigravity"], confirmed a simple task that would have picked antigravity via its difficulty band instead fell through to the band's next entry.allowedProviders: ["claude-code"], confirmed routing landed on claude-code despite the matched band's chain listing antigravity and codex first.visiontask-type band even though its classified difficulty (50) would not have matched the0-25difficulty band at all.Development note: the three pieces above (exclude/allowlist, task-type, escalation) were implemented in parallel across three isolated git worktrees against a shared, frozen
settings.tsschema, each producing only new standalone functions (no shared function bodies touched), then merged and wired intoresolveRoutein one integration pass — avoided the conflict risk of multiple parallel edits to the same lines of the same files.Tests: 70/70 passing (was 54 before this update).
tscclean,bb plugin buildproduces realdist/output.