Skip to content

Commit 54fa3a1

Browse files
authored
Add nadir-caliper submission (#173)
Consensus-verified cascade (qwen3-235b / grok-4.3 / gemini-3.1-flash-lite) with correct grok-4.3 pricing. Acc 75.84%, $0.22/1K, arena 0.7455. Supersedes nadir-tumbler. Audited: 8400 unique rows, real tokens.
1 parent d513354 commit 54fa3a1

4 files changed

Lines changed: 20 additions & 0 deletions

File tree

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"pipeline_params": {
3+
"router_name": "nadir-caliper",
4+
"router_cls_name": "NadirCaliperRouter",
5+
"models": [
6+
"qwen/qwen3-235b-a22b-2507",
7+
"google/gemini-3.1-flash-lite",
8+
"grok-4.3",
9+
"deepseek/deepseek-v3.2"
10+
],
11+
"description": "Nadir-Caliper, from Nadir Lab. A consensus-verified cascade: two cheap probes (qwen3-235b-a22b-2507 and gemini-3.1-flash-lite) answer every prompt at temperature 0. When their extracted final answers agree, the router ships the cheaper probe. When they disagree, grok-4.3 (low reasoning effort) arbitrates and the router ships the cheapest model whose answer the arbiter confirms; if it confirms neither, grok-4.3 is shipped. An answer counts only when both jaws touch it. Every design choice (probe pair, arbiter, branch policy) was selected on RouterBench only, family-grouped out-of-fold, at current model prices, and frozen before any RouterArena prompt was routed; two alternatives (gpt-5-mini arbiter, other probe pairs) were tested and rejected on RouterBench samples under a pre-registered criterion. A declared product constraint keeps at least 30% of traffic below the strongest menu model (measured: 84% downroute). All responses are self-generated (temperature 0, byte-exact prompts, token usage recorded); rows generated after 2026-08-05 carry per-row provenance (requested_model, model_used, provider, invoked_at). deepseek-v3.2 appears only as a declared per-row probe fallback where gemini-3.1-flash-lite generation was rate-limited. The robustness split is a genuine re-run of the identical rule on freshly generated probe and arbiter responses for all 420 perturbed prompts. Supersedes nadir-tumbler; please keep a single Nadir Lab entry.",
12+
"router_homepage": "https://getnadir.com",
13+
"contact_email": "info@getnadir.com",
14+
"affiliation": "Nadir Lab",
15+
"github": "https://github.com/NadirRouter"
16+
}
17+
}

‎router_inference/predictions/nadir-caliper-robustness.json‎

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.

‎router_inference/predictions/nadir-caliper.json‎

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.

‎universal_model_names.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@
8989
# xAI models
9090
"grok-4",
9191
"grok-4-1-fast-reasoning",
92+
"grok-4.3",
9293
# R2-Router
9394
"qwen/qwen3-235b-a22b-2507",
9495
"qwen/qwen3-next-80b-a3b-instruct",

0 commit comments

Comments
 (0)