Skip to content

Commit 8b41012

Browse files
steventohmeclaudeyl231
authored
Update Weave Router scores to v0.62 (#1) (#41)
* Update Weave Router scores to v0.62 Weave Router v0.62 submission re-took #1 on the leaderboard. Updated the headline leaderboard entry with the new RouterArena eval results: - Arena Score: 74.61 → 76.09 - Accuracy: 78.43 → 79.32 - Cost per 1k: $0.92 → $0.61 - Robustness Score: 79.05 → 79.76 - Opt. Selection: 1.27 → 5.60 - Opt. Cost: 12.10 → 17.19 - Opt. Accuracy: 89.56 → 89.21 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Update Weave Router metadata to v0.62 (modelPool + description) Mirrors the v0.62 submission config (RouterArena PR #102): - 12-model pool -> 7 models (drops claude-sonnet/haiku-4-5, gpt-5.4-mini, gpt-4.1, qwen3.5-flash-02-23, kimi-k2.5; adds qwen3-235b-a22b-2507) - description updated to mention v0.62 specifics: Jina v2 INT8 ONNX (768-dim) embeddings, top-p=4 cluster sum, k=160 clusters, alpha=0.25 Category scores and flip labels are intentionally left at v0.27-derived values for this PR; can be refreshed in a follow-up once v0.62 predictions are available locally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: yl231 <yl231@users.noreply.github.com>
1 parent d028d27 commit 8b41012

3 files changed

Lines changed: 11 additions & 21 deletions

File tree

‎src/data/routerMetrics/leaderboard.json‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12,14 +12,14 @@
1212
},
1313
{
1414
"Router Name": "Weave Router",
15-
"Arena Score": 74.61,
16-
"Optimal Selection Score": 1.27,
17-
"Optimal Cost Score": 12.1,
18-
"Optimal Acc. Score": 89.56,
19-
"Robustness Score": 79.05,
15+
"Arena Score": 76.09,
16+
"Optimal Selection Score": 5.6,
17+
"Optimal Cost Score": 17.19,
18+
"Optimal Acc. Score": 89.21,
19+
"Robustness Score": 79.76,
2020
"Latency Score": null,
21-
"Accuracy": 78.43,
22-
"Cost per 1k": 0.92
21+
"Accuracy": 79.32,
22+
"Cost per 1k": 0.61
2323
},
2424
{
2525
"Router Name": "R2-Router",

‎src/data/routers.json‎

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,21 +15,16 @@
1515
"Weave Router": {
1616
"name": "Weave Router",
1717
"type": "open-source",
18-
"description": "Cluster-routing over a 12-model BYOK pool spanning Anthropic, OpenAI, Google, and OpenRouter providers. Embeds each prompt, scores against per-cluster model rankings, and selects the cost-quality optimum via an alpha-blended score.",
18+
"description": "Cluster-routing (v0.62) over a 7-model BYOK pool spanning Anthropic, OpenAI, Google, and OpenRouter providers. Embeds each prompt with Jina v2 INT8 ONNX (768-dim), runs top-p=4 cluster sum against per-cluster rankings trained on the RouterArena full split with k=160 clusters, then selects the cost-quality optimum via an alpha-blended score (alpha=0.25).",
1919
"affiliation": "Weave",
2020
"modelPool": [
2121
"claude-opus-4-7",
22-
"claude-sonnet-4-5",
23-
"claude-haiku-4-5",
2422
"gpt-5.5",
25-
"gpt-5.4-mini",
26-
"gpt-4.1",
2723
"gemini-3.1-pro-preview",
2824
"gemini-3.1-flash-lite-preview",
2925
"deepseek/deepseek-v4-pro",
3026
"deepseek/deepseek-v4-flash",
31-
"qwen/qwen3.5-flash-02-23",
32-
"moonshotai/kimi-k2.5"
27+
"qwen/qwen3-235b-a22b-2507"
3328
],
3429
"websiteUrl": "https://weaverouter.com",
3530
"githubUrl": "https://github.com/workweave/router"

‎src/data/routers.yaml‎

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,16 @@
11
Weave Router:
22
name: Weave Router
33
type: open-source
4-
description: Cluster-routing over a 12-model BYOK pool spanning Anthropic, OpenAI, Google, and OpenRouter providers. Embeds each prompt, scores against per-cluster model rankings, and selects the cost-quality optimum via an alpha-blended score.
4+
description: Cluster-routing (v0.62) over a 7-model BYOK pool spanning Anthropic, OpenAI, Google, and OpenRouter providers. Embeds each prompt with Jina v2 INT8 ONNX (768-dim), runs top-p=4 cluster sum against per-cluster rankings trained on the RouterArena full split with k=160 clusters, then selects the cost-quality optimum via an alpha-blended score (alpha=0.25).
55
affiliation: Weave
66
modelPool:
77
- claude-opus-4-7
8-
- claude-sonnet-4-5
9-
- claude-haiku-4-5
108
- gpt-5.5
11-
- gpt-5.4-mini
12-
- gpt-4.1
139
- gemini-3.1-pro-preview
1410
- gemini-3.1-flash-lite-preview
1511
- deepseek/deepseek-v4-pro
1612
- deepseek/deepseek-v4-flash
17-
- qwen/qwen3.5-flash-02-23
18-
- moonshotai/kimi-k2.5
13+
- qwen/qwen3-235b-a22b-2507
1914
websiteUrl: https://weaverouter.com
2015
githubUrl: https://github.com/workweave/router
2116

0 commit comments

Comments
 (0)