Skip to content

Commit cd531bd

Browse files
github-actions[bot]claude
github-actions[bot]
and
claude
committed
Merge main into klaud/auto-5a3ab4e2462f0912-1e6fa8d804bf7b88
合并 main，并将 perf-changelog 条目保留在文件末尾。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 parents b2498df + c08eb51 commit cd531bd

14 files changed

Lines changed: 430 additions & 5 deletions

File tree

‎inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/musespark/sglang/b200-fp4-mtp/agentic.yaml‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
base:
2+
schema: 2
3+
name: musespark-fp4-b200-sglang-agentic
4+
resources:
5+
gpu_type: b200
6+
gpus_per_node: 8
7+
frontend:
8+
type: sglang
9+
enable_multiple_frontends: false
10+
observability:
11+
enabled: false
12+
tachometer:
13+
enabled: false
14+
engine: sglang
15+
roles:
16+
agg:
17+
nodes: 1
18+
workers: 1
19+
gpus: 4
20+
args:
21+
served-model-name: meta-models/Muse-Spark-Open-816B-A42B-NVFP4
22+
trust-remote-code: true
23+
tensor-parallel-size: 4
24+
expert-parallel-size: 4
25+
quantization: modelopt_fp4
26+
moe-runner-backend: flashinfer_trtllm
27+
kv-cache-dtype: fp8_e4m3
28+
enable-layernorm-sp: true
29+
enable-megamoe-for-layernorm-sp: true
30+
# 133 GB of weights per GPU leave the KV pool what prefill graphs would take.
31+
mem-fraction-static: 0.88
32+
disable-prefill-cuda-graph: true
33+
swa-full-tokens-ratio: 0.1
34+
# DFlash2 verifies 8 tokens per step: the current token plus 7 drafts.
35+
speculative-algorithm: DFLASH
36+
speculative-draft-model-path: RadixArk/Muse-Spark-Open-DFlash2
37+
speculative-draft-attention-backend: fa4
38+
speculative-num-draft-tokens: 8
39+
mm-feature-transport: cpu
40+
reasoning-parser: muse
41+
tool-call-parser: muse
42+
watchdog-timeout: 3600
43+
enable-metrics: true
44+
env:
45+
PYTHONNOUSERSITE: '1'
46+
PYTHONUNBUFFERED: '1'
47+
# Outlast AIPerf's pooled connections past Uvicorn's keep-alive.
48+
SGLANG_TIMEOUT_KEEP_ALIVE: '900'
49+
health_check:
50+
max_attempts: 1440
51+
interval_seconds: 5
52+
benchmark:
53+
env:
54+
# The gated Hub repo is unreadable to the client; use the staged checkpoint.
55+
TOKENIZER: /model
56+
# Warmup can leave bytes unacknowledged past AIPerf's 30 s default.
57+
AIPERF_HTTP_TCP_USER_TIMEOUT: '900000'
58+
AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'sglang:'
59+
AGENTIC_WARMUP_GRACE_PERIOD: '3600'
60+
61+
62+
override_tp4_c1:
63+
roles:
64+
agg:
65+
args:
66+
max-running-requests: 2
67+
cuda-graph-max-bs-decode: 2
68+
benchmark:
69+
env:
70+
CONC: '1'
71+
72+
override_tp4_c2:
73+
roles:
74+
agg:
75+
args:
76+
max-running-requests: 4
77+
cuda-graph-max-bs-decode: 4
78+
benchmark:
79+
env:
80+
CONC: '2'
81+
82+
override_tp4_c4:
83+
roles:
84+
agg:
85+
args:
86+
max-running-requests: 8
87+
cuda-graph-max-bs-decode: 8
88+
benchmark:
89+
env:
90+
CONC: '4'
91+
92+
override_tp4_c8:
93+
roles:
94+
agg:
95+
args:
96+
max-running-requests: 16
97+
cuda-graph-max-bs-decode: 16
98+
benchmark:
99+
env:
100+
CONC: '8'
101+
102+
override_tp4_c12:
103+
roles:
104+
agg:
105+
args:
106+
max-running-requests: 24
107+
cuda-graph-max-bs-decode: 24
108+
benchmark:
109+
env:
110+
CONC: '12'
111+
112+
override_tp4_c16:
113+
roles:
114+
agg:
115+
args:
116+
max-running-requests: 32
117+
cuda-graph-max-bs-decode: 32
118+
benchmark:
119+
env:
120+
CONC: '16'
121+
122+
override_tp4_c24:
123+
roles:
124+
agg:
125+
args:
126+
max-running-requests: 48
127+
cuda-graph-max-bs-decode: 48
128+
benchmark:
129+
env:
130+
CONC: '24'
131+
132+
override_tp4_c32:
133+
roles:
134+
agg:
135+
args:
136+
max-running-requests: 64
137+
cuda-graph-max-bs-decode: 64
138+
benchmark:
139+
env:
140+
CONC: '32'

‎inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/musespark/sglang/gb200-fp4-mtp/agentic.yaml‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
base:
2+
schema: 2
3+
name: musespark-fp4-gb200-sglang-agentic
4+
resources:
5+
gpu_type: gb200
6+
gpus_per_node: 4
7+
frontend:
8+
type: sglang
9+
enable_multiple_frontends: false
10+
observability:
11+
enabled: false
12+
tachometer:
13+
enabled: false
14+
engine: sglang
15+
roles:
16+
agg:
17+
nodes: 1
18+
workers: 1
19+
gpus: 4
20+
args:
21+
served-model-name: meta-models/Muse-Spark-Open-816B-A42B-NVFP4
22+
trust-remote-code: true
23+
tensor-parallel-size: 4
24+
expert-parallel-size: 4
25+
quantization: modelopt_fp4
26+
moe-runner-backend: flashinfer_trtllm
27+
kv-cache-dtype: fp8_e4m3
28+
enable-layernorm-sp: true
29+
enable-megamoe-for-layernorm-sp: true
30+
# 133 GB of weights per GPU leave the KV pool what prefill graphs would take.
31+
mem-fraction-static: 0.88
32+
disable-prefill-cuda-graph: true
33+
swa-full-tokens-ratio: 0.1
34+
# DFlash2 verifies 8 tokens per step: the current token plus 7 drafts.
35+
speculative-algorithm: DFLASH
36+
speculative-draft-model-path: RadixArk/Muse-Spark-Open-DFlash2
37+
speculative-draft-attention-backend: fa4
38+
speculative-num-draft-tokens: 8
39+
mm-feature-transport: cpu
40+
enable-hierarchical-cache: true
41+
hicache-size: 120
42+
reasoning-parser: muse
43+
tool-call-parser: muse
44+
watchdog-timeout: 3600
45+
enable-metrics: true
46+
env:
47+
PYTHONNOUSERSITE: '1'
48+
PYTHONUNBUFFERED: '1'
49+
# Outlast AIPerf's pooled connections past Uvicorn's keep-alive.
50+
SGLANG_TIMEOUT_KEEP_ALIVE: '900'
51+
# 533 GB of weights load from Lustre before graph capture.
52+
health_check:
53+
max_attempts: 1440
54+
interval_seconds: 5
55+
benchmark:
56+
env:
57+
# The gated Hub repo is unreadable to the client; use the staged checkpoint.
58+
TOKENIZER: /model
59+
# Warmup can leave bytes unacknowledged past AIPerf's 30 s default.
60+
AIPERF_HTTP_TCP_USER_TIMEOUT: '900000'
61+
AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'sglang:'
62+
AGENTIC_WARMUP_GRACE_PERIOD: '3600'
63+
64+
65+
override_tp4_c1:
66+
roles:
67+
agg:
68+
args:
69+
max-running-requests: 2
70+
cuda-graph-max-bs-decode: 2
71+
benchmark:
72+
env:
73+
CONC: '1'
74+
75+
override_tp4_c2:
76+
roles:
77+
agg:
78+
args:
79+
max-running-requests: 4
80+
cuda-graph-max-bs-decode: 4
81+
benchmark:
82+
env:
83+
CONC: '2'
84+
85+
override_tp4_c4:
86+
roles:
87+
agg:
88+
args:
89+
max-running-requests: 8
90+
cuda-graph-max-bs-decode: 8
91+
benchmark:
92+
env:
93+
CONC: '4'
94+
95+
override_tp4_c6:
96+
roles:
97+
agg:
98+
args:
99+
max-running-requests: 12
100+
cuda-graph-max-bs-decode: 12
101+
benchmark:
102+
env:
103+
CONC: '6'
104+
105+
override_tp4_c8:
106+
roles:
107+
agg:
108+
args:
109+
max-running-requests: 16
110+
cuda-graph-max-bs-decode: 16
111+
benchmark:
112+
env:
113+
CONC: '8'
114+
115+
override_tp4_c10:
116+
roles:
117+
agg:
118+
args:
119+
max-running-requests: 20
120+
cuda-graph-max-bs-decode: 20
121+
benchmark:
122+
env:
123+
CONC: '10'
124+
125+
override_tp4_c12:
126+
roles:
127+
agg:
128+
args:
129+
max-running-requests: 24
130+
cuda-graph-max-bs-decode: 24
131+
benchmark:
132+
env:
133+
CONC: '12'
134+
135+
override_tp4_c16:
136+
roles:
137+
agg:
138+
args:
139+
max-running-requests: 32
140+
cuda-graph-max-bs-decode: 32
141+
benchmark:
142+
env:
143+
CONC: '16'
144+
145+
override_tp4_c24:
146+
roles:
147+
agg:
148+
args:
149+
max-running-requests: 48
150+
cuda-graph-max-bs-decode: 48
151+
benchmark:
152+
env:
153+
CONC: '24'
154+
155+
override_tp4_c32:
156+
roles:
157+
agg:
158+
args:
159+
max-running-requests: 64
160+
cuda-graph-max-bs-decode: 64
161+
benchmark:
162+
env:
163+
CONC: '32'

‎inferencex-e2e/configs/nvidia-master.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8365,6 +8365,35 @@ dsv41flash-fp4-gb200-sglang-agentic-dspark:
83658365
- { tp: 4, ep: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16], srt-recipe: agentic.yaml }
83668366
- { tp: 4, ep: 4, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [32, 64, 128, 192], router: { name: sglang-router, version: "0.3.2" }, srt-recipe: agentic.yaml }
83678367

8368+
musespark-fp4-gb200-sglang-agentic-dflash:
8369+
image: lmsysorg/sglang:dev-muse-spark-open@sha256:8cfc5a72cb296771f5f8a4a4688cb6d05685610a830cbb616fa4ab0eb7bb2b0a
8370+
model: meta-models/Muse-Spark-Open-816B-A42B-NVFP4
8371+
model-prefix: musespark
8372+
runner: cluster:gb200-nv
8373+
precision: fp4
8374+
framework: sglang
8375+
multinode: false
8376+
srt-recipe-dir: musespark/sglang/gb200-fp4-mtp
8377+
scenarios:
8378+
agentic-coding:
8379+
- dram-utilization: 0.80
8380+
search-space:
8381+
- { tp: 4, ep: 4, kv-offloading: dram, kv-offload-backend: { name: hicache }, spec-decoding: draft_model, conc-list: [1, 2, 4, 6, 8, 10, 12, 16, 24, 32], srt-recipe: agentic.yaml }
8382+
8383+
musespark-fp4-b200-sglang-agentic-dflash:
8384+
image: lmsysorg/sglang:dev-muse-spark-open@sha256:8cfc5a72cb296771f5f8a4a4688cb6d05685610a830cbb616fa4ab0eb7bb2b0a
8385+
model: meta-models/Muse-Spark-Open-816B-A42B-NVFP4
8386+
model-prefix: musespark
8387+
runner: cluster:b200-nscale
8388+
precision: fp4
8389+
framework: sglang
8390+
multinode: false
8391+
srt-recipe-dir: musespark/sglang/b200-fp4-mtp
8392+
scenarios:
8393+
agentic-coding:
8394+
- search-space:
8395+
- { tp: 4, ep: 4, kv-offloading: none, spec-decoding: draft_model, conc-list: [1, 2, 4, 8, 12, 16, 24, 32], srt-recipe: agentic.yaml }
8396+
83688397
dsv41flash-fp4-b300-vllm-agentic-dspark:
83698398
image: vllm/vllm-openai:nightly-dev-x86_64-cu130-ac9126e58aa7
83708399
model: deepseek-ai/DeepSeek-V4.1-Flash

‎inferencex-e2e/configs/runners.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -460,6 +460,7 @@ clusters:
460460
Kimi-K3: {root: scratch, dir: Kimi-K3}
461461
MiniMax-M3-MXFP8: {root: scratch, dir: MiniMax-M3-MXFP8}
462462
MiniMax-M3-NVFP4: {root: scratch, dir: MiniMax-M3-NVFP4}
463+
Muse-Spark-Open-816B-A42B-NVFP4: {root: data-models, dir: Muse-Spark-Open-816B-A42B-NVFP4}
463464
Qwen3.5-397B-A17B-FP8: {root: scratch, dir: Qwen3.5-397B-A17B-FP8}
464465
Qwen3.5-397B-A17B-NVFP4: {root: scratch, dir: Qwen3.5-397B-A17B-NVFP4}
465466
Qwen3.5-397B-A17B-NVFP4-V2: {root: scratch, dir: Qwen3.5-397B-A17B-NVFP4-V2}
@@ -474,6 +475,7 @@ clusters:
474475
hf-hub-cache: {path: /data/home/sa-shared/gharunners/hf-hub-cache}
475476
aiperf-cache: {path: /data/home/sa-shared/gharunners/aiperf-cache}
476477
scratch: {path: /scratch/models, visibility: node-local}
478+
data-models: {path: /data/models, visibility: shared}
477479
squash:
478480
dir: /data/home/sa-shared/containers
479481
visibility: shared
@@ -577,6 +579,7 @@ clusters:
577579
MiniMax-M3-NVFP4: {root: lustre-models, dir: MiniMax-M3-NVFP4}
578580
Qwen3.5-397B-A17B-FP8: {root: lustre-models, dir: Qwen3.5-397B-A17B-FP8}
579581
Qwen3.5-397B-A17B-NVFP4-V2: {root: lustre-models, dir: Qwen3.5-397B-A17B-NVFP4-V2}
582+
Muse-Spark-Open-816B-A42B-NVFP4: {root: lustre-models, dir: Muse-Spark-Open-816B-A42B-NVFP4}
580583
GLM-5.2-NVFP4: {root: sa-shared-models, dir: GLM-5.2-NVFP4}
581584
DeepSeek-R1-0528: {root: numa1, dir: DeepSeek-R1-0528}
582585
DeepSeek-R1-0528-NVFP4-v2: {root: numa1, dir: DeepSeek-R1-0528-NVFP4-v2}
@@ -613,6 +616,7 @@ clusters:
613616
rdma-gid-index: 3
614617
job-tag: gb200
615618
default-time-limit: "6:00:00"
619+
single-node-models: staged
616620
preflight: false
617621
segment-directive: false
618622
container-aliases: [dynamo-trtllm, dynamo-sglang]

‎inferencex-e2e/docs/MODELS.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ Other offloading tiers, including NVMe KV cache offloading, are outside the init
110110

111111
| Model architecture class | Prefix | Date added | Active scenarios | Deprecated scenarios |
112112
|---|---|---|---|---|
113+
| Muse Spark Open 816B-A42B | `musespark` | 2026-10-15 | Agentic coding (SGLang with DFlash2 draft `RadixArk/Muse-Spark-Open-DFlash2`; golden AL curve `musespark_dflash.yaml`) | |
113114
| DeepSeek-V4.1-Flash | `dsv41flash` | 2026-09-10 | Agentic coding (vLLM: DSpark, Engram UVA offload; SGLang: DSpark arms added per SKU from 2026-09-17; ATOM: MI355X TP2/TP4 DSpark added 2026-09-23, removed in #3463, restored on srt-slurm 2026-09-30; GPU validation pending) | |
114115
| GLM-5.3 | `glm5.3` | 2026-09-22 ([#3366](https://github.com/SemiAnalysisAI/InferenceX/pull/3366)) | Agentic coding (MTP only, per the deprecation policy) | Single-turn 1k1k (deprecated for all models before this model was added; never run) |
115116
| Qwen3.8-Flash-Next | `qwen3.8next` | 2026-08-26 ([#2742](https://github.com/SemiAnalysisAI/InferenceX/pull/2742)) | Agentic coding | |

‎inferencex-e2e/infx/golden_al_distribution/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,7 @@ Before accepting an updated curve, reviewers should verify:
145145
| GLM-5.2 | MTP | [`glm5.2_mtp.yaml`](glm5.2_mtp.yaml) | [28058352479](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28058352479) |
146146
| GLM-5.3 | MTP (provisional, K=3 copied from GLM-5.2) | [`glm5.3_mtp.yaml`](glm5.3_mtp.yaml) | [28058352479](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28058352479) (GLM-5.2) |
147147
| Qwen3.8-Flash-Next | MTP (native) | [`qwen3.8next_mtp.yaml`](qwen3.8next_mtp.yaml) | [33034290269](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33034290269) |
148+
| Muse Spark Open | DFlash2 | [`musespark_dflash.yaml`](musespark_dflash.yaml) | [38808046215](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/38808046215) |
148149

149150
## Using golden curves in srt-slurm runs
150151

‎inferencex-e2e/infx/golden_al_distribution/README_zh.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,7 @@ gh workflow run speedbench-al.yml \
138138
| GLM-5.2 | MTP | [`glm5.2_mtp.yaml`](glm5.2_mtp.yaml) | [28058352479](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28058352479) |
139139
| GLM-5.3 | MTP（临时，K=3 取自 GLM-5.2） | [`glm5.3_mtp.yaml`](glm5.3_mtp.yaml) | [28058352479](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/28058352479)（GLM-5.2） |
140140
| Qwen3.8-Flash-Next | MTP (native) | [`qwen3.8next_mtp.yaml`](qwen3.8next_mtp.yaml) | [33034290269](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33034290269) |
141+
| Muse Spark Open | DFlash2 | [`musespark_dflash.yaml`](musespark_dflash.yaml) | [38808046215](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/38808046215) |
141142

142143
## 在 srt-slurm 运行中使用黄金曲线
143144

0 commit comments

Comments
 (0)