Repository navigation
Commit 402aba1
Repin DSv4.1-Flash MI355X vLLM AgentX to nightly-rocm100-ac9126e5 (includes #58671)
Enables vllm-project/vllm#58671's ROCm paged MXFP4 sparse-logits indexer via
--attention-config and --block-size 128, replacing the dense fp8 indexer
path. A live A/B test (TP2 c16, matched 900s window) measured #58671 giving
+14.7/+14.9% p50/p90 interactivity and -9.5/-10.8% p50/p90 e2e latency over
the dense path, with throughput/GPU unchanged. Also drops c128 from both
TP2 and TP4 conc-lists for this recipe.
Co-authored-by: Cursor <cursoragent@cursor.com>1 parent 1c5e6b9 commit 402aba1
3 files changed
Lines changed: 32 additions & 41 deletions
File tree
- inferencex-e2e
- benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/mi355x-fp4-mtp
- configs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
10 | 13 | | |
11 | 14 | | |
12 | 15 | | |
| |||
40 | 43 | | |
41 | 44 | | |
42 | 45 | | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
43 | 52 | | |
44 | 53 | | |
45 | 54 | | |
| |||
54 | 63 | | |
55 | 64 | | |
56 | 65 | | |
| 66 | + | |
57 | 67 | | |
58 | 68 | | |
59 | 69 | | |
| |||
70 | 80 | | |
71 | 81 | | |
72 | 82 | | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
78 | 88 | | |
79 | 89 | | |
80 | 90 | | |
| |||
166 | 176 | | |
167 | 177 | | |
168 | 178 | | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
174 | | - | |
175 | | - | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | 179 | | |
183 | 180 | | |
184 | 181 | | |
| |||
269 | 266 | | |
270 | 267 | | |
271 | 268 | | |
272 | | - | |
273 | | - | |
274 | | - | |
275 | | - | |
276 | | - | |
277 | | - | |
278 | | - | |
279 | | - | |
280 | | - | |
281 | | - | |
282 | | - | |
283 | | - | |
284 | | - | |
285 | | - | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1457 | 1457 | | |
1458 | 1458 | | |
1459 | 1459 | | |
1460 | | - | |
1461 | | - | |
| 1460 | + | |
| 1461 | + | |
| 1462 | + | |
1462 | 1463 | | |
1463 | 1464 | | |
1464 | 1465 | | |
| |||
1470 | 1471 | | |
1471 | 1472 | | |
1472 | 1473 | | |
1473 | | - | |
1474 | | - | |
1475 | | - | |
1476 | | - | |
1477 | | - | |
1478 | | - | |
| 1474 | + | |
| 1475 | + | |
| 1476 | + | |
| 1477 | + | |
| 1478 | + | |
| 1479 | + | |
1479 | 1480 | | |
1480 | 1481 | | |
1481 | 1482 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9168 | 9168 | | |
9169 | 9169 | | |
9170 | 9170 | | |
| 9171 | + | |
| 9172 | + | |
| 9173 | + | |
| 9174 | + | |
| 9175 | + | |
| 9176 | + | |
| 9177 | + | |
0 commit comments