Skip to content

Commit 9cee89c

Browse files
nvpohanhclaude
nvpohanh
and
claude
committed
feat(dsxe): use NIXL for B300 Dynamo+SGLang disaggregated KV transfer
Switch the B300 DeepSeek-V4-Pro-0813 disaggregated recipes (1P1D DEP8 c64/c240 and 1P2D DEP8 c64) from Mooncake to NIXL with the LIBFABRIC backend for the DSXE EFA fabric. The NIXL build already in the pinned lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 image provides the libfabric plugin, so the container runs as shipped: drop the install-mooncake-efa-cu13.sh setup script, which replaced the image's Mooncake package with the EFA wheel, along with its waiver and the Mooncake-only MOONCAKE_PROTOCOL, MC_MS_FILTERS and WITH_NVIDIA_PEERMEM settings. Prefill and decode both set SGLANG_DISAGGREGATION_NIXL_BACKEND=LIBFABRIC, and the master config reports kv-p2p-transfer: nixl. 将 B300 DeepSeek-V4-Pro-0813 分离式配方（1P1D DEP8 c64/c240 与 1P2D DEP8 c64）的 KV 传输由 Mooncake 改为使用 LIBFABRIC 后端的 NIXL，以适配 DSXE 的 EFA 网络。固定镜像 lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 中自带的 NIXL 已包含 libfabric 插件，容器可按原样运行：移除用 EFA wheel 替换镜像内 Mooncake 包的 install-mooncake-efa-cu13.sh 初始化脚本及其豁免文件，并移除仅 供 Mooncake 使用的 MOONCAKE_PROTOCOL、MC_MS_FILTERS 与 WITH_NVIDIA_PEERMEM 设置。Prefill 与 decode 均设置 SGLANG_DISAGGREGATION_NIXL_BACKEND=LIBFABRIC， 主配置中的 kv-p2p-transfer 改为 nixl。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 84278f4 commit 9cee89c

6 files changed

Lines changed: 13 additions & 86 deletions

File tree

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/configs/install-mooncake-efa-cu13.sh‎

Lines changed: 0 additions & 5 deletions
This file was deleted.

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p1d-dep8-dep8-c240-mtp-kvoffload.yaml‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,6 @@ identity:
1919
dynamo:
2020
install: false
2121

22-
setup_script: install-mooncake-efa-cu13.sh
23-
2422
health_check:
2523
max_attempts: 1440
2624
interval_seconds: 10
@@ -65,6 +63,7 @@ roles:
6563
SGLANG_ENABLE_PREFILL_WAR_READ_DONE: "1"
6664
SGLANG_RAGGED_VERIFY_MODE: "static"
6765
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "900"
66+
SGLANG_DISAGGREGATION_NIXL_BACKEND: LIBFABRIC
6867
SGLANG_DSV4_MHC_PREWARM: '1'
6968
PIP_BREAK_SYSTEM_PACKAGES: '1'
7069
PYTHONUNBUFFERED: '1'
@@ -81,9 +80,6 @@ roles:
8180
SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '9216'
8281
SGLANG_OPT_USE_ONLINE_COMPRESS: '0'
8382
NCCL_CUMEM_ENABLE: '1'
84-
MOONCAKE_PROTOCOL: efa
85-
MC_MS_FILTERS: rdmap191s0
86-
WITH_NVIDIA_PEERMEM: '0'
8783
NCCL_TIMEOUT: '100000'
8884
SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW: '1'
8985
DYN_SKIP_SGLANG_LOG_FORMATTING: '1'
@@ -110,7 +106,7 @@ roles:
110106
moe-dense-tp-size: 1
111107
moe-a2a-backend: megamoe
112108
enable-deepseek-v4-fp4-indexer: true
113-
disaggregation-transfer-backend: mooncake
109+
disaggregation-transfer-backend: nixl
114110
disaggregation-mode: prefill
115111
load-balance-method: total_tokens
116112
mem-fraction-static: 0.85
@@ -140,6 +136,7 @@ roles:
140136
env:
141137
SGLANG_RAGGED_VERIFY_MODE: "static"
142138
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "900"
139+
SGLANG_DISAGGREGATION_NIXL_BACKEND: LIBFABRIC
143140
PIP_BREAK_SYSTEM_PACKAGES: '1'
144141
PYTHONUNBUFFERED: '1'
145142
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
@@ -154,9 +151,6 @@ roles:
154151
SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '4096'
155152
SGLANG_OPT_USE_ONLINE_COMPRESS: '0'
156153
NCCL_CUMEM_ENABLE: '1'
157-
MOONCAKE_PROTOCOL: efa
158-
MC_MS_FILTERS: rdmap191s0
159-
WITH_NVIDIA_PEERMEM: '0'
160154
NCCL_TIMEOUT: '100000'
161155
SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW: '1'
162156
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
@@ -183,7 +177,7 @@ roles:
183177
moe-dense-tp-size: 1
184178
moe-a2a-backend: megamoe
185179
enable-deepseek-v4-fp4-indexer: true
186-
disaggregation-transfer-backend: mooncake
180+
disaggregation-transfer-backend: nixl
187181
disaggregation-mode: decode
188182
load-balance-method: total_tokens
189183
mem-fraction-static: 0.9

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,6 @@ identity:
1919
dynamo:
2020
install: false
2121

22-
setup_script: install-mooncake-efa-cu13.sh
23-
2422
health_check:
2523
max_attempts: 1440
2624
interval_seconds: 10
@@ -65,6 +63,7 @@ roles:
6563
SGLANG_ENABLE_PREFILL_WAR_READ_DONE: "1"
6664
SGLANG_RAGGED_VERIFY_MODE: "static"
6765
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "900"
66+
SGLANG_DISAGGREGATION_NIXL_BACKEND: LIBFABRIC
6867
SGLANG_DSV4_MHC_PREWARM: '1'
6968
PIP_BREAK_SYSTEM_PACKAGES: '1'
7069
PYTHONUNBUFFERED: '1'
@@ -81,9 +80,6 @@ roles:
8180
SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '9216'
8281
SGLANG_OPT_USE_ONLINE_COMPRESS: '0'
8382
NCCL_CUMEM_ENABLE: '1'
84-
MOONCAKE_PROTOCOL: efa
85-
MC_MS_FILTERS: rdmap191s0
86-
WITH_NVIDIA_PEERMEM: '0'
8783
NCCL_TIMEOUT: '100000'
8884
SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW: '1'
8985
DYN_SKIP_SGLANG_LOG_FORMATTING: '1'
@@ -110,7 +106,7 @@ roles:
110106
moe-dense-tp-size: 1
111107
moe-a2a-backend: megamoe
112108
enable-deepseek-v4-fp4-indexer: true
113-
disaggregation-transfer-backend: mooncake
109+
disaggregation-transfer-backend: nixl
114110
disaggregation-mode: prefill
115111
load-balance-method: total_tokens
116112
mem-fraction-static: 0.85
@@ -140,6 +136,7 @@ roles:
140136
env:
141137
SGLANG_RAGGED_VERIFY_MODE: "static"
142138
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "900"
139+
SGLANG_DISAGGREGATION_NIXL_BACKEND: LIBFABRIC
143140
PIP_BREAK_SYSTEM_PACKAGES: '1'
144141
PYTHONUNBUFFERED: '1'
145142
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
@@ -154,9 +151,6 @@ roles:
154151
SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '4096'
155152
SGLANG_OPT_USE_ONLINE_COMPRESS: '0'
156153
NCCL_CUMEM_ENABLE: '1'
157-
MOONCAKE_PROTOCOL: efa
158-
MC_MS_FILTERS: rdmap191s0
159-
WITH_NVIDIA_PEERMEM: '0'
160154
NCCL_TIMEOUT: '100000'
161155
SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW: '1'
162156
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
@@ -183,7 +177,7 @@ roles:
183177
moe-dense-tp-size: 1
184178
moe-a2a-backend: megamoe
185179
enable-deepseek-v4-fp4-indexer: true
186-
disaggregation-transfer-backend: mooncake
180+
disaggregation-transfer-backend: nixl
187181
disaggregation-mode: decode
188182
load-balance-method: total_tokens
189183
mem-fraction-static: 0.9

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p2d-dep8-dep8-c64-mtp-kvoffload.yaml‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,6 @@ identity:
2121
dynamo:
2222
install: false
2323

24-
setup_script: install-mooncake-efa-cu13.sh
25-
2624
health_check:
2725
max_attempts: 1440
2826
interval_seconds: 10
@@ -67,6 +65,7 @@ roles:
6765
SGLANG_ENABLE_PREFILL_WAR_READ_DONE: "1"
6866
SGLANG_RAGGED_VERIFY_MODE: "static"
6967
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "900"
68+
SGLANG_DISAGGREGATION_NIXL_BACKEND: LIBFABRIC
7069
SGLANG_DSV4_MHC_PREWARM: '1'
7170
PIP_BREAK_SYSTEM_PACKAGES: '1'
7271
PYTHONUNBUFFERED: '1'
@@ -83,9 +82,6 @@ roles:
8382
SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '9216'
8483
SGLANG_OPT_USE_ONLINE_COMPRESS: '0'
8584
NCCL_CUMEM_ENABLE: '1'
86-
MOONCAKE_PROTOCOL: efa
87-
MC_MS_FILTERS: rdmap191s0
88-
WITH_NVIDIA_PEERMEM: '0'
8985
NCCL_TIMEOUT: '100000'
9086
SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW: '1'
9187
DYN_SKIP_SGLANG_LOG_FORMATTING: '1'
@@ -112,7 +108,7 @@ roles:
112108
moe-dense-tp-size: 1
113109
moe-a2a-backend: megamoe
114110
enable-deepseek-v4-fp4-indexer: true
115-
disaggregation-transfer-backend: mooncake
111+
disaggregation-transfer-backend: nixl
116112
disaggregation-mode: prefill
117113
load-balance-method: total_tokens
118114
mem-fraction-static: 0.85
@@ -142,6 +138,7 @@ roles:
142138
env:
143139
SGLANG_RAGGED_VERIFY_MODE: "static"
144140
SGLANG_DISAGGREGATION_WAITING_TIMEOUT: "900"
141+
SGLANG_DISAGGREGATION_NIXL_BACKEND: LIBFABRIC
145142
PIP_BREAK_SYSTEM_PACKAGES: '1'
146143
PYTHONUNBUFFERED: '1'
147144
SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0'
@@ -156,9 +153,6 @@ roles:
156153
SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '4096'
157154
SGLANG_OPT_USE_ONLINE_COMPRESS: '0'
158155
NCCL_CUMEM_ENABLE: '1'
159-
MOONCAKE_PROTOCOL: efa
160-
MC_MS_FILTERS: rdmap191s0
161-
WITH_NVIDIA_PEERMEM: '0'
162156
NCCL_TIMEOUT: '100000'
163157
SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW: '1'
164158
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
@@ -185,7 +179,7 @@ roles:
185179
moe-dense-tp-size: 1
186180
moe-a2a-backend: megamoe
187181
enable-deepseek-v4-fp4-indexer: true
188-
disaggregation-transfer-backend: mooncake
182+
disaggregation-transfer-backend: nixl
189183
disaggregation-mode: decode
190184
load-balance-method: total_tokens
191185
mem-fraction-static: 0.9

‎inferencex-e2e/configs/nvidia-master.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1229,7 +1229,7 @@ dsv4-fp4-b300-dynamo-sglang-agentic-disagg:
12291229
precision: fp4
12301230
framework: dynamo-sglang
12311231
router: { name: dynamo-router, version: "1.5.0.dev20260914" }
1232-
kv-p2p-transfer: mooncake
1232+
kv-p2p-transfer: nixl
12331233
multinode: true
12341234
disagg: true
12351235
scenarios:

‎inferencex-e2e/docs/waiver/3631.md‎

Lines changed: 0 additions & 50 deletions
This file was deleted.

0 commit comments

Comments
 (0)