Skip to content

Commit f9d0605

Browse files
nvpohanhclaude
andcommitted
docs(waiver): add the PR #3631 Mooncake EFA package waiver
The B300 DSXE disaggregated recipes replace the image's mooncake-transfer-engine-cuda13 0.3.13 with the public EFA build mooncake-transfer-engine-efa-cuda13 0.3.13.post1 at container setup, as SGLang's AWS EFA documentation prescribes, because the nodes connect over AWS EFA and the standard build lacks the EFA transport. Document what is changed, why the unmodified image cannot run the disaggregated points, the upstream links, and the removal plan. B300 DSXE 分离式配方在容器初始化时,按照 SGLang 的 AWS EFA 文档,将镜像中的 mooncake-transfer-engine-cuda13 0.3.13 替换为公开的 EFA 版本 mooncake-transfer-engine-efa-cuda13 0.3.13.post1:节点之间通过 AWS EFA 互联, 而标准版本不包含 EFA 传输。本豁免说明所做的改动、未经修改的镜像无法运行分离式 测试点的原因、上游链接以及移除计划。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent f7dfe80 commit f9d0605

1 file changed

Lines changed: 50 additions & 0 deletions

File tree

‎inferencex-e2e/docs/waiver/3631.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Serving-stack package waiver — PR #3631
2+
3+
Filed per [`docs/PR_REVIEW_CHECKLIST.md`](../PR_REVIEW_CHECKLIST.md) (no-patch item): the disaggregated
4+
recipes in this PR replace one package in the pinned serving image at container setup. The checklist
5+
requires the pinned image to run as shipped unless such a change is covered by a filled-out waiver.
6+
7+
## Config covered
8+
9+
- **Master config entry:** `dsv4-fp4-b300-dynamo-sglang-agentic-disagg` in
10+
[`configs/nvidia-master.yaml`](../../configs/nvidia-master.yaml) (1P1D DEP8 c64 and c240, 1P2D DEP8 c64).
11+
The aggregate entry `dsv4-fp4-b300-dynamo-sglang-agentic-agg` does not run the script.
12+
- **Pinned image:** `lmsysorg/sglang:nightly-dev-20260916-c9a8fba9`
13+
- **Setup entrypoint:** `setup_script: install-mooncake-efa-cu13.sh` in
14+
[`disagg-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml`](../../benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml),
15+
[`disagg-1p1d-dep8-dep8-c240-mtp-kvoffload.yaml`](../../benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p1d-dep8-dep8-c240-mtp-kvoffload.yaml), and
16+
[`disagg-1p2d-dep8-dep8-c64-mtp-kvoffload.yaml`](../../benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p2d-dep8-dep8-c64-mtp-kvoffload.yaml)
17+
- **Script:** [`benchmarks/multi_node/srt-slurm-recipes/configs/install-mooncake-efa-cu13.sh`](../../benchmarks/multi_node/srt-slurm-recipes/configs/install-mooncake-efa-cu13.sh)
18+
19+
## What is changed
20+
21+
The script uninstalls the image's `mooncake-transfer-engine-cuda13` 0.3.13 and installs the public EFA
22+
build of the same Mooncake release line, `mooncake-transfer-engine-efa-cuda13==0.3.13.post1`, from PyPI
23+
with `--no-deps`. Both packages provide the same `mooncake` Python module; the EFA build adds the AWS EFA
24+
(libfabric) transport. The commands are the ones in SGLang's documentation.
25+
26+
Nothing else changes. No SGLang, Dynamo, or Mooncake source is patched, no forked or rebuilt wheel is
27+
installed, and no other package is added or removed. Mooncake only moves KV-cache pages from prefill to
28+
decode workers; it performs no model computation, so the change cannot alter model numerics.
29+
30+
## Why the unmodified upstream image cannot run this benchmark
31+
32+
The B300 DSXE nodes connect to each other over AWS EFA; they have no InfiniBand fabric. The image's
33+
standard `mooncake-transfer-engine-cuda13` build does not include the EFA transport, so the disaggregated
34+
recipes cannot transfer KV cache between the prefill and decode nodes with it. SGLang's PD-disaggregation
35+
documentation prescribes replacing the standard CUDA 13 Mooncake package with the EFA build on AWS EFA
36+
instances, using these exact commands, and the recipes set `MOONCAKE_PROTOCOL=efa` as that documentation
37+
describes.
38+
39+
## Upstream PR / issue links
40+
41+
- SGLang documentation, AWS EFA section: https://docs.sglang.io/docs/advanced_features/pd_disaggregation#aws-efa,
42+
added in https://github.com/sgl-project/sglang/pull/39973
43+
- Mooncake EFA wheel: https://pypi.org/project/mooncake-transfer-engine-efa-cuda13/0.3.13.post1/
44+
45+
## Removal plan
46+
47+
Retire `install-mooncake-efa-cu13.sh` once an upstream `lmsysorg/sglang` image ships an EFA-capable
48+
Mooncake build. At that point, bump the pinned `image:` for `dsv4-fp4-b300-dynamo-sglang-agentic-disagg`
49+
to that image, remove the `setup_script` lines from the three disaggregated recipes, delete the script,
50+
and delete this waiver in the same PR.

0 commit comments

Comments
 (0)