|
| 1 | +# Serving-stack package waiver — PR #3631 |
| 2 | + |
| 3 | +Filed per [`docs/PR_REVIEW_CHECKLIST.md`](../PR_REVIEW_CHECKLIST.md) (no-patch item): the disaggregated |
| 4 | +recipes in this PR replace one package in the pinned serving image at container setup. The checklist |
| 5 | +requires the pinned image to run as shipped unless such a change is covered by a filled-out waiver. |
| 6 | + |
| 7 | +## Config covered |
| 8 | + |
| 9 | +- **Master config entry:** `dsv4-fp4-b300-dynamo-sglang-agentic-disagg` in |
| 10 | + [`configs/nvidia-master.yaml`](../../configs/nvidia-master.yaml) (1P1D DEP8 c64 and c240, 1P2D DEP8 c64). |
| 11 | + The aggregate entry `dsv4-fp4-b300-dynamo-sglang-agentic-agg` does not run the script. |
| 12 | +- **Pinned image:** `lmsysorg/sglang:nightly-dev-20260916-c9a8fba9` |
| 13 | +- **Setup entrypoint:** `setup_script: install-mooncake-efa-cu13.sh` in |
| 14 | + [`disagg-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml`](../../benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p1d-dep8-dep8-c64-mtp-kvoffload.yaml), |
| 15 | + [`disagg-1p1d-dep8-dep8-c240-mtp-kvoffload.yaml`](../../benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p1d-dep8-dep8-c240-mtp-kvoffload.yaml), and |
| 16 | + [`disagg-1p2d-dep8-dep8-c64-mtp-kvoffload.yaml`](../../benchmarks/multi_node/srt-slurm-recipes/dsv4/sglang/b300-fp4/agentx/disagg-1p2d-dep8-dep8-c64-mtp-kvoffload.yaml) |
| 17 | +- **Script:** [`benchmarks/multi_node/srt-slurm-recipes/configs/install-mooncake-efa-cu13.sh`](../../benchmarks/multi_node/srt-slurm-recipes/configs/install-mooncake-efa-cu13.sh) |
| 18 | + |
| 19 | +## What is changed |
| 20 | + |
| 21 | +The script uninstalls the image's `mooncake-transfer-engine-cuda13` 0.3.13 and installs the public EFA |
| 22 | +build of the same Mooncake release line, `mooncake-transfer-engine-efa-cuda13==0.3.13.post1`, from PyPI |
| 23 | +with `--no-deps`. Both packages provide the same `mooncake` Python module; the EFA build adds the AWS EFA |
| 24 | +(libfabric) transport. The commands are the ones in SGLang's documentation. |
| 25 | + |
| 26 | +Nothing else changes. No SGLang, Dynamo, or Mooncake source is patched, no forked or rebuilt wheel is |
| 27 | +installed, and no other package is added or removed. Mooncake only moves KV-cache pages from prefill to |
| 28 | +decode workers; it performs no model computation, so the change cannot alter model numerics. |
| 29 | + |
| 30 | +## Why the unmodified upstream image cannot run this benchmark |
| 31 | + |
| 32 | +The B300 DSXE nodes connect to each other over AWS EFA; they have no InfiniBand fabric. The image's |
| 33 | +standard `mooncake-transfer-engine-cuda13` build does not include the EFA transport, so the disaggregated |
| 34 | +recipes cannot transfer KV cache between the prefill and decode nodes with it. SGLang's PD-disaggregation |
| 35 | +documentation prescribes replacing the standard CUDA 13 Mooncake package with the EFA build on AWS EFA |
| 36 | +instances, using these exact commands, and the recipes set `MOONCAKE_PROTOCOL=efa` as that documentation |
| 37 | +describes. |
| 38 | + |
| 39 | +## Upstream PR / issue links |
| 40 | + |
| 41 | +- SGLang documentation, AWS EFA section: https://docs.sglang.io/docs/advanced_features/pd_disaggregation#aws-efa, |
| 42 | + added in https://github.com/sgl-project/sglang/pull/39973 |
| 43 | +- Mooncake EFA wheel: https://pypi.org/project/mooncake-transfer-engine-efa-cuda13/0.3.13.post1/ |
| 44 | + |
| 45 | +## Removal plan |
| 46 | + |
| 47 | +Retire `install-mooncake-efa-cu13.sh` once an upstream `lmsysorg/sglang` image ships an EFA-capable |
| 48 | +Mooncake build. At that point, bump the pinned `image:` for `dsv4-fp4-b300-dynamo-sglang-agentic-disagg` |
| 49 | +to that image, remove the `setup_script` lines from the three disaggregated recipes, delete the script, |
| 50 | +and delete this waiver in the same PR. |
0 commit comments