feat(deepep-efa): vLLM DeepEP-V2 MoE all-to-all over EFA (eager + non-eager) - #1230
Open
dmvevents wants to merge 6 commits into
Open
feat(deepep-efa): vLLM DeepEP-V2 MoE all-to-all over EFA (eager + non-eager)#1230dmvevents wants to merge 6 commits into
dmvevents wants to merge 6 commits into
Conversation
README + benchmarks (verified eager 4.8->301.8 / non-eager 5.0->239.7 tok/s + provenance table + honest caveats), recipe/ (build_deepep, serve, noneager-fix-stack, benchmark_probe), kubernetes/ 2-node yaml, .gitignore (raw/ matches write path). Mirrors ../../sglang/dsr1-deepep-efa layout, built to KeitaW's awslabs#1215 review rubric. TODO (Workflow A finish): setup_deepep_v2_efa.sh (distinct name — dodges the 567632d vendor-sync gate), primary Dockerfile (vendored-script), recipe wrappers benchmark.sh/verify-image.sh, setup/build-push.sh+env_vars.example, MIT-0 headers all files, rubric-scrub copied scripts (no HF_TOKEN under set -x, no ||true-swallow, no hardcoded NIC).
…-eager)
Complete test-case (finishes the WIP scaffold): Dockerfile (NGC-from-scratch, house-pattern
COPY setup_deepep_v2_efa.sh + ldconfig single-NCCL fail-loud guard), setup_deepep_v2_efa.sh
(distinctly named to stay outside the setup_deepep_efa.sh/567632d vendor-sync gate; aws-ofi GIN
9c44d34 + PR#1354 param + DeepEP b306af06 + PR#612 immutable-SHA), recipe/{build_deepep,serve
(eager+non-eager, EP_REUSE_NCCL_COMM=0 segfault fix),apply-noneager-fix-stack,benchmark,
verify-image (fail-loud),benchmark_probe}, setup/{build-push,env_vars.example (user ECR)},
kubernetes/2-node yaml (placeholder image), README + benchmarks/README (measured eager 4.8->301.8 /
non-eager 5.0->239.7 tok/s + provenance table + caveats). Built to KeitaW's awslabs#1215 review rubric;
MIT-0 headers all files; docker build --check clean; all versions ARG-pinned, no 'latest'.
dmvevents
marked this pull request as ready for review
August 14, 2026 17:20
…e (KeitaW-rubric pass) - benchmark_probe.py: add --url/--out argparse + write JSONL (benchmark.sh grep'd a never-written file → exited 1 on every good run) - kubernetes YAML: /opt/serve_vllm.sh → /opt/serve.sh (CrashLoop), drop dead ../docker + GUIDE-PROVENANCE + docs refs - add recipe/run-kernel-test.sh: pre-model cross-node EFA smoke (DeepEP test_ep.py + efa-direct banner assert), wired into Dockerfile + README - README/serve.sh: fix serve_vllm.sh self-name, StatefulSet (not LWS) wording, drop unshipped-Slurm claim, note vendor-sync exemption - vllm index row: EKS-only (was over-claiming Slurm)
KeitaW
self-requested a review
August 17, 2026 13:23
…r path only Per the repo's no-patches rule: remove recipe/apply-noneager-fix-stack.sh (cherry-pick + build-time source rewrite of vLLM). Eager (--enforce-eager) is the shipped + supported path. Default compilation is documented as pending the empty-ExpertTokensMetadata guard now filed upstream (vLLM PR <PENDING-PR>); once merged, a pin bump enables it with no recipe change. Historical non-eager measurements in benchmarks/ relabeled as taken with the then-unmerged guard. Signed-off-by: Anton Alexander <dmvevents@gmail.com>
Signed-off-by: Anton Alexander <dmvevents@gmail.com>
Contributor
Author
|
Re-scoped per the repo's no-patches rule — thank you for stating it plainly rather than letting it surface in review. What changed (this push):
This keeps the case alive across pin bumps by construction: nothing in the build rewrites upstream source. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
3.test_cases/pytorch/vllm/deepep-v2-efa/— the V2/NCCL-GIN counterpart to the mergedsglang/dsr1-deepep-efa(#1215). Serves an MoE on vLLM with DeepEP-V2ElasticBufferall-to-all over EFA via the aws-ofi-nccl GIN CPU-proxy (NCCL_GIN_TYPE=2), no NVSHMEM/IBGDA.Both execution modes measured on 2× p5en (H200), Qwen3-30B-A3B-FP8 DP16/EP16: eager 4.8→301.8 tok/s (121/121 HTTP 200), non-eager 5.0→239.7 tok/s (153/153 HTTP 200). Full tables + environment provenance in
benchmarks/README.md.Two integration fixes are baked in + documented: (1)
EP_REUSE_NCCL_COMM=0(stock DeepEP-V2 null-derefsncclTeamWorldunder vLLM's lazy torch comms — deterministic serve-init segfault; the guard is also filed upstream as deepseek-ai/DeepEP#727); (2) the gdrcopy forced-PCIe capability via aws-ofi-nccl#1354. Non-eager needs vLLM #46404+#46432 (merged) + a one-line meta guard, applied byrecipe/apply-noneager-fix-stack.sh.Follows the sglang exemplar layout + conventions: in-tree
setup_deepep_v2_efa.sh(distinctly named — not vendor-synced to the 567632d NVSHMEM setup), ARG-pinned versions (nolatest), user-supplied ECR, MIT-0 headers. Draft while I attach the raw benchmark logs + a fresh-pod re-measure.Draft — feedback welcome, especially on placement + the eager-vs-non-eager framing.