|
9402 | 9402 | - "TP4 c8 adds a HiCache DRAM tier (ratio 2.75, write_through, direct I/O, page_first_direct, as in the B200 TP8 AgentX recipe on the same image) because TP4 c4 already fills about 60% of the GPU KV pool. 1P2D DEP8 c64 is the 1P1D c64 recipe with a second decode worker, which halves each decode rank's load." |
9403 | 9403 | - "The aggregate DEP8 c384 recipe follows the single-node SGLang DEP8 c384 point (attention DP8 + EP8 Mega-MoE, FP4 indexer, prefill delayer with interval 20, chunked prefill 65536, max-running-requests 768, mem-fraction-static 0.88, HiCache ratio 3 write_back DRAM tier) on the newer image, with cuda-graph-max-bs-decode set to the 96-request per-rank cap (768 / 8), behind the Dynamo KV router with per-DP-rank KV events and session affinity." |
9404 | 9404 | - "Mega-MoE uses the default FP8xFP4 kernels in every DEP8 recipe; the W4A4 MXFP4 Mega-MoE path is not enabled." |
9405 | | - - "Use Mooncake for disaggregated KV transfer and install mooncake-transfer-engine-efa-cuda13 0.3.13.post1 during container setup; rely on the DSXE-managed fabric injection without explicit host EFA/OFI mounts." |
9406 | | - - "Use the DSXE cluster's 24 CPUs per GPU, HiCache ratio 3, decode max-running-requests at twice concurrency, persistent AgentX/Hugging Face caches, and exclude the known degraded DSXE node." |
| 9405 | + - "Use NIXL with the LIBFABRIC backend for disaggregated KV transfer over the DSXE EFA fabric. The pinned image runs as shipped, with no setup script, no package changes and no explicit host EFA/OFI mounts." |
| 9406 | + - "Use the DSXE cluster's 24 CPUs per GPU, HiCache ratio 3, decode max-running-requests at twice concurrency, and exclude the known degraded DSXE nodes." |
9407 | 9407 | - "Serve B300 DSXE Dynamo+SGLang DeepSeek-V4-Pro-0813 weights from the node-local NVMe copy (DeepSeek-V4-Pro-0813@scratch, as for vLLM) instead of the shared /data/models copy, which slowed startup by hours when several points loaded concurrently." |
9408 | 9408 | - "新增 DeepSeek-V4-Pro-0813 FP4 B300 Dynamo+SGLang AgentX 配方:聚合式 TP8 c1、TP4 c4、TP4 c8 和 DEP8 c384,以及分离式 1P1D DEP8 c64/c240 和 1P2D DEP8 c64。" |
9409 | 9409 | - "由于 TP4 c4 已占用约 60% 的 GPU KV 池,TP4 c8 增加 HiCache DRAM 层(比例 2.75、write_through、direct I/O、page_first_direct,与同一镜像上的 B200 TP8 AgentX 配方相同)。1P2D DEP8 c64 即 1P1D c64 配方增加第二个 decode worker,使每个 decode rank 的负载减半。" |
9410 | 9410 | - "聚合式 DEP8 c384 配方沿用单节点 SGLang DEP8 c384 测试点(注意力 DP8 + EP8 Mega-MoE、FP4 indexer、间隔为 20 的 prefill delayer、chunked prefill 65536、max-running-requests 768、mem-fraction-static 0.88、HiCache 比例 3 的 write_back DRAM 层),改用更新的镜像,并将 cuda-graph-max-bs-decode 设为每个 rank 96 个请求的上限(768 / 8),并由 Dynamo KV 路由器基于各 DP rank 的 KV 事件和会话亲和性进行路由。" |
9411 | 9411 | - "所有 DEP8 配方的 Mega-MoE 均使用默认的 FP8xFP4 kernel,不启用 W4A4 MXFP4 Mega-MoE 路径。" |
9412 | | - - "分离式 KV 传输使用 Mooncake,并在容器初始化时安装 mooncake-transfer-engine-efa-cuda13 0.3.13.post1;依赖 DSXE 管理的网络栈注入,不显式挂载主机 EFA/OFI 目录。" |
9413 | | - - "使用 DSXE 集群的每 GPU 24 个 CPU、HiCache 比例 3,decode 的 max-running-requests 为并发数两倍,挂载持久化 AgentX/Hugging Face 缓存,并排除已知异常 DSXE 节点。" |
| 9412 | + - "分离式 KV 传输使用 LIBFABRIC 后端的 NIXL,经由 DSXE 的 EFA 网络传输。固定镜像按原样运行,不使用初始化脚本、不改动软件包,也不显式挂载主机 EFA/OFI 目录。" |
| 9413 | + - "使用 DSXE 集群的每 GPU 24 个 CPU、HiCache 比例 3,decode 的 max-running-requests 为并发数两倍,并排除已知异常 DSXE 节点。" |
9414 | 9414 | - "B300 DSXE Dynamo+SGLang 的 DeepSeek-V4-Pro-0813 权重改为从节点本地 NVMe 副本(DeepSeek-V4-Pro-0813@scratch,与 vLLM 相同)加载,而非共享的 /data/models 副本;后者在多个测试点并发加载时会使启动慢数小时。" |
9415 | | - pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3631 |
| 9415 | + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX |
0 commit comments