[NVIDIA][AgentX] Refresh GB200 DeepSeek-V4.1-Flash SGLang aggregate curve / [NVIDIA][AgentX] 更新 GB200 DeepSeek-V4.1-Flash SGLang 聚合曲线

[shyeh25]

Summary

Refresh the DeepSeek-V4.1-Flash FP4 GB200 SGLang AgentX aggregate curve (dsv41flash-fp4-gb200-sglang-agentic-dspark):

• repin the SGLang image to lmsysorg/sglang:nightly-dev-cu13-20261011-d32d8fa0, pinned by multiarch digest;
• replace the TP4/EP1 and TP2/EP1 C1–C128 sweeps with TP4/EP4 points at C1, C4, C8 and C16 and aggregate TP4/EP4 points with DP attention (DP4) at C32, C64, C128 and C192;
• use the validated per-point prefill-decode interval, prefill chunking, admission, memory and SWA prefix-tail settings;
• route the DP4 points through sglang-router 0.3.2 with cache-aware, correlation-ID-affine routing;
• append the matching performance-changelog entry.

The deployment remains single-node and aggregate. No cluster-route or runner configuration changes are included. DSpark remains the checkpoint-bundled draft at its shipped precision with block size 5.

AI model disclosure

• Model/version: Claude Opus 5.5 (claude-opus-5-5)
• Role: benchmark tuning and analysis, recipe and matrix integration, validation, and PR preparation. No delegated agents contributed to this PR patch.

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• For every change that can affect benchmark performance and every recipe addition or modification, I have appended a new entry to the physical end of inferencex-e2e/perf-changelog.yaml and have not edited historical entries
• If this PR can affect benchmark performance or adds or modifies a recipe, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

更新 DeepSeek-V4.1-Flash FP4 GB200 SGLang AgentX 聚合曲线（dsv41flash-fp4-gb200-sglang-agentic-dspark）：

• 将 SGLang 镜像更新至 lmsysorg/sglang:nightly-dev-cu13-20261011-d32d8fa0，并以 multiarch digest 固定；
• 以 TP4/EP4 的 C1、C4、C8、C16，以及启用 DP attention 的聚合式 TP4/EP4（DP4）的 C32、C64、C128、C192，取代原本的 TP4/EP1 与 TP2/EP1 C1–C128 扫描；
• 使用已验证的逐点 prefill-decode interval、prefill chunk、运行请求上限、显存与 SWA 前缀尾部设置；
• DP4 测试点通过 sglang-router 0.3.2 进行缓存感知及 correlation-ID 亲和路由；
• 在 performance changelog 末尾追加对应条目。

部署仍为单节点聚合式部署。本 PR 不包含集群路由或 runner 配置改动。DSpark 继续使用 checkpoint 自带的 draft 及其原始精度，block size 为 5。

AI 模型披露：本 PR 使用 Claude Opus 5.5（claude-opus-5-5）进行 benchmark 调优与分析、recipe/矩阵整合、验证与 PR 准备；没有委派其他 agent 参与此 PR patch。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38266927261

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38266927261

[claude]

I reviewed this PR and found no bugs. Because it touches CODEOWNER-owned nvidia-master.yaml and the GB200 srt-slurm recipe, and reworks the entire TP4/EP1+TP2/EP1 sweep into a new TP4/EP4 + DP4-behind-router search space, a human look is still worthwhile.

• Checked image consistency: the master config's image: and the recipe's model.container: both now pin the identical sha256:8346c975... digest.
• Checked the new DP4/router knobs (expert-parallel-size, dp-attention, load-balance-method, sglang-router args) for draft/KV precision drift — none touch quantization or draft precision.
• Checked the c192 point's cuda-graph-max-bs-decode (inherited at 64) against its max-running-requests: 384 — most decode batches will run eager rather than graphed, a possible throughput understatement, not a correctness bug.
• Confirmed the deleted tp2/tp4 override blocks were removed outright rather than archived, matching the repo's deprecation convention.

Extended reasoning...

This change touches two CODEOWNER-owned paths (inferencex-e2e/configs/nvidia-master.yaml and the gb200-fp4-mtp srt-slurm recipe) plus an append-only perf-changelog entry; it is a pure config/YAML rework (image bump, new TP4/EP4 and DP4-via-sglang-router sweep points, replacing the old TP4/EP1 and TP2/EP1 sweeps) with no shell or application code changed, so there is no injection/auth/data-exposure surface. The perf-changelog entry still carries a placeholder pull/XXX pr-link that needs to be filled before merge, and the changelog's claim of no precision change to the DSpark draft checks out against the recipe's new args (none alter quantization or KV/draft dtype). Given the CODEOWNERS ownership and the scope of the sweep-shape rewrite, I deferred to a human reviewer rather than approving, independent of the one benign-performance item noted above.

[nvpohanh]

/use 38266927261

[github-actions]

@nvpohanh staged run 38266927261.

• DeepSeek-V4.1-Flash · agentic-traces · FP4 · gb200_sglang · 2026-10-12

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (dsv41flash-fp4-gb200-sglang-agentic-dspark) will happen in sgl-project/sglang#43983, in the Agentic Long-Context Deployment section of the DeepSeek-V4.1 cookbook page.

The commands mirror the launch measured in run 38266927261, pinned by this PR's /use comment.

[nvpohanh]

[by Claude Code] sgl-project/sglang#43983 has been merged. This PR's AgentX config (dsv41flash-fp4-gb200-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang DeepSeek-V4.1 cookbook: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4_1#agentic-long-context-deployment

The commands mirror the launch measured in run 38266927261, pinned by this PR's /use comment.

[nvpohanh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• Verified that every draft model and draft head is served as it ships: the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling, with the shipped and effective draft precision recorded in the additional detail section. No submission-side quantization, dtype override, checkpoint substitution, or patch may lower draft precision below that default, regardless of eval results or AL. Explicitly verified that SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not enabled in the effective recipe, including inherited settings; enabling it is prohibited going forward, and historical runs do not grant an exception. See Draft-model precision for what counts as the default and the MLPerf comparison.
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in infx/golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; target/verifier FLOPs at lower precisions is fine, given that the config passes private evals, but this does not permit lowering draft-model or draft-head precision below what ships. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every affected vLLM/SGLang production serving configuration in this PR, including Dynamo deployments, is documented in the official vLLM recipes and/or the SGLang cookbook, for both single-node and multi-node aggregated and disaggregated serving. The documentation covers production-useful flags, environment variables, and launch commands (including prefill, decode, and router/frontend commands for disaggregation); InferenceX-only benchmark/harness settings are excluded with reasoning in the additional detail section:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang cookbook source repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If this PR uses append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
• Reported measured throughput/E2EL Pareto counts and evidence per affected curve (≥5 points strongly recommended). Below 5 or unverifiable: tag a core maintainer for review; recorded admin bypass required before merge. N/A if no curves are affected. Details.

Additional detail section:

• Recipe publication (merged): every point in this PR is documented in the SGLang cookbook by sgl-project/sglang#43983, which is MERGED (mergedAt: 2026-10-13T06:39:46Z, merge commit dbad07a06fbedfc63fd54282d2989ef4feb03c23).
  • Published page: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4_1#agentic-long-context-deployment (hardware GB200, checkpoint deepseek-ai/DeepSeek-V4.1-Flash, KV offload "None"). The live page already shows both GB200 cells with this PR's head and run.
  • Data: docs/src/snippets/agentx/deepseek-ai/deepseek-v4_1.jsx records this PR's head e5d1fa07b165c344a6fb32b739f5fd1697564e78 and run 38266927261 (status: verified-pr) as the source of both GB200 deployments, "Agg TP4·EP4" and "Agg DEP4". It lists the same image digest.
  • GB200 "Agg TP4·EP4" (C1, C4, C8, C16), worker: python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V4.1-Flash --served-model-name deepseek-ai/DeepSeek-V4.1-Flash --tensor-parallel-size 4 --expert-parallel-size 4 --speculative-algorithm DSPARK --speculative-dspark-block-size 5 --chunked-prefill-size 8192 --cuda-graph-max-bs-decode 64 --mem-fraction-static 0.76 --reasoning-parser auto --tool-call-parser auto --trust-remote-code --watchdog-timeout 3600, plus the per-point settings below.
    • --max-running-requests 2 / 8 / 16 / 32.
    • --swa-prefix-tails 128 / 128 / 256 / 512.
    • --prefill-decode-interval 16 / 384 / 384 / 96.
  • GB200 "Agg DEP4" (C32, C64, C128, C192), worker: the TP4·EP4 command plus --data-parallel-size 4 --enable-dp-attention --enable-dp-lm-head --enable-dp-attention-local-control-broadcast --load-balance-method total_requests --tokenizer-worker-num 8, plus the per-point settings below.
    • --chunked-prefill-size 32768 / 65536 / 65536 / 65536.
    • --max-running-requests 256 / 256 / 256 / 384.
    • --swa-prefix-tails 2048 / 2048 / 2048 / 3072.
    • --prefill-decode-interval 88 / 72 / 24 / 12.
  • GB200 "Agg DEP4", router: python3 -m sglang_router.launch_router --policy cache_aware --request-id-headers x-correlation-id --dp-aware --connect-timeout-secs 900 --request-timeout-secs 14400 --disable-health-check --disable-retries --worker-urls ... (sglang-router 0.3.2, as pinned in the master config and logged by the run). "Agg TP4·EP4" has no router; the recipe's base uses frontend.type: sglang.
  • Environment, both deployments: SGLANG_DEFAULT_THINKING=1, SGLANG_DSV41_REASONING_EFFORT=high, SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE=1, SGLANG_DSV41_ENGRAM_HOST_TABLE_LAYOUT=per_rank, SGLANG_TIMEOUT_KEEP_ALIVE=900.
  • I resolved the cookbook command for each of the 8 points and diffed it against the logged srt-slurm launch command, router command and worker environment in run 38266927261. They are identical except for the InferenceX-only settings below.
    • The pinned image auto-resolves quantization, KV-cache dtype and kernel backends. The logs show quantization=None, which falls back to the checkpoint's FP8/MXFP4 quantization_config, kv_cache_dtype=fp8_e4m3, attention_backend=dsv4, moe_runner_backend=flashinfer_mxfp4 and fp8_gemm_runner_backend=flashinfer_cutedsl. Neither the recipe nor the cookbook sets them.
    • Under DP attention, SGLang splits --chunked-prefill-size and --max-running-requests across the 4 attention-DP ranks. The logs show per-rank chunked_prefill_size 8192 (C32) and 16384 (C64 to C192), and per-rank max_running_requests 64 (C32 to C128) and 96 (C192). The recipe and the cookbook pass the same CLI values.
  • Excluded as InferenceX-only:
    • --enable-metrics on the worker and --prometheus-port/--prometheus-host on the router. They are the Prometheus export that AIPerf scrapes for result collection (AIPERF_REQUIRED_SERVER_METRIC_PREFIX=sglang:) and do not affect scheduling or kernels.
    • --host, --port, --nccl-port and worker URLs. These are srt-slurm process placement; the cookbook uses placeholders.
    • PYTHONNOUSERSITE and PYTHONUNBUFFERED, which only set Python path isolation and log buffering in the container.
    • SGLANG_CUDA_COREDUMP_BEFORE_CRASH=0 and SGLANG_PYSPY_DUMP_BEFORE_CRASH=0. These srt-slurm crash-diagnostic toggles only control dumps after a crash.
    • SGLANG_SIMULATE_ACC_*, the synthetic acceptance control. The cookbook discloses it as "synthetic-acceptance".
    • The NCCL_* network variables in the worker fingerprint (NCCL_IB_HCA, NCCL_SOCKET_IFNAME, NCCL_IB_SL, NCCL_IB_TIMEOUT, NCCL_MNNVL_ENABLE, NCCL_CUMEM_ENABLE, NCCL_NET_GDR_C2C). They come from the GB200 cluster node environment; the recipe, master config and runner do not set them.
    • The AIPerf client settings (CONC, AGENTIC_WARMUP_GRACE_PERIOD, AIPERF_*) and the srt-slurm health_check timeout.
• PR validation: Run Sweep 38266927261 ran on head e5d1fa07b165c344a6fb32b739f5fd1697564e78 (pull_request, full-sweep-fail-fast) and completed with success on attempt 3. It is pinned with /use 38266927261. Its bmk_agentic_*, eval_results_all and server-log artifacts are unexpired (they expire 2027-01-10).
  • All 8 dsv41flash-fp4-gb200-sglang-agentic-dspark benchmark jobs succeeded on attempt 3: TP4/EP4 C1/C4/C8/C16 and TP4/EP4/DP4 with DP attention at C32/C64/C128/C192 behind sglang-router 0.3.2. Every point recorded 0 error-dropped requests.
  • Both agentic eval / jobs succeeded.
  • Attempts 1 and 2 were infrastructure failures on the same head and image. The C128 server (attempts 1 and 2) and the attempt-1 eval servers exited at startup with RuntimeError: Cannot find any model weights with deepseek-ai/DeepSeek-V4.1-Flash from the shared Hugging Face cache (attempt 1, attempt 2). Fail-fast cancelled the other points, and attempt 3 reran all of them.
  • The skipped jobs are the empty fixed-sequence and multi-node lanes, the canary and reuse gates, and collect-results, compare-results and calc-success-rate, which run only for fixed-sequence results.
• Evals: gsm8k em_strict was 0.9712 at TP4/EP4 C16 (job) and 0.9757 at TP4/EP4/DP4 C192 (job), n=1319 each, infrastructure_success: true, from eval_results_all.
  • Both clear the default 0.90 threshold in infx/evals/thresholds.yaml, which has no dsv41flash override.
  • Both ran on the same image digest with real DSpark verification. The eval server logs show speculative_algorithm='DSPARK' and no SGLANG_SIMULATE_ACC_*.
• Image: upstream lmsysorg/sglang:nightly-dev-cu13-20261011-d32d8fa0@sha256:8346c975515af7ec6b2692faadd5332f789001f78f958335175d4d34e170537c, with SGLang d32d8fa0d1693189fda6727dd1b02caa3d26e367 (SGLANG_BUILD_COMMIT in the logged environment). The GB200 nodes ran it on aarch64. The config is framework: sglang, so the engine-ordering item is satisfied by this submission itself.
• Chat template: AgentX replay runs aiperf profile --scenario agentx, and the AIPerf export records endpoint type chat with streaming. The server logs of all 8 points show only POST /v1/chat/completions, plus the harness's /freeze_gc admin call. They show no /v1/completions or /generate requests.
  • The served revision ships chat_template.jinja. The server logs "Using default HuggingFace chat template" and auto-detects the deepseek-v41 reasoning parser and deepseekv41 tool-call parser from it. Thinking is on (SGLANG_DEFAULT_THINKING=1) with reasoning effort high.
• Draft precision: the served draft is the DSpark head bundled in deepseek-ai/DeepSeek-V4.1-Flash rev 5254448103b6254fc646dacbca1cbbf20e9216c1, the current Hugging Face main (mtp.0 to mtp.2, block size 5).
  • The logs show the local snapshot snapshots/5254448103b6254fc646dacbca1cbbf20e9216c1 and "DSpark draft weights are bundled in the target checkpoint; defaulting --speculative-draft-model-path to --model-path".
  • All 48 safetensors shards at this revision have the same LFS object IDs as rev dba1be0a40aa45a94ad051997016db3960a90277, and config.json is identical.
  • Shipped, from the safetensors headers of shards 44 to 46 at this revision (all 2,401 mtp.* tensors): the attention (wq_a, wq_b, wkv, wo_a, wo_b), shared-expert and main_proj weights are F8_E4M3 with F8_E8M0 scales. The routed experts are MXFP4 (expert_dtype: fp4, packed as I8 with F8_E8M0 scales). Norms, the gate weight, markov_head and confidence_head are BF16. attn_sink, the gate biases and the hc_* tensors are F32. The checkpoint quantization_config (fp8, 32x32 blocks, ue8m0) has no exclusions.
  • Effective: the same, using the pinned image's default handling. deepseek_v4_dspark.py builds the draft with the target's quant config and wo_a_fp8=False. It dequantizes wo_a from FP8 to BF16 at load (_dequant_fp8_wo_a_streaming), matching DeepSeek's reference. The logs show type=DeepseekV4ForCausalLMDSpark, quant=fp8. Routed experts run MXFP4 through flashinfer_mxfp4, and the KV cache is the default FP8 for both target and draft.
  • The logged server args of all 8 points show speculative_draft_model_quantization=None, speculative_draft_kv_cache_dtype=None, quantization=None, dtype=auto and speculative_draft_model_path=deepseek-ai/DeepSeek-V4.1-Flash, which is the same checkpoint.
  • SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not set in the recipe, the launch path or the logged worker environment of any point. It defaults to EnvBool(False) in environ.py at the pinned SHA.
• Golden AL: infx/golden_al_distribution/dsv41flash_dspark.yaml gives 3.51 for thinking_on with draft length 5.
  • Every benchmark point logged SGLANG_SIMULATE_ACC_LEN=3.51, SGLANG_SIMULATE_ACC_METHOD=match-expected and SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token, with SGLANG_DEFAULT_THINKING=1. The values are injected by infx/srt_slurm/synthetic_acceptance.py from the golden curve, not set in the recipe.
  • The mean accept len in the decode logs is 3.508 to 3.513 on every point.
• MODELS.md: dsv41flash agentic coding is active, with SGLang DSpark arms per SKU.
• No architecture hacks: there is no --hf-overrides, and json_model_override_args='{}'.
  • enable_deepseek_v4_fp4_indexer=False, so the indexer uses the image's default path.
  • SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE=1 with per_rank layout only moves where Engram tables live; the data and computation are unchanged.
  • swa-prefix-tails sizes headroom in the SWA KV pool; the sliding window is unchanged.
  • publish_events_and_metrics is a Dynamo TRT-LLM srt-slurm field and does not apply to this SGLang path. enable-metrics is on.
• Code quality: the diff only changes this recipe and its master-config entry and appends one changelog entry. The changelog entry uses the XXX pr-link placeholder that validate_perf_changelog.py accepts, and check-changelog passed.
  • The recipe's sizing comments match every TP4 point: admission is 2x CONC (2/8/16/32), below the graph batch of 64, and SWA prefix tails are 32x CONC with a 128 floor. The DP4 points run behind sglang-router, use 64K prefill chunks from C64, and get the longer 3600 s warmup drain.
  • DP4 C192 admits 96 per rank with cuda-graph-max-bs-decode: 64. The per-rank decode batch never exceeded 49 in 6,187 decode log lines, all with cuda graph: True.
  • The only merge conflict with main is in perf-changelog.yaml, and this PR's change to it is append-only.
• No engine patches: the diff adds no patch files, git apply, heredoc rewrites, site-packages edits or wheel installs. The launch path has no setup script, and the pinned image runs as shipped. No waiver is needed.
• append-only: N/A, because the single new changelog entry does not use append-only: true.
• Pareto: one curve, dsv41flash / agentic-coding / gb200_sglang / fp4 / run 38266927261, counted with infx.workflows.pareto_coverage (x = E2EL in seconds, y = total token throughput per GPU).
  • Source: the 8 bmk_agentic_* artifacts of the run (request_metrics.throughput.per_gpu.total_tput_tps and request_metrics.latency.e2el). This agentic-only run has no results_bmk artifact because collect-results did not run.
  • P90 E2EL frontier: 7 of 8 points (TP4 C4/C8/C16 and DP4 C32/C64/C128/C192). TP4 C1 is dominated by C4.
  • P75 E2EL frontier: 6 of 8 points (TP4 C4/C16 and DP4 C32/C64/C128/C192). TP4 C1 and C8 are dominated by C4 and C16.
  • Per point (tok/s/GPU, P90 E2EL s, P75 E2EL s): C1 7105.5, 9.159, 5.412; C4 9747.4, 7.804, 4.181; C8 15717.0, 7.900, 4.316; C16 28604.3, 8.674, 4.277; C32 55465.5, 15.143, 7.423; C64 102457.5, 19.281, 9.776; C128 148250.9, 29.911, 14.613; C192 192061.2, 46.223, 20.948.
  • Helper output (python -m infx.workflows.pareto_coverage from inferencex-e2e/ on upstream main, fed the 8 points above): dsv41flash/agentic-coding/gb200_sglang/fp4/run38266927261/p90_e2el is PASS with measuredPoints: 8, invalidPoints: 0, frontierPoints: 7; .../p75_e2el is PASS with measuredPoints: 8, invalidPoints: 0, frontierPoints: 6.
  • Dashboard: https://inferencex.semianalysis.com/inference?unofficialRun=38266927261
• NVIDIA overview (advisory): the NVIDIA recipes overview lists DeepSeek-V4 (SGLang, linking the DeepSeek-V4 cookbook page), DeepSeek-V4-Flash and DeepSeek-V4-Pro, but has no DeepSeek-V4.1-Flash row yet. The advisory overview check is therefore expected to warn for DeepSeek-V4.1-Flash with SGLang on GB200; it does not affect the items above.

Signed: nvpohanh

[github-actions]

❌❌❌ REJECTED ❌❌❌

The verifier did not produce a valid verdict. Retry the sign-off verification.

Assessed commit: e5d1fa07b165c344a6fb32b739f5fd1697564e78.