[NVIDIA][AgentX] Refresh B300 DeepSeek-V4.1-Flash SGLang aggregate curve / [NVIDIA][AgentX] 更新 B300 DeepSeek-V4.1-Flash SGLang 聚合曲线

[shyeh25]

Description

Refresh the DeepSeek-V4.1-Flash FP4 B300 SGLang AgentX aggregate curve:

• repin the SGLang image to the digest-pinned 20261004 nightly;
• publish the selected TP4/EP1, TP2/EP1, and aggregate DP4 point set;
• use the validated per-point scheduling and memory settings;
• route the aggregate DP4 point through sglang-router with cache-aware, correlation-ID-affine routing;
• append the matching performance-changelog entry.

The deployment remains single-node and aggregate. No cluster-route or runner configuration changes are included.

Validation

• Parsed all changed YAML files.
• Generated the exact-key matrix and confirmed every row selects one intended recipe override.
• Validated every selected override with the vendored srt-slurm dry-run path.
• Passed the focused matrix and single-node SRT tests.
• Confirmed the changelog change is append-only and the complete diff passes git diff --check.

AI model disclosure

• Model/version: GPT-5 (Codex)
• Role: Prepared the upstream configuration update, reviewed repository policy and comparable submissions, validated the launch matrix and recipe overrides, and drafted the PR text.

Related Issue

None.

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• For every change that can affect benchmark performance and every recipe addition or modification, I have appended a new entry to the physical end of inferencex-e2e/perf-changelog.yaml and have not edited historical entries
• If this PR can affect benchmark performance or adds or modifies a recipe, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

说明

更新 DeepSeek-V4.1-Flash FP4 B300 SGLang AgentX 聚合曲线：

• 将 SGLang 镜像更新至通过 digest 固定的 20261004 nightly；
• 发布选定的 TP4/EP1、TP2/EP1 与聚合式 DP4 测试点；
• 使用已验证的逐点调度与显存设置；
• 聚合式 DP4 测试点通过 sglang-router 进行缓存感知及 correlation-ID 亲和路由；
• 在 performance changelog 末尾追加对应条目。

部署仍为单节点聚合式部署。本 PR 不包含集群路由或 runner 配置改动。

验证

• 已解析所有修改的 YAML 文件。
• 已生成精确配置键矩阵，并确认每一行仅选择一个预期的配方 override。
• 已通过 vendored srt-slurm dry-run 路径验证所有选定的 override。
• 已通过相关 matrix 与单节点 SRT 测试。
• 已确认 changelog 仅在末尾追加，且完整 diff 通过 git diff --check。

AI 模型披露

• 模型/版本：GPT-5（Codex）
• 工作内容：准备 upstream 配置更新、检查仓库规则与同类提交、验证启动矩阵和配方 override，并撰写 PR 说明。

关联 Issue

无。

改动类型

配置变更。

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

I reviewed this PR and didn't find a definitive bug, but a human look is still worthwhile before merging. The new override_dp4_c192 point switches to DP4 decode (data-parallel-size: 4, max-running-requests: 384) but doesn't raise cuda-graph-max-bs-decode above the inherited base value of 64. The only other DP-attention recipe in this repo (dsv4/sglang/b300-fp4-mtp/agentic.yaml) explicitly raises this to 544 for its DP8 points, and perf-changelog.yaml documents the exact reason for a prior similar change: "the CONC-capped value of 64 does not cover the padded MTP decode batch across DP ranks." Worth a maintainer double-check that this was intentionally validated for the DSpark/DP4 case here (the file's own comment notes the point admits "384 globally," well above the per-rank captured graph size).

Extended reasoning...

The diff refreshes a B300 SGLang AgentX config (repinned nightly image, revised TP4/TP2 points, and a new aggregated DP4/EP4 point behind sglang-router) plus matching docs and an append-only changelog entry; it touches no auth/crypto/permission surface. I independently cross-checked a candidate concern (already investigated and marked ruled-out upstream) against a sibling recipe in the same repo that explicitly raises cuda-graph-max-bs-decode for its DP-attention points, plus a changelog entry explaining why the CONC-capped default is insufficient under DP — that precedent is not obviously addressed in this PR's new override_dp4_c192 block, so despite the automated ruling, I think it merits a maintainer's second look rather than silent approval.

This review covers commit 358f5e1, which is no longer the latest commit on this pull request; later commits are not covered by it.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=37816190252

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=37816190252

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (dsv41flash-fp4-b300-sglang-agentic-dspark) will happen in sgl-project/sglang#43083, in the Agentic Long-Context Deployment section of the DeepSeek-V4.1 cookbook page. This PR has no authorized /use <run_id> comment yet, so the cookbook shows the PR-head recipe as "in progress" until a measured run is pinned.

[nvpohanh]

[by Claude Code] sgl-project/sglang#43083 has been merged. This PR's AgentX config (dsv41flash-fp4-b300-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang DeepSeek-V4.1 cookbook: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4_1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (ec20ed566) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

[nvpohanh]

/use 37816190252

[github-actions]

@nvpohanh staging run 37816190252. Existing staged runs will be preserved; a completion link will be posted here.

[nvpohanh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• Verified that every draft model and draft head is served as it ships: the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling, with the shipped and effective draft precision recorded in the additional detail section. No submission-side quantization, dtype override, checkpoint substitution, or patch may lower draft precision below that default, regardless of eval results or AL. Explicitly verified that SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not enabled in the effective recipe, including inherited settings; enabling it is prohibited going forward, and historical runs do not grant an exception. See Draft-model precision for what counts as the default and the MLPerf comparison.
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in infx/golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; target/verifier FLOPs at lower precisions is fine, given that the config passes private evals, but this does not permit lowering draft-model or draft-head precision below what ships. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every affected vLLM/SGLang production serving configuration in this PR, including Dynamo deployments, is documented in the official vLLM recipes and/or the SGLang cookbook, for both single-node and multi-node aggregated and disaggregated serving. The documentation covers production-useful flags, environment variables, and launch commands (including prefill, decode, and router/frontend commands for disaggregation); InferenceX-only benchmark/harness settings are excluded with reasoning in the additional detail section:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang cookbook source repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If this PR uses append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
• Reported measured throughput/E2EL Pareto counts and evidence per affected curve (≥5 points strongly recommended). Below 5 or unverifiable: tag a core maintainer for review; recorded admin bypass required before merge. N/A if no curves are affected. Details.

Additional detail section:

• Recipe publication (merged): every point in this PR is documented in the SGLang cookbook by sgl-project/sglang#43083, which is MERGED (mergedAt: 2026-10-09T05:55:37Z, merge commit 1fc6b8962c02f8b8413aac33c57ad82a4df3c8b7).
  • Published page: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4_1#agentic-long-context-deployment (hardware B300, checkpoint deepseek-ai/DeepSeek-V4.1-Flash, KV offload "None").
  • Data: docs/src/snippets/agentx/deepseek-ai/deepseek-v4_1.jsx records this PR's head ec20ed5660994bc20a94d68a04488aa5bc2b33ea, the commit of the pinned run, as the source of all three B300 deployments ("Agg TP2", "Agg TP4" and "Agg DEP4"). It lists the same image digest. The entry is marked in-progress only because the cookbook sync ran before /use was posted.
  • B300 "Agg TP2" (C1, C4, C16, C24, C32, C48), worker: python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V4.1-Flash --served-model-name deepseek-ai/DeepSeek-V4.1-Flash --tensor-parallel-size 2 --expert-parallel-size 1 --speculative-algorithm DSPARK --speculative-dspark-block-size 5 --chunked-prefill-size 16384 --cuda-graph-max-bs-decode 64 --reasoning-parser auto --tool-call-parser auto --trust-remote-code --watchdog-timeout 3600 --weight-loader-prefetch-checkpoints, plus the per-point settings below.
    • --max-running-requests 2 / 8 / 32 / 48 / 64 / 64.
    • --mem-fraction-static 0.7 / 0.7 / 0.8 / 0.8 / 0.85 / 0.8.
    • --swa-prefix-tails 128 / 256 / 1024 / 1536 / 2048 / 3072.
    • --prefill-decode-interval 16 / 96 / 192 / 96 / 64 / 64.
  • B300 "Agg TP4" (C1, C4), worker: the TP2 command with --tensor-parallel-size 4, with --max-running-requests 2 / 8, --mem-fraction-static 0.7, --swa-prefix-tails 128 / 256 and --prefill-decode-interval 16 / 192.
  • B300 "Agg DEP4" (C192), worker: the TP4 command plus --data-parallel-size 4 --enable-dp-attention --enable-dp-lm-head --enable-dp-attention-local-control-broadcast --load-balance-method total_requests --tokenizer-worker-num 8 --expert-parallel-size 4, with --chunked-prefill-size 32768 --max-running-requests 384 --mem-fraction-static 0.76 --swa-prefix-tails 3072 --prefill-decode-interval 16.
  • B300 "Agg DEP4", router: python3 -m sglang_router.launch_router --policy cache_aware --request-id-headers x-correlation-id --dp-aware --connect-timeout-secs 900 --request-timeout-secs 14400 --disable-health-check --disable-retries --worker-urls ... (sglang-router 0.3.2, as pinned in the master config). The TP2 and TP4 deployments have no router; the recipe's base uses frontend.type: sglang.
  • Environment, all three deployments: SGLANG_DEFAULT_THINKING=1, SGLANG_DSV41_REASONING_EFFORT=high, SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE=1, SGLANG_DSV41_ENGRAM_HOST_TABLE_LAYOUT=per_rank, SGLANG_TIMEOUT_KEEP_ALIVE=900.
  • I resolved the cookbook command for each of the 9 points and diffed it against the logged srt-slurm launch command, router command and worker environment in run 37816190252. They are identical except for the InferenceX-only settings below.
    • The pinned image auto-resolves quantization, KV-cache dtype and kernel backends. The logs show quantization=None, which falls back to the checkpoint's FP8/MXFP4 quantization_config, kv_cache_dtype=fp8_e4m3, attention_backend=dsv4 and moe_runner_backend=flashinfer_mxfp4. Neither the recipe nor the cookbook sets them.
    • The cookbook header's "8 GPUs" is the B300 node size. The commands run on 2 or 4 GPUs (--tensor-parallel-size), as the run did.
  • Excluded as InferenceX-only:
    • --enable-metrics on the worker and --prometheus-port/--prometheus-host on the router. They are the Prometheus export that AIPerf scrapes for result collection (AIPERF_REQUIRED_SERVER_METRIC_PREFIX=sglang:) and do not affect scheduling or kernels.
    • --host, --port, --nccl-port, --model-path /model, worker URLs, CUDA_VISIBLE_DEVICES and HEAD_NODE_IP. These are srt-slurm process placement and the local model mount; the cookbook uses the Hugging Face ID and placeholders.
    • PYTHONNOUSERSITE and PYTHONUNBUFFERED, which only set Python path isolation and log buffering in the container.
    • SGLANG_CUDA_COREDUMP_BEFORE_CRASH=0 and SGLANG_PYSPY_DUMP_BEFORE_CRASH=0. These srt-slurm crash-diagnostic toggles only control dumps after a crash.
    • SGLANG_SIMULATE_ACC_*, the synthetic acceptance control. The cookbook discloses it as "synthetic-acceptance".
    • The AIPerf client settings (CONC, AGENTIC_WARMUP_GRACE_PERIOD, AIPERF_*).
• PR validation: Run Sweep 37816190252 ran on head ec20ed5660994bc20a94d68a04488aa5bc2b33ea (attempt 1, success, full-sweep-fail-fast) and is pinned with /use 37816190252. Its bmk_agentic_*, eval_results_all and server-log artifacts are unexpired (they expire 2027-01-06).
  • All 9 dsv41flash-fp4-b300-sglang-agentic-dspark benchmark jobs succeeded: TP4/EP1 C1/C4, TP2/EP1 C1/C4/C16/C24/C32/C48, and TP4/EP4/DP4 with DP attention at C192 behind sglang-router 0.3.2. Every point recorded 0 error-dropped requests.
  • Both agentic eval / jobs succeeded.
  • The skipped jobs are the empty fixed-sequence and multi-node lanes, collect-results (which runs only for fixed-sequence or canary results), and the canary. The run was triggered by re-adding the full-sweep-fail-fast label, which skips reuse-sweep-gate, so canary-select was also skipped and all 9 points ran directly.
• Evals: gsm8k em_strict was 0.9735 at TP2 C48 (job) and 0.9682 at TP4/EP4/DP4 C192 (job), n=1319 each, from eval_results_all.
  • Both clear the default 0.90 threshold in infx/evals/thresholds.yaml, which has no dsv41flash override.
  • Both ran on the same image digest with real DSpark verification, with no SGLANG_SIMULATE_ACC_* set.
• Image: upstream lmsysorg/sglang:nightly-dev-cu13-20261004-295a53c9@sha256:7cfd4dd8e0e902d2a3c0560565963c4e172091e1586aaed15ff5a0a131394d34, the tag's multiarch index (amd64 and arm64), with SGLang 295a53c90ea4adb2588084dc0e3004cd3145e414. The config is framework: sglang, so the engine-ordering item is satisfied by this submission itself.
• Chat template: AgentX replay runs aiperf profile --scenario agentx with endpoint type chat (streaming). The server logs of all 9 points show only POST /v1/chat/completions and no /v1/completions or /generate requests.
  • The served revision has no Jinja chat template. The pinned SGLang resolves chat encoding dsv41 from the auto-detected deepseekv41 tool-call parser (chat_encoding.resolve_chat_encoding_spec) and renders every request with its built-in DeepSeek-V4.1 encoder, with thinking on and reasoning effort high.
• Draft precision: the served draft is the DSpark head bundled in deepseek-ai/DeepSeek-V4.1-Flash rev dba1be0a40aa45a94ad051997016db3960a90277 (mtp.0 to mtp.2, block size 5). The logs of every point verify model.revision: dba1be0a40aa.
  • All 48 safetensors shards at this revision have the same LFS object IDs as rev 2cba9e42aa026125f3ed06c6d98c1db82f7ca027, which only adds chat_template.jinja.
  • Shipped, from the safetensors headers of shards 44 to 46: the attention (wq_a, wq_b, wkv, wo_a, wo_b), shared-expert and main_proj weights are F8_E4M3 with F8_E8M0 32x32 block scales. The routed experts are MXFP4 (expert_dtype: fp4, packed as I8 with F8_E8M0 scales). Norms, the gate weight, markov_head and confidence_head are BF16. The checkpoint quantization_config has no exclusions.
  • Effective: the same, using the pinned image's default handling. deepseek_v4_dspark.py builds the draft with the target's quant config and wo_a_fp8=False, so it dequantizes wo_a from FP8 to BF16 at load (_dequant_fp8_wo_a_streaming), matching DeepSeek's reference. Routed experts run MXFP4 through flashinfer_mxfp4, and the KV cache is the default FP8 for both target and draft.
  • The logged server args of all 9 points show speculative_draft_model_quantization=None, speculative_draft_kv_cache_dtype=None, quantization=None, dtype=auto and speculative_draft_model_path=/model, which is the same checkpoint.
  • SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not set in the recipe, the launch path or the logged worker environment of any point. It defaults to EnvBool(False) in environ.py at the pinned SHA.
• Golden AL: infx/golden_al_distribution/dsv41flash_dspark.yaml gives 3.51 for thinking_on with draft length 5.
  • Every benchmark point logged SGLANG_SIMULATE_ACC_LEN=3.51, SGLANG_SIMULATE_ACC_METHOD=match-expected and SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token, with SGLANG_DEFAULT_THINKING=1.
  • The mean accept len in the decode logs is 3.509 to 3.511 on every point.
• MODELS.md: dsv41flash agentic coding is active, with SGLang DSpark arms per SKU.
• No architecture hacks: there is no --hf-overrides, and json_model_override_args='{}'.
  • The head commit removed enable-deepseek-v4-fp4-indexer, so the indexer uses the image's default path. The pinned run's logged commands confirm that the option is absent.
  • SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE=1 with per_rank layout only moves where Engram tables live; the data and computation are unchanged.
  • swa-prefix-tails sizes headroom in the SWA KV pool; the sliding window is unchanged.
  • publish_events_and_metrics is a Dynamo TRT-LLM srt-slurm field and does not apply to this SGLang path. enable-metrics is on.
• Code quality: the diff only changes this recipe and its master-config entry and appends one changelog entry.
  • The recipe's sizing comments match every point: TP admission is 2x CONC capped at 64, DP4 C192 admits 384, and SWA prefix tails are 64x per-rank CONC with a 128 floor.
  • The earlier review note was about DP4 keeping cuda-graph-max-bs-decode: 64. Per-rank admission is 96 (the log shows max_running_requests=96 per DP rank), but the per-rank decode batch never exceeded 44 in 7,917 decode log lines, all with cuda graph: True.
• No engine patches: the diff adds no patch files, git apply, heredoc rewrites, site-packages edits or wheel installs, and the pinned image runs as shipped. No waiver is needed.
• append-only: N/A, because the single new changelog entry does not use append-only: true.
• Pareto: one curve, dsv41flash / agentic-coding / b300_sglang / fp4 / run 37816190252, counted with infx.workflows.pareto_coverage (x = E2EL, y = total token throughput per GPU).
  • Source: the 9 bmk_agentic_* artifacts of the run (request_metrics.throughput.per_gpu.total_tput_tps and request_metrics.latency.e2el). This agentic-only run has no results_bmk artifact because collect-results did not run.
  • P90 E2EL frontier: 7 of 9 points (TP4 C4, TP2 C4/C16/C24/C32/C48 and DP4 C192).
  • P75 E2EL frontier: 7 of 9 points, the same set.
  • TP4 C1 and TP2 C1 are dominated by the C4 points.
  • Dashboard: https://inferencex.semianalysis.com/inference?unofficialRun=37816190252

Signed: nvpohanh

[github-actions]

✅✅✅ Verdict: PASS ✅✅✅

Passed and not applicable checks

✅ Check 0 (CODEOWNER): PASS — @nvpohanh is a specific owner of inferencex-e2e/configs/nvidia-master.yaml and of /inferencex-e2e/benchmarks/*_node/srt-slurm-recipes/**/b300-*/; perf-changelog.yaml falls only under the catch-all.

✅ Check 1 (Sweep on in-PR commit): PASS — Run 37816190252 ran on head ec20ed56 (still the PR tip), and all 9 agentic / and both agentic eval / jobs report success; this agentic-only config leaves the fixed-sequence single-node */ and eval / lanes empty and skipped.

✅ Check 2 (Evals): PASS — eval_results_all shows gsm8k em_strict 0.9735 (TP2 C48) and 0.9682 (TP4/EP4/DPA C192), n=1319 each, infrastructure_success: true, above the 0.90 default in thresholds.yaml (no dsv41flash override), on the same lmsysorg/sglang:nightly-dev-cu13-20261004-295a53c9@sha256:7cfd4dd8… image with EVAL_ONLY and real DSpark verification.

✅ Check 3 (Upstream recipe): PASS — sgl-project/sglang#43083 is MERGED (2026-10-09T05:55:37Z, 1fc6b896), and the B300 "Agg TP2", "Agg TP4" and "Agg DEP4" cells in deepseek-v4_1.jsx match the pinned recipe (image digest, base worker args, per-point MRR/mem-fraction/swa-prefix-tails/prefill-decode-interval, DP4 args, sglang-router command, worker environment); the only differences are the justified InferenceX-only settings (metrics export, process placement, Python path/buffering, crash dumps, synthetic acceptance, AIPerf client).

✅ Check 4 (Reuse command): PASS — /use 37816190252 was posted by nvpohanh (COLLABORATOR) on 2026-10-09.

✅ Check 5 (Latest template): PASS — Every item in the current PR_REVIEW_CHECKLIST.md template, including the recipe sub-item and the Pareto item, is present and checked.

✅ Check 6 (Upstream image / ordering): PASS — dsv41flash-fp4-b300-sglang-agentic-dspark is framework: sglang on an upstream digest-pinned lmsysorg/sglang image; as an open-source engine entry it is not subject to the ordering rule.

✅ Check 7 (Deprecated models): PASS — In MODELS.md, dsv41flash agentic coding is active (as of 2026-10-09), including the SGLang DSpark arms per SKU.

✅ Check 8 (Architecture hacks / event publication): PASS — There is no --hf-overrides, model-config edit or layer/expert trimming, the Engram host table and swa-prefix-tails only change memory placement, and publish_events_and_metrics exists only in Dynamo TRT-LLM srt-slurm recipes, not on this SGLang path.

✅ Check 9 (Spec-decode chat template): PASS — AgentX replay uses aiperf profile --scenario agentx (infx/bench/agentic/replay.py), whose scenario sets the chat-completions endpoint; evals use local-chat-completions with --apply_chat_template.

✅ Check 10 (Engine patches): PASS — The diff adds no patch files, git apply, heredoc rewrites, site-packages edits or wheel installs.

✅ Check 11 (Golden AL): PASS — synthetic_acceptance.py derives the AL from speculative-dspark-block-size: 5, giving 3.51 in dsv41flash_dspark.yaml (thinking_on, length 5), and the C192 benchmark job log shows THINKING_MODE: thinking_on, SGLANG_SIMULATE_ACC_LEN=3.51, match-expected and real-draft-token.

➖ Check 12 (Append-only): N/A — The single new changelog entry does not set append-only: true.

✅ Check 13 (Draft as shipped): PASS — The draft is the DSpark head bundled in the served deepseek-ai/DeepSeek-V4.1-Flash checkpoint (revision dba1be0a, no --speculative-draft-model-path), and its precision matches the shipped baseline: the recipe, shared launchers and job logs have no draft-quantization, dtype or KV-dtype override, SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is unset on this path and defaults to EnvBool(False) in environ.py at the pinned SGLang 295a53c9, and the sign-off records the image's default wo_a FP8→BF16 load as unchanged.

✅ Check 14 (Pareto coverage): PASS — dsv41flash / agentic-coding / b300_sglang / fp4 / run 37816190252 has 7 frontier points at P90 E2EL and 7 at P75 E2EL from the 9 bmk_agentic_* artifacts (total_tput_tps vs e2el, 0 error-dropped records, only TP4 C1 and TP2 C1 dominated); the count was computed by hand with the pareto_coverage helper's upper_right rule because the helper could not run in this sandbox.

✅ Check 15 (PR description): PASS — The body's claims match the code at ec20ed56: the 20261004 digest-pinned image, the TP4/EP1, TP2/EP1 and aggregated DP4 point set, sglang-router with cache-aware and correlation-ID routing, single-node aggregated serving, and no runner changes.

Assessed commit: ec20ed5660994bc20a94d68a04488aa5bc2b33ea.

[functionstackx]

/reuse-sweep-run 37816190252