docs(cookbook): add Qwen3.8 GB300 long-context recipe

docs/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next.mdx

@@ -218,3 +218,45 @@ Pending update — a sample transcript will be added here once the weights are p
 ```
 
 </Accordion>
+
+### 2.3 GB300 long-context serving with HiCache and MTP
+
+Use this single-GPU, aggregated TP1/EP1 configuration for long-context, multi-turn workloads on GB300. It combines the NVIDIA NVFP4 checkpoint's embedded MTP head, CPU DRAM KV offload through HiCache, and pinned-memory PLE embedding offload.
+
+This serving configuration comes from [InferenceX #3816](https://github.com/SemiAnalysisAI/InferenceX/pull/3816). [Run 37714835620, attempt 1](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/37714835620) passed concurrency 1, 2, 4, 8, 16, 32, 48 and 64 on commit `e67ca6fc9ecf4cd57f27654bda8a3e9059923820`. Its [matching GSM8K eval](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/37714835620/job/113109083376) used real verification and scored 97.35% on 1,319 questions.
+
+The throughput sweep used InferenceX's simulated acceptance; the generated command uses normal MTP verification and does not claim those throughput numbers for production traffic.
+
+#### Prerequisites and memory budget
+
+- Use a Linux ARM64 GB300 host with Docker, NVIDIA Container Toolkit, and a driver compatible with the image's CUDA 13.0 runtime. See [Docker installation](/docs/get-started/install#method-3-using-docker).
+- Reserve one GB300 GPU. The recipe keeps TP, PP, DP and EP at 1.
+- Budget 224 GiB of CPU memory for the combined target/draft KV and Mamba HiCache host pool, plus approximately 48 GiB for the offloaded FP8 PLE table and additional headroom for checkpoint loading and host processes. `--hicache-size` sets the combined cache budget; it is separate from PLE weight offload. Keep sufficient free host memory for both.
+- The Docker command mounts your Hugging Face cache to retain downloaded model files across restarts. The image digest and both target/draft checkpoint revisions are pinned to the validated configuration.
+
+#### Generate the launch command
+
+In the [Deployment panel](#deployment), select **GB300 → NVFP4 (NVDA) → Low Latency → Single Node**, and leave **PLE Offload** at **Auto** or **On**. The matched cell includes all five serving environment variables and the full argument set, including ReplaySSM, HiCache and pinned PLE offload. Choose **Docker** to use the pinned image, or **Python** for a matching local installation. Changing PLE Offload to **Off** produces an unverified configuration.
+
+The Docker wrapper comes from the shared command builder and is adapted from the validated Slurm/Enroot launch; it has not been run separately on a GB300 host. Use the command generated by the panel rather than a separate static copy.
+
+<Accordion title="Reference startup output">
+
+The validated run loaded the target and embedded draft through the stock image's mixed-precision loader. This excerpt is from the real-verification eval; timings and memory availability vary by host.
+
+```text Output
+[2026-10-08 02:22:36] Load weight end. elapsed=1011.56 s, type=Qwen4ExpForConditionalGeneration, quant=modelopt_mixed, quant_algo=MIXED_PRECISION, avail mem=202.03 GB, mem usage=73.64 GB.
+[2026-10-08 02:23:03] Load weight end. elapsed=25.97 s, type=Qwen4ExpForCausalLMMTP, quant=modelopt_mixed, quant_algo=MIXED_PRECISION, avail mem=197.09 GB, mem usage=4.89 GB.
+```
+
+</Accordion>
+
+#### Serving choices
+
+- **Draft precision:** revision `fc694b54` stores the MTP routed experts in block-scaled FP8 and the rest of the head in BF16. The pinned image, built from SGLang `e00930c5`, preserves this per-layer `modelopt_mixed` configuration. The draft uses the same checkpoint revision as the target. `SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0` disables the optional draft precision conversion path; target and draft KV caches stay in BF16.
+- **Hybrid attention:** the recipe selects FlashInfer for linear-attention prefill/decode, Triton for linear verification, and `trtllm_mha` as the attention backend. ReplaySSM speculation, `extra_buffer_lazy`, a tracking interval of 256 and 480 Mamba cache slots retain the qualified state-cache configuration.
+- **Admission and graphs:** the server admits up to 96 requests, uses 16,384-token prefill chunks, and captures full decode graphs at the listed batch sizes up to 96. Prefill graphs remain disabled. The qualification varied client concurrency while retaining these server settings.
+- **CPU offload:** HiCache uses the `kernel` I/O backend, `page_first` layout and `write_back` policy. PLE uses a separate pinned-memory embedding table. See [HiCache design](/docs/advanced_features/hicache_design) for the cache tiers and policies.
+- **API and telemetry:** send requests through the chat-completions API at `http://localhost:30000/v1`, using model name `nvidia/Qwen3.8-Flash-Next-NVFP4`. `--reasoning-parser auto` separates reasoning from answer content. Engine metrics and cache reporting remain enabled.
+
+The generated command omits arguments that match the pinned image's defaults, including TP/PP/DP/EP=1, the inherited served-model name and draft MoE backend, full decode graphs, and default Mamba tracking, HiCache layout, PLE backend, tokenizer and scheduler settings. It retains the qualified lane's non-default serving settings. InferenceX-only trace loading, simulated-acceptance controls, client concurrency overrides, result collection and Slurm allocation settings are excluded.

docs/src/snippets/configs/Qwen/qwen3.8-flash-next.jsx

@@ -103,16 +103,22 @@ export const config = {
           disableReason: (sel) => sel.hw === "rtx6000"
             ? "RTX PRO 6000 (96 GB) only fits this checkpoint with the 47.7 GiB FP8 N-gram table in pinned host RAM; the verified cells pass --ple-offload-embedding explicitly, so On is the only pick."
             : "DGX Spark is unified memory: PLE offload to RAM frees nothing (the pinned table shares the 128 GB pool with the weights). The verified settings are Off for the two-node cells and On (NVMe file) for a single Spark.",
-          hints: ["PLE Offload: auto-enabled for BF16 on CUDA, off otherwise"] },
+          hints: (sel) => sel.hw === "gb300" && sel.quant === "nvfp4-nvda" && sel.strategy === "low-latency"
+            ? ["PLE Offload: enabled in the GB300 base recipe; reserve approximately 48 GiB of host RAM in addition to the 224 GiB HiCache pool."]
+            : ["PLE Offload: auto-enabled for BF16 on CUDA, off otherwise"] },
         { id: "on",   label: "On",
           disabled: (sel) => sel.hw === "dgx-spark",
           disableReason: "DGX Spark is unified memory: PLE offload to RAM frees nothing (the pinned table shares the 128 GB pool with the weights). The verified settings are Off for the two-node cells and On (NVMe file) for a single Spark.",
+          stripPrefixes: (sel) => sel.hw === "gb300" && sel.quant === "nvfp4-nvda" && sel.strategy === "low-latency"
+            ? ["--ple-offload-embedding"] : [],
           flags: ["--ple-offload-embedding"] },
         { id: "off",  label: "Off",
           disabled: (sel) => sel.hw === "rtx6000" || (sel.hw === "dgx-spark" && sel.nodes === "single"),
           disableReason: (sel) => sel.hw === "dgx-spark"
             ? "A single DGX Spark cannot hold the 126 GiB checkpoint in its 128 GB of unified memory; the verified single-Spark cells keep the 47.7 GiB FP8 N-gram table in a file on the local NVMe (On (NVMe file))."
             : "RTX PRO 6000 (96 GB) cannot hold the 47.7 GiB FP8 N-gram table alongside the other 78 GiB of the checkpoint; the table must be offloaded to pinned host RAM (On).",
+          stripPrefixes: (sel) => sel.hw === "gb300" && sel.quant === "nvfp4-nvda" && sel.strategy === "low-latency"
+            ? ["--ple-offload-embedding", "--ple-offload-backend"] : [],
           flags: ["--no-ple-offload-embedding"] },
         // File-backed table (sgl-project/sglang#37068, merged into qwen4-main-squashed):
         // a sparse 47.7 GiB file under $SGLANG_CACHE_DIR/ple/<model> (override
@@ -206,6 +212,7 @@ export const config = {
     "b300|nvfp4-nvda": "lmsysorg/sglang:latest",
     gb300:  "lmsysorg/sglang:qwen38flashnext",
     "gb300|nvfp4-nvda": "lmsysorg/sglang:latest",
+    "gb300|nvfp4-nvda|low-latency": "lmsysorg/sglang:v0.5.21-cu130@sha256:1f176b3a1dd1205d2a462ab80ea9fc17769a32e53cb6f3c690efa2dc2c384977",
     mi350x: "lmsysorg/sglang-rocm:qwen38flashnext",
     mi355x: "lmsysorg/sglang-rocm:qwen38flashnext",
   },
@@ -803,20 +810,60 @@ export const config = {
     },
     {
       match: { hw: "gb300", variant: "default", quant: "nvfp4-nvda", strategy: "low-latency", nodes: "single" },
-      verified: false,
-      warn: "Requires SGLang v0.5.20 or later for this NVIDIA ModelOpt MIXED_PRECISION export. Quantization and MoE backends are selected automatically from the checkpoint.",
-      env: [],
+      verified: true,
+      verificationStatus: (sel) => sel.pleOffload === "off" ? "unverified" : "verified",
+      warn: "Long-context recipe: reserve 224 GiB of host RAM for HiCache, approximately 48 GiB for pinned PLE offload, and loading headroom. See [GB300 configuration notes](#23-gb300-long-context-serving-with-hicache-and-mtp). The Docker wrapper is adapted from the qualified Slurm/Enroot launch.",
+      env: [
+        "PYTHONNOUSERSITE=1",
+        "PYTHONUNBUFFERED=1",
+        "NO_COLOR=1",
+        "SGLANG_TIMEOUT_KEEP_ALIVE=1800",
+        "SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0",
+      ],
+      // Omit defaults from the pinned v0.5.21 build: TP/PP/DP/EP=1,
+      // model-name and draft-MoE inheritance, full decode graphs, and
+      // the standard Mamba, HiCache, PLE, tokenizer and scheduler settings.
       flags: [
         "--model-path {{MODEL_NAME}}",
-        "--tp 1",
+        "--moe-runner-backend flashinfer_trtllm",
+        "--disable-shared-experts-fusion",
+        "--revision fc694b54fb0174e0913e6adf86691ef85a4ead47",
+        "--speculative-draft-model-revision fc694b54fb0174e0913e6adf86691ef85a4ead47",
+        "--trust-remote-code",
+        "--quantization modelopt_mixed",
+        "--context-length 262144",
+        "--attention-backend trtllm_mha",
         "--linear-attn-prefill-backend flashinfer",
         "--linear-attn-decode-backend flashinfer",
+        "--linear-attn-verify-backend triton",
         "--mamba-ssm-dtype bfloat16",
+        "--mamba-radix-cache-strategy extra_buffer_lazy",
+        "--max-mamba-cache-size 480",
+        "--mem-fraction-static 0.88",
+        "--max-running-requests 96",
+        "--page-size 64",
+        "--chunked-prefill-size 16384",
         "--speculative-algorithm NEXTN",
         "--speculative-num-steps 3",
         "--speculative-eagle-topk 1",
         "--speculative-num-draft-tokens 4",
+        "--enable-linear-replayssm-spec",
+        "--cuda-graph-backend-prefill disabled",
+        "--cuda-graph-max-bs-decode 96",
+        "--cuda-graph-bs-decode 1 2 4 8 16 24 32 40 48 56 64 72 80 88 96",
+        "--kv-cache-dtype bfloat16",
+        "--speculative-draft-kv-cache-dtype bfloat16",
+        "--enable-hierarchical-cache",
+        "--hicache-size 224",
+        "--hicache-write-policy write_back",
+        "--ple-offload-embedding",
+        "--enable-metrics",
+        "--enable-cache-report",
         "--reasoning-parser auto",
+        "--decode-log-interval 10",
+        "--stream-interval 20",
+        "--watchdog-timeout 1800",
+        "--random-seed 35",
         "--host {{HOST_IP}}",
         "--port {{PORT}}",
       ],