feat(agentx): add native Kimi-K3 PD recipes / feat(agentx): 新增原生 Kimi-K3 PD 配置 (#3582)

* feat(launch): support read-only Kimi-K3 assets

为 Kimi-K3 PD 声明模型资源与只读 draft/provider 挂载；保留现有可写缓存目录的创建行为，不改共享镜像准备和任务生命周期。

* feat(config): add native Kimi-K3 PD recipes

Add topology-named native variants with a digest-pinned official worker image, role-owned serving settings and default evaluation selection. Preserve upstream request-ID randomization and keep benchmark and cluster configuration separate.

新增按拓扑命名的原生配置变体，使用 digest 固定的官方 worker 镜像、按角色管理的 serving 参数及默认精度评测；遵循上游 request-ID 随机化，将 benchmark 和集群配置分层。

* fix(srt-slurm): backport discovery connector template binding

Carry NVIDIA/srt-slurm #508 through the existing patches directory at the shared submodule API. Exercise real job-local patch application and conflict handling, then run the upstream P/D command-rendering regressions in an isolated checkout. Remove the patch once the shared dependency includes it.

沿用既有 patches 目录回补 NVIDIA/srt-slurm #508，与共享 submodule API 保持一致。验证任务私有 checkout 的补丁应用和冲突退出，再在隔离进程中执行上游 P/D 命令渲染回归；共享依赖包含该修复后删除补丁。

* refactor: use native asset mounts for Kimi-K3 recipes

Declare draft checkpoint and Ionic provider mounts through existing recipe and cluster configuration. Keep shared launch code unchanged, carry only the #508 runtime backend fix, and record recipe details in YAML and the changelog.

中文：通过现有 recipe 和集群配置声明 draft checkpoint 与 Ionic provider 挂载。共享 launch 代码保持不变，仅携带 #508 的运行时 backend 修复，将 recipe 信息统一记录在 YAML 和 changelog。

* perf(config): tune and simplify Kimi-K3 PD recipe settings

Enable INT4 QuickReduce with a 256 MiB limit and FlyDSL stage-2 FP8 on both roles. Use default nonblocking waits and breakable-graph settings. Scope draft and Ionic mount paths to the recipe. Configure 1P1D c48/c56, 1P2D c24/c48 and 1P3D c12/c24 with prefill-only CPU offload and point-specific decode graph coverage. Update the existing performance changelog entry.

中文：在 P/D 两侧启用 INT4 QuickReduce、256 MiB 大小上限和 FlyDSL stage-2 FP8，使用默认非阻塞等待与 breakable graph 设置。将 draft 和 Ionic 挂载路径收束到 recipe。配置 1P1D c48/c56、1P2D c24/c48 和 1P3D c12/c24，使用仅 prefill 侧 CPU offload 与各点对应的 decode graph 覆盖，并更新当前 PR 的性能变更记录。

* fix(config): use default AgentX failure threshold for Kimi-K3 PD

Remove the recipe's live failure threshold override so all Kimi-K3 PD variants use the pinned AgentX scenario's 10% default. Keep the final result threshold, both roles' stream interval, and optional server-metrics validation unchanged.

移除 Kimi-K3 PD 配方的运行中错误率覆盖，所有变体沿用固定版本 AgentX 场景的默认 10% 门槛。最终结果校验门槛、P/D 流式输出间隔及可选 server-metrics 校验保持不变。

* fix(srt): clarify discovery patch scope

说明 #508 仅修复 srt-slurm 启动器的 discovery 配置渲染，不修改推理引擎、kernel、模型计算或镜像。补丁应用后的代码保持不变。

* fix(kimik3): migrate PD recipe to srt-recipe selection and AgentX fragments

Main now selects srt-slurm variants with srt-recipe-dir/srt-recipe and binds
model, image, precision, client env and concurrencies from the master row, so
the recipe drops those keys and the shared agentic.yaml supplies the client
command.

---------

Co-authored-by: Cam Quilici <cjquilici@gmail.com>
Co-authored-by: billishyahao <bill.he@amd.com>

Author: 3 people

inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/kimik3/vllm/mi355x-fp4/agentx/disagg-variants.yaml

@@ -0,0 +1,213 @@
+# Kimi-K3 AgentX with TP8 workers, MoRIIO READ and prefill-only CPU offload.
+base:
+  schema: 2
+  name: kimik3-fp4-mi355x-vllm-disagg-agentic
+  slurm:
+    time_limit: '08:00:00'
+  sbatch_directives:
+    mem: '0'
+  srun_options:
+    container-remap-root: ''
+    container-writable: ''
+    mem: '0'
+  container_mounts:
+    /dev/infiniband: /dev/infiniband
+  extra_mount:
+    - /it-share/data/Inferact-Kimi-K3-DSpark:/models/Inferact-Kimi-K3-DSpark
+    # Match the host's Ionic provider without replacing the image's verbs core.
+    - /usr/lib/x86_64-linux-gnu/libibverbs/libionic-rdmav34.so:/usr/lib/x86_64-linux-gnu/libibverbs/libionic-rdmav34.so:ro
+  resources:
+    gpu_type: mi355x
+    gpus_per_node: 8
+  frontend:
+    type: vllm-router
+    enable_multiple_frontends: false
+    placement:
+      node: head
+    # Native Pyxis/Enroot digest reference; imported in the frontend's job step.
+    container_image: registry-1.docker.io#vllm/vllm-router:sha256:1fbf06701abce8c8cd459414e472d63999c05b201baffc62e00727c10583f5ea
+    args:
+      policy: consistent_hash
+      prefill-policy: consistent_hash
+      decode-policy: consistent_hash
+      log-level: info
+  observability:
+    enabled: false
+    tachometer:
+      enabled: false
+  health_check:
+    interval_seconds: 10
+    # Match the engine's 7200-second weight-loading budget.
+    max_attempts: 720
+  engine:
+    type: vllm
+    connector: moriio
+  roles:
+    prefill:
+      nodes: 1
+      workers: 1
+      gpus: 8
+      args: &common_args
+        served-model-name: moonshotai/Kimi-K3
+        trust-remote-code: true
+        tensor-parallel-size: 8
+        decode-context-parallel-size: 1
+        cp-kv-cache-interleave-size: 1
+        dcp-comm-backend: a2a
+        moe-backend: auto
+        load-format: fastsafetensors
+        gpu-memory-utilization: 0.90
+        language-model-only: true
+        enable-auto-tool-choice: true
+        tool-call-parser: kimi_k3
+        reasoning-parser: kimi_k3
+        max-model-len: 1048576
+        stream-interval: 10
+        enable-prefix-caching: true
+        prefix-match-unit: 128
+        kv-cache-dtype: fp8
+        mamba-ssm-cache-dtype: float32
+        block-size: 128
+        attention-backend: ROCM_AITER_MLA
+        attention-config: '{"mla_prefill_backend":"ROCM_AITER_FA","use_prefill_query_quantization":true}'
+        # The native srt driver supplies the measured AL for the selected DSpark
+        # draft length in throughput runs and keeps real block rejection for eval.
+        speculative-config: '{"model":"/models/Inferact-Kimi-K3-DSpark","method":"dspark","num_speculative_tokens":4,"attention_backend":"ROCM_AITER_MLA","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}'
+        kv-transfer-config: '{"kv_connector":"MultiConnector","kv_role":"kv_both","kv_load_failure_policy":"fail","kv_connector_extra_config":{"connectors":[{"kv_connector":"MoRIIOConnector","kv_role":"kv_producer","kv_load_failure_policy":"fail","kv_connector_extra_config":{"backend":"rdma"}},{"kv_connector":"SimpleCPUOffloadConnector","kv_role":"kv_both","kv_connector_extra_config":{"cpu_bytes_to_use":1799000000000,"lazy_offload":false}}]}}'
+      env: &common_env
+        NCCL_IB_HCA: rdma0,rdma1,rdma2,rdma3,rdma4,rdma5,rdma6,rdma7
+        NCCL_SOCKET_IFNAME: eno0
+        GLOO_SOCKET_IFNAME: eno0
+        # Use the nightly's default SiTUv2 activation selection.
+        VLLM_ROCM_USE_AITER: '1'
+        AITER_BF16_FP8_MOE_BOUND: '0'
+        AITER_FLYDSL_STAGE2_FP8: '1'
+        VLLM_ROCM_AITER_MLA_ASM_PADDING: asm
+        VLLM_ROCM_AITER_MLA_DCP_VERIFY: asm
+        VLLM_ROCM_QUICK_REDUCE_QUANTIZATION: INT4
+        VLLM_ROCM_QUICK_REDUCE_MAX_SIZE_BYTES_MB: '256'
+        SAFETENSORS_FAST_GPU: '1'
+        VLLM_SSM_CONV_STATE_LAYOUT: DS
+        VLLM_KV_CACHE_LAYOUT: HND
+        NCCL_DMABUF_ENABLE: '0'
+        HSA_ENABLE_IPC_MODE_LEGACY: '1'
+        HIP_FORCE_DEV_KERNARG: '1'
+        PYTHONHASHSEED: '42'
+        # Keep DCP collectives on the graph-compatible RCCL path.
+        VLLM_USE_DIRECT_DCP_A2A: '0'
+        VLLM_USE_DIRECT_DCP_Q_GATHER: '0'
+        VLLM_USE_DIRECT_DCP_KV_GATHER: '0'
+        VLLM_ENGINE_READY_TIMEOUT_S: '7200'
+        VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS: '1200'
+        PYTHONNOUSERSITE: '1'
+        VLLM_SERVER_DEV_MODE: '0'
+        # Bound RDMA queue resources and the wait for available send slots.
+        MORI_IO_SQ_BACKOFF_TIMEOUT_US: '50000'
+        MORI_IO_QP_MAX_SEND_WR: '16384'
+        MORI_IO_QP_MAX_CQE: '32768'
+        MORI_IO_QP_MAX_SGE: '2'
+        MORI_IO_TC_DISABLE: '0'
+    decode:
+      nodes: 1
+      workers: 1
+      gpus: 8
+      args:
+        <<: *common_args
+        kv-transfer-config: '{"kv_connector":"MoRIIOConnector","kv_role":"kv_consumer","kv_load_failure_policy":"fail","kv_connector_extra_config":{"backend":"rdma"}}'
+      env: *common_env
+  benchmark:
+    placement:
+      node: head
+
+override_1p3d_c12:
+  roles:
+    prefill:
+      args:
+        max-num-seqs: 24
+        max-num-batched-tokens: 8192
+        compilation-config: '{"mode":3,"cudagraph_mode":"NONE","max_cudagraph_capture_size":0,"custom_ops":["+fused_rms_norm_gated","+quant_fp8","+grouped_topk","+sparse_attn_indexer","none"]}'
+      env:
+        HSA_NO_SCRATCH_RECLAIM: '0'
+    decode:
+      nodes: 3
+      workers: 3
+      args:
+        max-num-seqs: 8
+        max-num-batched-tokens: 512
+        compilation-config: '{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":40,"cudagraph_capture_sizes":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],"custom_ops":["+fused_rms_norm_gated","+quant_fp8","+grouped_topk","+sparse_attn_indexer","none"]}'
+
+override_1p2d_c24: &tp8_c24
+  roles: &tp8_c24_roles
+    prefill:
+      args:
+        decode-context-parallel-size: 8
+        max-num-seqs: 48
+        max-num-batched-tokens: 8192
+        compilation-config: '{"mode":3,"cudagraph_mode":"NONE","max_cudagraph_capture_size":0,"custom_ops":["+fused_rms_norm_gated","+quant_fp8","+grouped_topk","+sparse_attn_indexer","none"]}'
+    decode: &tp8_c24_decode
+      nodes: 2
+      workers: 2
+      args: &tp8_c24_decode_args
+        decode-context-parallel-size: 8
+        max-num-seqs: 24
+        max-num-batched-tokens: 512
+        compilation-config: '{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":120,"cudagraph_capture_sizes":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,70,80,90,100,110,120],"custom_ops":["+fused_rms_norm_gated","+quant_fp8","+grouped_topk","+sparse_attn_indexer","none"]}'
+
+# Keep the c24 serving policy; scale only the decoder replicas and per-D limit.
+override_1p3d_c24:
+  <<: *tp8_c24
+  roles:
+    <<: *tp8_c24_roles
+    decode:
+      <<: *tp8_c24_decode
+      nodes: 3
+      workers: 3
+      args:
+        <<: *tp8_c24_decode_args
+        max-num-seqs: 16
+
+override_1p1d_c48: &tp8_c48
+  roles: &tp8_c48_roles
+    prefill: &tp8_c48_prefill
+      args: &tp8_c48_prefill_args
+        decode-context-parallel-size: 8
+        max-num-seqs: 96
+        max-num-batched-tokens: 8192
+        compilation-config: '{"mode":3,"cudagraph_mode":"NONE","max_cudagraph_capture_size":0,"custom_ops":["+fused_rms_norm_gated","+quant_fp8","+grouped_topk","+sparse_attn_indexer","none"]}'
+      env:
+        # Short-tail segmented MLA can require ~4.1 GB of non-reclaimable scratch.
+        HSA_NO_SCRATCH_RECLAIM: '0'
+    decode: &tp8_c48_decode
+      args: &tp8_c48_decode_args
+        decode-context-parallel-size: 8
+        max-num-seqs: 96
+        max-num-batched-tokens: 512
+        compilation-config: '{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":240,"cudagraph_capture_sizes":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,130,140,150,160,170,180,190,200,210,220,230,240],"custom_ops":["+fused_rms_norm_gated","+quant_fp8","+grouped_topk","+sparse_attn_indexer","none"]}'
+
+override_1p1d_c56:
+  <<: *tp8_c48
+  roles:
+    <<: *tp8_c48_roles
+    prefill:
+      <<: *tp8_c48_prefill
+      args:
+        <<: *tp8_c48_prefill_args
+        max-num-seqs: 112
+    decode:
+      <<: *tp8_c48_decode
+      args:
+        <<: *tp8_c48_decode_args
+        max-num-seqs: 112
+        compilation-config: '{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":280,"cudagraph_capture_sizes":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,130,140,150,160,170,180,190,200,210,220,230,240,250,260,270,280],"custom_ops":["+fused_rms_norm_gated","+quant_fp8","+grouped_topk","+sparse_attn_indexer","none"]}'
+
+override_1p2d_c48:
+  <<: *tp8_c48
+  roles:
+    <<: *tp8_c48_roles
+    decode:
+      <<: *tp8_c48_decode
+      nodes: 2
+      workers: 2
+      args:
+        <<: *tp8_c48_decode_args
+        max-num-seqs: 48

inferencex-e2e/configs/amd-master.yaml

@@ -1601,3 +1601,107 @@ dsv41flash-fp4-mi355x-sglang-agentic-dspark:
       - { tp: 2, ep: 1, dp-attn: false, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16], srt-recipe: agentic.yaml }
       - { tp: 4, ep: 1, dp-attn: false, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16], srt-recipe: agentic.yaml }
       - { tp: 2, ep: 1, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [32, 64], srt-recipe: agentic.yaml }
+
+# Native Kimi-K3 PD with TP8 workers and MoRIIO READ.
+kimik3-fp4-mi355x-vllm-disagg-agentic:
+  image: vllm/vllm-openai-rocm:nightly-81198e97ba7eee2a22540caaa756b7fdddcb4d93@sha256:a401e4f46872dcde77e07ae3d7fa2d3a71b7f2773714316116532e2b39deb179
+  model: moonshotai/Kimi-K3
+  model-prefix: kimik3
+  runner: cluster:mi355x-amds
+  precision: fp4
+  framework: vllm-disagg
+  router: { name: vllm-router, version: nightly-20260913-83944c4 }
+  kv-p2p-transfer: moriio
+  multinode: true
+  disagg: true
+  srt-recipe-dir: kimik3/vllm/mi355x-fp4/agentx
+  scenarios:
+    agentic-coding:
+    - dram-utilization: 0.6
+      search-space:
+      - spec-decoding: mtp
+        conc-list: [12]
+        kv-offloading: dram
+        kv-offload-backend: { name: vllm-simple }
+        srt-recipe: disagg-variants.yaml:override_1p3d_c12
+        prefill:
+          num-worker: 1
+          tp: 8
+          dcp-size: 1
+          ep: 1
+          dp-attn: false
+          additional-settings:
+          - TOTAL_CPU_DRAM_GB=1799
+        decode: { num-worker: 3, tp: 8, dcp-size: 1, ep: 1, dp-attn: false }
+      - spec-decoding: mtp
+        conc-list: [24]
+        kv-offloading: dram
+        kv-offload-backend: { name: vllm-simple }
+        srt-recipe: disagg-variants.yaml:override_1p3d_c24
+        prefill:
+          num-worker: 1
+          tp: 8
+          dcp-size: 8
+          ep: 1
+          dp-attn: false
+          additional-settings:
+          - TOTAL_CPU_DRAM_GB=1799
+        decode: { num-worker: 3, tp: 8, dcp-size: 8, ep: 1, dp-attn: false }
+      - spec-decoding: mtp
+        conc-list: [24]
+        kv-offloading: dram
+        kv-offload-backend: { name: vllm-simple }
+        srt-recipe: disagg-variants.yaml:override_1p2d_c24
+        prefill:
+          num-worker: 1
+          tp: 8
+          dcp-size: 8
+          ep: 1
+          dp-attn: false
+          additional-settings:
+          - TOTAL_CPU_DRAM_GB=1799
+        decode: { num-worker: 2, tp: 8, dcp-size: 8, ep: 1, dp-attn: false }
+      - spec-decoding: mtp
+        conc-list: [48]
+        kv-offloading: dram
+        kv-offload-backend: { name: vllm-simple }
+        srt-recipe: disagg-variants.yaml:override_1p1d_c48
+        prefill:
+          num-worker: 1
+          tp: 8
+          dcp-size: 8
+          ep: 1
+          dp-attn: false
+          additional-settings:
+          # Recipe enables P-only scratch reclaim; D and GMU0.90 are unchanged.
+          - TOTAL_CPU_DRAM_GB=1799
+        decode: { num-worker: 1, tp: 8, dcp-size: 8, ep: 1, dp-attn: false }
+      - spec-decoding: mtp
+        conc-list: [56]
+        kv-offloading: dram
+        kv-offload-backend: { name: vllm-simple }
+        srt-recipe: disagg-variants.yaml:override_1p1d_c56
+        prefill:
+          num-worker: 1
+          tp: 8
+          dcp-size: 8
+          ep: 1
+          dp-attn: false
+          additional-settings:
+          - TOTAL_CPU_DRAM_GB=1799
+        decode: { num-worker: 1, tp: 8, dcp-size: 8, ep: 1, dp-attn: false }
+      - spec-decoding: mtp
+        conc-list: [48]
+        kv-offloading: dram
+        kv-offload-backend: { name: vllm-simple }
+        srt-recipe: disagg-variants.yaml:override_1p2d_c48
+        prefill:
+          num-worker: 1
+          tp: 8
+          dcp-size: 8
+          ep: 1
+          dp-attn: false
+          additional-settings:
+          # Inherits the same P-only scratch-reclaim setting as 1P1D c48.
+          - TOTAL_CPU_DRAM_GB=1799
+        decode: { num-worker: 2, tp: 8, dcp-size: 8, ep: 1, dp-attn: false }

inferencex-e2e/configs/runners.yaml

@@ -874,6 +874,7 @@ clusters:
       MORI_RDMA_TC: '@fabric.rdma-tc'
     models:
       entries:
+        Kimi-K3: {root: it-share-data, dir: Kimi-K3}
         DeepSeek-R1-0528-MXFP4-v2: {root: it-share-data, dir: DeepSeek-R1-0528-MXFP4-v2}
         DeepSeek-V4-Pro-0813: {root: it-share-data, dir: DeepSeek-V4-Pro-0813}
         GLM-5.3: {root: it-share-data, dir: GLM-5.3}

inferencex-e2e/perf-changelog.yaml

@@ -9705,3 +9705,16 @@
   - Pin minimaxm3 B300 FP4 vllm mtp AgentX to the 2026-10-13 image vllm/vllm-openai:nightly-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e5c9e6bb82f645e8fca04e4d395743ee5ed473a62b98942d65f83c35d0c69301.
   - Preserve deployment topology, speculative mode, concurrency points, KV-offload settings and evaluation coverage.
   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4000
+
+- config-keys:
+    - kimik3-fp4-mi355x-vllm-disagg-agentic
+  scenario-type:
+    - agentic-coding
+  description:
+    - "Add native Kimi-K3 PD with FP8 KV and GMU 0.90 on official ROCm nightly 81198e97ba7eee2a22540caaa756b7fdddcb4d93, pinned by digest in master and recipe."
+    - "Use TP8 per worker: 1P1D c48/c56, 1P2D c24/c48 and 1P3D c12/c24, with automatic golden acceptance, default request-ID randomization and default evaluation selection."
+    - "Use the AgentX scenario's default 10% live failure threshold, retaining a stream interval of 10 on both roles and optional server-metrics validation."
+    - "Set the prefill sequence limit to twice the global concurrency and divide that capacity across decode workers; 1P1D c56 uses 112 sequences per role with a decode graph capture limit of 280."
+    - "Enable INT4 QuickReduce with a 256 MiB size limit and FlyDSL stage-2 FP8 on prefill and decode workers."
+    - "Configure MoRIIO READ and prefill-only SimpleCPUOffloadConnector through native role arguments. Use recipe-scoped draft checkpoint and RDMA device/provider mounts. Use the upstream discovery-template fix from srt-slurm #508 through the shared v2.51.1 dependency."
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3582