[Docs][ROCm] Proposal: a shared kernel substrate for persistent mono decode kernels

[Fangzhou-Ai]

Overview

A draft design note, doc only, stacked on dsv4-pro-mono (vllm-project#61482): a shared kernel substrate for the persistent ("mono") decode kernels. The note is docs/design/mono_kernel_substrate.md. It is meant for discussion in this fork first; it does not touch any code.

Related: RFC vllm-project#60904, vllm-project#61311 (the runtime-layer draft), vllm-project#60397, vllm-project#61482.

What it argues

• Five codebases (DeepSeek-V4.x, Kimi-K3 with Qwen3.8 on top, GLM-5.x, GLM-5.3-Flash, MiniMax-M3) each carry 0.5-1.3k lines of the same plumbing: a tagged-pair mailbox with an epoch tag, a peer-memory buffer, a build/JIT key, CTA placement, device helpers and a rank-ordered in-kernel all-reduce.
• The wire format is the same everywhere. The contracts around it have diverged: three epoch/tag protocols, three wait policies (unbounded, 10 s timeout, retry-bounded), two peer allocators, four Mailbox constructors, and build keys that exist in some copies and not in others.
• [ROCm][Perf][DSv4] Mono decode layer for DeepSeek-V4-Pro on MI355X, on shared mono kernels vllm-project/vllm#61482 itself has two Mailbox classes (attention/device.py and common/sync.py), which is the cheapest place to start.
• Proposal: one substrate with a compile-time policy at each divergence (so a model that does not opt in keeps today's machine code), ATOM's atom/mono device/plan/runtime layering as the API baseline, consolidation inside vLLM first, and a phased migration (P0-P4) gated by a compile-only machine-code diff.

What to review

• The capability matrix and the "divergences that matter" list: are they right? Several rows describe other authors' PRs and come from reading alone.
• Decisions D1-D8 (wait policy, epoch strategies, typed addresses, one peer allocator, build key, layout/ABI, all-reduce primitives vs policy, one device-helper set).
• Whether the first step should be P0 (dedupe inside [ROCm][Perf][DSv4] Mono decode layer for DeepSeek-V4-Pro on MI355X, on shared mono kernels vllm-project/vllm#61482) before anything is shared across models.
• The open questions: peer-buffer hazards under the three parity schemes, MiniMax's flag-publication ordering, tag range and wrap, and the license mix if the package ever moves to FlyDSL.

Validation

None, by design: this is static analysis (no GPU was available). Nothing was compiled, run or measured; the note lists what each migration phase must show before it can land. PR heads and reference-repo commits that were read are listed in the note's "Method and caveats" section.

AI assistance (Claude Code) was used to investigate the codebases and to write the note; the submitter has not yet reviewed every statement, so claims about other authors' PRs should be read as unconfirmed.

🤖 Generated with Claude Code