feat(qwen35moe): pipelined hybrid MoE decode with GPU/CPU overlap#289

Open

howard0su wants to merge 8 commits into

Luce-Org:mainfrom

howard0su:pipeline_moe

Commits on May 30, 2026

feat(qwen35moe): pipelined hybrid MoE decode with GPU/CPU overlap

howard0su
and
Copilot
committed
fix: distribute dummy MoE routing evenly to avoid MMQ stream-k OOB

howard0su
and
Copilot
committed
fix: free cached FFN graphs between requests to prevent GPU memory corruption

howard0su
and
Copilot
committed
feat: add decode time-breakdown telemetry to generate path

howard0su
and
Copilot
committed
fix(moe): normalize expert weights by sum (fixes wrong math output)

howard0su
and
Copilot
committed
chore: remove noisy pipelined init log

howard0su
and
Copilot
committed
fix: address PR #289 review comments

howard0su
and
Copilot
committed
fix(qwen35moe): sub-batch hybrid prefill FFN to avoid MMQ mul_mat_id OOB

authored and
howard0su
committed