refactor: extract MoE hybrid mode into common layer for qwen and laguna#305

Merged

davide221 merged 21 commits into

Luce-Org:mainfrom

howard0su:layersplit_refactor

Jun 4, 2026

Commits on Jun 3, 2026

refactor: extract MoE hybrid mode into common layer for qwen35moe + laguna

howard0su
and
Copilot
committed
feat(pipeline): add routed FFN fast path with cached graphs

howard0su
and
Copilot
committed
feat(pipeline): GPU-resident act_cur with async logits projection

howard0su
and
Copilot
committed
fix(test): pass MoeHybridStorage to init_pipelined_decode_state

howard0su
and
Copilot
committed
feat(laguna): print VRAM allocation budget and hot/cold split info

howard0su
and
Copilot
committed
fix(common): skip dense layers in MoE hybrid placement and storage

howard0su
and
Copilot
committed
feat(laguna): hybrid prefill path + DFLASH_EXPERT_BUDGET_PCT support

howard0su
and
Copilot
committed
feat(laguna): match qwen35moe dynamic placement print format

howard0su
and
Copilot
committed
feat(laguna): auto dynamic placement like qwen35moe

howard0su
and
Copilot
committed
perf(hybrid-prefill): remove sub-batching and reuse gallocr across layers

howard0su
and
Copilot
committed
telemetry: instrument AR decode routed path + per-token breakdown

howard0su
and
Copilot
committed
fix: skip batched prefill FFN when layer has cold experts

howard0su
and
Copilot
committed
perf: extend routed FFN cold-masking to attention layers

howard0su
and
Copilot
committed
feat: add DFLASH_COLD_COMPUTE option for exact cold expert computation

howard0su
and
Copilot
committed
feat: fused cold FFN kernel with OpenMP parallelization

howard0su
and
Copilot
committed
feat: hybrid routed+cold path for inline cold compute

howard0su
and
Copilot
committed
Remove agents.md
howard0su
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: extract MoE hybrid mode into common layer for qwen and laguna#305

refactor: extract MoE hybrid mode into common layer for qwen and laguna#305
davide221 merged 21 commits into
Luce-Org:mainfrom
howard0su:layersplit_refactor

Commits on Jun 3, 2026

refactor: extract MoE hybrid mode into common layer for qwen35moe + laguna

feat(pipeline): add routed FFN fast path with cached graphs

feat(pipeline): GPU-resident act_cur with async logits projection

fix(test): pass MoeHybridStorage to init_pipelined_decode_state

feat(laguna): print VRAM allocation budget and hot/cold split info

fix(common): skip dense layers in MoE hybrid placement and storage

feat(laguna): hybrid prefill path + DFLASH_EXPERT_BUDGET_PCT support

feat(laguna): match qwen35moe dynamic placement print format

feat(laguna): auto dynamic placement like qwen35moe

perf(hybrid-prefill): remove sub-batching and reuse gallocr across layers

telemetry: instrument AR decode routed path + per-token breakdown

fix: skip batched prefill FFN when layer has cold experts

perf: extend routed FFN cold-masking to attention layers

feat: add DFLASH_COLD_COMPUTE option for exact cold expert computation

feat: fused cold FFN kernel with OpenMP parallelization

feat: hybrid routed+cold path for inline cold compute

Remove agents.md

Commits on Jun 4, 2026

fix(laguna): link test/bench targets against the active ggml backend

fix(moe-hybrid): avoid ggml-cuda MMQ mul_mat_id crash on reduced hot stacks

Merge main into layersplit_refactor

docs(server): document hybrid MoE hot/cold expert split