Repository navigation

perf: Reduce MXFP8 refit weight-transfer overheads - #3294

Merged

terrykong merged 136 commits into

NVIDIA-NeMo:mainfrom

seonjinn:sna/pr-mxfp8-refit-optimization

Oct 18, 2026

Merged

perf: Reduce MXFP8 refit weight-transfer overheads#3294
terrykong merged 136 commits into
NVIDIA-NeMo:mainfrom
seonjinn:sna/pr-mxfp8-refit-optimization

Commits on Jul 21, 2026
- perf(vllm): optimize quantized refit paths
  seonjinn
  committed
  b88aa2c
- Merge branch 'main' into sna/pr-mxfp8-refit-optimization
  seonjinn
  authored
  438e20d
Commits on Jul 27, 2026
- Merge branch 'main' into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  96ba299
- fix(trtllm): align refit metadata interface
  seonjinn
  committed
  0fb59b6
- test(vllm): include MXFP8 refit tests in L0
  seonjinn
  committed
  c4a5e14
- fix(refit): complete prequant metadata handshake
  seonjinn
  committed
  5c53bd4
- test(refit): cover MXFP8 optimization paths
  seonjinn
  committed
  98fc197
- test(refit): fix vLLM test import order
  seonjinn
  committed
  b237fd2
Commits on Jul 28, 2026
- Merge upstream/main into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  d1f9fc9
- test(fp8): cover MXFP8 MoE padding path
  seonjinn
  committed
  ab030c3
- Merge upstream/main into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  2c147ac
Commits on Jul 29, 2026
Commits on Jul 30, 2026
Commits on Jul 31, 2026
- Merge remote-tracking branch 'upstream/main' into sna/pr-mxfp8-refit-optimization-squashed
  seonjinn
  committed
  2339686
- fix(vllm): preserve MXFP8 refit on vLLM 0.25
  seonjinn
  committed
  5880499
- style(vllm): sort MXFP8 backend import
  seonjinn
  committed
  123cc07
- fix(vllm): allow partial configs in quant validation
  seonjinn
  committed
  ef3fa08
- Merge remote-tracking branch 'upstream/main' into sna/pr-mxfp8-refit-optimization-squashed
  seonjinn
  committed
  7989113
- feat(recipe): add async Qwen3 30B MXFP8 rollout
  seonjinn
  committed
  c4a7b06
- feat(recipe): add async Qwen3 235B MXFP8 rollout
  seonjinn
  committed
  69f7995
- ci: allowlist async recipe result path
  seonjinn
  committed
  d53bfdf
- fix(vllm): initialize async driver FP8 config
  seonjinn
  committed
  2405bf5
- fix(vllm): propagate FP8 config to refit workers
  seonjinn
  committed
  0a11e94
- fix(recipe): keep Qwen router gate in BF16
  seonjinn
  committed
  81ce860
- fix(recipe): exclude Qwen MoE routers from MXFP8
  seonjinn
  committed
  9e43378
Commits on Aug 1, 2026
- fix(vllm): patch FP8 in RayExecutorV2 workers
  seonjinn
  committed
  6e582b8
- fix(vllm): serialize FP8 worker pre-init hook
  seonjinn
  committed
  93394c2
- test(vllm): share RayExecutorV2 patch recorder
  seonjinn
  committed
  f3c9196
- feat(vllm): configure refit runtime optimizations
  seonjinn
  committed
  e3384d2
- fix(refit): propagate runtime configuration
  seonjinn
  committed
  c4d453e
- Merge remote-tracking branch 'upstream/main' into sna/pr-mxfp8-refit-optimization-squashed
  seonjinn
  committed
  d43372e
- fix(refit): cover standalone algorithm paths
  seonjinn
  committed
  2b80a85
- fix(vllm): scope refit optimization config
  seonjinn
  committed
  0b6b5cc
- fix(refit): allow matching fp8 reshard storage
  seonjinn
  committed
  cd7732c
Commits on Aug 13, 2026
- Merge remote-tracking branch 'upstream/main' into sna/pr-mxfp8-refit-optimization-squashed
  seonjinn
  committed
  80cd8fe
- fix(refit): validate MXFP8 prequant wire format
  seonjinn
  committed
  7705d22
Commits on Aug 18, 2026
- Merge remote-tracking branch 'origin/main' into review/pr3294-latest-20260817
  seonjinn
  committed
  88721ce
Commits on Aug 25, 2026
Commits on Aug 26, 2026
- Merge branch 'main' into pr-3294-fixes
  seonjinn
  committed
  1d6a10a
- fix(dynamo): align prepare_refit_info override with interface
  seonjinn
  committed
  8dc4c36
- Merge branch 'main' into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  db442ef
- Merge main into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  4427f4c
- fix: resolve CI unit-test failures after main merge
  seonjinn
  committed
  c162cfd
- Merge remote branch update, keeping local merge resolution and CI fixes
  seonjinn
  committed
  382fa8b
- Merge main into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  125f178
- fix(refit): derive prequantize metadata without re-exporting weights
  seonjinn
  committed
  e28239c
- Fix pyrefly no-matching-overload false positive on masked_fill
  seonjinn
  committed
  232da17
Commits on Aug 27, 2026
- test: update refit cleanup fixture after main merge
  seonjinn
  committed
  6b2ab6e
- refactor(refit): limit optimized recipes to sync RL
  seonjinn
  committed
  06b6fbb
Commits on Sep 4, 2026
- Merge remote-tracking branch 'origin/main' into codex/pr3294-push-20260904
  seonjinn
  committed
  7c9b881
- fix(mxfp8): handle mixed grouped expert refit
  seonjinn
  committed
  bf03429
- test(mxfp8): align refit fixtures with vllm
  seonjinn
  committed
  d9246a9
- test(mxfp8): model routed expert config
  seonjinn
  committed
  443a9f9
- fix(vllm): reroute grouped MXFP8 scale sidecars
  seonjinn
  committed
  b8159de
- test(mxfp8): patch modular kernel factory
  seonjinn
  committed
  8d38b28
- test(mxfp8): use block-aligned hidden size
  seonjinn
  committed
  658d385
- test(mxfp8): exercise unpadded kernel reuse
  seonjinn
  committed
  172d346
- test(mxfp8): use current refit loader contract
  seonjinn
  committed
  546d346
- test(mxfp8): update remaining loader fixtures
  seonjinn
  committed
  db15416
Commits on Sep 7, 2026
- test(refit): reject native loader cache wrapping
  seonjinn
  committed
  1d4d013
- fix(refit): preserve vLLM online loaders
  seonjinn
  committed
  1b5e642
- Merge remote-tracking branch 'origin/main' into codex/pr3294-push-20260904
  seonjinn
  committed
  f377f0a
- test(refit): avoid duplicate kv scale finalization
  seonjinn
  committed
  b7d4ce6
- test(refit): avoid duplicate fp8 kv finalization
  seonjinn
  committed
  201ad9e
- test(refit): use a realized module fixture
  seonjinn
  committed
  7438153
- test(refit): expect serialized FP8 config
  seonjinn
  committed
  08e7d10
- test(refit): follow collective sender contract
  seonjinn
  committed
  d003483
- fix(refit): preserve MXFP8 kernel scale storage
  seonjinn
  committed
  238f241
- test(refit): update offload call expectation
  seonjinn
  committed
  b72f4d4
Commits on Sep 12, 2026
- Merge branch 'main' into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  bc024e8
- style: apply lint fixes
  seonjinn
  committed
  0122151
Commits on Sep 15, 2026
- fix(vllm): accept weights keyword in refit loader callback
  seonjinn
  committed
  07eda91
Commits on Sep 17, 2026
- Merge main into MXFP8 refit optimization
  seonjinn
  committed
  85a9a53
- test: restore mock call import after main merge
  seonjinn
  committed
  0982f5d
- fix(refit): renegotiate metadata after communicator rebuild
  seonjinn
  committed
  83011b1
- refactor(refit): define MXFP8 wire contract
  seonjinn
  committed
  b3e36e9
- test(refit): exercise both MXFP8 quantization paths
  seonjinn
  committed
  f61f498
- test(refit): require vLLM MXFP8 quantizer backend
  seonjinn
  committed
  b65f89d
- test(refit): complete MXFP8 quantizer test double
  seonjinn
  committed
  1c5ece7
- test(refit): model MXFP8 tensor size API
  seonjinn
  committed
  0ac5b7d
- test(refit): cover metadata renegotiation
  seonjinn
  committed
  aabd384
- fix(refit): align prequantization lifecycle
  seonjinn
  committed
  c619ac4
- fix(refit): restore grouped MXFP8 value shape
  seonjinn
  committed
  1d2a023
- fix(refit): preserve canonical MXFP8 tensor identity
  seonjinn
  committed
  385cbf0
- test(refit): compare MXFP8 values by storage bits
  seonjinn
  committed
  8f2d78a
Commits on Sep 21, 2026
- Merge latest main into MXFP8 refit optimization
  seonjinn
  committed
  5409fb4
Commits on Sep 28, 2026
- Merge latest main into MXFP8 refit optimization
  seonjinn
  committed
  9643037
- fix: align MXFP8 refit quantization with vLLM selection
  seonjinn
  committed
  c24bef9
- style: sort MXFP8 prequant test imports
  seonjinn
  committed
  9c59f6e
- test: align MXFP8 refit fixtures with current dependencies
  seonjinn
  committed
  9c9628d
- Merge latest main into PR #3294
  seonjinn
  committed
  b319b7f
- fix: preserve unpadded MXFP8 W13 storage
  seonjinn
  committed
  077a48a
Commits on Sep 29, 2026
Commits on Sep 30, 2026
- Merge latest main into PR #3294
  seonjinn
  committed
  d45552f
- fix(vllm): reject prequantization with native reload
  seonjinn
  committed
  190a521
Commits on Oct 1, 2026
- test(vllm): cover refit scratch release
  seonjinn
  committed
  dc4e355
- fix(vllm): release MXFP8 shuffle scratch after refit
  seonjinn
  committed
  01e05c7
- fix(vllm): fence refit finalization before scratch release
  seonjinn
  committed
  64b91c4
- fix(vllm): keep MXFP8 shuffle scratch in weight pool
  seonjinn
  committed
  9084313
Commits on Oct 3, 2026
- Merge latest main into MXFP8 refit optimization
  seonjinn
  committed
  54ed892
Commits on Oct 4, 2026
- chore: merge latest main for vLLM 0.29 (PR3294)
  seonjinn
  committed
  5b3a7d1
Commits on Oct 6, 2026
- chore: merge latest main into MXFP8 refit optimization
  seonjinn
  committed
  fd294b5
- style: match pinned Ruff formatting in refit test
  seonjinn
  committed
  09987a9
- docs: clarify MXFP8 refit options
  seonjinn
  committed
  90a35e4
Commits on Oct 8, 2026
Commits on Oct 13, 2026
- Address MXFP8 refit review feedback
  seonjinn
  committed
  5c9dab7
- Merge remote-tracking branch 'origin/main' into fix/pr3294-review-20261013
  seonjinn
  committed
  1d3ff56
- Fix MXFP8 refit tests and lint after review cleanup
  seonjinn
  committed
  69ac028
- Align refit metadata return annotations across backends
  seonjinn
  committed
  9b04a62
Commits on Oct 15, 2026
- Merge upstream/main into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  f420b01
- Remove stale MXFP8 shuffle verification references
  seonjinn
  committed
  4520c78
Commits on Oct 16, 2026
- Merge upstream/main into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  9e2ab4b
- Fix MXFP8 CI test collection
  seonjinn
  committed
  f555a77
- Merge upstream/main into sna/pr-mxfp8-refit-optimization
  seonjinn
  committed
  21a70a6
- Install vLLM extra for GB200 MXFP8 CI tests
  seonjinn
  committed
  d30a813
Commits on Oct 18, 2026
- Merge latest main and repair MXFP8 GB200 L1 config
  seonjinn
  committed
  5e451db
- Match pinned Ruff formatter for refit test
  seonjinn
  committed
  7bc2096
- Fix MXFP8 checkpoint handshake unit fixture after main merge
  seonjinn
  committed
  ecb9aa0
- Update MXFP8 refit test for source metadata API
  seonjinn
  committed
  501b5d5
- test(refit): exercise prequant handshake through real worker
  seonjinn
  committed
  b1a8060

Commits

Commits on Jul 21, 2026

Commits on Jul 27, 2026

Commits on Jul 28, 2026

Commits on Jul 29, 2026

Commits on Jul 30, 2026

Commits on Jul 31, 2026

Commits on Aug 1, 2026

Commits on Aug 13, 2026

Commits on Aug 18, 2026

Commits on Aug 25, 2026

Commits on Aug 26, 2026

Commits on Aug 27, 2026

Commits on Sep 4, 2026

Commits on Sep 7, 2026

Commits on Sep 12, 2026

Commits on Sep 15, 2026

Commits on Sep 17, 2026

Commits on Sep 21, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Sep 30, 2026

Commits on Oct 1, 2026

Commits on Oct 3, 2026

Commits on Oct 4, 2026

Commits on Oct 6, 2026

Commits on Oct 8, 2026