Repository navigation
FP8 fix after vllm 0.11 upgrade #1368
Description
After #1334 goes in, FP8 support will need to be fixed since APIs for fp8 have changed in vllm. In order to minimize the amount of changes in #1334, @guyueh1 will fix fp8 as a short follow
repro:
bash tests/test_suites/llm/grpo-llama3.1-8b-instruct-1n8g-megatron-fp8-rollouts.v2.sh
error message
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/executor/executor_base.py", line 54, in __init__
�[36m(VllmGenerationWorker pid=294858)�[0m self._init_executor()
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/executor/uniproc_executor.py", line 55, in _init_executor
�[36m(VllmGenerationWorker pid=294858)�[0m self.collective_rpc("load_model")
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
�[36m(VllmGenerationWorker pid=294858)�[0m return [run_method(self.driver_worker, method, args, kwargs)]
�[36m(VllmGenerationWorker pid=294858)�[0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/utils/__init__.py", line 3122, in run_method
�[36m(VllmGenerationWorker pid=294858)�[0m return func(*args, **kwargs)
�[36m(VllmGenerationWorker pid=294858)�[0m ^^^^^^^^^^^^^^^^^^^^^
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 213, in load_model
�[36m(VllmGenerationWorker pid=294858)�[0m self.model_runner.load_model(eep_scale_up=eep_scale_up)
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2635, in load_model
�[36m(VllmGenerationWorker pid=294858)�[0m self.model = model_loader.load_model(
�[36m(VllmGenerationWorker pid=294858)�[0m ^^^^^^^^^^^^^^^^^^^^^^^^
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/model_executor/model_loader/base_loader.py", line 51, in load_model
�[36m(VllmGenerationWorker pid=294858)�[0m process_weights_after_loading(model, model_config, target_device)
�[36m(VllmGenerationWorker pid=294858)�[0m File "/opt/ray_venvs/nemo_rl.models.generation.vllm.vllm_worker.VllmGenerationWorker/lib/python3.12/site-packages/vllm/model_executor/model_loader/utils.py", line 112, in process_weights_after_loading
�[36m(VllmGenerationWorker pid=294858)�[0m quant_method.process_weights_after_loading(module)
�[36m(VllmGenerationWorker pid=294858)�[0m File "/tmp/ray/session_2025-10-14_23-48-06_424410_277188/runtime_resources/working_dir_files/_ray_pkg_f44cda39f5f054bc/nemo_rl/models/generation/fp8.py", line 421, in process_weights_after_loading
�[36m(VllmGenerationWorker pid=294858)�[0m weight = self._maybe_pad_weight(weight)
�[36m(VllmGenerationWorker pid=294858)�[0m ^^^^^^^^^^^^^^^^^^^^^^
�[36m(VllmGenerationWorker pid=294858)�[0m AttributeError: 'Fp8LinearMethod' object has no attribute '_maybe_pad_weight'
Activity
qiaochuz-nv commentedon May 5, 2026
QA RCCA Analysis
RCCA Status: Done
RCCA Category: covered
Test Action: none
Fix PR: #1432
Root Cause: fix PR includes upstream test coverage.
Trigger Config: FP8 fix after vllm 0.11 upgrade
Coverage Assessment:
Conclusion: RCCA classification completed for this closed NeMo-RL GitHub issue.