Repository navigation
fix(utils): share the Saver frequency decision across models in a step - #1802
Merged
sitabulaixizawaluduo merged 1 commit intoOct 19, 2026
Conversation
PPOTrainer._save_hf calls Saver.save for the actor and then for the critic in the same training step. Each call ran the shared EpochStepTimeFreqCtl.check, so the step/epoch counters advanced twice per step and the time window was reset by the first call. With freq_steps=2 (or freq_epochs=2) the critic was saved every step or epoch and the actor was never saved; with freq_secs the critic was never saved. Make the frequency decision once per global step and reuse it for the other models saved in that step. Trainers with a single model are unaffected. Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com>
MohammadHijjawi97
requested review from
fishcrap,
garrett4wade and
sitabulaixizawaluduo
as code owners
October 17, 2026 13:19
sitabulaixizawaluduo
approved these changes
Oct 19, 2026
Le8r0nJames
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
PPOTrainer._save_hfcallsSaver.savefor the actor and then for the critic in the same training step. Each call ran the sharedEpochStepTimeFreqCtl.check, so the step and epoch counters advanced twice per step, and the time window was reset by the first call. As a result, when a critic is used:saver.freq_steps=2(orfreq_epochs=2): the critic is saved every step (or epoch) and the actor is never saved.saver.freq_secs=N: the first call resets the window, so the actor is saved and the critic is not.This PR makes the frequency decision once per global step in
Saverand reuses it for the other models saved in that step. Trainers that save a single model behave exactly as before, and the savedfreq_ctlstate format is unchanged.Reproduction (
freq_steps=2, actor + critic saved each step, 6 steps):Related Issue
None; no existing issue or PR covers it.
Type of Change
Checklist
pre-commit run --all-files)./docs/build_all.sh)main/review-prcommand/create-prAdditional Context
Added
test_save_applies_frequency_once_per_step_for_all_modelstotests/test_saver.py, parametrized overfreq_steps=2andfreq_epochs=2. It fails onmain(the actor is never saved) and passes with this change. All oftests/test_saver.pypasses (6 tests).pre-commit run --fileson the two changed files passes (ruff lint/format, SPDX header). These are CPU-only unit tests; I did not run a full PPO training job.