fix: --from-stage without --output now finds existing run directory#217
Closed
dennis-lynch-nv wants to merge 2 commits intoaiming-lab:mainfrom
Closed
fix: --from-stage without --output now finds existing run directory#217dennis-lynch-nv wants to merge 2 commits intoaiming-lab:mainfrom
dennis-lynch-nv wants to merge 2 commits intoaiming-lab:mainfrom
Conversation
When using --from-stage without --output, the CLI generated a new empty run directory. The StageContract input_files check then failed immediately because prior stage artifacts (e.g., exp_plan.yaml) didn't exist in the fresh directory. This extends the BUG-119 checkpoint-search logic to also apply when --from-stage is specified, so it finds the most recent matching run directory with a checkpoint. Fixes aiming-lab#216
5 test cases covering: - --from-stage without --output finds existing run dir (the fix) - old behavior would NOT find it (proving the bug) - --resume still works (no regression) - explicit --output skips search - multiple runs: picks newest All pass on Python 3.11.
Jiaaqiliu
added a commit
that referenced
this pull request
Apr 10, 2026
…217) Extend BUG-119 checkpoint-search logic to also apply when --from-stage is used without --output. Includes 5 new tests. Fixes #216. Contributed by @dennis-lynch-nv.
Collaborator
|
Merged manually via cherry-pick in commit 7b57457. Thank you @dennis-lynch-nv for the fix and tests! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When using
--from-stagewithout--output, the CLI generates a new empty run directory via_generate_run_id(). TheStageContract.input_filescheck then fails immediately because prior stage artifacts (e.g.,exp_plan.yamlfrom stage 9) don't exist in the fresh directory.Fix
Extend the BUG-119 checkpoint-search logic (already working for
--resume) to also apply when--from-stageis specified. One-line condition change:Tests
5 new test cases in
tests/test_from_stage_run_dir.py:test_from_stage_without_output_finds_existing_run--from-stagefinds existing run dirtest_from_stage_without_output_old_behavior_failstest_resume_still_works--resumetest_explicit_output_skips_search--outputbypasses searchtest_picks_newest_run_when_multiple_existFixes #216