Skip to content

Honor zero-hour PaperBench checkpoint selection - #169

Open
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix-paperbench-zero-target-duration
Open

Honor zero-hour PaperBench checkpoint selection#169
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix-paperbench-zero-target-duration

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

  • preserve target_duration_hr=0 as a valid PaperBench checkpoint target
  • keep None as the sentinel for selecting the latest available checkpoint
  • add regression coverage showing that a zero-hour target selects the earliest submission

PBTask._select_checkpoint() currently uses a truthiness check when choosing the duration passed to get_file_at_duration(). Because 0 is falsey, an explicit target_duration_hr=0 is replaced with 10000, which selects the latest checkpoint instead of the checkpoint at the start of the run.

The configuration type and get_file_at_duration() both support zero as a valid non-negative duration. This change switches the fallback check to is not None, so zero keeps its documented meaning while the existing None behavior remains unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant