Skip to content

Fix PaperBench zero prior-node limit - #165

Open
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix-paperbench-zero-prior-nodes
Open

Fix PaperBench zero prior-node limit#165
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix-paperbench-zero-prior-nodes

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

  • make max_prior_nodes=0 return no prior rubric nodes
  • preserve the existing unlimited behavior when the limit is None
  • preserve positive limits as a tail slice of the available prior context

TaskNode.get_prior_nodes() currently applies required_nodes[-max_prior_nodes:] whenever a limit is provided. In Python, -0 is 0, so [-0:] is the same as [0:]. Explicitly requesting zero prior nodes therefore returns every prior node instead of none.

This can inject all available prior rubric context into SimpleJudge when a caller intentionally configures a zero-node context budget.

The fix handles zero explicitly while leaving positive and unlimited behavior unchanged.

Regression coverage verifies unlimited, zero, and positive-limit cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant