End-of-run computer-task logging treats partial scores as correct

[sylvesterkaczmarek]

Summary

PythonCodingEval defines correctness through FinalResult.correct, which is true only when grade.score == 1. The optional log_at_end path instead records correctness with bool(score).

For continuous or partial grades, any positive score such as 0.5 is therefore recorded as fully correct even though FinalResult.correct is false.

Current behavior

FinalResult.correct:

return self.grade.score == 1

_log_results():

score = result.grade.score
...
recorder.record_match(correct=bool(score), ...)

Grade.score is a float and explicitly supports continuous grading.

Impact

When PythonCodingEval(log_at_end=True) is used with a partial/continuous grade, recorder match data disagrees with the eval's canonical correctness definition. Downstream per-sample correctness views can therefore report a partial failure as a success.

Proposed resolution

Use result.correct when recording the end-of-run match, matching the normal _evaluate_inner() recording path.

Add a regression with grade.score=0.5 verifying _log_results() records correct=False.