PaperBench token aggregation crashes on explicit null usage metadata

[sylvesterkaczmarek]

Summary

SimpleJudge.grade_leaf() always includes a token_usage key in judge_metadata, but the value can legitimately be None when the configured completer is not OpenAICompletionsTurnCompleter.

get_total_token_usage() later recurses through leaves and checks only whether the key exists:

if task.judge_metadata is not None and "token_usage" in task.judge_metadata:
    return [TokenUsage.from_dict(task.judge_metadata["token_usage"])]

TokenUsage.from_dict(None) then attempts data.items() and raises AttributeError.

grade_submission() calls get_total_token_usage() for every judge_type == "simple", so a successful simple-judge run can fail during post-grade usage aggregation solely because usage metadata is unavailable.

Impact

PaperBench's generic TurnCompleter support can grade successfully but fail to produce a JudgeOutput when token usage is absent/null.

Proposed resolution

Treat null/missing token usage as no usage for that leaf. Only deserialize the metadata when the token_usage value is a dictionary.

Add regressions for a leaf with {"token_usage": None} and a leaf with normal usage data, verifying null usage is skipped while valid usage is still summed.