Skip to content

Add coverage for successful generalization gap calculation #2329

Description

@harrythomasben

Problem

The runner has a successful path that computes generalization_gap only when both tuned and held-out partitions score, but there is no direct test pinning that calculation.

Proposed coverage

Add a tests-only regression case with mocked successful partitions and assert the reported gap equals tuned minus held-out composite means.

Found by locally pulling upstream/test and reviewing the generalization runner path; this is not based on an existing issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkEvaluation harnesstestsTest suite

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions