Skip to content

packaging: ship judge_corpus and score_corpus JSON via package-data #2318

Description

@kurosawareiji7007-hub

Summary

benchmark.judge_corpus and benchmark.score_corpus are listed under [tool.setuptools] packages, but there is still no [tool.setuptools.package-data]. An installed wheel/sdist therefore contains only __init__.py for those packages — not manifest.json or scenarios/*.json.

After pip install, MANIFEST_PATH / SCENARIOS_DIR point at missing files, so load_corpus() and tools like python -m scripts.calibrate_judge fail on an installed distribution (they only work from a source checkout).

Expected

Declare package data in pyproject.toml only, e.g.:

oml [tool.setuptools.package-data] "benchmark.judge_corpus" = ["manifest.json", "scenarios/*.json"] "benchmark.score_corpus" = ["manifest.json", "scenarios/*.json"]

Note

#2307 / #2310 covered this earlier and were closed without a review comment while CI was green. Re-raising because the packaging gap is still present on test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkEvaluation harnessbugSomething isn't workingtestsTest suite

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions