Skip to content

validate_bundle trusts summary.json n_total over the canonical ParamSpec, so a wrong-size theta passes #167

Description

@philluiz2323

Bug

validate_bundle in src/trinity/validate_submission.py is supposed to reject a best_theta.npy whose length does not match the canonical head size (params.make_spec().n_total, 13312). Since f56e7af ("make submission size check warning-only", #150) it no longer does:

  • the summary's n_total overrides the canonical expected length (n_expected = summary_n_total), and
  • a theta-length mismatch is only a warning, not an error.

So a malformed bundle passes validation as long as summary.json claims a matching (wrong) n_total — the submitter's own claim is trusted over the real spec, defeating the check.

Reproduce (offline)

from pathlib import Path; import json, numpy as np
from trinity.validate_submission import validate_bundle
d = Path("/tmp/b"); d.mkdir(exist_ok=True)
np.save(d/"best_theta.npy", np.zeros(128))               # wrong length (canonical is 13312)
(d/"summary.json").write_text(json.dumps({"n_total": 128}))  # summary claims the wrong length
print(validate_bundle(d).ok)   # -> True  (should be False)

This also leaves tests/test_validate_submission.py::test_wrong_theta_length_fails red on main (CI test-router fails for every PR).

Expected

The theta length is validated against the canonical ParamSpec.n_total (or an explicit expected_n_total override), the summary's value does not redefine it, and a mismatch is a hard error. A summary/theta disagreement stays a separate warning.

Location

src/trinity/validate_submission.py, validate_bundle (the n_total handling ~line 181 and the theta-length check ~line 195).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions