Skip to content

fix(strategy): print a summary for --compare --quiet instead of nothing#228

Merged
zeokin merged 1 commit into
zeokin:mainfrom
statxc:fix/compare-quiet-summary
Jul 14, 2026
Merged

fix(strategy): print a summary for --compare --quiet instead of nothing#228
zeokin merged 1 commit into
zeokin:mainfrom
statxc:fix/compare-quiet-summary

Conversation

@statxc

@statxc statxc commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

PR kind

  • fix
  • feat

Summary

strategy --compare --quiet produced no output at all. runner.compare() only prints its report when verbose (i.e. not --quiet), and main() discarded compare()'s return value and return 0'd before the --quiet summary block that the normal run path has:

if args.compare:
    runner.compare(...)   # prints nothing under --quiet
    return 0              # <- before the quiet-summary block

So a scripted --compare --quiet run silently computed both engines and printed nothing.

Fix

Capture compare()'s returned dict and, under --quiet, print a one-line summary (exact/smart seconds, speedup, rel_err) — mirroring the run path's --quiet line. Non-quiet behaviour is unchanged (compare() still prints its full report; no double output).

Touches only strategy/cli.py and a new test — no protected paths.

Validation

uv run --extra test python -m pytest tests/test_compare_quiet.py -q             # 2 passed
uv run --extra test python -m pytest tests/ strategy/tests/ eval/tests/ -q      # 283 passed, 24 skipped

The test stubs runner.compare via monkeypatch, so it runs on CPU with no GPU/torch — it exercises the flag handling and summary formatting (and asserts non-quiet mode does not double-print).

Checklist

  • CPU-safe validation passed.
  • My commits do not include Co-authored-by footers for coding agents such as Cursor, Codex, Claude, Copilot, or similar tools.
  • Not a feat PR — no scorecard required.

runner.compare() only prints when verbose (i.e. not --quiet), and main()
discarded its return value and returned before the quiet-summary block -- so
`strategy --compare --quiet` produced no output at all. Capture compare()'s
result and, under --quiet, print a one-line exact/smart/speedup/rel_err
summary (mirroring the run path's --quiet line). Add a CPU test that stubs
runner.compare so it needs no GPU.
@github-actions github-actions Bot added area:strategy Smart strategies / transforms (strategy/) area:tests Correctness gates and test suites (tests/) status:needs-review Awaiting maintainer review type:bug Something is incorrect or broken status:ready-non-gpu Fix/docs PR cleared non-GPU triage; maintainer review only labels Jul 13, 2026

@zeokin zeokin left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--compare --quiet now prints a one-line summary instead of nothing; no double-print.

@zeokin zeokin merged commit c477e9e into zeokin:main Jul 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:strategy Smart strategies / transforms (strategy/) area:tests Correctness gates and test suites (tests/) status:needs-review Awaiting maintainer review status:ready-non-gpu Fix/docs PR cleared non-GPU triage; maintainer review only type:bug Something is incorrect or broken

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants