Repository navigation
Leave the monthly CHS current run to the results repository - #481
Merged
Merged
Conversation
openwatersio/chs-current-validation now runs the full CHS current fit benchmark every month and publishes each gate's tier and bounds in its releases, so this repository's benchmarks workflow no longer runs it on main. Branch pushes still run the quick check as a regression check, with NO_FLOORS and the reversed-axis check unchanged. The validation record, README and CONTRIBUTING described the results repository as upcoming and this workflow as keeping the rows for 90 days. They now describe what the results repository does and link its releases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
openwatersio/chs-current-validation is live, so the benchmark's code comments that said it "will pool" rows, "will check" across shards or "will find" a gate's worst slacks now say what it does. The comment about its cross-shard check says "recipe window", because that check leaves out the report-only 365-day window. Comments only, no code change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The grid steps 35, 35, 35 and 7 days, so a calendar month holds at most two grid points; "several" overstated it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main no longer runs the full benchmark, so CONTRIBUTING says to run it locally and that the results repository scores only the stamps its targets.json names. The README's October 1 figures are labelled as this repository's scorer version 1 run, so they don't read as a results-repository release, and the NO_FLOORS note now names the quick run, the only one the workflow makes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The benchmarks workflow no longer runs the full CHS current benchmark on
main. openwatersio/chs-current-validation now does that every month, and published its first release,fit-2026-10-22-8f85912-r1, today. Branch pushes keep the quick check. The docs and benchmark comments now describe the results repository in the present tense.Part of openwatersio/slackwater-ios#752 (Rollout step 1:
chs-current-fitout of the main-branch and scheduled runs). It is the last task of the results-repository plan, openwatersio/slackwater-ios#813.Why
This workflow's monthly run scored only the newest grid point and kept its rows as an artifact for 90 days. It was a stopgap until the results repository existed. That repository now:
A second full run here would only duplicate those IWLS requests.
What changes
.github/workflows/benchmarks.yml: the CHS current step runs only whenFASTis set, which means branch refs. It keepsNO_FLOORS. Thescheduleandworkflow_dispatchtriggers stay for the other benchmarks, and so does the upload step, which carries their outputs.docs/validation/chs-currents.md,README.md,CONTRIBUTING.md: the sentences about this workflow's monthly run and its 90-day artifact are gone, and every sentence about the results repository is in the present tense, linking its releases. CONTRIBUTING says to run the benchmark locally to score every gate and held-out week, and that the results repository scores the recipe stamps itstargets.jsonnames. The README labels its October 1 figures as this repository's scorer version 1 run.chs-currents.md,chs-current-fit.ts), "window" is now "recipe window". The results repository's check leaves out the 365-day window, which is reported only.benchmarks/chs-current-{grid,score,fit}.tssay the results repository pools, checks and finds, where they used to say it will.Verification
npm run lint,npm run buildandgit diff --checkpass.benchmarks.ymlparses as YAML.mainonly;FAST's definition.Review
During implementation the change had its own review against its brief. That review checked each claim about the results repository against that repository's code, and found one overstatement, "several" grid points a month, fixed in 80a1eff. One
review-branchround followed.Round 1: Codex, Claude Opus and Claude Sonnet. Copilot was skipped because its CLI isn't installed. Codex read the results repository and verified the new claims against it, and found no issues. Opus and Sonnet couldn't read outside this worktree, so they couldn't verify those claims. Opus and Sonnet found only low issues, so no second round ran.
Fixed in cf67900:
mainskips it (Opus, Sonnet). It says to run the benchmark locally, and that the results repository scores the stamps itstargets.jsonnames.NO_FLOORS" (Sonnet). It now names the quick run, the only one the workflow makes.Not fixed:
workflow_dispatchonmaincan no longer produce a full CHS current run (Opus). This is the point of the change, as the plan specifies. A full run isnpm run chs-current-fitlocally.reversed_axisdisqualifier in the release, not as a failed run. That is the design of the rows-only mode (Add a rows-only mode to the CHS current benchmark for the results repository #471).cf67900 had no model review after it. Lint and
git diff --checkran on it.🤖 Generated with Claude Code