Repository navigation
v1.91.119.0 test(seed): refresh the free-suite duration seed (#3224) - #3238
Merged
Merged
Conversation
…rding (#3224) 54 free test files had landed since the 2026-10-15 recording, so CEO-10 failed again (limit 5). Recorded with `bun run test:ubicloud --record-durations` (standard-16, recorder exit 0, no failed files) on origin/main cadbd0a: 1219 -> 1273 durations, none removed. test/setup-codex-scope.test.ts is back under the 60 s ceiling (58.9 s), so its allowlist entry is removed as the shrink-only ratchet requires. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
|
😎 This pull request was merged. |
Contributor
E2E Evals: ✅ PASS0 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $0 rule/judge cost | reconcile exit: 0 Profile: pr / mode Rule and judge shards
Fail-closed reconciliationSliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry. |
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In plain words
Since the 2026-10-15 recording (#3212), 54 new free test files have landed. CI therefore guesses their run time as the 99th-percentile duration, and the weekly health check fails CEO-10 again (limit 5). This PR re-records every free file's duration on a CI-mirrored Ubicloud VM, so the shard packer has real numbers for all of them. Only data files change.
What changed
scripts/free-test-durations.json: recorded withUBI_OWNER=gsta33 bun run test:ubicloud --record-durations(standard-16) on origin/maincadbd0af(v1.91.116.0). The recorder exited 0 after 316 s, and no file failed or timed out. Durations went from 1219 to 1273: 54 added, none removed. The serial sum went from 3202 s to 3515 s.scripts/free-test-seed-allowlist.json: removestest/setup-codex-scope.test.ts, which now records 58.9 s, under the 60 s ceiling. The shrink-only ratchet requires dropping entries that no longer exceed it. The two other files over 60 s from v1.91.105.0 fix(tests): the free suite stops needing its retry pass; test:health tells the truth (#3154) #3212 remain allowlisted with their split todos:setup-install-registryat 86.5 s andhost-renders-pruneat 63.5 s.free-tests.ymlis untouched; #3236 owns its curation. A standing recorder is planned as a separate lane (W4).Validation
unseededCheck(the CEO-10 check inscripts/test-health-report.ts) on this tree:pass 0 unseeded file(s) (limit 5). On main it reportsfailwith 54 unseeded files.GSTACK_FREE_SEED_BASE=$(git merge-base HEAD origin/main) bun test test/free-seed-ratchet.test.ts test/test-health-report.test.ts: 26 pass, 0 fail.Behavior changes
None. The shard packer reads the new durations.
Refs #3224. Expect CEO-10 to go red again as new test files accumulate until the standing recorder lands.