Repository navigation
v1.91.102.0 fix(pregate): run Dart and Flutter test lanes - #3193
arlophoenix wants to merge 21 commits into
Conversation
A Dart repo got PREGATE_NO_LANE for every touched file: .dart was not a source extension, _test.dart was not a test, and package: URIs never resolved. Dart directives now resolve through every tracked pubspec.yaml, and Dart tests run per package root with flutter/dart test (through fvm when pinned), judged on the +N -M verdict line. Two policy gaps blocked the same repos after mapping worked: a linux-only CI lane is requires-remote from a Mac with no way to clear it when CI is off (new `local` key), and a repo that gitignores .gstack/ could never carry a base policy (fallback to <GSTACK_HOME>/projects/<slug>/pregate.json, read only when the base has none; outside the repo, so a PR cannot reach it). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dart lists gstack-diff-scope matched no category for .dart, so a Flutter widget PR with a test set SCOPE_TESTS alone: the design, performance and security passes were skipped with no SCOPE_ERROR, because the test file matched. The extension cannot separate a widget from a repository, so .dart is both (lib/frontend-scope.ts mirrors the frontend half; impeccable still scans web sources only). pubspec.lock is config. pregate's workflow and selection-manifest parsers now read _test.dart paths, and pubspec.yaml/.lock, analysis_options.yaml and dart_test.yaml are conventional root files, not strays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…etection The review-army stack hint and TEST_FW knew Ruby, Node, Python, Go and Rust, so a Flutter repo was described to every specialist as "node" (from its package.json) with an unknown test framework. Test bootstrap had no Dart runtime marker and its test-file count missed *_test.dart outside a test/ directory. pubspec.yaml now reports dart or flutter (by the Flutter SDK dependency), flutter-test/dart-test, RUNTIME:dart, FRAMEWORK:flutter and dart_test.yaml, with flutter test / dart test (through fvm when pinned) as the command to offer. Generated docs regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # bin/gstack-pregate # docs/pregate.md # lib/pregate/checks/lanes.ts # lib/pregate/config.ts
…policy paths Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…SDK, user-policy scope CI jobs that run flutter test or dart test are test lanes. integration_test/ and test_driver/ are not auto-selected (they need a device). A missing Dart SDK is incomplete and named, not a 127 test failure. The user-owned pregate.json covers run only and refuses sealed/price_file, with a fix line that points at the user file. planRuns groups linearly and memoises package roots; the incomplete detail names the unclaimed files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, fvm hint only when pinned A touched integration_test/ or test_driver/ file now gets a fix naming the active policy file (the user-owned copy when that is the one in force) and the `[]` declaration that clears it; the docs said to declare a lane, which could never select a non-test. `dart run test`, `melos test` and `very_good test` count as CI test commands. The missing-SDK hint offers fvm only when the repo pins a version, and a null `price_file` in the user policy is not refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ir pattern The fix text and docs now recommend "**/integration_test/**": [] (and test_driver), which also matches a sub-package's device tests; the root-only glob left them failing. Device detection lives in import-graph (any .dart file under integration_test/ or test_driver/), shared by isTestFile and the remedy, so driver entry points get the hint too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uses, per-run zero-run
`dart test` runs files under test/ only, so a production lib/src/ab_test.dart
is no longer selected and run as a test (a false lane failure). A user-owned
policy that exists but cannot be read now refuses instead of running on
defaults, which silently dropped its guards. One package whose tests all skip
makes the lane ZERO-RUN even when another package's count would hide it, and
{files} is substituted with a function so a `$&` in a file name stays literal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y error names the file The test/ anchor let the manifest regex start inside a directory name, so integration_test/app_test.dart became a phantom test/app_test.dart in that lane's set. A left boundary keeps the match on the whole path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…est paths; guarded pubspec reads
- `+0 ~N: All tests skipped.` parses as 0 ran, so a fully skipped package is
ZERO-RUN instead of a pass.
- Test paths a workflow step names resolve against the step's (or job's, or
workflow's) working-directory, so a sub-package's platform lane keeps its
obligation; a `- run: |` block no longer swallows a sibling key.
- Several Dart directives on one line are all recorded.
- pubspec.yaml reads skip non-regular files, escaping symlinks and files over
1 MiB.
- fvm pins are found from the package up to the repo root, and flow-style
`flutter: {sdk: flutter}` selects `flutter test`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y budgets The Dart/Flutter stack and test-framework lines pushed ship, review and plan-eng-review past their parity caps. Tighter prose and a one-line `--numstat | awk` diff-size count replace the three-line `--stat | grep` form (same total), so all three fit without raising a threshold. Regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lute/.. dirs unresolved; flutter sdk match skips comments Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ubspec.yaml, as test infrastructure Every Dart package has a pubspec.yaml, so listing it reported test infrastructure for untested projects. Regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-directory and missing SDK in pregate and troubleshooting docs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CHANGELOG conflict: both sides added 1.91.101.0; this branch's entry moves to 1.91.102.0 above upstream's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
/trunk merge |
|
An error occurred while submitting your PR to the queue: |
|
Thanks @arlophoenix, this is a substantial piece of work, and we'd like Dart and Flutter as a first-class pregate stack, including the user-owned policy file outside the repo. Decision: yes, split into three PRs, each rebased on current main:
Main's lanes now run through |
|
Split into three PRs as requested, each rebased on current main with VERSION and CHANGELOG unstamped:
Closing this one in their favour. |
Why (in your own words)
gstack-pregatenow works on Dart and Flutter repos. Before this, every touched file in a Flutter repo endedPREGATE_NO_LANE, sogstack-ship-receipt write --pregaterefused on every ship..dartwas not a source extension,_test.dartwas not a test andpackage:imports never resolved, and a lane declared in.gstack/pregate.jsondidn't help, because a declared edge only counts when it points at a recognised test. This branch resolves imports through every trackedpubspec.yaml, runs each package's tests from its own root withflutter testordart test(throughfvmwhen pinned), and treats Flutter and Dart CI jobs as lanes. A repo that keeps.gstack/out of git can keep its policy in<GSTACK_HOME>/projects/<slug>/pregate.json, includinglocallanes for CI jobs a Mac can't otherwise clear. Separately,gstack-diff-scopematched no category for.dart, so a Flutter widget PR skipped the design, performance and security reviewers..dartnow counts as frontend and backend, and/reviewand/shipdetect a Dart stack and test framework.Live evidence
A throwaway local clone of a real Flutter app (flutter-bbtm), one commit touching
test/ui/widgets/bbtm_drawer_widget_test.dart, and the user policy{ "local": ["test.yml/flutter", "test.yml/guards", "qa-driver.yml/smoke"] }in a scratchGSTACK_HOME:Same clone, one commit touching
lib/theme.dart,lib/widgets/bbtm_drawer/view/bbtm_drawer.dartand the drawer test:On an earlier build of this branch, the import graph for the full repo recognised 984 tests in about 490 ms.
lib/theme.dartselected 783 tests, andpackages/cache/lib/cache.dartselected 743 throughpackage:cache.Scope
flutter test,dart test,dart run test,melos test,very_good test, with test paths resolved against each step'sworking-directory) and the user-owned policy withlocal. Device tests underintegration_test/andtest_driver/are named with a fix but never run, and a missing SDK isincompleterather than exit 127. Also.dartingstack-diff-scopeandlib/frontend-scope.ts(gstack-design-detectskips.dart, since impeccable reads web sources), Dart stack and test-framework detection in/reviewand/ship,docs/pregate.mdanddocs/troubleshooting.md. To keepship,reviewandplan-eng-reviewinside the parity-suite byte caps, the diff-size count became onegit diff --numstat | awkline (same total), and some existing resolver prose was shortened.test/pregate-dart.test.ts(15 tests covering the Dart graph, per-package run groups, an all-skipped run asZERO-RUN, fvm pins, working directories, device tests and the user policy; PATH shims stand in for the SDK).lib/theme.dartselects 783 files, which may exceed the 600 s default and endincomplete. The pregate tests stage on this PR's own diff, which selects 596 files and ran past the 600 s cap on this Mac. Paid evals: the PR profile selects 108 probes and 24 judges, and a fork can't dispatch them.bun run teston the final tree failed on this Mac: 367 failing tests across 91 files, mostly CSO, browse cookies, QA evidence and doctor, with other work loading the machine. I re-ran 30 of those files one at a time on this branch and on upstream/main. Every failure that reproduces alone fails with the same test names on upstream/main, and the rest pass alone. The other 61 files (1 to 6 failures each, none in code this branch touches) were not re-checked. Seven pregate tests inpregate-selection,pregate-invalidationandpregate-guardsfail the same way on upstream/main because they assume a Linux host, whereubuntu-*lanes are tierunit.Codex reviewed the whole branch once, and its two P1 findings are fixed. Fixes made after that pass were reviewed by Claude on the fix diff only.
Liveness proof (required for external contributors)
Screenshot attached by @arlophoenix:

Checklist
GSTACK PRtyped live into a real surface) or PR author is @garrytan (owner exemption)Not addressed
flutter testchildren can outlive it. An fvm pin whose SDK isn't installed falls back to the PATH SDK without saying so.pubspec.lockin the evidence lockfiles, a Dart case ingstack-first-task-detect, pub support in/cso, a Dart option in the test-bootstrap runtime question, and per-glob runners are left for follow-ups.Test plan
test/pregate-dart.test.ts15/15, plusdiff-scope,frontend-scopeparity,ship-test-detection-markers,gstack-design-detect,gen-skill-docs,skill-validation,touchfilesand the parity and budget suites, all passingbun run typecheckclean;gstack-regen checkfreshbun run testfull free suite: red on this Mac; see Scope for what was checked against upstream/maineval:bg:pr): deferred, not dispatchable from a fork🤖 Generated with Claude Code