fix(quality): name a missing mutation tool instead of reporting non-zero - #4576
Merged
Conversation
The run left a print-based script at the repository root instead of a test, so nothing asserted the new behaviour and the diff carried no test file at all. Replace it with pytest cases that cover both halves: that exit code 127 survives the return path, and that GateRunner turns it into a command_not_found status rather than the fail it used to report. The two-tuple cases pin the shape every existing caller unpacks.
The mutation gate unpacked the exit code and never read it, so a run whose tool is not installed fell through the unparseable-score branch and printed "Exit: non-zero" -- the same line a mutation run that genuinely scored badly produces. Report 127 by name. Also drops two debug scripts the change left at the repository root.
bernstein-the-conductor
Bot
requested a review
from chernistry
as a code owner
August 25, 2026 20:47
chernistry
enabled auto-merge
August 25, 2026 20:48
Contributor
Docs drift reportAGENTS.md / CLAUDE.md / CONVENTIONS.md sync drift
Run |
Contributor
Context-file staleness reportChecked 7 curated context file(s) at
|
| Commit | Lines | Subject |
|---|---|---|
62c88b2b7444 |
287 | feat(review): run the fix-until-green contour inside review --pipeline (#4481) (#4518) |
86e08b047942 |
162 | feat(evolve): add --dry-run flag and failure-pattern draft GitHub sync (#4550) |
e5912b1a8ab2 |
158 | fix(pr): describe the pull request from the change, not the session (#4486) |
c52f6752c4fb |
70 | feat(routes): remove legacy /dashboard in favor of gui serve (#4395) (#4430) |
a6b0c3382eed |
31 | feat(skills): recognize Agent Plugins directory layout in skills install (#3772) (#4448) |
src/bernstein/core/quality/AGENTS.md (newly flagged in this range)
- Scope:
src/bernstein/core/quality/ - Last touched:
62c88b2b7444(2026-08-25T10:29:00Z) — feat(review): run the fix-until-green contour inside review --pipeline (review --pipeline stops at a verdict; the fix-until-green loop and its provenance live outside the product #4481) (feat(review): run the fix-until-green contour inside review --pipeline (#4481) #4518) - Since then: 7 commit(s) in scope; net diff 5 file(s), +214/-27 (241 lines, threshold 200)
- Top commits by churn:
| Commit | Lines | Subject |
|---|---|---|
40a14c82ea8a |
101 | [WIP] backend-9877ebfe partial work |
9dc77120d5ac |
76 | fix(janitor): resolve test_passes paths against the tree before running them (#4554) |
b3b8e407c901 |
63 | refactor(quality): return one shape from the gate command runner |
35c6c18e1672 |
38 | fix(janitor): judge signal-less tasks instead of skipping them (#4563) |
10288597d603 |
31 | [WIP] backend-21dcf9f9 partial work |
src/bernstein/core/security/AGENTS.md
- Scope:
src/bernstein/core/security/ - Last touched:
34090c678fb6(2026-08-24T19:45:14Z) — docs: reset staleness clock for curated context files (context-file staleness: curated context files lag their subtrees #4442) (docs: reset staleness clock for curated context files (#4442) #4477) - Since then: 2 commit(s) in scope; net diff 4 file(s), +252/-23 (275 lines, threshold 200)
- Modules added:
src/bernstein/core/security/key_derivation.py - Top commits by churn:
| Commit | Lines | Subject |
|---|---|---|
b01b291c82f0 |
244 | feat(security): add HKDF-SHA256 per-store key derivation with scheme versioning (#4483) |
4190a9212f0c |
31 | feat(security): opt-in host restriction for third-party-derived URLs (#4555) |
tests/AGENTS.md
- Scope:
tests/ - Last touched:
34090c678fb6(2026-08-24T19:45:14Z) — docs: reset staleness clock for curated context files (context-file staleness: curated context files lag their subtrees #4442) (docs: reset staleness clock for curated context files (#4442) #4477) - Since then: 37 commit(s) in scope; net diff 79 file(s), +7617/-437 (8054 lines, threshold 200)
- Modules added:
tests/fixtures/receipt-vectors/_build_audit_receipt_vectors.py,tests/unit/cli/test_evolve_dry_run.py,tests/unit/core/agents/test_reap_merge_quality_gates.py,tests/unit/core/replay/test_journal_unauthenticated_fields.py,tests/unit/lineage/test_run_graph.py,tests/unit/quality/review_pipeline/test_contour.py,tests/unit/quality/review_pipeline/test_ruleset.py,tests/unit/quality/review_pipeline/test_runner_ruleset.py,tests/unit/quality/test_gate_command_not_found.py,tests/unit/review/test_review_receipt_chain.py(+20 more) - Modules removed:
tests/unit/test_dashboard_external_scripts.py,tests/unit/test_dashboard_responsive.py - Top commits by churn:
| Commit | Lines | Subject |
|---|---|---|
62c88b2b7444 |
977 | feat(review): run the fix-until-green contour inside review --pipeline (#4481) (#4518) |
b01b291c82f0 |
618 | feat(security): add HKDF-SHA256 per-store key derivation with scheme versioning (#4483) |
5887cef21f48 |
535 | fix(config): resolve run overrides from an untracked overlay (#4487) |
e5912b1a8ab2 |
457 | fix(pr): describe the pull request from the change, not the session (#4486) |
3fd846aaf46a |
456 | feat(receipt): commit run-receipt test vectors exercised by CI (#4204) (#4489) |
To clear a flag, review the context file against its scope and touch it in a commit — update it, or make a reconfirmation-only edit. Either way the "is this still true?" review leaves a commit that resets the clock.
`bernstein agents-md sync` writes the checked-out branch as the repository's default, so running it inside a fleet run rewrote five context files to say `Default branch: run-<stamp>`. The drift guard compares them against a generator that says `main`, so the run failed Repo hygiene and two test shards on a line no part of the change touched. Restore them.
`_run_command` returned two values for an ordinary result and three when the exit code was 127, so every caller had to re-derive the shape before it could read anything: eight sites carried `result[2] if len(result) == 3 else None`. Two callers did not - the dead-code gate and the impacted-tests gate unpacked two values, and would have raised `ValueError` on the very exit code the signal exists to report. The variable shape also broke two existing tests that unpack the runner directly. It now always returns `(ok, detail, exit_code)`. `NO_EXIT_CODE` stands in for the paths where no process reported one - a timeout kill, an OS spawn failure - so the third element is never absent and never a guess. The eight guards collapse into plain unpacking, and the two latent crash sites go with them. Tests assert the exit code on every return path: 127 for a missing command, 3 for `exit 3`, 0 for success, and `NO_EXIT_CODE` for a timeout, so an ordinary failure stays distinguishable from a missing tool in both directions.
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Aug 25, 2026
Contributor
Bisect-on-red triageThe CI workflow failed on
Auto-applied label: Posted automatically by |
github-actions
Bot
added
the
regression
Contributor
Bisect-on-red triageThe CI workflow failed on
Auto-applied label: Posted automatically by |
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4548
Problem
A quality gate whose command cannot be found fails identically to a gate whose command ran and found violations. The orchestrator then treats a misconfiguration as a code defect and spends the run trying to fix code that was never the problem.
Change
No changes recorded for this session.
Verification
Provenance
sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855unrecordedbernstein review-receipt verify --pr <this PR> --issue <issue.md> --diff <pr.diff>Generated from Bernstein session
unknown.bernstein-session-id: unknown
Made by bernstein v3.18.0 - unattended run
run-20260825T124209p2795288Z, no operator in the loop.