Skip to content

feat(quality): add dynamic test quality runners - #165

Merged
vicotrbb merged 6 commits into
mainfrom
codex/m0-09-dynamic-test-quality-runners
Jul 30, 2026
Merged

feat(quality): add dynamic test quality runners#165
vicotrbb merged 6 commits into
mainfrom
codex/m0-09-dynamic-test-quality-runners

Conversation

@vicotrbb

Copy link
Copy Markdown
Contributor

Closes #10.

Scope

Semantic owner: Quality Engineering.

This implements the EG-DYNAMIC descriptor registry and its public cargo xtask quality / immutable evidence-report seam. It adds bounded target identities, retained corpus/seed/schedule/minimized-failure/output-protocol data, controlled target execution, and fail-closed cancellation, timeout, malformed-output, bounded-capture, missing-tool, stale-registry, and no-retry behavior. It also adds a focused EG-COVERAGE detector-availability contract audit without changing coverage policy or production coverage commands.

Qualification status

Implemented, not Qualified. This change makes no Qualification Cell, merge-eligibility, approval, or release-readiness claim.

Evidence

Focused checks passed:

  • cargo test --locked --package xtask --test foundational_scope_activation -- m0_09_ — 15 tests
  • cargo test --locked --package xtask dynamic_quality — registry unit coverage
  • cargo fmt --check
  • cargo clippy --locked --package xtask --all-targets --all-features -- -D warnings
  • policy JSON validation and git diff --check
  • clean isolated cargo xtask quality --profile pr for ea26c13; EG-DYNAMIC and EG-TEST passed.

EG-COVERAGE is intentionally not selected by the PR profile. Its contract is covered through existing public EXT/focused tests plus the new detector-availability fixture; no coverage threshold, selection, exception, or command changed.

Preserved diagnostic failures were not erased: an initial root-target PR attempt failed EG-EVIDENCE on historical retained evidence, and an isolated b05a142 attempt failed EG-TEST because its summary parser classified 3 slow as noncanonical skips. The final exact ea26c13 isolated PR-profile attempt passed.

@vicotrbb

Copy link
Copy Markdown
Contributor Author

M0-09 final review remediation is pushed at exact head 4637e3bd9d8260bfebef6d4509fd5d1a93b4659e.

Implemented narrowly:

  • frozen seven-kind dynamic capability catalog, separate from the two active M0 targets;
  • independent retained-evidence verifier with its own grammar parsing and target/plan/tool/version/argv/environment/input/catalog/registry/result recomputation;
  • fail-closed copied-source tamper coverage for plan, command, environment, result, and target order/cardinality;
  • EG-DYNAMIC remains PR/EXT only and is absent from QUAL execution;
  • canonical Nextest (N slow) passing summaries are accepted while malformed, skipped, and failure-bearing summaries remain rejected.

Focused evidence:

  • M0-09 public-seam suite: 22/22 passed;
  • independent verifier subset: 4/4 passed;
  • dynamic unit tests: 2/2 passed;
  • focused llvm-cov smoke passed;
  • fmt, clippy, JSON, and diff checks passed.

Fresh unprepared exact-head push from a new detached worktree and new external Cargo target passed every authoritative PR gate: 501/501 Nextest tests passed (2 canonical slow annotations), 16 doctests passed, EG-DYNAMIC/EG-EVIDENCE/EG-TEST passed. Diagnostic evidence: /private/tmp/positron-m0-09-final-push-worktree.9kanE5/target/quality/evidence/1785400960192-4637e3bd9d82-71800.json (local diagnostic, not a trusted-CI merge claim).

Preserved failed attempts: the shared worktree first failed stale historical EG-EVIDENCE plus two unrelated M0-08 reap timing tests; the first isolated run passed 501/501 but exposed the pre-existing slow-summary parser regression. Neither failure was erased. GitHub CI run 30528207118 is now running for the exact pushed head.

@vicotrbb
vicotrbb merged commit 542f383 into main Jul 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[M0-09] Implement dynamic and test-quality runners

1 participant