chore(standards): guard-test fixtures must use the real persisted shape (ag-jwtx #test-standard-names-real-shape-requirement)#776
Merged
Conversation
…ed shape (ag-jwtx #test-standard-names-real-shape-requirement) Skip/dedup/consumed/idempotency/regression guard tests must build fixtures by round-tripping a real persisted sample (production writer -> reader) or asserting against a checked-in real example -- never a hand-built in-memory constructor that sets a marker at a granularity the on-disk format never emits. Encodes the ag-mjlg / PR #652 false-green: a guard test set `consumed` at item-level while next-work.jsonl marks it at batch-level, so CI passed on a shape production can't produce. - skills/standards/references/test-pyramid.md: new "Fixture Fidelity" section - skills/standards/references/go.md + .claude/rules/go.md: Test-Conventions bullet - skills/pre-mortem/references/mandatory-checks.md: two flags in the Test Pyramid Coverage Check (Mandatory) Closes-scenario: ag-jwtx#test-standard-names-real-shape-requirement Bounded-context: BC1-Corpus Evidence: skills/standards/references/test-pyramid.md
…(ag-jwtx)
The references edits in this PR (pre-mortem/references/mandatory-checks.md,
standards/references/{go,test-pyramid}.md) drifted the codex twin hashes
for both skills. Regenerated via scripts/regen-codex-hashes.sh --only
pre-mortem,standards. Fixes the skill-gates 'codex hashes (no drift)' check.
Bounded-context: BC2-Skills
boshu2
added a commit
that referenced
this pull request
Jun 6, 2026
…nk ../ (ag-eatf #safe-paths-doclink-filter) (#796) ## What `scripts/skill-eval.sh` treated ms's `safe-paths` rule as blocking on **every** `../`. That rule is a blunt regex, and `../` relative markdown doc-links are the repo-wide SKILL.md convention (47+ skills, e.g. `[x](../other/SKILL.md)`, `../../docs/...md`). SKILL.md is documentation, not executed code, so those are false positives — they redded `skill-eval` → `summary` on **every skills-touching PR** (blocked #776; gated #759/#764/#767 in this session's drain). ## Fix Added a file-level filter: `safe-paths` stays **blocking only when a `../` survives stripping (1) markdown inline-link targets `](...)` and (2) relative doc-path tokens (`*.md/.markdown/.mdx/.txt/.rst`)** — i.e. a real, non-doc `../`. Otherwise the findings downgrade to advisory annotations (announced, not silenced). Net effect: **purely removes false positives; real protection is preserved.** A genuine traversal (e.g. `../../../../etc/passwd` in a description) still blocks — verified. No regression to the other blocking rules. ## Evidence - `bats tests/scripts/skill-eval.bats` → 12/12 pass (10 prior + 2 new ag-eatf cases: doc-links downgrade→PASS; real non-doc `../`→BLOCK). - Manual: `agent-native` and `domain` skills (previously red on safe-paths) now PASS; a mixed real-threat fixture BLOCKs. Closes-scenario: ag-eatf#safe-paths-doclink-filter Bounded-context: BC2-Validation Evidence: scripts/skill-eval.sh
boshu2
added a commit
that referenced
this pull request
Jun 6, 2026
…d hole) The embedded-sync step ran only on hooks/go/ci changes, but the embedded files mirror skills/standards/references/*, skills/using-agentops/SKILL.md and skills/compile/scripts/compile.sh. A skills-only PR could drift the embeds undetected — exactly how #776 reached main with empty embedded standards files (main-red 2026-06-06). Add skills + contracts to the gate condition so this class can't slip through PR CI again.
boshu2
added a commit
that referenced
this pull request
Jun 6, 2026
…k follow-up) (#799) ## Main is red — hotfix `cli/embedded/skills/standards/references/{go,test-pyramid}.md` were **empty (0 bytes)** on main HEAD; #776's merge truncated them. `contracts-sync` / `correctness` run the full embedded-sync check on main-push (changed-files-scoped on PRs, so it slipped through during the batch drain) → main Validate failed (run on 3df599d, def8fc9). ## Fix - Re-synced both embedded files from their sources via the `cli/Makefile` recipe (`cp ../skills/standards/references/*`). `scripts/validate-embedded-sync.sh` now reports **All embedded files are in sync**. - Corrected that script's stale fix hint: `make sync-embedded` (no such target) → `make sync-hooks` (the real recipe). ## Evidence `bash scripts/validate-embedded-sync.sh` → "All embedded files are in sync." Bounded-context: BC5-Runtime Evidence: cli/embedded/skills/standards/references/go.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Encodes a test standard from ag-jwtx: skip/dedup/consumed/idempotency/regression guard tests must build fixtures from the real persisted data shape — round-trip a real sample (production writer → reader) or assert against a checked-in real example — never a hand-built in-memory constructor that sets a marker at a granularity the on-disk format never emits.
Why
ag-mjlg(PR #652) shipped a next-work materialize over-creation bug (44 vs 16) green becauseTestNextWorkMaterialize_SkipsConsumedsetconsumedat the per-item level while realnext-work.jsonlmarks it at the batch level. The guard exercised a shape production never produces → false green. This converts that one-off into a durable standard + a pre-mortem flag (catch→encode→never-repeat).Changes (docs/standards only)
skills/standards/references/test-pyramid.md— new Fixture Fidelity section (the standard + how-to-comply + the pre-mortem flag).skills/standards/references/go.md+.claude/rules/go.md— Test-Conventions bullet (kept in parity).skills/pre-mortem/references/mandatory-checks.md— two flags added to the Test Pyramid Coverage Check (Mandatory) table.Verification
markdownlint --config .markdownlint.jsonon all four files: clean.scripts/generate-registry.sh --check: registry up to date (no new reference files; existing files edited).Scenario
Closes-scenario: ag-jwtx#test-standard-names-real-shape-requirement
Bounded-context: BC1-Corpus
Evidence: skills/standards/references/test-pyramid.md