Enforce Preset/Extension coding standards through scaffolds, validation, CI, and review

[bigsmartben]

Problem

The repository now has a maintainer coding standard for Preset and Extension implementation at docs/preset-extension-coding-standard.md, covering component ownership, command/template/schema/validator separation, platform-neutral command sources, lifecycle safety, and testing.

Documentation alone does not make these requirements enforceable. Contributors can still introduce platform-specific command logic, incomplete manifests, prompt-only contracts, unsafe paths, lifecycle regressions, or new components that copy historical patterns instead of the intended architecture.

Goal

Turn the Preset/Extension coding standard into an enforceable contribution loop:

scaffold → local validator → contract tests → required CI check → maintainer review → release gate

New files must comply. Modified legacy files must converge within the changed scope. Untouched historical violations should be reported without forcing unrelated cleanup.

Scope

1. Component-standard validator

Add a dependency-free repository validator, for example:

scripts/validate-component-standard.py
tests/test_component_coding_standard.py

The validator must support changed-file enforcement, for example:

.venv/bin/python scripts/validate-component-standard.py --changed-from <base-ref>

Blocking checks should include objectively testable rules:

• Extension and Preset manifests parse and contain required fields.
• Declared command/template/schema/script paths exist and remain inside the package root.
• Extension command IDs follow speckit.<extension-id>.<command>.
• Command frontmatter is valid YAML.
• Universal command sources use supported placeholders and do not leave unresolved placeholders after projection.
• Preset/Extension command sources do not hardcode agent output directories such as .claude/commands, .agents/skills, or .github/agents.
• Package-controlled paths reject absolute paths, .. traversal, and symlink escape.
• Structured artifact declarations have traceable schema and contract-test coverage where required by the component contract.
• Preset strategies are compatible with their asset type.

Heuristic checks, such as prompt length, should report warnings rather than block automatically.

2. Incremental adoption

Implement enforcement without requiring a repository-wide migration in the same change:

• New files receive the full check set.
• Modified legacy files must comply within the modified scope.
• Untouched legacy violations are reported but do not block the PR.
• Any exception mechanism must be explicit and include a reason, owner, and follow-up issue; ad hoc inline suppression is not sufficient.

3. Correct-by-default scaffolds

Update the maintained examples/scaffolds:

• extensions/template/
• presets/scaffold/

They should demonstrate:

• Manifest/file alignment.
• Command, Contract, Template, Schema, Validator, and Script ownership.
• Platform-neutral placeholders.
• The concise command structure: User Input, Goal, Normative Authority, Operating Boundaries, Procedure, Validation, Report.
• Focused contract tests.
• README, CHANGELOG, and LICENSE expectations for distributable components.

4. Contract-test requirements

Add or extend tests that prove:

• Manifest and declared-file consistency.
• Valid command frontmatter.
• Valid and invalid structured fixtures.
• PASS/BLOCKED states and stable blocker codes.
• Install, repeated install, failed-update rollback, enable/disable, and remove behavior.
• Hook registration, ordering, and orphan cleanup.
• Preset priority and replace/prepend/append/wrap behavior.
• Path traversal, symlink, and user-modified-file protection.
• Shared registrar changes preserve Markdown, TOML, YAML, Skills, and Copilot outputs.

Tests must not access the real network.

5. CI enforcement

Add a dedicated CI check for component standards. It should:

• Resolve the PR base commit reliably.
• Run changed-file validation.
• Run focused component-standard tests.
• Produce actionable path-and-rule diagnostics.
• Be suitable for configuration as a required branch-protection check.

Community integration and release validation paths that package Presets or Extensions should call the same validator instead of implementing divergent checks.

6. PR review evidence

Update .github/PULL_REQUEST_TEMPLATE.md with concise sections for:

• Component type and rationale.
• Contract authority map.
• Platform-specific behavior, if any.
• Install/update/rollback/remove evidence.
• Prompt behavior and manual agent test evidence when command behavior changes.

Subjective architecture decisions remain maintainer-review responsibilities; the validator should not attempt to decide whether prose is well designed through fragile regex checks.

Non-goals

• Migrating every existing Preset and Extension in one PR.
• Treating line count as a hard quality metric.
• Adding a new runtime dependency.
• Changing the public Preset/Extension manifest schema unless a concrete enforcement requirement cannot be implemented without it.
• Replacing component-specific contract tests with one generic linter.
• Changing release versions or catalog release URLs outside release preparation.

Implementation constraints

• Reuse PyYAML and existing repository utilities where possible.
• Keep CLI/CI orchestration separate from importable validation logic.
• Preserve offline test execution.
• Keep path validation and symlink handling consistent with existing shared policies.
• Do not duplicate agent metadata; the integration registry remains the platform source of truth.
• Context/instruction file handling remains owned by the opt-in agent-context extension.

Acceptance criteria

• A maintainer can run one documented local command to validate changed Preset/Extension files.
• New or modified non-compliant files fail with an actionable rule ID and path.
• Untouched historical violations do not block unrelated PRs.
• Extension and Preset scaffolds generate examples that pass the validator.
• Objective rules are automated; semantic component-boundary decisions remain explicit review items.
• Focused tests cover validator behavior, including false-positive and false-negative regression cases.
• Lifecycle, security, and cross-agent projection invariants remain covered.
• CI runs the validator against the PR diff and is ready to become a required check.
• The PR template requests component architecture, lifecycle, and prompt-test evidence.
• Community/release validation uses the shared validator where applicable.
• Documentation explains local execution, CI behavior, warnings, failures, and exceptions.
• git diff --check, Ruff, Markdown checks, focused tests, and the repository test suite pass.

Suggested rollout

1. Land the validator in report-only mode and update scaffolds.
2. Enable blocking for new and modified files.
3. Reduce the historical report backlog in separate focused issues.
4. Enable broader enforcement only after the baseline is intentionally cleared.

AI disclosure

Posted on behalf of @bigsmartben by OpenAI Codex (model: GPT-5, autonomous). Codex analyzed the repository conventions and drafted this issue; implementation and review remain subject to maintainer validation.