Component
Documentation
Bug Description
docs/architecture/workflows.md was updated in commit 18ce4c6 (PR #1834 — feat(agents): add Vally evaluation agents and prompts), but the npm command reference table is missing approximately 20 eval-related commands that exist in package.json. The documented eval commands cover only the lint and basic run variants:
Documented (10 eval entries):
eval:lint:vally, eval:lint:schema, eval:lint:text, eval:lint:safety, eval:lint:skills, eval:run, eval:run:skills, eval:run:agents, eval:run:scripts, eval:compare
Missing from the reference table:
| Command |
Purpose |
eval:presence |
Checks that every changed AI artifact has an eval-spec backlink |
eval:execute |
Runs vally evals for each changed AI artifact per the manifest |
eval:moderate |
Runs content moderation on a single artifact |
eval:moderate:corpus |
Runs corpus-wide content-moderation evaluation |
eval:moderate:artifacts |
Runs artifact-level moderation per manifest |
eval:moderate:test |
Runs Pester tests for the moderation runner |
eval:run:equivalence |
Runs the baseline-equivalence eval suite |
eval:behavior-prompts |
Runs behavior-conformance evals for prompts |
eval:behavior-instructions |
Runs behavior-conformance evals for instructions |
eval:behavior-skills |
Runs behavior-conformance evals for skills |
eval:equivalence |
Runs the baseline equivalence comparison |
eval:dashboard |
Renders equivalence results into a dashboard |
eval:agent |
Runs the agent-matrix evaluation |
eval:agent:matrix |
Runs the full nightly agent-matrix suite |
eval:agent:matrix:dryrun |
Dry-run mode for the nightly agent-matrix suite |
eval:agent:changed |
Runs agent-matrix only for changed AI artifacts |
eval:agent:dashboard |
Renders agent-matrix results into a dashboard |
eval:agent:dashboard:open |
Renders and opens the agent-matrix dashboard |
eval:agent:report |
Full agent-matrix run and dashboard open |
eval:agent:report:dryrun |
Dry-run agent-matrix run and dashboard open |
Expected Behavior
docs/architecture/workflows.md should include a complete row for each eval-related npm command so contributors can discover and understand the full evaluation toolchain from the reference table.
Steps to Reproduce
- Open
docs/architecture/workflows.md.
- Locate the npm command reference table.
- Compare against
package.json — eval:presence, eval:execute, all eval:moderate:*, all eval:agent:*, all eval:behavior-*, and eval:run:equivalence are absent.
Additional Context
Generated by Documentation Update Check · 1K AIC · ⌖ 22.3 AIC · ⊞ 34.4K · ◷
Component
Documentation
Bug Description
docs/architecture/workflows.mdwas updated in commit18ce4c6(PR #1834 — feat(agents): add Vally evaluation agents and prompts), but the npm command reference table is missing approximately 20 eval-related commands that exist inpackage.json. The documented eval commands cover only the lint and basic run variants:Documented (10 eval entries):
eval:lint:vally,eval:lint:schema,eval:lint:text,eval:lint:safety,eval:lint:skills,eval:run,eval:run:skills,eval:run:agents,eval:run:scripts,eval:compareMissing from the reference table:
eval:presenceeval:executeeval:moderateeval:moderate:corpuseval:moderate:artifactseval:moderate:testeval:run:equivalenceeval:behavior-promptseval:behavior-instructionseval:behavior-skillseval:equivalenceeval:dashboardeval:agenteval:agent:matrixeval:agent:matrix:dryruneval:agent:changedeval:agent:dashboardeval:agent:dashboard:openeval:agent:reporteval:agent:report:dryrunExpected Behavior
docs/architecture/workflows.mdshould include a complete row for each eval-related npm command so contributors can discover and understand the full evaluation toolchain from the reference table.Steps to Reproduce
docs/architecture/workflows.md.package.json—eval:presence,eval:execute, alleval:moderate:*, alleval:agent:*, alleval:behavior-*, andeval:run:equivalenceare absent.Additional Context
18ce4c6(PR feat(agents): add Vally evaluation agents and prompts #1834)..github/workflows/pr-validation.ymland.github/workflows/eval-validation.yml.scripts/README.mdgap for evals orchestration scripts.docs/contributing/evals-ci.mddocuments the eval CI contract but does not serve as a full npm command reference.