Skip to content

v1.91.135.0 fix(cso-eval): producer artifact inventory recognizes dependency-closure file names - #3265

Merged
capy-ai[bot] merged 2 commits into
mainfrom
cso-producer-closure-names
Oct 18, 2026
Merged

capy-ai[bot] merged 2 commits into
mainfrom
cso-producer-closure-names

Conversation

@garrytan

Copy link
Copy Markdown
Owner

In the second paid /cso runtime qualification (evaluator run 39107626021), the rails-constantize-param fixed-variant comprehensive cell failed with REDACTION_FAILED: artifact path component 4 under "ID/ID/dependency-closures" looks like a secret. That failure counts against matchedCompleteMatrix, mandatoryReports and allCoreColdStarts.

Cause. Repairs reach verify since v1.91.123.0 (#3240). Once they do, the helper retains dependency closures as <finding id>-<before|after>-<plan hash>.json (closureStateFile in lib/cso/cli.ts). assertProducerArtifactPath accepts a path component without redaction only when it matches HELPER_ID_COMPONENT. That pattern allows a word prefix before hex IDs, but not a word between two of them. So the closure name went to the secret redactor, which read the 64-hex finding ID as a credential.

Fix. HELPER_ID_COMPONENT gains one alternative: exactly [a-f0-9]{64}-(before|after)-[a-f0-9]{16}.json. Any other word in that position still goes through the redactor.

Test. The existing artifact-name test in test/cso-eval.test.ts now uses a closure name chosen so the redactor flags it. It checks that the before and after forms are accepted, and that -during- still fails with "looks like a secret".

Checks: bun run typecheck; test/cso-eval.test.ts, test/cso-eval-producer.test.ts and the module-size ratchet (74 pass).

Stamped v1.91.135.0 (slot from GStack Minder).

Open in Capy

@trunk-io

trunk-io Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Evals: ✅ PASS

[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 108/108 · behavior 0/0 · judge 24/24 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 8m41s · cost $93.34 known + 54 of 132 trial(s) cost unknown

121 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $93.34 rule/judge cost | reconcile exit: 0

Profile: pr / mode dependents; selected behaviors: 108, judges: 24. Deferred to scheduled/release coverage: 138 behaviors and 0 changed prompt files. Deferred checks did not run and receive no PR-pass credit.

Derived dependents (scripts/pr-dependencies.ts):

  • scripts/cso-eval-producer.ts: 246 case(s), 24 judge(s)
Rule and judge shards
Shard Automated result Manual/unscored Executed Reused Status Cost
e2e-deploy/skill-e2e-deploy--setup-deploy-workflow 1/1 0 1 0 ✅ $0.43
e2e/skill-e2e-coverage-audit 2/2 0 2 0 ✅ $0.78
llm-judge/skill-llm-eval 24/24 0 24 0 ✅ $1.44
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-current 1/1 0 1 0 ✅ $3.92
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-store 1/1 0 1 0 ✅ $3.59
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-completion 1/1 0 1 0 ✅ $3.9
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-plan-checks 1/1 0 1 0 ✅ $2.02
e2e/skill-e2e-test-value 3/3 0 3 0 ✅ $2.23
e2e-qa-callers/skill-e2e-qa-callers--review-exploratory-small-cli 1/1 0 1 0 ✅ $2.25
e2e-qa-workflow/skill-e2e-qa-workflow--qa-only-no-fix 1/1 0 1 0 ✅ $1.75
e2e/skill-e2e-shared-libs--shared-libs-read-only 1/1 0 1 0 ✅ $0.67
e2e-docsync-spawned/skill-e2e-docsync-spawned 1/1 0 1 0 ✅ $1.28
e2e-deploy/skill-e2e-deploy--land-and-deploy-first-run 1/1 0 1 0 ✅ $0.85
e2e-deploy/skill-e2e-deploy--land-and-deploy-review-gate 1/1 0 1 0 ✅ $1.2
e2e-review-army/skill-e2e-review-army--review-army-migration-safety 1/1 0 1 0 ✅ $1.27
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-stale-after 1/1 0 1 0 ✅ $1.07
e2e-learnings/skill-e2e-learnings 1/1 0 1 0 ✅ $0.32
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-late-result 1/1 0 1 0 ✅ $0.94
e2e-review-army/skill-e2e-review-army--review-army-json-findings 1/1 0 1 0 ✅ $0.24
e2e-deploy/skill-e2e-deploy--benchmark-workflow 1/1 0 1 0 ✅ $0.65
e2e/skill-e2e-investigate-owned-completion 1/1 0 1 0 ✅ $0.24
e2e/skill-e2e-shared-libs-paths--shared-libs-review-index-flags 1/1 0 1 0 ✅ $2.58
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-small-cli 1/1 0 1 0 ✅ $1.89
e2e-review-army/skill-e2e-review-army--review-army-perf-n-plus-one 1/1 0 1 0 ✅ $1.01
e2e-triage/skill-e2e-triage 1/1 0 1 0 ✅ $0.48
e2e-qa-workflow/skill-e2e-qa-workflow--qa-bootstrap 1/1 0 1 0 ✅ $0.25
e2e-third-party-actions/skill-e2e-third-party-actions 5/5 0 5 0 ✅ $1.46
e2e-design/skill-e2e-design--plan-design-review-no-ui-scope 1/1 0 1 0 ✅ $0.43
e2e-deploy/skill-e2e-deploy--canary-workflow 1/1 0 1 0 ✅ $0.4
e2e/skill-e2e-investigate-owned-termination 2/2 0 2 0 ✅ $0.51
e2e/skill-e2e-ship-hook-consent 2/2 0 2 0 ✅ $0.7
e2e-review-army/skill-e2e-review-army--review-army-delivery-audit 1/1 0 1 0 ✅ $0.35
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-timeout-unsettled 1/1 0 1 0 ✅ $0.53
e2e-session-intelligence/skill-e2e-session-intelligence 4/4 0 4 0 ✅ $0.75
e2e-design/skill-e2e-design--design-review-detector-shim 1/1 0 1 0 ✅ $0.4
e2e-plan-tune/skill-e2e-plan-tune 1/1 0 1 0 ✅ $0.39
e2e-safety-pair-agent/skill-e2e-safety-pair-agent 1/1 0 1 0 ✅ $0.29
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-launch-failure 1/1 0 1 0 ✅ $0.51
e2e/skill-e2e-ship-hook-refresh 1/1 0 1 0 ✅ $0.26
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-unavailable 1/1 0 1 0 ✅ $1.86
e2e/skill-e2e-plan--plan-review-report 1/1 0 1 0 ✅ $1.09
e2e-retro/skill-e2e-retro 1/1 0 1 0 ✅ $1.18
e2e-qa-functional-fix/skill-e2e-qa-functional-fix 2/2 0 2 0 ✅ $3.1
e2e/skill-e2e-ship-skip 1/1 0 1 0 ✅ $0.6
e2e-qa-workflow/skill-e2e-qa-workflow--qa-quick 1/1 0 1 0 ✅ $1.75
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-stale-before 1/1 0 1 0 ✅ $1.04
e2e-review-attribution/skill-e2e-review-attribution 3/3 0 3 0 ✅ $0.93
e2e/skill-e2e-shared-libs-paths--shared-libs-review-prior-coverage 1/1 0 1 0 ✅ $2.43
e2e/skill-e2e-shared-libs--shared-libs-review-lifecycle 1/1 0 1 0 ✅ $2.37
e2e-safety-codex-consult/skill-e2e-safety-codex-consult 1/1 0 1 0 ✅ $0.99
e2e-deploy/skill-e2e-deploy--land-and-deploy-workflow 1/1 0 1 0 ✅ $1.88
e2e/skill-e2e-workflow 4/4 0 4 0 ✅ $3.06
e2e-review-army/skill-e2e-review-army--review-army-quality-score 1/1 0 1 0 ✅ $0.35
e2e-hermetic/skill-e2e-hermetic-canary 2/2 0 2 0 ✅ $0.03
e2e-browse/skill-e2e-bws 6/6 0 6 0 ✅ $0.86
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-missing-asset 1/1 0 1 0 ✅ $1.21
e2e-design/skill-e2e-design--design-review-plugin-handoff 1/1 0 1 0 ✅ $0.5
e2e-design/skill-e2e-design--design-review-detector-shim-dom 1/1 0 1 0 ✅ $0.72
e2e-safety-ship-evidence/skill-e2e-safety-ship-evidence 1/1 0 1 0 ✅ $0.51
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-missing-marker 1/1 0 1 0 ✅ $0.48
e2e/skill-e2e-shared-libs--shared-libs-unsupported-git 1/1 0 1 0 ✅ $0.72
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-recovery 1/1 0 1 0 ✅ $1.02
e2e-review/skill-e2e-review 2/2 0 2 0 ✅ $1.14
e2e/skill-e2e-shared-libs--shared-libs-review-revalidation 1/1 0 1 0 ✅ $4.82
e2e-qa-functional/skill-e2e-qa-functional 2/2 0 2 0 ✅ $2.34
e2e-cso/skill-e2e-cso 1/1 0 1 0 ✅ $1.31
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-late-input 1/1 0 1 0 ✅ $2.36
e2e-skillify/skill-e2e-skillify 3/3 0 3 0 ✅ $2.51
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-failure 1/1 0 1 0 ✅ $1.99
e2e/skill-e2e-shared-libs-paths--shared-libs-review-path-eligibility 1/1 0 1 0 ✅ $3.96
Fail-closed reconciliation
6s  test/skill-e2e-triage.test.ts
  slice 14  passed             67s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-timeout-unsettled
  slice 14  passed             56s  test/skill-e2e-ask-user-question-format-compliance.test.ts
  slice 14  passed             64s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-launch-failure
  slice 14  passed             40s  test/skill-e2e-session-intelligence.test.ts
  slice 14  passed             46s  test/skill-e2e-ship-hook-consent.test.ts
  slice 14  passed             43s  test/skill-e2e-investigate-owned-termination.test.ts
  slice 14  passed             38s  test/skill-e2e-design.test.ts#design-review-detector-shim
  slice 14  passed             38s  test/skill-e2e-deploy.test.ts#canary-workflow
  slice 14  passed             34s  test/skill-e2e-ship-hook-refresh.test.ts
  slice 14  passed             41s  test/skill-e2e-design.test.ts#plan-design-review-no-ui-scope
  slice 14  passed             36s  test/skill-e2e-review-army.test.ts#review-army-delivery-audit
  slice 14  passed             31s  test/skill-e2e-safety-pair-agent.test.ts
  slice 14  passed             33s  test/skill-e2e-qa-workflow.test.ts#qa-bootstrap
  slice 15  passed             45s  test/skill-e2e-learnings.test.ts
  slice 15  passed             28s  test/skill-e2e-diagram.test.ts
[test:paid] collector final outcomes: 121 executed, 0 reused; 121 passed, 0 failed, 0 manual accepted (unscored; no score-cache credit) (121 attempt records from 70 collectors; every record counts)
[test:paid] deselected testcases, zero credit (312):
[test:paid]    298  sibling case of a case shard (its own shard runs it)
[test:paid]      8  periodic-tier case (this lane runs gate)
[test:paid]      6  Bun hook placeholder of a skipped describe block (not a test)
[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 108/108 · behavior 0/0 · judge 24/24 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 8m41s · cost $93.34 known + 54 of 132 trial(s) cost unknown
[test:paid] exercised-path map (shadow, non-blocking): 10 MAP_GAP line(s) over 100 executed case(s) with recorded commands
  MAP_GAP document-release bin/gstack-banned-terms: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP land-and-deploy-review-gate bin/gstack-approval-classes: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP plan-ceo-review-plan-mode bin/gstack-plan-reality: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP review-army-migration-safety bin/gstack-approval-classes: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP review-enum-completeness bin/gstack-approval-classes: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP review-gate-integrity bin/gstack-approval-classes: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP review-gate-integrity bin/gstack-banned-terms: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP review-gate-integrity bin/gstack-contributor-mode: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP review-sql-injection bin/gstack-approval-classes: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
  MAP_GAP shared-libs-review-revalidation bin/gstack-approval-classes: non-blocking for this PR; the next census refresh widens the map (docs/TESTING_INTERNALS.md#exercised-path-map)
[test:paid] report: every planned shard accounted and passed

Sliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry.

@capy-ai
capy-ai Bot merged commit 25dbe34 into main Oct 18, 2026
82 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant