Skip to content

fix(catalog): hold every control a served manifest can name - #27

Closed
antonbabenko wants to merge 2 commits into
masterfrom
fix/catalog-served-controls
Closed

antonbabenko wants to merge 2 commits into
masterfrom
fix/catalog-served-controls

Conversation

@antonbabenko

Copy link
Copy Markdown
Member

Problem

ctfkit report exited 2 on modules the compliance.tf registry serves today:

ctfkit: ctf.coverage.delegation_unresolved: ctfkit: finding ctf.coverage.delegation_unresolved names control vpc_security_group_remote_administration_ipv4, which the catalog 2026.9.3 does not hold

The embedded catalog held 294 controls, and served manifests name more. delegation_unresolved copied the manifest's control id into ctf.controlIds. ValidateFor then refused it at the emit boundary, and the whole run failed.

Changes

  • Catalog: regenerated from the current control catalog. It now holds 703 controls (version 2026.11.1), and clause text on the existing entries is refreshed. Existing resource-type placements do not change, apart from es_domain_error_logging_enabled, which moves to aws_opensearch_domain. Merged ids that the rule pack still references are kept.
  • report: when a manifest delegates a control id the catalog does not hold, the delegation_unresolved warning names the id in its message, carries no ctf.controlIds, and the report completes. ValidateFor stays strict for analyzer bugs.
  • module_source: a raw-resource finding lists only the controls the suggested module enforces, as ModuleRef.Controls already documents. The larger catalog places more controls on some covered types (for example s3_bucket_enforces_ssl on aws_s3_bucket). Without this filter, those findings would claim the module closes controls it does not.
  • Guard: TestEveryFixtureManifestControlIDResolves now also checks build.controls_not_applicable and the post_init fixtures. A fixture re-record from the live registry fails while the catalog lags the served manifests.
  • Re-recorded SARIF goldens. The diffs are the catalog version, plus clauses and frameworks for a few controls. No control id or rule changes.

Tests

  • TestReportToleratesAControlTheCatalogLacks (end to end) and TestDelegationUnresolvedToleratesAControlTheCatalogLacks: both fail with the exit-2 error before the fix.
  • TestRawResourceCarriesOnlyTheModulesControls: fails before the module_source change.
  • make check, go test -race ./..., golangci-lint v2.15.0 (0 issues), make vulncheck and prek run --all-files pass.
  • The reported reproduction (vpc, kms, sns, ecs, alb, ec2-instance and route53 from the ISO 27001 endpoint) now writes the coverage table. Run against the 2026.9.3 catalog, the unknown ids appear as warnings with no control id.

…exiting 2

A served manifest newer than the embedded catalog can delegate a control
id the catalog does not hold. delegation_unresolved put that id in
ctf.controlIds, the runner's emit check refused it, and report exited 2.
The finding now names the id in its message, carries no control id, and
the report completes.
The embedded catalog held 294 controls; served manifests name controls
beyond them, such as vpc_security_group_remote_administration_ipv4.
Regenerate it from the current control catalog (703 controls, version
2026.11.1), refreshing clause text, and re-record the SARIF goldens.

module_source now lists on a raw-resource finding only the controls the
suggested module enforces, since the catalog places further controls on
some covered types. The fixture-manifest guard also checks the
not-applicable list and the post_init fixtures.
@antonbabenko

Copy link
Copy Markdown
Member Author

Superseded by a smaller change; closing.

@antonbabenko
antonbabenko deleted the fix/catalog-served-controls branch November 9, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant