Skip to content

Add SAST scanner failure and SARIF integrity gates#2283

Open
Errordog2 wants to merge 1 commit into
UnitOneAI:mainfrom
Errordog2:codex/sast-sarif-failure-integrity-gates
Open

Add SAST scanner failure and SARIF integrity gates#2283
Errordog2 wants to merge 1 commit into
UnitOneAI:mainfrom
Errordog2:codex/sast-sarif-failure-integrity-gates

Conversation

@Errordog2

Copy link
Copy Markdown

Skill Improvement ($50-150 Bounty)

Skill Modified

Skill name: sast-config
Skill path: skills/devsecops/sast-config/

What Was Wrong

Issue #2278 notes that the current SAST skill covers rule quality, coverage, suppression hygiene, and CI integration, but it can over-trust a green CI result or SARIF upload.

A workflow can satisfy surface checks while the scanner failure is hidden by continue-on-error, shell wrappers, or upload/reporting steps. SARIF upload can still succeed with stale, empty, partial, or fallback output, making branch protection look green even when analysis did not complete for the current commit.

What This PR Fixes

  • Bumps sast-config to 1.0.1.
  • Adds CI result integrity review to the skill use cases.
  • Adds a dedicated Scanner Failure and SARIF Integrity Gates section under CI integration review.
  • Requires scanner commands to fail closed for blocking checks.
  • Requires branch protection / required checks to bind to scanner success, not only SARIF upload or dashboard ingestion.
  • Requires SARIF freshness and current-commit/run binding evidence.
  • Distinguishes successful zero findings from skipped, failed, partial, stale, cached, or fallback SARIF output.
  • Adds SAST-CI-FAIL finding triggers for hidden scanner failures, upload-only required checks, stale/empty SARIF, missing zero-finding evidence, partial analysis, and monitor-only gates presented as blocking.
  • Extends the output report with a Scanner Failure and SARIF Integrity table.
  • Adds common pitfalls and changelog coverage.

Evidence

Before: Reviewers could treat SAST as passing when a wrapper/upload job succeeded, even if the scanner failed or produced stale/empty SARIF.

After: Reviewers must document scanner exit handling, required-check binding, current commit/run binding, SARIF freshness, empty-result semantics, and partial/skipped analysis handling before trusting a clean SAST result.

Test Cases Added/Updated

  • Added vulnerable test cases (tests/vulnerable/)
  • Added benign test cases (tests/benign/)
  • Existing checks still pass

This existing skill stores examples in Markdown guidance files; the change keeps scope to SKILL.md.

Validation

  • git diff --check
  • Frontmatter required-field check matching .github/workflows/lint-skills.yml
  • index.yaml file-existence check matching .github/workflows/validate-index.yml
  • Markdown code fence balance check for changed file
  • Workflow-equivalent prompt-injection scan over skills/ and roles/
  • Marker check for version 1.0.1, Scanner Failure and SARIF Integrity Gates, continue-on-error: true, upload-sarif, current commit binding, SARIF freshness, empty result semantics, SAST-CI-FAIL triggers, required-check binding, and stale/partial SARIF handling

Bounty Tier

  • Minor ($50) - Doc update, small logic tweak, typo fix
  • Moderate ($100) - New edge case coverage, FP reduction with evidence
  • Substantial ($150) - Rewritten detection logic, major coverage expansion

Bounty Info

  • I have read and agree to the CONTRIBUTING.md bounty terms
  • Preferred payment method: Payment details can be provided privately after maintainer acceptance

Fixes #2278

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[REVIEW] sast-config: add scanner failure and SARIF integrity gates

1 participant