Skip to content

agt red-team: report verdict and CI exit code use different criteria #3237

Description

@imran-siddique

Found during a code-quality pass (not a security issue).

In agent_compliance/cli/red_team.py, the printed/JSON verdict and overall_passed are derived from each playbook's r.passed (around lines 317, 345-364), but the process exit code is gated on all(r.resilience_score >= threshold for r in results) (around line 366).

When a playbook's passed flag diverges from the score-vs-threshold comparison (e.g. score 75 with a default threshold of 70 but passed=False, or vice-versa), CI can exit 0 while the report prints FAIL, or exit 1 while it prints PASS — an unreliable gate.

Suggestion: drive both the printed verdict and the exit code from a single criterion.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions