ATR (Agent Threat Rules) has 108 open-source detection rules for AI agent threats. Cisco AI Defense ships 34 of them in production. Proposing a dual integration with Garak:
As detector: garak/detectors/atr.py — scan LLM outputs for credential leaks, prompt injection artifacts, tool poisoning indicators. 108 regex patterns, <1ms per check, zero API keys.
As probe source: ATR's 824 verified test cases (true_positive payloads) are ready-made attack inputs covering prompt injection, tool poisoning, credential exfiltration, and 6 more categories. Each payload is labeled with expected behavior.
This gives Garak users red+blue in one tool — attack with ATR payloads, detect with ATR rules.
Numbers: 96.9% recall on SKILL.md benchmark (498 samples), 99.7% precision on PINT (850 samples), 0% FP on 53K real-world skills.
Would the team be open to a PR? Happy to follow existing detector/probe patterns.
ATR (Agent Threat Rules) has 108 open-source detection rules for AI agent threats. Cisco AI Defense ships 34 of them in production. Proposing a dual integration with Garak:
As detector:
garak/detectors/atr.py— scan LLM outputs for credential leaks, prompt injection artifacts, tool poisoning indicators. 108 regex patterns, <1ms per check, zero API keys.As probe source: ATR's 824 verified test cases (true_positive payloads) are ready-made attack inputs covering prompt injection, tool poisoning, credential exfiltration, and 6 more categories. Each payload is labeled with expected behavior.
This gives Garak users red+blue in one tool — attack with ATR payloads, detect with ATR rules.
Numbers: 96.9% recall on SKILL.md benchmark (498 samples), 99.7% precision on PINT (850 samples), 0% FP on 53K real-world skills.
Would the team be open to a PR? Happy to follow existing detector/probe patterns.