feat: shadow AI detection for repositories - #53
Merged
Merged
Conversation
Add agent-strace audit-tools — scans a git repository for AI tool usage signatures using file presence and commit message patterns. No network calls required; git-only by default. agent-strace audit-tools [--repo .] [--since '90 days ago'] [--approved cursor,copilot] Detected tools: Claude Code, Cursor, GitHub Copilot, Codex/ChatGPT, Windsurf, Aider — identified via file signals (.cursorrules, CLAUDE.md, .github/copilot-instructions.md, etc.) and commit message patterns. Flags unapproved tools against a configurable --approved list. Scans .env git history for unknown LLM API endpoints. Reports PII patterns found in recently committed files. Closes #46 Co-authored-by: Ona <no-reply@ona.com>
Co-authored-by: Ona <no-reply@ona.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #46
What
Adds
agent-strace audit-tools— scans a git repository for AI tool usage signatures using file presence and commit message patterns. No network calls required; git-only by default.Usage
Detection signals (no LLM calls, pure static analysis)
CLAUDE.md,.claude/settings.json\bClaude\bin message.cursorrules,.cursor/cursor aiin message.github/copilot-instructions.mdCo-authored-by: GitHub Copilotcodex.md,.openai/generated by chatgpt.windsurfrules,.codeium/.aider.conf.yml,.aider/Confidence levels:
confirmed(file present) >likely(commit patterns only).Changes
src/agent_trace/shadow_ai.py(new)detect_ai_tools(): main detection functionformat_audit_tools(): report formatter_file_exists_in_repo(): checks file/dir presence_scan_commit_messages(): regex scan of git log_scan_env_history(): scans.envgit history for LLM endpoints_scan_pii_in_commits(): detects PII patterns in recent commitscmd_audit_tools(): CLI handlersrc/agent_trace/cli.pyaudit-toolssubcommand with--repo,--since,--approvedtests/test_shadow_ai.py: 9 tests covering file detection, approval filtering, confidence levels, and output formatting.