Skip to content

feat: shadow AI detection for repositories - #53

Merged
Siddhant-K-code merged 2 commits into
mainfrom
feat/issue-46-audit-tools
Apr 19, 2026
Merged

Siddhant-K-code merged 2 commits into
mainfrom
feat/issue-46-audit-tools

Conversation

@Siddhant-K-code

Copy link
Copy Markdown
Owner

Closes #46

What

Adds agent-strace audit-tools — scans a git repository for AI tool usage signatures using file presence and commit message patterns. No network calls required; git-only by default.

Usage

$ agent-strace audit-tools --repo . --approved "Claude Code"

AI Tool Usage Report
───────────────────────────────────────────────────────
Repo:  /path/to/repo
Since: 90 days ago
Approved: Claude Code
───────────────────────────────────────────────────────
Detected tools:

  ✅ Claude Code          (confirmed)
       file present: CLAUDE.md
  ⚠️  Cursor              (confirmed)
       file present: .cursorrules

Unapproved tool signals:
  ❌ Cursor detected but not in approved list
───────────────────────────────────────────────────────

Detection signals (no LLM calls, pure static analysis)

Tool File signals Commit patterns
Claude Code CLAUDE.md, .claude/settings.json \bClaude\b in message
Cursor .cursorrules, .cursor/ cursor ai in message
GitHub Copilot .github/copilot-instructions.md Co-authored-by: GitHub Copilot
Codex / ChatGPT codex.md, .openai/ generated by chatgpt
Windsurf .windsurfrules, .codeium/ —
Aider .aider.conf.yml, .aider/ —

Confidence levels: confirmed (file present) > likely (commit patterns only).

Changes

src/agent_trace/shadow_ai.py (new)

  • detect_ai_tools(): main detection function
  • format_audit_tools(): report formatter
  • _file_exists_in_repo(): checks file/dir presence
  • _scan_commit_messages(): regex scan of git log
  • _scan_env_history(): scans .env git history for LLM endpoints
  • _scan_pii_in_commits(): detects PII patterns in recent commits
  • cmd_audit_tools(): CLI handler

src/agent_trace/cli.py

  • audit-tools subcommand with --repo, --since, --approved

tests/test_shadow_ai.py: 9 tests covering file detection, approval filtering, confidence levels, and output formatting.

Add agent-strace audit-tools — scans a git repository for AI tool
usage signatures using file presence and commit message patterns.
No network calls required; git-only by default.

  agent-strace audit-tools [--repo .] [--since '90 days ago'] [--approved cursor,copilot]

Detected tools: Claude Code, Cursor, GitHub Copilot, Codex/ChatGPT,
Windsurf, Aider — identified via file signals (.cursorrules, CLAUDE.md,
.github/copilot-instructions.md, etc.) and commit message patterns.

Flags unapproved tools against a configurable --approved list.
Scans .env git history for unknown LLM API endpoints.
Reports PII patterns found in recently committed files.

Closes #46

Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code Siddhant-K-code added enhancement New feature or request security labels Apr 19, 2026
@Siddhant-K-code
Siddhant-K-code merged commit dfc8770 into main Apr 19, 2026
4 checks passed
@Siddhant-K-code
Siddhant-K-code deleted the feat/issue-46-audit-tools branch April 19, 2026 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request security

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Shadow AI detection: identify which AI tools are active in a repo

1 participant