Skip to content

feat(tools): add TestRunner, GitAnalysis, DependencyAnalyzer, CodeAnalysis tools#1475

Closed
skyhighbg22-jpg wants to merge 1 commit into
Gitlawb:mainfrom
skyhighbg22-jpg:feat/new-analysis-tools
Closed

feat(tools): add TestRunner, GitAnalysis, DependencyAnalyzer, CodeAnalysis tools#1475
skyhighbg22-jpg wants to merge 1 commit into
Gitlawb:mainfrom
skyhighbg22-jpg:feat/new-analysis-tools

Conversation

@skyhighbg22-jpg

Copy link
Copy Markdown
Contributor

Summary

Adds 4 new analysis tools to the agent's tool pool. Each tool is self-contained with its own schema, UI, prompt, and constants. Includes focused permission tests.

New Tools

TestRunnerTool (TestRunner)

Structured test execution with framework auto-detection. Accepts command (optional — auto-detects from project files), args, and pattern (test name filter).

  • Auto-detects: jest, vitest, pytest, go test, cargo test, make test
  • Parses output into structured results: pass/fail/skip counts, failure details, duration
  • isReadOnly: false, isConcurrencySafe: false
  • checkPermissions returns { behavior: 'ask' } — goes through the permission system like BashTool

GitAnalysisTool (GitAnalysis)

Read-only git history analysis. Operations:

  • blame — line-by-line authorship with optional line range
  • log — search commits by message, author, date range, file
  • diff-range — compare two refs with stat + full diff
  • show-commit — full commit details (author, date, diff, stats)
  • isReadOnly: true (no bisect — it mutates .git state)

DependencyTool (DependencyAnalyzer)

Dependency analysis with auto-detected package manager (npm/cargo/pip/go). Operations:

  • audit — security vulnerability scanning
  • outdated — check for outdated packages
  • graph — dependency tree visualization
  • license — license info for all dependencies
  • info — details for a specific package
  • isReadOnly: false, checkPermissions returns { behavior: 'ask' }

CodeAnalysisTool (CodeAnalysis)

Static code analysis for refactoring decisions. Operations:

  • complexity — cyclomatic complexity metrics (Python-aware with indentation-based scope tracking)
  • dead-code — detect potentially unused exports/modules
  • imports — import/dependency graph analysis (Go import block parsing, not overbroad regex)
  • duplicates — detect duplicate code blocks across files
  • size — file sizes and line counts
  • isReadOnly: true

Tool Registration

  • All 4 tools added to getAllBaseTools() in src/tools.ts
  • TestRunnerTool and DependencyTool excluded from ASYNC_AGENT_ALLOWED_TOOLS (require permission prompt for subprocess execution)
  • GitAnalysisTool and CodeAnalysisTool included in ASYNC_AGENT_ALLOWED_TOOLS (read-only)

Tests

src/tools/TestRunnerTool/permissions.test.ts — 6 tests:

  • TestRunnerTool: isReadOnly() === false, isConcurrencySafe() === false, checkPermissions → ask
  • DependencyTool: isReadOnly() === false, isConcurrencySafe() === false, checkPermissions → ask

…lysis tools

New tools for the agent:

TestRunnerTool (TestRunner):
- Structured test execution with auto-detection (jest, vitest, pytest, go test, cargo test)
- Parses results into pass/fail counts, failure details, durations
- checkPermissions returns 'ask' (routes through permission system)
- isReadOnly: false, isConcurrencySafe: false

GitAnalysisTool (GitAnalysis):
- blame: who wrote each line (with line range support)
- log: search commit messages with author/date/file filters
- diff-range: compare arbitrary refs
- show-commit: full commit analysis
- isReadOnly: true (no bisect — removed as it mutates .git)

DependencyTool (DependencyAnalyzer):
- audit, outdated, graph, license, info operations
- Auto-detects package manager (npm/cargo/pip/go)
- checkPermissions returns 'ask' (routes through permission system)
- isReadOnly: false, isConcurrencySafe: false

CodeAnalysisTool (CodeAnalysis):
- complexity, dead-code, imports, duplicates, size operations
- Python-aware indentation-based scope tracking
- Go import block parsing (not overbroad regex)
- isReadOnly: true

All tools registered in tool pool. TestRunner and DependencyAnalyzer removed from ASYNC_AGENT_ALLOWED_TOOLS (require permission prompt). Includes focused permission tests (6 tests).

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found issues that need to be addressed before this is ready.

Findings

  • [P1] Keep CodeAnalysis inside approved project roots
    src/tools/CodeAnalysisTool/CodeAnalysisTool.ts:232
    CodeAnalysis is registered as read-only, auto-allowed, and available to async agents, but it resolves any supplied path with path.resolve(cwd, inputPath) and then reads it without checking that the target stays inside the current workspace or an explicitly allowed directory. A model can therefore call CodeAnalysis on an absolute path or .. path outside the project and get file-derived content back without a permission prompt. For example, the imports operation reads a target file and returns matching export lines, so a source-like file outside the repo can be disclosed through the tool result. Please route this through the same project/file permission boundary used by existing file-reading tools, or reject paths outside approved roots before any statSync/readFileSync/directory walk occurs.

  • [P2] Show the real dependency commands before running them
    src/tools/DependencyTool/DependencyTool.ts:121
    The dependency tool asks for permission using only the high-level operation name, but the implementation may execute materially different commands, including package-manager installs and npx execution. For instance, operation: "license" on npm runs npx license-checker --json, Go audit first runs go install github.com/securego/gosec/v2/cmd/gosec@latest, and Cargo license/outdated operations install helper crates before running the analysis. A user approving “Run dependency command: license” or “audit” is not being shown the network/install/executable command they are actually approving. Please compute and display the exact command sequence in checkPermissions, or split install/bootstrap steps into explicit prompts.

@skyhighbg22-jpg

Copy link
Copy Markdown
Contributor Author

Superseded by focused PRs #1476 (bug fixes), #1477 (security), #1478 (performance), #1479 (DX), #1487 (type safety).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants