Skip to content

feat(miner-hands): tree-sitter-based repo map builder #4280

Description

@JSONbored

A repo-map builder gives a coding-agent driver (or the acceptance-criteria/prompt-packet builders upstream of it) a compact, structural view of the target repository — function/class/symbol signatures across the codebase — without paying the token cost of dumping every file's full contents into the prompt. This is a well-established convergent pattern (SWE-agent, Aider, and others independently arrived at a tree-sitter-parsed repo map for exactly this reason). Verified via grep -rn "tree-sitter" package.json packages/*/package.json — zero hits, so tree-sitter (in whatever binding form) is not a dependency anywhere in this repo today; this issue adds it.

Deliverables

  • Add a tree-sitter dependency, preferring the WASM binding (web-tree-sitter) over native bindings given this repo also ships a Cloudflare Workers deployment target ("dev": "wrangler dev", "deploy": "wrangler deploy" in package.json) where native Node addons are not an option — confirm whether this module actually needs to run in that context before committing to a binding; if it only ever runs in the local miner/CLI process, a native binding may be acceptable and simpler.
  • A repo-map builder that walks a target repository's source files, parses each with the appropriate language grammar, and extracts a compact symbol/signature outline (not full file bodies), bounded to a fixed output size so it can't blow out a prompt budget on a large repo.
  • Graceful degradation for languages/files without an available grammar (skip, don't crash the whole map) and for tree-sitter itself failing to initialize (fall back to a simpler heuristic, e.g. a plain file listing, rather than blocking the whole driver invocation).
  • Tests against fixture files per supported language, asserting the extracted symbol outline is stable (a golden-fixture style test, similar in spirit to test/contract/engine-parity.test.ts's golden-JSON comparisons, though this is a new, separate suite).
  • Document which languages are supported at launch (likely starting with TypeScript/JavaScript, given that's what this repo and most target repos are written in) and how to add another grammar later.

References

  • Verified via grep -rn "tree-sitter" package.json packages/*/package.json — zero hits, confirming this is a net-new dependency
  • package.json: "dev": "wrangler dev", "deploy": "wrangler deploy" (confirms a Workers runtime target exists in this repo — relevant to the native-vs-WASM binding choice, though verify this module's actual runtime context before assuming the constraint applies)
  • test/contract/engine-parity.test.ts:1-11 (existing golden-fixture test STYLE this issue's fixture tests can follow, not a shared harness to extend)
  • Feeds the acceptance-criteria file (feat(miner-hands): immutable acceptance-criteria file written before the coding agent starts #4271, this batch) and packages/gittensory-engine/src/prompt-packet.ts as an additional context source, though wiring it into those is separable follow-up, not required by this issue

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.gittensor:priorityMaintainer-selected Gittensor priority — scores a 1.5x multiplier.help wantedExtra attention is needed

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions