TraceRoot — Vectorless RAG Codebase Archaeologist

Trace decisions, not just code. Ask why your codebase is the way it is.

---

The Problem

Engineering teams lose the why behind their code every single day.

You can read what a function does — but not why it was designed that way, who pushed back in the PR review, what tradeoff was made, or which customer bug forced this decision three years ago.

Existing RAG tools embed code and search by similarity. They collapse causal chains into blurry similarity scores. That is the wrong tool for understanding provenance.

---

The Solution

TraceRoot builds a typed provenance graph from your GitHub history and lets you ask natural language questions about decisions, not just code.

Question: "Why was the dependency injection system changed?"

Retrieved decision chain:

  issue:421   — customer reported broken behaviour     (2023-04-11)
      |
  pr:88       — tradeoff debated in review thread      (2023-04-12)
      |
  commit:a3f9 — fix merged, approved by reviewer       (2023-04-12)

No embeddings. No cosine similarity. Pure graph traversal + BM25 + Groq Llama in-context reasoning.

---

Architecture

 Phase 1 — Ingestion (run once in terminal)
 ┌──────────────┐  ┌─────────────────┐  ┌─────────────┐
 │  GitHub API  │  │ tree-sitter AST │  │  git blame  │
 │ commits, PRs │  │ files, functions│  │ line -> SHA │
 └──────┬───────┘  └────────┬────────┘  └──────┬──────┘
        └───────────────────┴───────────────────┘
                            │
                  ┌─────────▼──────────┐
                  │   Normalizer +     │
                  │   Deduplicator     │  ingest_github.py
                  └─────────┬──────────┘  ingest_ast.py
                            │              build_bm25.py

 Phase 2 — Graph Construction (run once in terminal)
                  ┌─────────▼──────────┐
                  │  Provenance Graph  │
                  │  NetworkX + SQLite │  build_graph.py
                  │  9 edge types      │
                  └─────────┬──────────┘

 Phase 3-5 — Query (streamlit run app.py)
                  ┌─────────▼──────────┐
                  │ Vectorless         │
                  │ Retriever          │  retriever.py
                  │ BM25 + k-hop walk  │
                  └─────────┬──────────┘
                            │
                  ┌─────────▼──────────┐
                  │  Groq Reasoning    │
                  │  Cited answers +   │  reasoner.py
                  │  Timeline output   │
                  └─────────┬──────────┘
                            │
                  ┌─────────▼──────────┐
                  │  Streamlit UI      │  app.py
                  │  Timeline + Export │
                  └────────────────────┘

---

Setup and Run

Prerequisites

• Python 3.10 or higher
• Git installed and available in PATH
• A GitHub personal access token
• A Groq API key (free at console.groq.com)

Step 1 — Clone the repo

git clone https://github.com/AbhiramAbbireddy/TraceRoot-Vectorless-RAG-Codebase-Archaeologist.git
cd TraceRoot-Vectorless-RAG-Codebase-Archaeologist

Step 2 — Create and activate virtual environment

python -m venv .venv

# Windows
.venv\Scripts\activate

# Mac / Linux
source .venv/bin/activate

Step 3 — Install dependencies

pip install -r requirements.txt

Step 4 — Configure .env

cp .env.example .env

Open .env and fill in your values:

GITHUB_TOKEN=your_github_token       # github.com/settings/tokens — needs repo: read scope
GROQ_API_KEY=your_groq_key           # console.groq.com — free tier available
TARGET_REPO=fastapi/fastapi          # the repo you want to analyse
INGEST_MONTHS=1                      # how many months of history to ingest (start with 1)
OUTPUT_DIR=./data                    # where generated files are stored

Step 5 — Run ingestion (once per repo)

# Fetch commits, PRs, issues, and authors from GitHub
python run_phase1.py --step github

# Clone the repo locally and parse all Python files with tree-sitter
python run_phase1.py --step ast

# Build the BM25 keyword index over commit messages and PR text
python run_phase1.py --step bm25

# Verify all files were created successfully
python run_phase1.py --verify

Step 6 — Build the provenance graph

python build_graph.py --verify

You should see a table showing node and edge counts. A healthy graph for 1 month of fastapi looks like:

• ~185 commits, ~231 PRs, ~15 issues, ~137 authors
• ~8000 nodes, ~29000 edges total

Step 7 — Test in terminal (optional but recommended)

python retriever.py "Why was the dependency injection system changed?"
python reasoner.py  "Why was the dependency injection system changed?"

Step 8 — Launch the UI

streamlit run app.py

Opens at http://localhost:8501. The graph is already loaded — just type a question and hit Dig in.

Switching to a different repo

Stop Streamlit, update TARGET_REPO and optionally INGEST_MONTHS in .env, then re-run steps 5 and 6. The data/ folder will be rebuilt automatically.

---

Example questions to try

These work well on the default fastapi repo with 1 month of history:

Who has been most active in reviewing PRs?
Why was the dependency injection system changed?
What commits touched the routing system?
Which pull requests had the most review discussion?
What bugs were fixed in the last month?
Who introduced the most changes recently?
What files changed the most recently?
Which issues were fixed by recent commits?

---

Project Structure

TraceRoot/
|
|-- models.py            # Typed dataclasses — CommitNode, PRNode, FileNode, etc.
|
|-- run_phase1.py        # Master runner for all Phase 1 ingestion steps
|-- ingest_github.py     # Fetch commits, PRs, issues, review comments from GitHub API
|-- ingest_ast.py        # Clone repo and extract functions via tree-sitter
|-- build_bm25.py        # Build BM25 keyword index over commit messages and PR text
|
|-- build_graph.py       # Construct typed provenance graph and save to graph.db
|
|-- retriever.py         # Vectorless retrieval: BM25 + k-hop graph traversal
|-- reasoner.py          # Groq Llama in-context reasoning over the retrieved subgraph
|-- app.py               # Streamlit UI — question box, timeline view, markdown export
|
|-- requirements.txt
|-- .env.example         # Template — copy to .env and fill in your keys
|-- .gitignore

---

Why Vectorless RAG

The relationships between a commit, a ticket, a PR review, and a file are explicit typed edges. Embedding them into a vector space loses exactly the structure that makes provenance traceable. Graph traversal preserves it.

	Vector RAG	TraceRoot
Retrieval method	Cosine similarity on embeddings	BM25 + typed graph traversal
Causal chain tracing	Collapses into similarity score	Follows edges in chronological order
Who approved this PR	Approximate match	Direct reviewed-by edge
What issue caused this commit	Finds similar text	issue -> fixed-by -> commit chain
Setup complexity	Embedding model + vector DB	SQLite + NetworkX only
Retrieval cost	Embedding API call per query	Free after one-time graph build

---

Graph Schema

Node types: commit · pr · issue · file · function · author

Edge	From	To	Meaning
modified	commit	file	this commit changed this file
authored-by	commit	author	who wrote this commit
references	commit	issue	commit message mentions an issue number
motivated-by	pr	issue	this PR was opened to address this issue
reviewed-by	pr	author	who reviewed this PR
fixed-by	issue	commit	the commit that closed this issue
merged-in	pr	commit	the merge commit for this PR
contains	file	function	functions defined in this file
calls	function	function	call graph edges between functions

---

Tech Stack

Layer	Tool
GitHub data	PyGithub, GitPython
AST parsing	tree-sitter, tree-sitter-python
Graph store	NetworkX, SQLite
Keyword search	rank-bm25
Query parsing	Groq llama-3.1-8b-instant
LLM reasoning	Groq llama-3.3-70b-versatile
Frontend	Streamlit
Serialization	orjson

---

License

MIT