Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ src/
foundation/ Arena allocator, hash table, string utils, platform compat
store/ SQLite graph storage (WAL mode, FTS5)
cypher/ Cypher query → SQL translation
mcp/ MCP server (JSON-RPC 2.0 over stdio, 14 tools)
mcp/ MCP server (JSON-RPC 2.0 over stdio, 15 tools)
pipeline/ Multi-pass indexing pipeline
pass_*.c Individual pipeline passes (definitions, calls, usages, etc.)
httplink.c HTTP route extraction (Go/Express/Laravel/Ktor/Python)
Expand Down
3 changes: 2 additions & 1 deletion Makefile.cbm
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,8 @@ TEST_STORE_SRCS = \
tests/test_store_arch.c \
tests/test_store_bulk.c \
tests/test_store_pragmas.c \
tests/test_store_checkpoint.c
tests/test_store_checkpoint.c \
tests/test_node_history.c

TEST_CYPHER_SRCS = \
tests/test_cypher.c
Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

**The fastest and most efficient code intelligence engine for AI coding agents.** Full-indexes an average repository in milliseconds, the Linux kernel (28M LOC, 75K files) in 3 minutes. Answers structural queries in under 1ms. Ships as a single static binary for macOS, Linux, and Windows — download, run `install`, done.

High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 159 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, and C++ — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 14 MCP tools. Zero dependencies. Plug and play across 11 coding agents.
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across all 159 languages, enhanced with [**Hybrid LSP** semantic type resolution](#hybrid-lsp) for Python, TypeScript / JavaScript / JSX / TSX, PHP, C#, Go, C, and C++ — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 15 MCP tools. Zero dependencies. Plug and play across 11 coding agents.

> **Research** — The design and benchmarks behind this project are described in the preprint [*Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP*](https://arxiv.org/abs/2603.27277) (arXiv:2603.27277). Evaluated across 31 real-world repositories: 83% answer quality, 10× fewer tokens, 2.1× fewer tool calls vs. file-by-file exploration.

Expand All @@ -37,7 +37,7 @@ High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-si
- **11 agents, one command** — `install` auto-detects Claude Code, Codex CLI, Gemini CLI, Zed, OpenCode, Antigravity, Aider, KiloCode, VS Code, OpenClaw, and Kiro — configures MCP entries, instruction files, and pre-tool hooks for each.
- **Built-in graph visualization** — 3D interactive UI at `localhost:9749` (optional UI binary variant).
- **Infrastructure-as-code indexing** — Dockerfiles, Kubernetes manifests, and Kustomize overlays indexed as graph nodes with cross-references. `Resource` nodes for K8s kinds, `Module` nodes for Kustomize overlays with `IMPORTS` edges to referenced resources.
- **14 MCP tools** — search, trace, architecture, impact analysis, Cypher queries, dead code detection, cross-service HTTP linking, ADR management, and more.
- **15 MCP tools** — search, trace, architecture, impact analysis, Cypher queries, dead code detection, cross-service HTTP linking, ADR management, and more.

## Quick Start

Expand Down Expand Up @@ -139,6 +139,7 @@ Removes all agent configs, skills, hooks, and instructions. Does not remove the
- **Architecture Decision Records**: `manage_adr` persists architectural decisions across sessions
- **Louvain community detection**: Discovers functional modules by clustering call edges
- **Git diff impact mapping**: `detect_changes` maps uncommitted changes to affected symbols with risk classification
- **Per-symbol git timeline**: `get_node_history` returns every commit that touched a symbol's line range — lazily computed, cached until HEAD moves, with on-demand diffs and temporal co-change coupling
- **Call graph**: Resolves function calls across files and packages (import-aware, type-inferred)
- **Dead code detection**: Finds functions with zero callers, excluding entry points
- **Cypher-like queries**: `MATCH (f:Function)-[:CALLS]->(g) WHERE f.name = 'main' RETURN g.name`
Expand Down Expand Up @@ -324,7 +325,7 @@ Add to `~/.claude/.mcp.json` (global) or project `.mcp.json`:
}
```

Restart your agent. Verify with `/mcp` — you should see `codebase-memory-mcp` with 14 tools.
Restart your agent. Verify with `/mcp` — you should see `codebase-memory-mcp` with 15 tools.

</details>

Expand Down Expand Up @@ -389,6 +390,7 @@ codebase-memory-mcp cli --raw search_graph '{"label": "Function"}' | jq '.result
| `search_graph` | Structured search by label, name pattern, file pattern, degree filters. Pagination via limit/offset. |
| `trace_path` | BFS traversal — who calls a function and what it calls (alias: `trace_call_path`). Depth 1-5. |
| `detect_changes` | Map git diff to affected symbols + blast radius with risk classification. |
| `get_node_history` | Git evolution timeline of a symbol: every commit that touched its line range (author, date, line stats), lazily computed and cached. Options: embedded diffs, temporal co-change coupling. |
| `query_graph` | Execute Cypher-like graph queries (read-only). |
| `get_graph_schema` | Node/edge counts, relationship patterns, property definitions per label. Run this first. |
| `get_code_snippet` | Read source code for a function by qualified name. |
Expand Down Expand Up @@ -527,7 +529,7 @@ Also supported (not yet benchmarked): Ada, Agda, Apex, Assembly (NASM), Astro, A
```
src/
main.c Entry point (MCP stdio server + CLI + install/update/config)
mcp/ MCP server (14 tools, JSON-RPC 2.0, session detection, auto-index)
mcp/ MCP server (15 tools, JSON-RPC 2.0, session detection, auto-index)
cli/ Install/uninstall/update/config (10 agents, hooks, instructions)
store/ SQLite graph storage (nodes, edges, traversal, search, Louvain)
pipeline/ Multi-pass indexing (structure → definitions → calls → HTTP links → config → tests)
Expand Down
6 changes: 3 additions & 3 deletions pkg/npm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

**The fastest and most efficient code intelligence engine for AI coding agents.** Full-indexes an average repository in milliseconds, the Linux kernel (28M LOC, 75K files) in 3 minutes. Answers structural queries in under 1ms. Ships as a single static binary — this package downloads and runs it automatically.

High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across 159 languages — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 14 MCP tools. Zero dependencies. Plug and play across 11 coding agents.
High-quality parsing through [tree-sitter](https://tree-sitter.github.io/tree-sitter/) AST analysis across 159 languages — producing a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. 15 MCP tools. Zero dependencies. Plug and play across 11 coding agents.

## Installation

Expand All @@ -30,7 +30,7 @@ Restart your agent. Say **"Index this project"** — done.
- **159 languages** — vendored tree-sitter grammars compiled into the binary. Nothing to install, nothing that breaks.
- **120x fewer tokens** — 5 structural queries: ~3,400 tokens vs ~412,000 via file-by-file search.
- **11 agents, one command** — `install` auto-detects Claude Code, Codex CLI, Gemini CLI, Zed, OpenCode, Antigravity, Aider, KiloCode, VS Code, OpenClaw, and Kiro.
- **14 MCP tools** — search, trace, architecture, impact analysis, Cypher queries, dead code detection, cross-service HTTP linking, ADR management, and more.
- **15 MCP tools** — search, trace, architecture, impact analysis, Cypher queries, dead code detection, cross-service HTTP linking, ADR management, and more.

## Supported Platforms

Expand Down Expand Up @@ -67,7 +67,7 @@ codebase-memory-mcp cli get_architecture '{}'
|----------|-------|
| **Indexing** | `index_repository`, `list_projects`, `delete_project`, `index_status` |
| **Querying** | `search_graph`, `trace_call_path`, `detect_changes`, `query_graph` |
| **Analysis** | `get_architecture`, `get_graph_schema`, `get_code_snippet`, `search_code` |
| **Analysis** | `get_architecture`, `get_graph_schema`, `get_code_snippet`, `get_node_history`, `search_code` |
| **Advanced** | `manage_adr`, `ingest_traces` |

## Performance
Expand Down
7 changes: 7 additions & 0 deletions scripts/security-allowlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ src/foundation/compat_fs.c:popen:cbm_popen wrapper definition (POSIX)
src/foundation/compat_fs.c:cbm_popen:cbm_popen function definition
src/foundation/compat_fs.c:fork:cbm_exec_no_shell — fork+execvp for shell-free subprocess execution
src/foundation/compat_fs.c:execvp:cbm_exec_no_shell — direct exec without shell interpretation
src/foundation/compat_fs.c:fork:cbm_proc_open — deadline-bounded command reader (kills runaway children)
src/foundation/compat_fs.c:execl:cbm_proc_open — sh -c in own process group so timeouts can kill the whole tree
src/foundation/compat_fs.c:_popen:cbm_proc_open Windows fallback (no deadline; documented follow-up)

# ── CLI: update command (user-initiated, interactive) ──────────────────────
src/cli/cli.c:cbm_popen:sha256 checksum verification (update cmd)
Expand All @@ -27,6 +30,10 @@ src/mcp/mcp.c:cbm_popen:search_code via grep (pattern in temp file, path validat
src/mcp/mcp.c:cbm_popen:detect_changes via git diff (args validated)
src/mcp/mcp.c:cbm_popen:git ls-files count for auto-index (session_root validated)
src/mcp/mcp.c:cbm_popen:update check to api.github.com (hardcoded URL)
src/mcp/mcp.c:cbm_proc_open:get_node_history git log -L timeline (root+file validated, since/until charset-validated, 30s deadline)
src/mcp/mcp.c:cbm_proc_open:get_node_history git rev-parse HEAD + shallow probe (root validated, 10s deadline)
src/mcp/mcp.c:cbm_proc_open:get_node_history git diff -U0 range mapping (root+file validated, 10s deadline)
src/mcp/mcp.c:cbm_proc_open:get_node_history git show co-change (shas from git output, root validated, 10s deadline)
src/mcp/mcp.c:popen:via cbm_popen wrapper calls

# ── Pipeline: git history parsing (fallback when libgit2 not available) ────
Expand Down
6 changes: 3 additions & 3 deletions src/cli/cli.c
Original file line number Diff line number Diff line change
Expand Up @@ -453,11 +453,11 @@ static const char skill_content[] =
"- High fan-in: `search_graph(min_degree=10, relationship=\"CALLS\", "
"direction=\"inbound\")`\n"
"\n"
"## 14 MCP Tools\n"
"## 15 MCP Tools\n"
"`index_repository`, `index_status`, `list_projects`, `delete_project`,\n"
"`search_graph`, `search_code`, `trace_path`, `detect_changes`,\n"
"`query_graph`, `get_graph_schema`, `get_code_snippet`, `get_architecture`,\n"
"`manage_adr`, `ingest_traces`\n"
"`query_graph`, `get_graph_schema`, `get_code_snippet`, `get_node_history`,\n"
"`get_architecture`, `manage_adr`, `ingest_traces`\n"
"\n"
"## Edge Types\n"
"CALLS, HTTP_CALLS, ASYNC_CALLS, IMPORTS, DEFINES, DEFINES_METHOD,\n"
Expand Down
151 changes: 151 additions & 0 deletions src/foundation/compat_fs.c
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,24 @@ int cbm_pclose(FILE *f) {
return _pclose(f);
}

/* Windows: no deadline enforcement — _popen exposes no process handle to
* kill, and the callers' git commands are already bounded by --max-count.
* Proper CreateProcess-based timeouts are a documented follow-up. */
bool cbm_proc_open(cbm_proc_t *p, const char *cmd, int timeout_ms) {
(void)timeout_ms;
p->timed_out = false;
p->fp = _popen(cmd, "r");
return p->fp != NULL;
}

bool cbm_proc_gets(cbm_proc_t *p, char *out, size_t cap) {
return fgets(out, (int)cap, p->fp) != NULL;
}

int cbm_proc_close(cbm_proc_t *p) {
return _pclose(p->fp);
}

bool cbm_mkdir_p(const char *path, int mode) {
(void)mode;
wchar_t *wpath = cbm_utf8_to_wide(path);
Expand Down Expand Up @@ -194,8 +212,11 @@ int cbm_exec_no_shell(const char *const *argv) {

#include <dirent.h>
#include <errno.h>
#include <poll.h>
#include <signal.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>

struct cbm_dir {
Expand Down Expand Up @@ -262,6 +283,136 @@ int cbm_pclose(FILE *f) {
return pclose(f);
}

enum { CBM_MS_PER_SEC = 1000, CBM_NS_PER_MS = 1000000, CBM_PROC_EXEC_FAIL = 127 };

static long long proc_now_ms(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (long long)ts.tv_sec * CBM_MS_PER_SEC + ts.tv_nsec / CBM_NS_PER_MS;
}

bool cbm_proc_open(cbm_proc_t *p, const char *cmd, int timeout_ms) {
memset(p, 0, sizeof(*p));
p->fd = -1;

int fds[2];
if (pipe(fds) != 0) {
return false;
}
pid_t pid = fork();
if (pid < 0) {
close(fds[0]);
close(fds[1]);
return false;
}
if (pid == 0) {
/* Own process group so a timeout can kill the shell AND whatever
* it spawned (git), not just the shell. */
setpgid(0, 0);
close(fds[0]);
dup2(fds[1], STDOUT_FILENO);
close(fds[1]);
execl("/bin/sh", "sh", "-c", cmd, (char *)NULL);
_exit(CBM_PROC_EXEC_FAIL);
}
close(fds[1]);
setpgid(pid, pid); /* mirror the child's call so a kill can never race it */
p->fd = fds[0];
p->pid = (int)pid;
p->deadline_ms = timeout_ms > 0 ? proc_now_ms() + timeout_ms : 0;
return true;
}

/* Emit up to cap-1 buffered bytes ending at the first newline (or the
* whole buffer when force is set / it is cap-filling). fgets semantics. */
static bool proc_emit(cbm_proc_t *p, char *out, size_t cap, bool force) {
size_t take = 0;
while (take < p->buf_len && p->buf[take] != '\n') {
take++;
}
if (take < p->buf_len) {
take++; /* include the newline */
} else if (!force && p->buf_len < cap - SKIP_ONE) {
return false; /* no full line buffered yet */
}
if (take == 0) {
return false;
}
if (take > cap - SKIP_ONE) {
take = cap - SKIP_ONE;
}
memcpy(out, p->buf, take);
out[take] = '\0';
p->buf_len -= take;
memmove(p->buf, p->buf + take, p->buf_len);
return true;
}

bool cbm_proc_gets(cbm_proc_t *p, char *out, size_t cap) {
if (cap < PAIR_LEN || p->timed_out) {
return false;
}
for (;;) {
if (proc_emit(p, out, cap, p->eof || p->buf_len == sizeof(p->buf))) {
return true;
}
if (p->eof) {
return false;
}
int wait_ms = -1;
if (p->deadline_ms > 0) {
long long remaining = p->deadline_ms - proc_now_ms();
if (remaining <= 0) {
p->timed_out = true;
return false;
}
wait_ms = (int)remaining;
}
struct pollfd pfd = {.fd = p->fd, .events = POLLIN};
int prc = poll(&pfd, 1, wait_ms);
if (prc == 0) {
p->timed_out = true;
return false;
}
if (prc < 0) {
if (errno == EINTR) {
continue;
}
p->eof = true;
continue;
}
ssize_t got = read(p->fd, p->buf + p->buf_len, sizeof(p->buf) - p->buf_len);
if (got > 0) {
p->buf_len += (size_t)got;
} else if (got == 0 || errno != EINTR) {
p->eof = true;
}
}
}

int cbm_proc_close(cbm_proc_t *p) {
if (p->fd >= 0) {
close(p->fd);
p->fd = -1;
}
if (p->pid <= 0) {
return -1;
}
if (p->timed_out && kill(-(pid_t)p->pid, SIGKILL) != 0) {
kill((pid_t)p->pid, SIGKILL);
}
int status = 0;
pid_t rc;
do {
rc = waitpid((pid_t)p->pid, &status, 0);
} while (rc < 0 && errno == EINTR);
p->pid = 0;
if (p->timed_out || rc < 0 || !WIFEXITED(status)) {
return -1;
}
return WEXITSTATUS(status);
}

bool cbm_mkdir_p(const char *path, int mode) {
/* Try direct mkdir first */
if (mkdir(path, (mode_t)mode) == 0) {
Expand Down
36 changes: 36 additions & 0 deletions src/foundation/compat_fs.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,42 @@ void cbm_closedir(cbm_dir_t *d);
FILE *cbm_popen(const char *cmd, const char *mode);
int cbm_pclose(FILE *f);

/* ── Deadline-bounded command reader ──────────────────────────── */

/* popen("r")-like handle whose reads observe a wall-clock deadline, so a
* runaway subprocess (e.g. an expensive git history walk) cannot hang the
* caller. POSIX: pipe + fork of `sh -c cmd` in its own process group —
* on expiry the group is killed and close() never blocks. Windows: plain
* _popen with no deadline enforcement (documented follow-up); timed_out
* stays false. */
enum { CBM_PROC_BUF = 4096 };

typedef struct {
#ifdef _WIN32
FILE *fp;
#else
int fd;
int pid;
long long deadline_ms; /* CLOCK_MONOTONIC, ms; 0 = no deadline */
size_t buf_len;
bool eof;
char buf[CBM_PROC_BUF];
#endif
bool timed_out;
} cbm_proc_t;

/* Start `sh -c cmd` with stdout piped to the handle. timeout_ms <= 0
* means no deadline. Returns false when the process could not start. */
bool cbm_proc_open(cbm_proc_t *p, const char *cmd, int timeout_ms);

/* Read the next line (newline kept, fgets-style: at most cap-1 bytes).
* Returns false on EOF or deadline expiry — check p->timed_out. */
bool cbm_proc_gets(cbm_proc_t *p, char *out, size_t cap);

/* Reap the child. Returns its exit code, or -1 when it was killed on
* timeout, did not exit normally, or could not be reaped. */
int cbm_proc_close(cbm_proc_t *p);

/* ── File operations ──────────────────────────────────────────── */

/* Create directory (and parents). mode is ignored on Windows. Returns true on success. */
Expand Down
16 changes: 16 additions & 0 deletions src/foundation/platform.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,22 @@
#include <stdint.h>
#include <stdbool.h>
#include <stdlib.h>
#include <sys/stat.h>

/* ── Platform-portable mtime_ns ───────────────────────────────── */

/* Nanosecond mtime from a stat buffer (second-granularity on Windows). */
static inline int64_t cbm_stat_mtime_ns(const struct stat *st) {
enum { CBM_STAT_NS_PER_SEC = 1000000000 };
#ifdef __APPLE__
return ((int64_t)st->st_mtimespec.tv_sec * CBM_STAT_NS_PER_SEC) +
(int64_t)st->st_mtimespec.tv_nsec;
#elif defined(_WIN32)
return (int64_t)st->st_mtime * CBM_STAT_NS_PER_SEC;
#else
return ((int64_t)st->st_mtim.tv_sec * CBM_STAT_NS_PER_SEC) + (int64_t)st->st_mtim.tv_nsec;
#endif
}

/* ── Safe memory ──────────────────────────────────────────────── */

Expand Down
Loading