macOS: two per-conversation servers run duplicate full index_repository workers on the same repo simultaneously (~400% CPU each, respawning every few minutes) — per-repo worker serialization still missing (0.9.0) #2673

Environment

• codebase-memory-mcp 0.9.0 (latest is v0.12.1 — happy to retest on it, but filing this as a macOS datapoint since the fix suggested in WAL grows unbounded (115 GB in 4.5 h, fills system drive): checkpoint starvation from accumulated per-conversation servers + 11 concurrent index workers on one repo #1083 does not appear to have landed)
• macOS 26 arm64 (Apple Silicon, 96 GB RAM)
• Install channel: install.sh → ~/.local/bin/codebase-memory-mcp
• Multiple concurrent MCP client sessions against the same repo (one long-lived CLI host), each conversation spawning its own codebase-memory-mcp server instance
• Repo: ~2.2 GB worktree, ~18.5k source files, actively modified during the workday (build outputs + tool-generated temp files). build/ dirs are not git-ignored (partly on us).

What happened

The machine sat at load average ~55 with 0% idle CPU while essentially idle from the user's perspective. Two distinct server instances (two different MCP conversations) each spawned a full index_repository worker for the same repo at the same time:

PID 13629  456.9% CPU  cli --index-worker index_repository {"repo_path":"/Users/<user>/Projects/<repo>"}   (parent server 46586)
PID 14093  401.4% CPU  cli --index-worker index_repository {"repo_path":"/Users/<user>/Projects/<repo>"}   (parent server 66866)

And this is not a one-off: paired skip-report logs in ~/.cache/codebase-memory-mcp/logs/ show one full extraction run per server, roughly every 1–8 minutes while the worktree is being modified:

12:41, 12:42   (×2 files)
12:43, 12:44   (×2)
12:46          (×2)
12:49          (×2)
12:53          (×2)
12:59          (×2)
13:01          (×2)

Each run re-scans/re-parses the whole tree (each skip report lists the same parse timeouts, e.g. UFIPanel/iosport/**/build/**/Intermediates.noindex/*-primary.d — the .d-parsed-as-D issue from #2414) and burns 400%+ CPU for minutes. Net effect: CPU never gets a break.

Aggravator observed the same day: the project DB was quarantined as ~/.cache/codebase-memory-mcp/<repo>.db.corrupt (669 MB, 12:51 local), throwing away the whole graph and forcing a from-scratch rebuild. The cache dir currently holds 9 .db.corrupt files across projects (Jul → Oct). Suspect: concurrent multi-process SQLite writers with no per-repo coordination — same root as the WAL starvation in #1083.

Diagnosis

Same root pattern as #1083, macOS variant, CPU-flavored instead of WAL-flavored:

1. No per-repo serialization of index workers across server instances — N conversations = N duplicate full indexes running concurrently.
2. Staleness triggers a full re-extract, not an incremental update, and there is no debounce/backoff while the tree is churning (feat(incremental): debounce artifact auto re-export #1884) — on an active worktree the index is stale again before a rebuild finishes, so workers respawn forever.
3. Corruption → silent discard → full rebuild (task: Silent index corruption / incomplete index reported as success (3 issues) #391 family), which resets all progress and feeds loop Fix C# support: wire tree-sitter-c-sharp grammar into parser #2.

Suggestions

1. Implement WAL grows unbounded (115 GB in 4.5 h, fills system drive): checkpoint starvation from accumulated per-conversation servers + 11 concurrent index workers on one repo #1083's suggestion 2 cross-platform: a per-repo file lock so a second index_repository queues or no-ops instead of running duplicate full indexes. Locking is not a Windows-only concern.
2. Debounce/backoff re-index triggers on hot worktrees; consider incremental (mtime/hash) extraction instead of full re-parses (feat(incremental): debounce artifact auto re-export #1884).
3. Harden DB writes against the recurring .db.corrupt quarantines; log a loud warning when it happens (users only find out via CPU).

Related issues

• WAL grows unbounded (115 GB in 4.5 h, fills system drive): checkpoint starvation from accumulated per-conversation servers + 11 concurrent index workers on one repo #1083 (per-conversation servers + 11 concurrent index workers, WAL starvation, Windows)
• Multiple codebase memory .exe running #2400 (multiple server processes), Very high CPU usage #45 / CPU memory usage is too high #1084 / Idle MCP frontend still burns ~19% of a core on v0.10.8 (Windows) — looks like a regression of #1764 #2093 (high CPU family)
• feat(incremental): debounce artifact auto re-export #1884 (debounce artifact re-export), task: Silent index corruption / incomplete index reported as success (3 issues) #391 (silent index corruption)
• fix(discover): make/cargo dep-info .d files are parsed with the D grammar #2414 (.d dep-info files parsed with the D grammar — the exact parse timeouts in our skip reports)
• index_repository 导致 oom #2649 (index_repository OOM)