v0.54.0 — Vault sync UI shows "synced" but mem_tree_chunks and mem_tree_ingested_sources stay at 0

[gestore-cpu]

Version: v0.54.0 (Beta build)
Platform: macOS
User UID: 6a146e7aefa23d1dbca7b440

Issue

After "Sync" on a Knowledge vault, UI reports "N file(s) · synced X minutes ago" but the underlying database remains empty.

Steps to reproduce

1. Add Knowledge vault pointing to a local folder with ~900 markdown/docx files
2. Click "Sync"
3. UI shows progress and finally "synced"
4. Check the database:
   sqlite3 ~/.openhuman/users//workspace/memory_tree/chunks.db
   "SELECT COUNT() FROM mem_tree_ingested_sources;
   SELECT COUNT() FROM mem_tree_chunks;"
5. Both queries return 0

Expected

mem_tree_ingested_sources should contain 1 row per vault
mem_tree_chunks should contain thousands of rows for ~1000 files

Actual

Both tables remain empty after multiple sync attempts. "Memory wiped" message confirms 0 rows removed = nothing was ever stored. "Build summary trees" button is cosmetic (no jobs queued).

Additional context

• Heartbeat loop: ON
• Subconscious inference: ON
• Memory tree workers: ON (4 workers polling every 5s, route to Anthropic claude-haiku)
• AI provider: Anthropic with own API key (verified $30 credit available)
• Tried: wipe memory + remove vaults + re-add vaults + re-sync. Same result.

Happy to provide more logs/diagnostics.

[justinhsu1477]

Picking this up.

The combined signal — both mem_tree_ingested_sources and mem_tree_chunks empty, "Memory wiped" reporting 0 rows removed (= nothing ever written), and "Build summary trees" producing no queued jobs — points at the break being upstream of the chunker/embedder path. If it were a chunker or embedding-provider failure, you'd typically see a row in mem_tree_ingested_sources (the source got registered) but a stalled / failed job downstream. Both tables being 0 looks more like the sync RPC either never enqueues an ingestion job, or the worker dispatch silently no-ops.

Plan:

1. Reproduce against v0.54.0 with a small (~10 file) local vault on macOS.
2. Add tracing across memory::sync → ingestion queue (memory::ingestion::queue) → worker pickup → mem_tree_ingested_sources insert in memory_store::chunks::store. Stable grep prefixes so the diagnostic survives the fix.
3. Locate the silent break, root-cause it, fix.
4. Regression test pinning the invariant: "files visible to memory_sync ⇒ exactly 1 row in mem_tree_ingested_sources per vault + N rows in mem_tree_chunks where N matches successfully-chunked files".
5. Keep the added instrumentation at debug! so the next silent-failure mode is triageable from logs alone (this is also why I'm flagging step 2 — even if the bug is one line, the missing observability is a bigger problem than the bug itself).

cc @senamakel

[justinhsu1477]

Root cause found — vault sync is wired to the wrong storage backend.

The full trace:

UI "Sync" → vault.sync RPC → vault::ops::vault_sync
            → spawned task: vault::sync::sync_vault
            → process_file → memory::ops::documents::doc_ingest
            → MemoryClient::ingest_doc
            → self.inner.ingest_document  (inner = Arc<UnifiedMemory>)
            → UnifiedMemory::ingest_document → upsert_document
            → INSERT INTO memory_docs    ← writes here

But the user is checking:

SELECT COUNT(*) FROM mem_tree_chunks;
SELECT COUNT(*) FROM mem_tree_ingested_sources;

These two tables belong to a completely different backend — memory_tree, populated by memory::ingest_pipeline::ingest_document. UnifiedMemory and memory_tree write to different schemas in different DB files, and there's no bridge between them.

The architecture trajectory in recent PRs (#2585 refactor, #2556 memory_tree consolidation, #2574 coverage) is migrating RAG primitives to memory_tree as the canonical layer. MCP tools (memory.search, tree.read_chunk, tree.browse) read from memory_tree. The vault sync path was not migrated alongside, so vault data lands in legacy UnifiedMemory and is invisible to every modern retrieval surface.

That explains every reported symptom:

Symptom	Explanation
mem_tree_chunks = 0	vault sync doesn't write there
mem_tree_ingested_sources = 0	same
"Memory wiped: 0 rows removed"	wipe clears memory_tree, which was already empty
"Build summary trees" produces no jobs	memory_tree has no chunks to summarise
UI "N file(s) · synced"	vault state correctly reports the UnifiedMemory ingest succeeded — but to the wrong table
Agent can't recall vault content	agent retrieval goes through memory_tree

Fix direction

Going with Option A: migrate vault sync to call memory::ingest_pipeline::ingest_document directly. The vault sync path was architecturally meant to feed memory_tree, the migration is the bug. Plan:

1. process_file calls ingest_pipeline::ingest_document (writes to mem_tree_chunks + mem_tree_ingested_sources) instead of the legacy doc_ingest.
2. Source id format: vault:{vault_id}:{rel_path} — stable per file path, so content-hash dedup at the vault layer continues to gate re-ingestion correctly.
3. On real content change: delete prior chunks for the source via memory_store::chunks::store::delete_chunks_by_source before the re-ingest so already_ingested doesn't short-circuit the new content.
4. On file deletion (vault walk doesn't see a previously-ledger'd path): same — delete_chunks_by_source against the orphan source_id.
5. Vault ledger keeps the same shape; the document_id field now stores the memory_tree source_id.
6. Regression test pins the invariant: "vault sync of N supported files ⇒ N rows in mem_tree_ingested_sources and ≥N rows in mem_tree_chunks".
7. Diagnostic logging stays as debug! along the full path so a future silent-failure mode surfaces from logs.

This does not touch the doc_ingest → UnifiedMemory path that other callers (Composio providers via memory_sync, agent_experience, etc.) rely on. Out of scope here:

• Composio / agent capture parity with memory_tree (if they have a similar gap, that's a separate audit).
• Removing the legacy UnifiedMemory path entirely (refactor(memory): remove UnifiedMemory from refactor(memory): separate tree policy from generic engine + E2E tests #2585's follow-up list — much bigger).

Starting the implementation now. PR incoming.

[gestore-cpu]

Update 2026-05-27 — Bug confirmed: not provider-related

Tried switching all background workloads to local Ollama provider:

• Memory summarization → Ollama gemma3:1b-it-qat
• Embeddings → Ollama all-minilm
• Heartbeat → Ollama gemma3:1b-it-qat
• Learning · Reflections → Ollama gemma3:1b-it-qat
• Subconscious → Ollama gemma3:1b-it-qat

After reset + re-sync of mercury-vault (904 files):

• UI reports: "Synced mercury-vault — Ingested 12, unchanged 916, removed 0, skipped 13 · 0.2s"
• But DB query:

sqlite3 chunks.db "SELECT COUNT(*) FROM mem_tree_chunks; SELECT COUNT(*) FROM mem_tree_ingested_sources;"
→ 0
→ 0

Ollama is reachable (verified with curl on /api/tags returning both models). Same behavior as with Anthropic cloud provider.

The bug appears to be downstream of the chunker, between the "Ingested N" counter and the actual SQL INSERT into mem_tree_chunks / mem_tree_ingested_sources.

The UI message is misleading because it suggests success.

Happy to share full logs if useful.