Skip to content

bug(store): single-conn pool from #271 has no recovery path when WAL-index goes stale #571

Description

@danielgap

📋 Pre-flight Checks

  • I have searched existing issues and this is not a duplicate
  • I understand this issue needs status:approved before a PR can be opened

📝 Bug Description

This is a follow-up on the architectural decision in #271 (fixed in v1.14.6). That fix correctly enforced db.SetMaxOpenConns(1) so the busy_timeout=5000 pragma always applies to every connection in the pool. The single-connection decision is sound for SQLite single-writer correctness.

However, the decision has an undocumented trade-off that amplifies the severity of #477 (stale WAL-index mmap in long-running processes): with a single connection, there is zero concurrency headroom. When the WAL-index mmap goes stale (per #477), the single connection is the ONLY connection, so every read AND write operation fails or hangs. There is no secondary path, and no code path triggers a reopen.

Concretely: /health continues to respond (trivial handler, no DB access), while every DB-backed endpoint (/search, /observations, writes) silently fails. The process cannot self-recover; it requires an external kill + restart.

This is not a duplicate of #477 or #206. It is the architectural gap between them:

🔄 Steps to Reproduce

  1. Run engram serve as a long-lived daemon (no cloud sync, no MCP needed to trigger this).
  2. After 1-7 days of uptime (reproduced here at ~1.7 days), any condition that replaces the -shm/-wal files on disk (corruption recovery, external checkpoint, or the natural staleness in bug(store): long-running processes keep stale WAL-index (-shm) mmap after file replacement — silent write loss + constraint failed (1555) on mem_session_summary #477) leaves the daemon operating on a divergent WAL view.
  3. Observe: /health returns ok, mem_doctor reports 4/4 checks passing (including checkpoint_log: 890), but /search hangs and returns SQLITE_BUSY after busy_timeout=5000.
  4. Confirm via lsof -p $(pgrep -f 'engram serve') | grep db-shm that the daemon holds a stale inode versus a fresh process.
  5. The daemon cannot recover without a manual kill + restart.

Reproduced on Linux (Ubuntu via Homebrew), Engram 1.16.0, single daemon, ~1.7 days uptime.

✅ Expected Behavior

A single-connection pool should be paired with a mechanism that detects WAL-index staleness and reopens the connection transparently (fsnotify on -shm/-wal, or inode verification before each transaction), as proposed in #206. With that companion, #271's single-conn decision becomes both correct AND resilient.

Additionally, /health should distinguish "process alive" from "DB reachable", so monitoring can detect the silent divergence instead of relying on mem_doctor (which checks store integrity, not handler liveness).

❌ Actual Behavior

🖥️ Environment

  • Operating System: Linux (Ubuntu/Debian)
  • Engram Version: 1.16.0
  • Agent / Client: Other (Pi via gentle-engram plugin)

📋 Relevant Logs

# mem_search failure surface (gentle-engram side):
gentle-engram could not reach the Engram HTTP server at http://127.0.0.1:7437.
The Pi-native mem_* tools are registered, but the native memory provider is not currently responding.

# Yet the trivial endpoint still responds:
$ curl http://127.0.0.1:7437/health
{"service":"engram","status":"ok","version":"0.1.0"}

# And mem_doctor reports a healthy store:
sqlite_lock_contention: ok (journal_mode=wal, busy_timeout_ms=5000, checkpoint_log=890)

💡 Additional Context

Related issues (this issue is the connective tissue between them):

Option B from the original #271 report (DSN pragmas via ?_pragma=busy_timeout(5000) plus a multi-connection read pool) was not chosen. That option would have provided read concurrency headroom, making staleness less catastrophic (reads could still succeed on a fresh connection). Either Option B is worth reconsidering, OR #206 should be prioritized as the necessary companion to the chosen Option A.

Downstream impact: the gentle-engram Pi plugin reports this failure as "could not reach Engram HTTP server", which is misleading (the server is reachable, only the DB-backed handlers are poisoned). Improving that error message would help users debug, but the underlying recovery gap remains.

Happy to contribute a fix in either direction (fsnotify-based reload aligned with #206, or a /healthz endpoint that probes the DB), pending maintainer guidance on which direction is preferred.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions