Skip to content

fix(store): journal_mode=WAL switch dies instantly on concurrent cold start #559

Description

@josegonzalezlopez

Pre-flight Checks

  • I have searched existing issues and this is not a duplicate
  • I understand this issue needs status:approved before a PR can be opened

Bug Description

When two engram processes cold-start against the same fresh engram.db at (almost) the same instant — the canonical case is the Claude Code plugin's SessionStart hook launching engram serve in the background while Claude Code itself spawns the engram mcp stdio server for the same session — one of them wins the race to switch journal_mode to WAL and the other dies immediately with "database is locked" instead of waiting it out. The MCP client sees the process exit and drops the connection for the rest of the session, with no retry.

PRAGMA busy_timeout does NOT protect this specific pragma: switching journal_mode to WAL requires exclusive access to the file, and SQLite returns SQLITE_BUSY immediately if another connection has the file open, regardless of busy_timeout. Confirmed by reproducing the exact race in a unit test (open one connection, hold a write transaction, concurrently call store.New() on a second connection against the same file) — reordering the pragmas alone does not fix it; retrying PRAGMA journal_mode = WAL at the application level does.

Steps to Reproduce

  1. Delete/rename ~/.engram/engram.db* so the next start is a cold start
  2. Trigger two engram processes to open the fresh DB at (almost) the same instant, e.g. the Claude Code plugin flow: SessionStart hook does engram serve & while Claude Code independently spawns engram mcp --tools=agent for the same session
  3. Observe the MCP server log for that session

Expected Behavior

The process that loses the race waits out the contention (bounded retry) and completes startup normally; both processes end up healthy.

Actual Behavior

Server stderr: engram: engram: pragma "PRAGMA journal_mode = WAL": database is locked (261)
Connection failed after 553ms: MCP error -32000: Connection closed

The MCP tools (mem_save, mem_search, etc.) are silently absent for the rest of that Claude Code session. Reproduced identically across 6 separate real occurrences in ~/.cache/claude-cli-nodejs/*/mcp-logs-plugin-engram-engram/*.jsonl (2026-06-20 x2, 2026-06-24, 2026-06-29, 2026-07-01), always right after a period where engram serve was not already running.

Operating System

Linux (Fedora)

Engram Version

1.15.13

Agent / Client

Claude Code

Relevant Logs

```
{"debug":"Starting connection with timeout of 30000ms", ...}
{"error":"Server stderr: engram: engram: pragma "PRAGMA journal_mode = WAL": database is locked (261)\n", ...}
{"debug":"Connection failed after 553ms: MCP error -32000: Connection closed", ...}
{"error":"Connection failed: MCP error -32000: Connection closed", ...}
```

Additional Context

Fix implemented and verified locally on branch fix/journal-mode-wal-startup-race: added an explicit bounded retry (execStartupPragmaWithRetry, reusing the existing isRetryableSQLiteLockError classifier) around the startup pragma loop in internal/store/store.go (New and newWithoutRepair), plus a regression test (TestNewSurvivesConcurrentJournalModeSwitchContention) that reproduces the exact contention with two real *sql.DB connections over the same file. Full suite green, PR to follow once this issue is approved.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions