fix(registry): give the SQLite contention retry a time budget

[e1010101]

The Windows leg failed on #105 with attempt to write a readonly database from one of the 16 processes in callback-concurrent-init. It is not #105's — that PR touches engine spawning — and it is not a new flake: it is the same class #104 addressed at journal_mode = WAL, surfacing one frame along at the schema-init transaction.

Raising this separately so #105 is not held behind it, and so the reasoning is reviewable on its own.

The retry was working; it ran out

Worth establishing before changing anything, since "add more retries" is the kind of fix that is easy to apply and hard to justify:

• the error code is SQLITE_READONLY (verified directly against better-sqlite3, not assumed), which isTransientSqliteError already matches;
• runSqliteBusyRetry is in the failing stack, so the wrapper engages;
• the ladder [10, 50, 200, 500, 1000] spends 1.76s, and the worker that died had been contending for 3.5s.

So it was giving up mid-race.

An attempt count is the wrong unit

What is being waited out is a window of contention whose length has nothing to do with how many times we have asked. This is now a time budget — 15s on Windows, 5s elsewhere — matched to the busy_timeout = 10000 already set on the connection, so the two agree about how long this class of contention is worth waiting for.

Backoff is exponential and jittered. Without jitter, peers that collide once back off by the same amount and collide again on every subsequent attempt, which is how a ladder that looks generous still exhausts itself.

What I can and cannot claim

I instrumented the give-up path and ran it at six times CI's concurrency (96 processes against one database):

RETRY-GAVEUP elapsed=15015ms code=SQLITE_READONLY

The loop engages, backs off, and exhausts the entire budget. That is the honest result, and it cuts both ways:

• it confirms the mechanism — this is real contention outlasting a bounded wait, not a predicate that fails to match or a wrapper that never runs;
• and it confirms that no budget is sufficient for unbounded concurrency. This raises a ceiling from 1.76s to 15s against observed contention of 3.5s. It does not remove one.

A straight comparison against main at that concurrency is within noise (1/12 vs 2/12), for the same reason: at 96 processes both exhaust whatever they are given. At CI-equivalent load (16 processes) I get 0 failures in 10 local runs, but I would not lean on that either — main also passes locally at that load, which is precisely why this only ever showed up on the runner.

The change that would remove the ceiling is serializing initialization across processes — a lock file around the migration, so peers wait once rather than colliding repeatedly. That is a larger change to the boot path and deserves its own review rather than being smuggled in behind a flake fix. Happy to take it if you want it; the comment in the code says the same thing so the next reader does not mistake this for a guarantee.

Why it matters outside CI

The gateway, the CLI and session workers all open this database. Sixteen simultaneous openers is a test construct, but two or three is an ordinary Windows session, and those were exposed to the same race with a 1.76s tolerance.

[hristo2612]

Gated on top of main: typecheck clean, 313 files / 3850 passing, and none of the three known load-dependent suites fired, so nothing is being quietly excused.

The change is sound and the reasoning in the body is right. I checked the thing I most wanted to check — whether the budget can truncate the existing ladder and reintroduce the race #104 fixed — and it cannot:

	before	after
POSIX	10+50+200 = 260 ms	5000 ms
win32	10+50+200+500+1000 = 1760 ms	15000 ms

The deadline is set once before the first attempt and every sleep is min(jittered, remainingMs), so cumulative sleep is bounded by the budget, and the budget strictly exceeds the old totals on both platforms. Even with every jitter draw at 0.5×, the first six sleeps total ~315 ms. There is no path that gives up earlier than main did. isTransientSqliteError is untouched so SQLITE_READONLY stays transient, and throw error rethrows the original object unwrapped, so a genuinely read-only database still surfaces the identical error after a bounded wait.

One blocker, one line: the budget uses a wall clock.

const deadline = performance.now() + SQLITE_RETRY_BUDGET_MS;
// ...
const remainingMs = deadline - performance.now();

This matters more here than it usually would, because this runs at process start and Atomics.wait blocks the thread outright. A backward wall-clock step during that window — Windows w32time resyncing at boot, an NTP correction, a VM snapshot restore — extends the synchronous block by the size of the step, unbounded, with no log output. The gateway just appears hung. A forward step silently truncates the budget back to main's behaviour. performance.now() is monotonic from process start and immune to both. It is a plain Node global, no import needed, and there is no non-test precedent in packages/jinn/src only because nothing else has needed a deadline.

Two non-blocking notes, your call:

• Nothing logs on the give-up path. You instrumented RETRY-GAVEUP elapsed=… code=… for the experiment and didn't ship it. A misconfigured read-only home now costs a silent 15 s stall at boot instead of 1.76 s. One warn on exhaustion turns a mystery hang into a diagnosable event, and I'd take it.
• No test, and I am explicitly not holding the PR for one. runSqliteBusyRetry is unexported and purely timing-based; a test here would pin the clock rather than the behaviour. Worth saying out loud that the correctness argument rests on reading rather than on a guard.

I also want to credit the framing in the PR body: you were straight that this raises a ceiling from 1.76 s to 15 s against observed 3.5 s contention, that it does not remove the contention, and that the real fix is serialising init behind a lock file. That is the right call for a flake fix and the right way to describe it.

Push the performance.now() change and I'll merge today. If you'd rather not spend a round trip on one line, say so and I'll apply it on merge with credit to you.

[hristo2612]

Applied the performance.now() change myself and rebased onto current main, as offered — you should not spend a round trip on one line. Pushed to your branch as b5a49d60, kept as a separate commit so it is clear what was yours and what was mine. The comment explaining why the monotonic clock matters here went in with it.

Re-gated after the rebase, since main moved a lot today (#105, #112, #115, #117, #118, #119, #120 all landed): typecheck clean, 314 files / 3876 passing, no flakes.

Merging.

I have also opened #121 for the real fix, and it is yours if you want it. Your framing is what the issue is built on: this raises a ceiling rather than removing one, and no bounded wait suffices for unbounded contention. What tipped it from a nice-to-have to a priority is that I watched the same bug fail different tests on two separate merges today — callback-concurrent-init on #105 and work-items/migrate on #118 — which is exactly what makes it read as unrelated noise instead of one root cause. I merged both after establishing ownership by hand, and that does not scale; a leg that goes red on a different unrelated test most runs is how a mandatory Windows check quietly becomes decorative.

The issue proposes the lock file you flagged, with crash-safety by pid liveness plus an mtime bound rather than a timeout, and acceptance at ten consecutive green Windows runs plus a killed-mid-migration test. No obligation at all — say either way and I will pick it up if you would rather not.