docs(server): design horizontal scale-out over centralized data stores

[davidwkeith]

Summary

Designs an opt-in horizontal scale-out mode for the container host, so high-traffic deployments can run N identical @dwk/server replicas behind a load balancer with all authoritative state in centralized data stores. Design only — nothing is adopted; the single-process local-storage default of spec/self-hosting.md is unchanged and stays recommended for single-owner deployments.

New spec: spec/scale-out.md. Key decisions:

• Assembly, not invention: reuses @dwk/deno-host's runtime-agnostic centralized-store shims (feat(deno-host): SQL shim — external libSQL/Turso behind D1Database/SqlStorage #397–feat(deno-host): R2Bucket-shaped object storage adapter over an S3-compatible store #400) — createD1Database over libSQL, SqlStorage over embedded replicas, the per-id lease + alarm machinery, the conforming queue broker, and the S3-compatible R2Bucket adapter — composed inside @dwk/server behind a new storage: { mode: "central" } config.
• Two centralized services, both self-hostable: a libSQL service (sqld/Turso) for D1, per-object DO databases, and a new DenoKvLike-over-libSQL coordination KV (order-preserving key encoding, CAS via per-key versionstamps, TTL) — deliberately avoiding a third service (Redis etc.); plus any S3-compatible object store (MinIO/R2/B2) for blobs.
• Single-writer generalizes from lockfile to lease: per-request DO leases with a sync-before-serve rule for embedded replicas; the data-directory lockfile is replaced by a mode marker + startup probes in central mode.
• Fleet lifecycle: every replica polls alarms/queues (claim-CAS makes concurrent pollers safe); cron handlers run once fleet-wide via a tick lease; drain-aware shutdown makes rolling deploys the normal case.
• Honest limits documented: per-request latency costs, the libSQL primary as the write ceiling (with the per-object-DB sharding lever), the WebSocket/lease pinning window (LB affinity in v1, a residency-lease v2 sketch deferred), and a multi-replica testing + live-verification plan per host-contract.md §9.

Cross-links the design from spec/self-hosting.md (the horizontal-scale non-goal, §8 mitigations, resolved decision 4) and spec/portability.md §4.4 (this unlocks Cloud Run-class disk-less platforms).

Packages affected

None (spec-only). The design touches @dwk/server (private) and proposes consuming @dwk/deno-host; no package code changes in this PR.

Checklist

• Read the relevant spec(s) under spec/packages/ and updated them if
  behaviour changed
• Added/updated colocated tests (src/*.test.ts) — not applicable, spec-only design doc
• Ran the local CI gate: pnpm lint && pnpm format:check && pnpm typecheck && pnpm build && pnpm test — format:check run on the changed markdown; no code changed
• Added a changeset (pnpm changeset) — not applicable, no publishable package touched
• Updated catalog.json / conformance/status.json — not applicable, no new worker

---

Generated by Claude Code

[davidwkeith]

CI status note: the github-advanced-security check failure on this PR is a GitHub-side infrastructure error, not something in this diff. The AI code-scanning agent crashed during its own setup with CAPIError: 400 model_not_supported (its backend rejected the model it is configured to use, sweagent-capi:claude-opus-4.6) before it analyzed any files — and this PR only touches three markdown spec files, most of which the scanner's file filters would exclude anyway. The run cannot be re-run via the API ("This workflow run cannot be retried"), so there is no fix to push from the repo side; a new push would re-trigger it, and otherwise it should resolve once GitHub fixes the scanning agent's model configuration.

Everything actionable is green or on track: Conformance passed, and the main CI workflow (lint → format → typecheck → build → test) is running normally.

---

Generated by Claude Code

[davidwkeith]

Reviewed. Spec-only design doc (spec/scale-out.md, new), no package code touched.

CONTRIBUTING.md conformance:

• Title/scope: docs(server): design horizontal scale-out over centralized data stores — correct type, lowercase, scope reflects the package this design targets even though no @dwk/server code changed in this PR (consistent with the PR body's "touches @dwk/server (private)").
• Checklist: spec read/updated ✓; tests/changeset/catalog correctly left unchecked with one-line "not applicable" reasons (spec-only, no publishable package, no new worker) rather than deleted — matches the required convention.
• Cross-links resolve: self-hosting.md, portability.md, host-contract.md, deno-deploy-design.md, and packages/deno-host.md are all real files at the paths referenced, and the new doc is correctly placed at spec/scale-out.md (cross-cutting, not under spec/packages/) since it isn't a single package's spec.
• The design correctly treats @dwk/deno-host as reused rather than duplicated (§3), and its claim that @dwk/cf-shims's queue broker currently auto-acks (used to justify switching to the conforming broker in central mode) checks out against packages/cf-shims/src/queue.ts's own comment ("Returned without an explicit decision → auto-ack").
• Non-goal language in self-hosting.md §4/§8 is updated consistently — the single-process default explicitly still stands, this is opt-in.

No blocking issues found. Note: the github-advanced-security check is failing on this PR, but it's an unrelated infra error (GitHub's Copilot code-scanning agent returning model_not_supported), not something this diff caused — the required ci.yml/release-gate checks are green.

---

Generated by Claude Code