Skip to content

feat(server): LibsqlKv — DenoKvLike coordination adapter over libSQL (scale-out phase 1) #428

Description

@davidwkeith

Phase 1 of the horizontal scale-out plan (spec/scale-out.md §15, merged in #427): implement the coordination-store adapter that the rest of the central storage mode builds on. This phase is deliberately standalone — no behavior change for existing users; nothing wires it into the host yet.

What

A DenoKvLike implementation (working name LibsqlKv) backed by one dedicated logical libSQL database, so @dwk/deno-host's lease / alarm / queue machinery (which is runtime-agnostic and only needs a CAS-capable KV behind that structural seam) can run against the same centralized SQL service scale-out mode already requires — avoiding a third service (Redis etc.). Design: spec/scale-out.md §8.

Single table:

CREATE TABLE IF NOT EXISTS kv (
  k          BLOB PRIMARY KEY,   -- order-preserving tuple encoding of the key array
  v          TEXT NOT NULL,      -- JSON value
  ver        INTEGER NOT NULL,   -- per-key monotonic versionstamp
  expires_at INTEGER             -- epoch ms, NULL = no TTL
);

Requirements (from the design)

  • Order-preserving key encoding across the key-part types the consumers use (strings, numbers): the alarm/queue due indexes range-scan [prefix … now] and depend on numeric parts sorting numerically (element-tagged encoding; big-endian, sign-flipped 8-byte numerics). This mirrors the guarantee Deno.Kv provides natively (spec/packages/deno-host.md, live-verification item 5).
  • atomic().check(...).set/delete(...).commit() maps to one libSQL transaction: SELECT ver per check (a versionstamp: null check asserts absence), return { ok: false } on any mismatch, else apply mutations bumping ver. client.batch(..., "write") (one implicit transaction, the existing LibsqlClientLike seam contract) is sufficient.
  • expireInexpires_at: reads and range scans treat expired rows as absent; lazy sweep from poll ticks. Ms-precise expiry is not required — only "expired is never returned."
  • list({ prefix, start, end }, { limit }) as an indexed range SELECT over the primary key.
  • Real Deno.Kv instances and @dwk/deno-host's FakeDenoKv remain the semantic reference: LibsqlKv must be a drop-in for the DenoKvLike consumers (lease.ts, alarms.ts, queue.ts).

Placement

A module in @dwk/server (private, first consumer) — mirroring how the Cloudflare shims lived in @dwk/server until a second consumer justified the @dwk/cf-shims extraction (#381). Extraction later is mechanical if the Deno host or another host wants it.

Testing (spec §14 item 1)

  • CAS under interleaving (two writers racing one key; absence checks; release-after-expiry no-op).
  • TTL expiry: expired rows absent from get and list; lazy sweep removes them.
  • Key-encoding order: property-style tests comparing list scan order against a sorted in-memory model, specifically numeric parts ordering numerically (the alarm/queue due-index correctness hinges on this).
  • Drive the real @dwk/deno-host acquireLease/setAlarm/pollAlarms/QueueBroker paths against LibsqlKv and assert the same behaviors their FakeDenoKv tests assert.
  • Unit tests run against node:sqlite-backed fakes through the existing LibsqlClientLike seam — no live service in unit tests.

Out of scope (later phases, spec §15)

  • Central-mode bindings assembly, storage: { mode: "central" } config, startup probes, mode marker (phase 2).
  • DO namespace wiring, embedded replicas, sync-before-serve (phase 3).
  • Fleet pollers, cron tick lease, drain (phase 4).
  • The §16.1 naming question (@dwk/deno-host rename / core extraction) — worth deciding before or alongside this issue since it introduces the @dwk/server@dwk/deno-host dependency, but it does not block the implementation.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions