@gmod/shared-read-cache

A promise cache for reads that several callers want at once. One read per key, shared by everyone who asks for it while it is in flight, cancelled only once every one of them has given up.

Why

When a cache memoizes a bare promise built from the first caller's signal, that caller's abort rejects everyone else awaiting it. In a genome browser, panning away from one block then fails its still-wanted siblings.

import { SharedReadCache } from '@gmod/shared-read-cache'

const cache = new SharedReadCache({
  cacheKey: chunk => chunk.toString(),
  fill: (chunk, signal) => readChunk(chunk, { signal }),
})

const data = await cache.get(chunk, opts.signal)

This package caches what a read produced, keyed however you like. @gmod/range-cache-filehandle sits below it and caches the raw HTTP byte ranges underneath a filehandle.

Docs

• docs/dataflow.md — how one get() flows
• docs/memory.md — how entries leave the cache, and the measurements behind the budgets
• docs/api.md — every option and method
• docs/consumers.md — which gmod packages use this, and where each measurement was taken

Budgets are opt-in

There is no default limit, because a sensible one depends on what you cache:

const cache = new SharedReadCache({
  maxSize: 100 * 2 ** 20,
  sizeOf: chunk => chunk.byteLength,
  cacheKey: chunk => chunk.toString(),
  fill: (chunk, signal) => readChunk(chunk, { signal }),
})

A budget bounds retained memory, not request size:

• The cache never turns a value away for being too large. A value bigger than the whole budget still goes in.
• Eviction never touches a read in flight, because dropping the promise would lose the de-duplication its callers rely on.
• Eviction only discards values the cache has already handed back once, so the worst a budget can cost is a re-read.

With no budget the cache grows for the life of the object. @gmod/tabix measured 2GB RSS panning a dense VCF before it bounded this. Pass a budget if the values are large or the object is long-lived.

Setting cache.maxSize = n later evicts immediately, so a consumer can shed memory under pressure.

idleTimeoutMs — reclaiming while nothing is happening

The cache checks maxSize when a read settles, so an idle cache stays at the level it reached, and no budget alone lowers it.

const cache = new SharedReadCache({
  maxSize: 1024 * 2 ** 20, // the ceiling under load
  idleTimeoutMs: 180_000, // ...but only while it is being used
  sizeOf: chunk => chunk.byteLength,
  fill: (chunk, signal) => readChunk(chunk, { signal }),
})

Use the two options together:

• Set maxSize generously. Below one request's working set it caches nothing, because the cache evicts each value before the next request can reuse it, while the reads in flight hold their memory anyway.
• idleTimeoutMs makes a generous ceiling affordable, because memory peaks under load and falls back when idle.

How the idle clock works:

• It runs from the last read of an entry, or from the fill settling if nothing has read it since.
• An entry fetched once and used every second never expires.
• A slow read still gets the full timeout for reuse after its download ends.
• The sweep skips reads in flight.
• cache.sweepIdle() reclaims on demand, for example when a tab goes hidden.

The sweep timer:

• The first read to settle arms it, and the first sweep that finds no settled entry stops it.
• It holds its cache weakly, so a dropped cache is collected without a dispose() call.
• It calls unref where that exists, so it never holds a Node process open.

One budget across several caches

A per-cache ceiling does not bound a consumer that opens many caches, such as one per open file. SharedBudget bounds them in aggregate and evicts the globally least-recently-used entry, so a busy cache can use the space an idle one is not using:

const budget = new SharedBudget(1024 * 2 ** 20)
const cache = new SharedReadCache({ budget, sizeOf, fill })

• SharedBudget works alongside maxSize; it does not replace it.
• It holds its members weakly, so there is no unregister to call.
• Every member must measure size in the same unit. A budget that adds one cache's bytes to another's entry counts bounds neither.

docs/memory.md has the measurement for dividing a ceiling by the file count instead.

Why sizeOf

sizeOf weighs a settled value against the budget; without it, the budget counts entries. The cache can only call sizeOf once a read settles, so a cache that budgets this way has to own its entries, which a plain LRU package cannot do. docs/consumers.md lists what each gmod package weighs.

Behaviour

• A caller with no signal pins the read. It can never abort, so aborting every other caller does not stop the read, and the read is uncancellable for everyone joined to it.
• The cache drops a rejection rather than caching it, so one transient failure does not make the key fail for the life of the cache.

Relationship to @gmod/abortable-promise-cache

This package replaces @gmod/abortable-promise-cache, which had two bugs:

• It never removed its listener from a caller's signal, so a long-lived signal accumulated one per key.
• It counted a caller that arrived already aborted as a waiter. An abort listener never fires on an already-aborted signal, so the count never reached zero and the read became uncancellable for everyone joined to it.