ai-sdk-cache

Record/replay cache middleware for the Vercel AI SDK.

You're building a UI over an LLM call. You tweak a margin, hot-reload, and pay for the same completion again. Your test suite hits the provider — so it's slow, flaky, needs secrets in CI, and bills you for the privilege. This middleware fixes both: identical requests replay from disk instantly. Change the prompt, and it's a miss — you get a real response, and it's cached for next time.

import { wrapLanguageModel } from "ai";
import { openai } from "@ai-sdk/openai";
import { cacheMiddleware } from "ai-sdk-cache";

const model = wrapLanguageModel({
  model: openai("gpt-5"),
  middleware: cacheMiddleware({
    enabled: () => process.env.NODE_ENV !== "production",
  }),
});

// Everything downstream is unchanged — and now free to re-run:
const { text } = await generateText({ model, prompt: "…" });

Works with streaming, tools, and structured output — it sits at the SDK's middleware seam, under all of that. Streams are recorded part-by-part as they pass through and replayed as a real stream; only fully-consumed streams are recorded, so an aborted generation never poisons the cache.

Install

npm i -D ai-sdk-cache   # AI SDK v6 ('ai' ^6) is a peer dependency

Deterministic tests, offline CI

Cache entries are plain JSON files under .ai-sdk-cache/ — inspectable, diffable, and safe to commit as fixtures. In CI, replay-only turns any cache miss into a loud failure instead of a surprise API call:

const model = wrapLanguageModel({
  model: openai("gpt-5"),
  middleware: cacheMiddleware({
    mode: process.env.CI ? "replay-only" : "default",
  }),
});

Mode	Hit	Miss
default	replay	call provider, record
record	call provider, overwrite	call provider, record
replay-only	replay	throw
bypass	call provider	call provider

Refresh fixtures intentionally with mode: "record" (or delete the JSON files).

How keys work

sha256(kind + provider + modelId + params) over a stable stringify (property order doesn't matter). Same model, same messages, same settings → same key. Need to ignore a volatile field? Supply your own:

cacheMiddleware({
  key: ({ type, provider, modelId, params }) =>
    myHash({ type, provider, modelId, params, ignore: "timestamps" }),
});

Stores

• fsStore(dir) — default; one JSON file per entry under .ai-sdk-cache/.
• memoryStore() — per-process, no disk; handy in unit tests.
• Bring your own: { get(key), set(key, value) } — Redis, SQLite, whatever.

Binary payloads (generated files/images as Uint8Array) survive the round trip via tagged base64.

Options

cacheMiddleware({
  store,     // CacheStore — default fsStore(dir)
  dir,       // fs store location — default ".ai-sdk-cache"
  mode,      // "default" | "record" | "replay-only" | "bypass"
  enabled,   // boolean | () => boolean — gate without rewiring the model
  key,       // custom cache-key function
});

Heads-up: this is a development and testing tool. For production response caching you want TTLs, invalidation, and shared storage — different problem, different tradeoffs.

License

MIT © Binaya Dhakal