Middleware support

[thisisaneeq]

Would love to see a middleware or plugin system that allows intercepting and transforming the chat flow.

So I'm building a memory integration package for Tanstack AI (already have for Vercel AI SDK using wrapLanguageModel). The goal is to:

1. Query relevant memories before each chat request
2. Inject them into the system prompt
3. Trigger memory extraction after the response completes

Without middleware, developers would need to manually orchestrate multiple steps but with middleware support, it could be as simple as:

const stream = chat({
  adapter: openai(),
  messages: modelMessages,
  middleware: [recallMiddleware()],
});

This would enable building integrations for cross-cutting concerns like memory injection, logging, caching, rate limiting, etc.

[AlemTuzlak]

What exactly would you want to transform and when? Would like to add this, but make it right and flexible for all use-cases

[thisisaneeq]

My use-case at the moment is RAG-based where when the user sends a new message, I need to extract the latest message, run a similarity search on that and include the top results in the system prompt (so transforming system prompt).

But middlewares broadly speaking would ideally need to support scenarios like:

• Guardrails: so injecting safety instructions into system prompt
• PII redaction: transforming messages to scrub sensitive data
• Supporting tool calls on the AI models that don't have native support for those etc.