You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
On every run, AbstractAgent.prepareRunAgentInput() serializes the entirethis.messages array into RunAgentInput.messages (it only strips activity-role messages — no bound on count). So a conversation with 1000 messages sends 1000 + 1 messages on the next turn, and 1001 + 1 on the one after that, and so on — an O(N) payload per turn to convey O(1) new content.
This is inherent to the base AbstractAgent, so it affects every transport (HttpAgent over SSE, WebSocket agents, etc.) and every integration (CopilotKit included).
Why this matters for stateful backends
Many backends already persist the full conversation and rebuild the LLM context server-side. The ADK middleware (ag-ui-adk) is a concrete example: it holds sessions in a store and, on each run, _get_unseen_messages()discards every message whose id is already in processed_message_ids — i.e. it deliberately ignores the re-sent history and only processes the new message. The LLM context comes from the persisted session, not from the re-sent array.
So for these backends the client is uploading the whole history on every turn only to have the server throw almost all of it away. The costs are real even though the LLM/token cost is not doubled:
Bandwidth / latency: the request body grows without bound; a 1000-message thread ships ~1000 messages of dead weight per turn.
Mobile / metered networks: uploading megabytes of already-known history per keystroke-turn is wasteful.
Relationship to existing issues
[Feature]: Add messageFilter to AgentConfig #2062 (messageFilter on AgentConfig) is the closest — a client-side callback to prune messages before send. It's a good escape hatch, but it's a manual, per-integration knob: every app has to know to set it, and it has no notion of what the server already has. The integrator has to guess the filter.
A protocol-level notion of incremental message sync, so the client doesn't have to resend what the server already holds. A few shapes, from least to most invasive:
Standardize "send only unconfirmed messages" — the client tracks which message ids the server has acknowledged (e.g. via MESSAGES_SNAPSHOT / run completion) and, when the agent is known to be stateful, sends only messages not yet acknowledged. A capability flag (GET /capabilities → statefulHistory: true) could gate this so stateless backends keep receiving the full array.
A sync cursor / sinceMessageId on RunAgentInput — the client sends messages after a cursor the server previously returned; the server fills the rest from its store.
Make messageFilter ([Feature]: Add messageFilter to AgentConfig #2062) first-class + defaulted — ship it, and provide a built-in filter for the "stateful backend" case rather than leaving it to each integrator.
Each has trade-offs (spec change vs. client-only, correctness on reconnect/rewind, stateless-backend compatibility). I'd value a maintainer's take on whether the client resending full history is considered acceptable by design, or whether a standard incremental path is wanted.
Repro (conceptual)
// After N turns, agent.messages.length === 2N (user+assistant).// Every runAgent() call:prepareRunAgentInput()// → RunAgentInput.messages = ALL 2N messages// Backend (ag-ui-adk) _get_unseen_messages() → keeps only the 1 new one.
I'm happy to prototype option 1 or 3 (client-side, opt-in via capability) in @ag-ui/client + ag-ui-adk if that's a direction you'd accept.
Summary
On every run,
AbstractAgent.prepareRunAgentInput()serializes the entirethis.messagesarray intoRunAgentInput.messages(it only stripsactivity-role messages — no bound on count). So a conversation with 1000 messages sends 1000 + 1 messages on the next turn, and 1001 + 1 on the one after that, and so on — an O(N) payload per turn to convey O(1) new content.This is inherent to the base
AbstractAgent, so it affects every transport (HttpAgentover SSE, WebSocket agents, etc.) and every integration (CopilotKit included).Why this matters for stateful backends
Many backends already persist the full conversation and rebuild the LLM context server-side. The ADK middleware (
ag-ui-adk) is a concrete example: it holds sessions in a store and, on each run,_get_unseen_messages()discards every message whose id is already inprocessed_message_ids— i.e. it deliberately ignores the re-sent history and only processes the new message. The LLM context comes from the persisted session, not from the re-sent array.So for these backends the client is uploading the whole history on every turn only to have the server throw almost all of it away. The costs are real even though the LLM/token cost is not doubled:
structuredClone_(this.messages)on every run (see also [Bug]: structuredClone_ deep-clones messages per subscriber per event — 20s+ main-thread blocks in CopilotKit chats #1644 — per-subscriber deep clones causing 20s+ main-thread blocks, and Performance: O(N) message cloning/serialization per streamed token causes blocking in long chats #1048 — O(N) cloning/serialization in long chats).Relationship to existing issues
messageFilteronAgentConfig) is the closest — a client-side callback to prune messages before send. It's a good escape hatch, but it's a manual, per-integration knob: every app has to know to set it, and it has no notion of what the server already has. The integrator has to guess the filter./agents/state). This issue is the upload counterpart: the same unbounded-history problem, but on the outboundRunAgentInput.Proposed direction (for discussion)
A protocol-level notion of incremental message sync, so the client doesn't have to resend what the server already holds. A few shapes, from least to most invasive:
MESSAGES_SNAPSHOT/ run completion) and, when the agent is known to be stateful, sends only messages not yet acknowledged. A capability flag (GET /capabilities→statefulHistory: true) could gate this so stateless backends keep receiving the full array.sinceMessageIdonRunAgentInput— the client sends messages after a cursor the server previously returned; the server fills the rest from its store.messageFilter([Feature]: Add messageFilter to AgentConfig #2062) first-class + defaulted — ship it, and provide a built-in filter for the "stateful backend" case rather than leaving it to each integrator.Each has trade-offs (spec change vs. client-only, correctness on reconnect/rewind, stateless-backend compatibility). I'd value a maintainer's take on whether the client resending full history is considered acceptable by design, or whether a standard incremental path is wanted.
Repro (conceptual)
I'm happy to prototype option 1 or 3 (client-side, opt-in via capability) in
@ag-ui/client+ag-ui-adkif that's a direction you'd accept.