Skip to content

Commit 121ed1b

Browse files
bloveclaude
blove
and
claude
authored
docs(website): measured runtime support matrix in the adapter guide (#904)
* docs(website): measured runtime support matrix in the adapter guide Additive section on /docs/choosing-an-adapter: the 4-runtime x 5-surface portability matrix (LangGraph, AWS Strands, Microsoft Agent Framework, Mastra) with a three-way cause column — protocol, upstream integration, or our adapter — plus per-cell notes, the measurement method, and links to the three cockpit/runtimes examples and the committed SSE fixtures. No existing headings, anchors, or content changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(website): review fixes on the runtime support matrix - Credit AWS Strands alongside Microsoft Agent Framework for reading the protocol-standard top-level resume array (verified against strands-resume.request.json and to-agent.ts, which serves both from one branch). - Note that the Mastra row was measured against a real server with live model calls and committed transcripts, but is not yet running in the hosted demo deployment. - Compact the cause column to scannable labels; the legend below the table carries the definitions. - State the adapter-cause history once, in current-state framing. - Two blog-register lines moved to reference register. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 4572875 commit 121ed1b

1 file changed

Lines changed: 70 additions & 0 deletions

File tree

  • apps/website/content/docs/choosing-an-adapter

‎apps/website/content/docs/choosing-an-adapter/index.mdx‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,76 @@ Three layers of test doubles, smallest scope first:
8484

8585
Reach for the smallest layer that covers your test. Don't stand up aimock when a `mockAgent` unit test suffices; don't hand-roll a transport when `provideFakeAgent` exists.
8686

87+
## Measured runtime support
88+
89+
The AG-UI adapter is protocol-first, so "any AG-UI backend plugs in" is a claim that can be tested rather than asserted.
90+
On 2026-08-31 we tested it against three runtimes that have nothing to do with LangGraph, in two languages.
91+
92+
| Runtime | Messages | Tool calls | State | Interrupts | Subagents | Cause of any gap |
93+
|---|---|---|---|---|---|---|
94+
| **LangGraph** (via the AG-UI bridge) | Yes | Yes | Yes | Yes | Yes | — |
95+
| **AWS Strands** (Python) | Yes | Yes | Partial | Yes | No | State + subagents: upstream |
96+
| **Microsoft Agent Framework** (Python) | Yes | Yes | Yes | Yes | No | Subagents: upstream |
97+
| **Mastra** (TypeScript) | Yes | Yes | Yes | Yes | No | Subagents: upstream |
98+
99+
Every gap in that table falls into one of three causes, and the distinction is the point of the column:
100+
101+
- **Protocol** — the AG-UI event vocabulary has no way to express the capability. Nothing an adapter or a backend author can do about it today.
102+
- **Upstream integration** — the protocol can express it, but the runtime's AG-UI bridge does not emit the events. Fixable upstream.
103+
- **Our adapter** — `@threadplane/ag-ui` failed to consume something the wire already carried. Our bug, and our responsibility.
104+
105+
No cell in the current matrix is caused by the protocol, and no cell is caused by a bug in our adapter — the two that were are fixed.
106+
107+
### Reading the individual cells
108+
109+
**State on AWS Strands is partial.**
110+
The Strands AG-UI bridge never emits `STATE_DELTA`, and outbound state exists only where a tool opts in through per-tool `ToolBehavior` hooks.
111+
Because the adapter applies a `STATE_SNAPSHOT` as a full replacement, every hook must return the complete state object; a partial return clobbers sibling keys.
112+
Shared state works on Strands, but it is snapshot-only and it is opt-in per tool.
113+
114+
**Interrupts arrive two different ways.**
115+
AWS Strands and Microsoft Agent Framework signal an interrupt only through the protocol-standard `RUN_FINISHED` outcome, `{ type: 'interrupt', interrupts: [...] }`.
116+
The LangGraph bridge signals it only through a `CUSTOM` event named `on_interrupt`.
117+
Mastra emits both.
118+
The adapter detects either convention, and within a single run the first signal wins.
119+
120+
**Resume payloads are not portable, so the adapter shapes them per runtime.**
121+
Mastra reads `forwardedProps.command.interruptEvent`, carrying a tool-call id and a run id.
122+
AWS Strands and Microsoft Agent Framework both read the protocol-standard top-level `resume` array, one `{ interruptId, status, payload }` entry per interrupt; Microsoft additionally expects an entry for every pending interrupt.
123+
The LangGraph bridge reads `forwardedProps.command.resume`.
124+
You pass one neutral `submit({ resume })`, and the adapter derives the wire shape from how the interrupt arrived.
125+
126+
**The Mastra row was measured, but not in the hosted demo.**
127+
Its cells come from a real Mastra server driven with live model calls, and its transcripts are committed and replayed like the others.
128+
Unlike the Strands and Microsoft Agent Framework rows, it is not yet running in the hosted demo deployment.
129+
130+
**Subagents are red for every third-party runtime, and none of those reds are a bug.**
131+
The three runtimes do not fail to implement one thing; they model delegation three different ways.
132+
Strands routes handoffs through a `CUSTOM` event plus step events, with no `ACTIVITY` events at all.
133+
Mastra reserves `ACTIVITY_*` for background tasks and observational memory.
134+
Microsoft Agent Framework emits coarse executor-level activity snapshots and constructs `ACTIVITY_DELTA` nowhere.
135+
The Threadplane subagent projection keys on an `activityType` of `subagent`, which is a convention our own demo backend adopts.
136+
The protocol has carried dedicated `SUBAGENT_STARTED`, `SUBAGENT_FINISHED`, and `SUBAGENT_ERROR` events since `@ag-ui/core` 0.0.59, and no runtime emits them yet.
137+
Treat server-declared subagents as a capability of LangGraph and of backends you control, until that changes.
138+
139+
### How this was measured
140+
141+
Each runtime got a real server, started against the upstream AG-UI integration and driven with live model calls.
142+
The raw Server-Sent Events were captured off the wire, then replayed event-for-event through the pinned `@ag-ui/client` — schema parse plus `verifyEvents` — and through the adapter reducer itself.
143+
144+
The captures are committed at [`libs/ag-ui/fixtures/runtime-transcripts/`](https://github.com/cacheplane/angular-agent-framework/tree/main/libs/ag-ui/fixtures/runtime-transcripts), verbatim from the wire.
145+
The adapter test suite replays them on every run.
146+
147+
### Runnable examples
148+
149+
Each runtime has a standalone Angular example bound through `@threadplane/ag-ui`, with its own backend:
150+
151+
- [`cockpit/runtimes/aws-strands`](https://github.com/cacheplane/angular-agent-framework/tree/main/cockpit/runtimes/aws-strands) — Python, hosted in the shared FastAPI deployment.
152+
- [`cockpit/runtimes/microsoft-agent-framework`](https://github.com/cacheplane/angular-agent-framework/tree/main/cockpit/runtimes/microsoft-agent-framework) — Python, hosted in that same FastAPI deployment, running beside the Strands and LangGraph topics in one process.
153+
- [`cockpit/runtimes/mastra`](https://github.com/cacheplane/angular-agent-framework/tree/main/cockpit/runtimes/mastra) — TypeScript, hosted by a Node AG-UI service, because the upstream Mastra integration ships an in-process bridge rather than an HTTP endpoint.
154+
155+
In all three, the Angular component is ordinary `<chat [agent]="agent" />` over `injectAgent()`.
156+
87157
## Why two adapters?
88158

89159
The `Agent` contract from `@threadplane/chat` is intentionally runtime-neutral. The two adapters exist because they bridge different on-the-wire protocols into that contract:

0 commit comments

Comments
 (0)