Skip to content

Commit 2818ec5

Browse files
bloveclaude
andcommitted
fix(website): review fixes on 'We Measured the Runtime Swap'
Factual: - Strands and Microsoft Agent Framework both read the protocol-standard top-level resume array; only Mastra needs the interruptEvent shape. Corrected the paragraph and the count ("Four runtimes, three answers"). Verified against strands-resume.request.json and to-agent.ts. - Name @ag-ui/core 0.0.59 for the subagent events, not a bare version. Editorial: - Cut the self-narration lines; state facts plainly instead. - Retitle the reds and outage sections; close on Conclusion. - Drop the competing superlative from the final line. - Trim two honesty declarations that were doing no work. - Break the longest lines; average prose line 102 -> 82 characters (predecessor is 84). Four "rather than" constructions converted to plain sentences. - Fold the editor's-note label into its first line in the predecessor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent bca65d3 commit 2818ec5

2 files changed

Lines changed: 54 additions & 35 deletions

File tree

apps/website/content/blog/2026-08-31-we-measured-the-runtime-swap.mdx

Lines changed: 52 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@ draft: false
1010

1111
The previous post in this series ended on an admission rather than a conclusion.
1212

13-
I had spent several thousand words showing that swapping the agent runtime under an Angular app costs one import line at the component level, and then I had to point out that every AG-UI backend in our repository was itself a LangGraph graph.
13+
I had spent several thousand words showing that swapping the agent runtime under an Angular app costs one import line at the component level.
14+
Then I had to point out that every AG-UI backend in our repository was itself a LangGraph graph.
1415
A transport swap over one runtime.
1516
Not a runtime swap.
1617
I wrote that I believed the neutral contract would hold against a genuinely different runtime, and that belief is not measurement.
@@ -34,7 +35,6 @@ Every server was driven with live model calls until it produced each surface we
3435
The raw event stream from each of those runs was saved.
3536
Those captures then went back through schema validation and the protocol client's own `verifyEvents`, and through the adapter reducer, event for event.
3637

37-
That last step is the one that matters for a portability claim.
3838
A screenshot of a working demo proves that one path worked once.
3939
A committed transcript replayed through the real client proves what the runtime put on the wire, and it keeps proving it every time the test suite runs.
4040

@@ -51,10 +51,11 @@ Messages, tool calls, and shared state crossed three non-LangGraph runtimes with
5151
Not one line.
5252
The Angular side of each example is the same `injectAgent()` call and the same `<chat>` element that the LangGraph demos use.
5353

54-
For me, that is the claim I actually wanted to test, and it is now a measurement rather than an argument from design.
54+
For me, that is the claim I actually wanted to test.
5555

56-
One honest note on the Mastra interrupt cell.
57-
It was verified live against a real model in the development harness, and its resume round trip is covered by tests that run the captured transcripts through the shipped protocol client.
56+
One note on the Mastra interrupt cell.
57+
It was verified live against a real model in the development harness.
58+
Its resume round trip is covered by tests that replay the captured transcripts through the shipped protocol client.
5859
The production service for it does not exist yet.
5960
I am not going to call that cell production-proven, because it is not.
6061

@@ -73,102 +74,120 @@ I assumed a runtime that did not adopt the convention would simply have no inter
7374

7475
What actually happens is worse.
7576

76-
Strands and Microsoft Agent Framework both signal an interrupt the way the protocol says to: the run-finished event carries an outcome object naming the pending interrupts.
77+
Strands and Microsoft Agent Framework both signal an interrupt the way the protocol says to.
78+
The run-finished event carries an outcome object naming the pending interrupts.
7779
Neither of them emits the custom event at all.
7880
Our handler for that event read the status and ignored the outcome.
7981

8082
So the run did not fail.
81-
It finalized as a success, with a tool call sitting there waiting for an approval that the UI never asked for, and with the adapter's `interrupt` signal still undefined.
82-
A silent wrong answer beats a loud failure only in the sense that it takes longer to notice.
83+
It finalized as a success, with a tool call sitting there waiting for an approval the UI never asked for.
84+
The adapter's `interrupt` signal stayed undefined.
85+
It failed silently, which only means it took longer to notice.
8386

8487
The fix was small once the cause was clear, and it landed in [#888](https://github.com/cacheplane/angular-agent-framework/pull/888).
85-
Both conventions are now recognized, and within one run the first signal to arrive wins, because Mastra emits both and we do not want a doubled interrupt.
88+
Both conventions are now recognized.
89+
Within one run the first signal to arrive wins, because Mastra emits both and a doubled interrupt helps nobody.
8690

8791
I want to name the mistake precisely, because "we had a bug" is not the lesson.
8892
We built a protocol adapter and then tested it exclusively against one bridge implementation of that protocol.
89-
A bridge convention and a protocol primitive are different things, and a test suite that only ever sees one backend cannot tell you which of the two you depend on.
93+
A bridge convention and a protocol primitive are different things.
94+
A test suite that only ever sees one backend cannot tell you which of the two you depend on.
9095

9196
### Our resume payload was shaped like LangGraph
9297

9398
The second defect is the same mistake wearing different clothes.
9499

95100
When the user approves the pending action, the adapter sends the resume value back.
96-
It sent exactly the shape the LangGraph bridge reads, nested under a command object in the forwarded properties.
101+
It sent exactly the shape the LangGraph bridge reads, under `forwardedProps.command.resume`.
97102
That works for LangGraph.
98103
It is a coincidence anywhere else.
99104

100-
Mastra needs the interrupt's identity carried with the value, specifically the tool-call id and the run id of the suspended run, or it cannot find the suspension to resume.
101-
Microsoft Agent Framework reads a top-level resume array before it looks at the forwarded properties, and it wants an entry addressing every pending interrupt.
102-
Send it nothing addressable and it raises an error rather than guessing.
105+
Mastra needs the interrupt's identity carried with the value, specifically the tool-call id and the run id of the suspended run.
106+
Without those it cannot find the suspension to resume.
103107

104-
Three runtimes, three answers to the question of how a resume identifies itself.
108+
Strands and Microsoft Agent Framework want something else entirely: the top-level `resume` array the protocol itself defines, carrying one entry per interrupt id.
109+
Microsoft additionally wants an entry addressing every pending interrupt.
110+
Send it nothing addressable and it raises an error.
111+
112+
Four runtimes, three answers to the question of how a resume identifies itself.
105113

106114
The adapter now derives the outgoing shape from how the interrupt arrived, which is the only signal available that is not a hardcoded runtime name.
107-
That work is in [#889](https://github.com/cacheplane/angular-agent-framework/pull/889), and [#891](https://github.com/cacheplane/angular-agent-framework/pull/891) upgraded the protocol packages so the standard top-level resume array became sendable at all.
115+
That work is in [#889](https://github.com/cacheplane/angular-agent-framework/pull/889).
116+
The protocol-package upgrade in [#891](https://github.com/cacheplane/angular-agent-framework/pull/891) is what made the standard top-level array sendable at all.
108117
The application-facing call did not change.
109118
You still write `submit({ resume })` and the adapter decides what goes on the wire.
110119

111120
Both defects share a root.
112121
Neither would have been found by another demo, another end-to-end test, or another review of our own code.
113122
They were only findable by pointing the adapter at software written by people who had never heard of us.
114123

115-
## What stayed red, and why that is not a bug list
124+
## What stayed red
116125

117-
Two cells did not go green, and I would rather explain them than quietly leave them off the page.
126+
Two cells did not go green.
118127

119128
**Shared state on Strands is partial.**
120129
Its bridge sends whole-document snapshots and never sends a patch, and a tool only contributes state if it opts in through a per-tool hook.
121-
Because a snapshot replaces the document rather than merging into it, every hook has to return the entire state object.
130+
A snapshot replaces the document; it does not merge into it.
131+
So every hook has to return the entire state object.
122132
A hook that returns just the key it touched will erase its siblings.
123-
That is a real hazard, it is documented in the example, and it is a property of the upstream bridge rather than of the protocol or of us.
133+
That is a real hazard.
134+
It is documented in the example.
135+
The cause is the upstream bridge, not the protocol and not us.
124136

125137
**Subagents are red for all three.**
126138
This one surprised me, and it is the finding I would most want another framework author to read.
127139

128140
It is tempting to write that as three bugs.
129141
It is not.
130142
Strands models delegation as a handoff custom event alongside step boundaries.
131-
Mastra treats the activity event family as belonging to background work and observational memory, which is a defensible reading of the same specification.
143+
Mastra treats the activity event family as belonging to background work and observational memory.
144+
That is a defensible reading of the same specification.
132145
Microsoft Agent Framework does emit activity snapshots, but at the granularity of an executor, and it never emits the incremental form.
133146

134147
Three runtimes, three different mental models of what a subagent is, and none of them wrong on their own terms.
135148
Meanwhile our own projection keys on an activity type of `subagent`, which is a convention our demo backends adopt and nobody else has any reason to.
136-
The protocol grew dedicated subagent events in version 0.0.59, and as of this measurement no runtime emits them.
149+
`@ag-ui/core` 0.0.59 added dedicated subagent events.
150+
As of this measurement, no runtime emits them.
137151

138152
I think that is an ecosystem finding, not a defect report.
139153
Delegation is the surface where the AG-UI vocabulary is agreed on in syntax and not yet in meaning.
140-
If you are building on server-declared subagents today, build on a backend you control, and know that you are ahead of the ecosystem rather than portable across it.
154+
If you are building on server-declared subagents today, build on a backend you control.
155+
You are ahead of the ecosystem, not portable across it.
141156

142-
## The finding nobody asked for
157+
## What the deploy check found
143158

144-
Measuring the deploy path surfaced something unrelated and much worse than anything in the matrix.
159+
Measuring the deploy path surfaced something unrelated.
145160

146161
One of our AG-UI demo topics had been dead in production for two and a half months.
147-
Its imports were written in a form that resolves when the topic is served from its own directory in development, and does not resolve in the aggregated deployment layout, so every image built since the middle of June crashed on startup.
148-
The deploy command uploads and detaches, which means it reports success at upload time.
162+
Its imports were written in a form that resolves when the topic is served from its own directory in development.
163+
In the aggregated deployment layout they do not resolve, so every image built since the middle of June crashed on startup.
164+
The deploy command uploads and detaches, so it reports success at upload time.
149165
The platform kept serving the last image that booted.
150-
Nothing was red anywhere, and the route had been returning a not-found for months.
166+
Nothing was red anywhere.
167+
The route had been returning a not-found for months.
151168

152169
The fix in [#899](https://github.com/cacheplane/angular-agent-framework/pull/899) was two import statements.
153-
The part worth keeping is the second half: the deploy workflow now installs the exact dependency set, imports the server, and fails the job if it cannot boot.
170+
The deploy workflow now installs the exact dependency set, imports the server, and fails the job if it cannot boot.
154171

155172
Verify the route, not the exit code.
156173
A deploy step that exits zero has told you that a file was uploaded.
157174
It has not told you that anything is running, and if your platform keeps the previous image alive on failure, the absence of an alarm is not evidence.
158175

159-
## Where this leaves the claim
176+
## Conclusion
160177

161-
The neutral `Agent` contract is portable across runtimes for messages, tool calls, and state, and that is now measured against three implementations in two languages rather than asserted from the shape of the types.
178+
The neutral `Agent` contract is portable across runtimes for messages, tool calls, and state.
179+
That is now measured against three implementations in two languages.
162180

163181
Interrupts are portable as of these fixes, and they were not before, in a way no amount of internal testing would have revealed.
164182

165183
Subagents are not portable today, and the reason is upstream disagreement rather than a missing feature on anyone's list.
166184

167-
The matrix now lives in the [adapter guide](/docs/choosing-an-adapter) with a cause column on every gap, split three ways: the protocol cannot express it, the upstream bridge does not emit it, or our adapter failed to consume it.
185+
The matrix now lives in the [adapter guide](/docs/choosing-an-adapter), with a cause column on every gap.
186+
Split three ways: the protocol cannot express it, the upstream bridge does not emit it, or our adapter failed to consume it.
168187
That third category is the one I care about keeping honest, and it is currently empty.
169188

170189
The transcripts are committed as test fixtures, verbatim from the wire.
171190
Interrupt detection and resume shaping are now pinned to what three real runtimes actually sent us on a specific day.
172191
The next regression in either one fails a test.
173192

174-
That is the difference between a portability claim and a portability guarantee, and it is the only part of this work I would insist on.
193+
That is the difference between a portability claim and a portability guarantee.

apps/website/content/blog/2026-08-31-what-changes-when-the-runtime-changes.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -301,8 +301,8 @@ Two smaller caveats.
301301
The interrupt path currently depends on a `CUSTOM` event name that the LangGraph bridge emits, so an unrelated AG-UI backend would need to adopt that convention.
302302
And the subagent path depends on the backend emitting native `ACTIVITY` events, which our demo backend does deliberately.
303303

304-
*Editor's note, added after publication.*
305-
*We went and measured it: three genuinely non-LangGraph backends, two languages, wire transcripts replayed through the shipped client.*
304+
*Editor's note, added after publication: we went and measured it.*
305+
*Three genuinely non-LangGraph backends, two languages, wire transcripts replayed through the shipped client.*
306306
*The results, including two adapter defects the exercise exposed, are in [We Measured the Runtime Swap](/blog/we-measured-the-runtime-swap).*
307307

308308
## Conclusion

0 commit comments

Comments
 (0)