You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(website): review fixes on 'We Measured the Runtime Swap'
Factual:
- Strands and Microsoft Agent Framework both read the protocol-standard
top-level resume array; only Mastra needs the interruptEvent shape.
Corrected the paragraph and the count ("Four runtimes, three answers").
Verified against strands-resume.request.json and to-agent.ts.
- Name @ag-ui/core 0.0.59 for the subagent events, not a bare version.
Editorial:
- Cut the self-narration lines; state facts plainly instead.
- Retitle the reds and outage sections; close on Conclusion.
- Drop the competing superlative from the final line.
- Trim two honesty declarations that were doing no work.
- Break the longest lines; average prose line 102 -> 82 characters
(predecessor is 84). Four "rather than" constructions converted to
plain sentences.
- Fold the editor's-note label into its first line in the predecessor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: apps/website/content/blog/2026-08-31-we-measured-the-runtime-swap.mdx
+52-33Lines changed: 52 additions & 33 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,8 @@ draft: false
10
10
11
11
The previous post in this series ended on an admission rather than a conclusion.
12
12
13
-
I had spent several thousand words showing that swapping the agent runtime under an Angular app costs one import line at the component level, and then I had to point out that every AG-UI backend in our repository was itself a LangGraph graph.
13
+
I had spent several thousand words showing that swapping the agent runtime under an Angular app costs one import line at the component level.
14
+
Then I had to point out that every AG-UI backend in our repository was itself a LangGraph graph.
14
15
A transport swap over one runtime.
15
16
Not a runtime swap.
16
17
I wrote that I believed the neutral contract would hold against a genuinely different runtime, and that belief is not measurement.
@@ -34,7 +35,6 @@ Every server was driven with live model calls until it produced each surface we
34
35
The raw event stream from each of those runs was saved.
35
36
Those captures then went back through schema validation and the protocol client's own `verifyEvents`, and through the adapter reducer, event for event.
36
37
37
-
That last step is the one that matters for a portability claim.
38
38
A screenshot of a working demo proves that one path worked once.
39
39
A committed transcript replayed through the real client proves what the runtime put on the wire, and it keeps proving it every time the test suite runs.
40
40
@@ -51,10 +51,11 @@ Messages, tool calls, and shared state crossed three non-LangGraph runtimes with
51
51
Not one line.
52
52
The Angular side of each example is the same `injectAgent()` call and the same `<chat>` element that the LangGraph demos use.
53
53
54
-
For me, that is the claim I actually wanted to test, and it is now a measurement rather than an argument from design.
54
+
For me, that is the claim I actually wanted to test.
55
55
56
-
One honest note on the Mastra interrupt cell.
57
-
It was verified live against a real model in the development harness, and its resume round trip is covered by tests that run the captured transcripts through the shipped protocol client.
56
+
One note on the Mastra interrupt cell.
57
+
It was verified live against a real model in the development harness.
58
+
Its resume round trip is covered by tests that replay the captured transcripts through the shipped protocol client.
58
59
The production service for it does not exist yet.
59
60
I am not going to call that cell production-proven, because it is not.
60
61
@@ -73,102 +74,120 @@ I assumed a runtime that did not adopt the convention would simply have no inter
73
74
74
75
What actually happens is worse.
75
76
76
-
Strands and Microsoft Agent Framework both signal an interrupt the way the protocol says to: the run-finished event carries an outcome object naming the pending interrupts.
77
+
Strands and Microsoft Agent Framework both signal an interrupt the way the protocol says to.
78
+
The run-finished event carries an outcome object naming the pending interrupts.
77
79
Neither of them emits the custom event at all.
78
80
Our handler for that event read the status and ignored the outcome.
79
81
80
82
So the run did not fail.
81
-
It finalized as a success, with a tool call sitting there waiting for an approval that the UI never asked for, and with the adapter's `interrupt` signal still undefined.
82
-
A silent wrong answer beats a loud failure only in the sense that it takes longer to notice.
83
+
It finalized as a success, with a tool call sitting there waiting for an approval the UI never asked for.
84
+
The adapter's `interrupt` signal stayed undefined.
85
+
It failed silently, which only means it took longer to notice.
83
86
84
87
The fix was small once the cause was clear, and it landed in [#888](https://github.com/cacheplane/angular-agent-framework/pull/888).
85
-
Both conventions are now recognized, and within one run the first signal to arrive wins, because Mastra emits both and we do not want a doubled interrupt.
88
+
Both conventions are now recognized.
89
+
Within one run the first signal to arrive wins, because Mastra emits both and a doubled interrupt helps nobody.
86
90
87
91
I want to name the mistake precisely, because "we had a bug" is not the lesson.
88
92
We built a protocol adapter and then tested it exclusively against one bridge implementation of that protocol.
89
-
A bridge convention and a protocol primitive are different things, and a test suite that only ever sees one backend cannot tell you which of the two you depend on.
93
+
A bridge convention and a protocol primitive are different things.
94
+
A test suite that only ever sees one backend cannot tell you which of the two you depend on.
90
95
91
96
### Our resume payload was shaped like LangGraph
92
97
93
98
The second defect is the same mistake wearing different clothes.
94
99
95
100
When the user approves the pending action, the adapter sends the resume value back.
96
-
It sent exactly the shape the LangGraph bridge reads, nested under a command object in the forwarded properties.
101
+
It sent exactly the shape the LangGraph bridge reads, under `forwardedProps.command.resume`.
97
102
That works for LangGraph.
98
103
It is a coincidence anywhere else.
99
104
100
-
Mastra needs the interrupt's identity carried with the value, specifically the tool-call id and the run id of the suspended run, or it cannot find the suspension to resume.
101
-
Microsoft Agent Framework reads a top-level resume array before it looks at the forwarded properties, and it wants an entry addressing every pending interrupt.
102
-
Send it nothing addressable and it raises an error rather than guessing.
105
+
Mastra needs the interrupt's identity carried with the value, specifically the tool-call id and the run id of the suspended run.
106
+
Without those it cannot find the suspension to resume.
103
107
104
-
Three runtimes, three answers to the question of how a resume identifies itself.
108
+
Strands and Microsoft Agent Framework want something else entirely: the top-level `resume` array the protocol itself defines, carrying one entry per interrupt id.
109
+
Microsoft additionally wants an entry addressing every pending interrupt.
110
+
Send it nothing addressable and it raises an error.
111
+
112
+
Four runtimes, three answers to the question of how a resume identifies itself.
105
113
106
114
The adapter now derives the outgoing shape from how the interrupt arrived, which is the only signal available that is not a hardcoded runtime name.
107
-
That work is in [#889](https://github.com/cacheplane/angular-agent-framework/pull/889), and [#891](https://github.com/cacheplane/angular-agent-framework/pull/891) upgraded the protocol packages so the standard top-level resume array became sendable at all.
115
+
That work is in [#889](https://github.com/cacheplane/angular-agent-framework/pull/889).
116
+
The protocol-package upgrade in [#891](https://github.com/cacheplane/angular-agent-framework/pull/891) is what made the standard top-level array sendable at all.
108
117
The application-facing call did not change.
109
118
You still write `submit({ resume })` and the adapter decides what goes on the wire.
110
119
111
120
Both defects share a root.
112
121
Neither would have been found by another demo, another end-to-end test, or another review of our own code.
113
122
They were only findable by pointing the adapter at software written by people who had never heard of us.
114
123
115
-
## What stayed red, and why that is not a bug list
124
+
## What stayed red
116
125
117
-
Two cells did not go green, and I would rather explain them than quietly leave them off the page.
126
+
Two cells did not go green.
118
127
119
128
**Shared state on Strands is partial.**
120
129
Its bridge sends whole-document snapshots and never sends a patch, and a tool only contributes state if it opts in through a per-tool hook.
121
-
Because a snapshot replaces the document rather than merging into it, every hook has to return the entire state object.
130
+
A snapshot replaces the document; it does not merge into it.
131
+
So every hook has to return the entire state object.
122
132
A hook that returns just the key it touched will erase its siblings.
123
-
That is a real hazard, it is documented in the example, and it is a property of the upstream bridge rather than of the protocol or of us.
133
+
That is a real hazard.
134
+
It is documented in the example.
135
+
The cause is the upstream bridge, not the protocol and not us.
124
136
125
137
**Subagents are red for all three.**
126
138
This one surprised me, and it is the finding I would most want another framework author to read.
127
139
128
140
It is tempting to write that as three bugs.
129
141
It is not.
130
142
Strands models delegation as a handoff custom event alongside step boundaries.
131
-
Mastra treats the activity event family as belonging to background work and observational memory, which is a defensible reading of the same specification.
143
+
Mastra treats the activity event family as belonging to background work and observational memory.
144
+
That is a defensible reading of the same specification.
132
145
Microsoft Agent Framework does emit activity snapshots, but at the granularity of an executor, and it never emits the incremental form.
133
146
134
147
Three runtimes, three different mental models of what a subagent is, and none of them wrong on their own terms.
135
148
Meanwhile our own projection keys on an activity type of `subagent`, which is a convention our demo backends adopt and nobody else has any reason to.
136
-
The protocol grew dedicated subagent events in version 0.0.59, and as of this measurement no runtime emits them.
I think that is an ecosystem finding, not a defect report.
139
153
Delegation is the surface where the AG-UI vocabulary is agreed on in syntax and not yet in meaning.
140
-
If you are building on server-declared subagents today, build on a backend you control, and know that you are ahead of the ecosystem rather than portable across it.
154
+
If you are building on server-declared subagents today, build on a backend you control.
155
+
You are ahead of the ecosystem, not portable across it.
141
156
142
-
## The finding nobody asked for
157
+
## What the deploy check found
143
158
144
-
Measuring the deploy path surfaced something unrelated and much worse than anything in the matrix.
159
+
Measuring the deploy path surfaced something unrelated.
145
160
146
161
One of our AG-UI demo topics had been dead in production for two and a half months.
147
-
Its imports were written in a form that resolves when the topic is served from its own directory in development, and does not resolve in the aggregated deployment layout, so every image built since the middle of June crashed on startup.
148
-
The deploy command uploads and detaches, which means it reports success at upload time.
162
+
Its imports were written in a form that resolves when the topic is served from its own directory in development.
163
+
In the aggregated deployment layout they do not resolve, so every image built since the middle of June crashed on startup.
164
+
The deploy command uploads and detaches, so it reports success at upload time.
149
165
The platform kept serving the last image that booted.
150
-
Nothing was red anywhere, and the route had been returning a not-found for months.
166
+
Nothing was red anywhere.
167
+
The route had been returning a not-found for months.
151
168
152
169
The fix in [#899](https://github.com/cacheplane/angular-agent-framework/pull/899) was two import statements.
153
-
The part worth keeping is the second half: the deploy workflow now installs the exact dependency set, imports the server, and fails the job if it cannot boot.
170
+
The deploy workflow now installs the exact dependency set, imports the server, and fails the job if it cannot boot.
154
171
155
172
Verify the route, not the exit code.
156
173
A deploy step that exits zero has told you that a file was uploaded.
157
174
It has not told you that anything is running, and if your platform keeps the previous image alive on failure, the absence of an alarm is not evidence.
158
175
159
-
## Where this leaves the claim
176
+
## Conclusion
160
177
161
-
The neutral `Agent` contract is portable across runtimes for messages, tool calls, and state, and that is now measured against three implementations in two languages rather than asserted from the shape of the types.
178
+
The neutral `Agent` contract is portable across runtimes for messages, tool calls, and state.
179
+
That is now measured against three implementations in two languages.
162
180
163
181
Interrupts are portable as of these fixes, and they were not before, in a way no amount of internal testing would have revealed.
164
182
165
183
Subagents are not portable today, and the reason is upstream disagreement rather than a missing feature on anyone's list.
166
184
167
-
The matrix now lives in the [adapter guide](/docs/choosing-an-adapter) with a cause column on every gap, split three ways: the protocol cannot express it, the upstream bridge does not emit it, or our adapter failed to consume it.
185
+
The matrix now lives in the [adapter guide](/docs/choosing-an-adapter), with a cause column on every gap.
186
+
Split three ways: the protocol cannot express it, the upstream bridge does not emit it, or our adapter failed to consume it.
168
187
That third category is the one I care about keeping honest, and it is currently empty.
169
188
170
189
The transcripts are committed as test fixtures, verbatim from the wire.
171
190
Interrupt detection and resume shaping are now pinned to what three real runtimes actually sent us on a specific day.
172
191
The next regression in either one fails a test.
173
192
174
-
That is the difference between a portability claim and a portability guarantee, and it is the only part of this work I would insist on.
193
+
That is the difference between a portability claim and a portability guarantee.
Copy file name to clipboardExpand all lines: apps/website/content/blog/2026-08-31-what-changes-when-the-runtime-changes.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -301,8 +301,8 @@ Two smaller caveats.
301
301
The interrupt path currently depends on a `CUSTOM` event name that the LangGraph bridge emits, so an unrelated AG-UI backend would need to adopt that convention.
302
302
And the subagent path depends on the backend emitting native `ACTIVITY` events, which our demo backend does deliberately.
303
303
304
-
*Editor's note, added after publication.*
305
-
*We went and measured it: three genuinely non-LangGraph backends, two languages, wire transcripts replayed through the shipped client.*
304
+
*Editor's note, added after publication: we went and measured it.*
305
+
*Three genuinely non-LangGraph backends, two languages, wire transcripts replayed through the shipped client.*
306
306
*The results, including two adapter defects the exercise exposed, are in [We Measured the Runtime Swap](/blog/we-measured-the-runtime-swap).*
0 commit comments