Skip to content

Commit 26eda2f

Browse files
committed
docs(ai): correct structured-output streaming guidance
1 parent 196088c commit 26eda2f

2 files changed

Lines changed: 13 additions & 9 deletions

File tree

docs/structured-outputs/streaming.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Streaming Structured Output UIs
33
id: structured-outputs-streaming
44
order: 3
5-
description: "Build a UI that fills in field by field as the model streams structured JSON. chat({ outputSchema, stream: true }) on the server, useChat({ outputSchema }) on the client — progressive partial state plus a validated terminal object."
5+
description: "Build a UI that fills in field by field as the model streams structured JSON. chat({ outputSchema, stream: true }) on the server, useChat({ outputSchema }) on the client — progressive partial state plus a typed terminal object."
66
keywords:
77
- tanstack ai
88
- structured outputs
@@ -16,7 +16,7 @@ keywords:
1616

1717
You have an existing chat-style endpoint and you want the structured response to populate a UI _while_ the model is generating — a form filling in field by field, a card whose ingredients list grows as JSON streams in, a typewriter preview of a JSON-typed report. Blocking on `await chat({ outputSchema })` would leave the UI dark until the whole object is ready; this guide is the alternative.
1818

19-
By the end you'll have a server endpoint streaming structured JSON as Server-Sent Events, and a client that reads a typed `partial` (progressive object) and `final` (validated terminal object) from `useChat`.
19+
By the end you'll have a server endpoint streaming structured JSON as Server-Sent Events, and a client that reads a typed `partial` (progressive object) and `final` (completed terminal object) from `useChat`.
2020

2121
> **Note:** This is the streaming counterpart of [One-Shot Extraction](./one-shot). If you don't need progressive UI updates, the one-shot path is simpler. If you want users to iterate on the object across multiple turns and keep history, see [Multi-Turn Chat](./multi-turn).
2222
@@ -48,11 +48,11 @@ export async function POST(request: Request) {
4848
}
4949
```
5050

51-
That's the entire server side. `chat({ outputSchema, stream: true })` returns a `StructuredOutputStream<InferSchemaType<typeof PersonSchema>>` — an `AsyncIterable` of standard streaming events plus a terminal `structured-output.complete` event carrying the validated object. `toServerSentEventsResponse` knows what to do with it.
51+
That's the entire server side. `chat({ outputSchema, stream: true })` returns a `StructuredOutputStream<InferSchemaType<typeof PersonSchema>>` — an `AsyncIterable` of standard streaming events plus a terminal `structured-output.complete` event carrying the completed object. `toServerSentEventsResponse` knows what to do with it.
5252

5353
## Client with `useChat`
5454

55-
Pass the same schema to `useChat`. The hook gives you a progressively-parsed `partial` and a validated `final`:
55+
Pass the same schema to `useChat`. The hook gives you a progressively-parsed `partial` and a typed `final`:
5656

5757
```tsx
5858
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
@@ -82,7 +82,7 @@ function PersonExtractor() {
8282
<p>Name: {partial.name ?? ""}</p>
8383
<p>Age: {partial.age ?? ""}</p>
8484
<p>Email: {partial.email ?? ""}</p>
85-
{final && <pre>Validated: {JSON.stringify(final, null, 2)}</pre>}
85+
{final && <pre>Completed: {JSON.stringify(final, null, 2)}</pre>}
8686
</form>
8787
);
8888
}
@@ -91,8 +91,8 @@ function PersonExtractor() {
9191
What the hook does for you:
9292

9393
- **`partial`** is `DeepPartial<z.infer<typeof PersonSchema>>` — every property optional, every nested array element optional. Updated from `TEXT_MESSAGE_CONTENT` deltas via the runtime's partial-JSON parser. The hook derives it from the latest assistant message's `structured-output` part (see [Multi-Turn Chat](./multi-turn) for why that distinction matters), so it reads `{}` between `sendMessage()` and the first chunk without any extra reset state.
94-
- **`final`** is `z.infer<typeof PersonSchema> | null` — the validated terminal payload from the `structured-output.complete` event. `null` until the run completes successfully.
95-
- **`outputSchema`** is used purely for client-side TypeScript inference. Validation still runs on the server against the schema you pass to `chat({ outputSchema })` on the server route — the client doesn't re-validate.
94+
- **`final`** is `z.infer<typeof PersonSchema> | null` — the completed terminal payload from the `structured-output.complete` event. `null` until the run completes successfully.
95+
- **`outputSchema`** is used purely for client-side TypeScript inference. The streaming path does not run Standard Schema validation; validate the completed object in the consumer when required.
9696
- The same shape works for **non-streaming adapters too**. If an adapter (Anthropic, Gemini, Ollama) returns a single `structured-output.complete` event with no incremental deltas, `partial` stays `{}` and `final` populates when the event arrives. Same consumer code.
9797

9898
`outputSchema` is optional: omit it and `useChat` returns its standard shape without `partial` / `final`.
@@ -144,7 +144,7 @@ return (
144144
type: "CUSTOM",
145145
name: "structured-output.complete",
146146
value: {
147-
object: T; // validated, parsed, typed
147+
object: T; // completed, parsed, typed
148148
raw: string; // full accumulated JSON text
149149
reasoning?: string; // present only for thinking/reasoning models
150150
},
@@ -164,6 +164,8 @@ Streaming structured output works with **every adapter**, but only some support
164164
| `@tanstack/ai-openrouter` | Native single-request stream (`response_format: json_schema`) |
165165
| `@tanstack/ai-grok` | Native single-request stream (Chat Completions, `response_format: json_schema`) |
166166
| `@tanstack/ai-groq` | Native single-request stream (Chat Completions, `response_format: json_schema`) |
167+
| `@tanstack/ai-bedrock` | Native stream through Converse or an OpenAI-compatible API |
168+
| `@tanstack/ai-byteplus` | Native single-request stream on supported models; unsupported models emit `RUN_ERROR` |
167169
| Other adapters (anthropic, gemini, ollama, …) | Fallback: runs non-streaming `structuredOutput` and emits the final object as one `structured-output.complete` event |
168170

169171
The fallback path keeps the consumer code identical across providers — you always read the final object off `structured-output.complete` — but you won't see incremental deltas unless the adapter implements `structuredOutputStream` natively.
@@ -192,7 +194,7 @@ const stream = chat({
192194

193195
for await (const chunk of stream) {
194196
if (chunk.type === "CUSTOM" && chunk.name === "structured-output.complete") {
195-
// Validated and typed against PersonSchema.
197+
// Typed against PersonSchema. Validate here when required.
196198
console.log(chunk.value.object.name);
197199
console.log(chunk.value.object.age);
198200
}

packages/ai/skills/ai-core/structured-outputs/SKILL.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,8 @@ The terminal event is a `CUSTOM` chunk: `{ type: 'CUSTOM', name: 'structured-out
189189
| `@tanstack/ai-grok` (Grok 4 family only) | **Native combined mode (#605)**`response_format: json_schema` + `tools`. Grok 2 / 3 fall back |
190190
| `@tanstack/ai-openrouter` | Native single-request stream (legacy `structuredOutputStream` path; per-call combined-mode lookup is a follow-up) |
191191
| `@tanstack/ai-groq` | Legacy `structuredOutputStream` only (no tools — Groq's API rejects schema + tools + stream) |
192+
| `@tanstack/ai-bedrock` | Separate native `structuredOutputStream` finalization through Converse or an OpenAI-compatible API |
193+
| `@tanstack/ai-byteplus` | Native combined mode on supported models; unsupported models emit `RUN_ERROR` |
192194
| All other adapters (ollama, older Claude, Gemini 2.x, Grok 2/3) | Fallback: runs non-streaming `structuredOutput`, emits one `structured-output.complete` event |
193195

194196
**Native combined mode vs fallback** is signaled by the adapter's

0 commit comments

Comments
 (0)