Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions docs/chat/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,95 @@ TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction)

> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](./thinking-content).

### Type-Safe Tool Call Events

When you pass typed tools (defined with `toolDefinition()` and Zod schemas) to `chat()`, the stream chunks automatically carry type information for tool call events. Prefer the AG-UI field `toolCallName` (or the deprecated `toolName` alias) — both narrow to the union of your tool name literals. The `input` field on `TOOL_CALL_END` is typed as the union of your tool input schemas (typically set on the adapter-emitted END once arguments are complete):

```typescript
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
});

const messages = [
{ role: "user" as const, content: "What's the weather in Paris?" },
];

const stream = chat({
adapter: openaiText("gpt-5.5"),
messages,
tools: [weatherTool],
});

for await (const chunk of stream) {
// `'type' in chunk` is required for control-flow narrowing across the
// StreamChunk union (AG-UI event types from `@ag-ui/core` use Zod
// passthrough, which otherwise hides the discriminant from property access).
if ("type" in chunk && chunk.type === "TOOL_CALL_END") {
chunk.toolCallName; // ✅ typed as "get_weather" (not string)
chunk.input; // ✅ typed as { location: string; unit?: "celsius" | "fahrenheit" } | undefined
}
}
```

Without typed tools, names default to `string` and `input`/`output` default to `unknown` — the same behavior as before. The type narrowing is automatic when you use `toolDefinition()` with Zod schemas.

When multiple tools are provided, tool call events form a **discriminated union** — checking `toolCallName` (or `toolName`) narrows `input` / `output` to that specific tool's type:

```typescript
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
});

const searchTool = toolDefinition({
name: "search",
description: "Search the web",
inputSchema: z.object({ query: z.string() }),
});

const messages = [
{ role: "user" as const, content: "Find the weather for Paris" },
];

const stream = chat({
adapter: openaiText("gpt-5.5"),
messages,
tools: [weatherTool, searchTool],
});

for await (const chunk of stream) {
if ("type" in chunk && chunk.type === "TOOL_CALL_END") {
if (chunk.toolCallName === "get_weather") {
// ✅ input is narrowed to { location: string; unit?: "celsius" | "fahrenheit" }
console.log(`Weather in ${chunk.input?.location}`);
}
if (chunk.toolCallName === "search") {
// ✅ input is narrowed to { query: string }
console.log(`Searched for: ${chunk.input?.query}`);
}
}
}
```

> **Tip:** The typed stream type is exported as `TypedStreamChunk<TTools>`. The default (no type args) matches `ChatStream`: standard chunks plus the known framework `CUSTOM` event union. Free-form `emitCustomEvent` names still flow at runtime; cast to `StreamChunk` if you need to read them.

### Thinking Chunks

Adapters emit reasoning as both the canonical `REASONING_MESSAGE_*` events and the older `STEP_STARTED` / `STEP_FINISHED` events. Rather than parsing those raw events yourself, read the reconciled `ThinkingPart` from `message.parts` — the stream processor merges both event families into a single part for you:
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/type-aliases/StreamChunk.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@ Defined in: [packages/ai/src/types.ts:1635](https://github.com/TanStack/ai/blob/

Chunk returned by the SDK during streaming chat completions.
Uses the AG-UI protocol event format.

For the tool-aware variant that narrows `TOOL_CALL_START`/`TOOL_CALL_END` events by tool name and `CUSTOM` events by tagged literal name, see [`TypedStreamChunk`](./TypedStreamChunk).
32 changes: 32 additions & 0 deletions docs/reference/type-aliases/TaggedCustomEvent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
id: TaggedCustomEvent
title: TaggedCustomEvent
---

# Type Alias: TaggedCustomEvent\<T\>

```ts
type TaggedCustomEvent<T = unknown> =
| StructuredOutputStartEvent
| StructuredOutputCompleteEvent<T>
| ApprovalRequestedEvent
| ToolInputAvailableEvent;
```

Defined in: [packages/typescript/ai/src/types.ts](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts)

Discriminated union of the orchestrator-tagged `CUSTOM` events. Each variant has a literal `name`, so a single narrow on `chunk.name` yields a typed `value` with no helper or cast:

```ts
if (chunk.type === 'CUSTOM' && chunk.name === 'approval-requested') {
chunk.value.toolCallId // typed as string
}
```

The `StructuredOutputCompleteEvent` value is parameterized by `T`, which the chat orchestrator narrows to the schema's inferred type after Standard Schema validation. Adapters always emit it with `T = unknown`.

`TaggedCustomEvent` is included in [`TypedStreamChunk`](./TypedStreamChunk)'s typed-tools branch so consumers iterating `chat()` streams get tagged narrowing alongside the per-tool `TOOL_CALL_START`/`TOOL_CALL_END` typing.

## Caveat: user-emitted custom events

Tools can emit arbitrary user-defined custom events via the `emitCustomEvent(name, value)` context API. Those flow through the stream at runtime but are intentionally absent from this union — including a bare `CustomEvent` (whose `value: any` would poison the union) would collapse `chunk.value` back to `any` after the narrow. If you rely on `emitCustomEvent`, branch on `CUSTOM` outside the literal-`name` narrows or cast the chunk to [`StreamChunk`](./StreamChunk) to recover the wider shape.
67 changes: 67 additions & 0 deletions docs/reference/type-aliases/TypedStreamChunk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
id: TypedStreamChunk
title: TypedStreamChunk
---

# Type Alias: TypedStreamChunk\<TTools\>

```ts
type TypedStreamChunk<
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<Tool<any, any, any>>,
> =
HasTypedTools<TTools> extends true
?
| Exclude<
StreamChunk,
| { type: 'TOOL_CALL_START' }
| { type: 'TOOL_CALL_END' }
| { type: 'CUSTOM' }
>
| DistributedToolCallStart<TTools>
| DistributedToolCallEnd<TTools>
| TaggedCustomEvent
: StreamChunk;
```

Defined in: [packages/typescript/ai/src/types.ts](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts)

A variant of [`StreamChunk`](./StreamChunk) parameterized by the tools array. When specific tool types are provided (e.g. from `chat({ tools: [myTool] })`):

- `TOOL_CALL_START` and `TOOL_CALL_END` events form a **discriminated union** over tool names.
- Checking `toolName === 'x'` narrows `input` to that specific tool's input type.
- `TOOL_CALL_END` events have `input` typed per-tool via Standard Schema inference.
- `CUSTOM` events with literal tagged names (`structured-output.start`, `structured-output.complete`, `approval-requested`, `tool-input-available`) narrow `value` to the corresponding payload via the [`TaggedCustomEvent`](./TaggedCustomEvent) union.

When tools are untyped or absent, `TypedStreamChunk` falls back to plain `StreamChunk` so existing consumers that pass streams as `AsyncIterable<StreamChunk>` keep working.

This is the type returned by `chat()` when streaming is enabled (the default). You don't typically need to reference it directly unless annotating function parameters or return types.

```ts
import { chat, toolDefinition, type TypedStreamChunk } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({ location: z.string() }),
});

const searchTool = toolDefinition({
name: "search",
description: "Search the web",
inputSchema: z.object({ query: z.string() }),
});

// Inferred from `chat()` — typed tool call events plus tagged CUSTOM events
const stream = chat({
adapter: openaiText("gpt-5.5"),
messages,
tools: [weatherTool, searchTool],
});

// Explicit annotation
type Chunk = TypedStreamChunk<[typeof weatherTool, typeof searchTool]>;
```

See [Streaming - Type-Safe Tool Call Events](../../chat/streaming) for a practical walkthrough.
2 changes: 2 additions & 0 deletions docs/tools/server-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,8 @@ const getUserData = getUserDataDef.server(async (args) => {

> **Note:** JSON Schema tools skip runtime validation. Zod schemas are recommended for full type safety and validation.

> **Tip:** When you pass typed tools (server, client, or definition) to `chat()`, the returned stream is fully typed — `toolName` narrows to your tool name literals and `input` narrows per-tool when you check the name. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Best Practices

1. **Keep tools focused** - Each tool should do one thing well
Expand Down
2 changes: 2 additions & 0 deletions docs/tools/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ const inputSchema: JSONSchema = {

> **Note:** When using JSON Schema, TypeScript infers `unknown` for input/output types (it cannot derive types from a JSON Schema at compile time), so you must narrow or cast `args` before use. Zod schemas are recommended for full type safety.
> **Tip:** Type safety from Zod schemas extends beyond tool execution — when you iterate over the stream returned by `chat()`, tool call events have typed `toolName` and `input` fields too. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).
## Tool Definition

Tools are defined using `toolDefinition()` from `@tanstack/ai`:
Expand Down
6 changes: 3 additions & 3 deletions packages/ai-bedrock/tests/converse/adapter.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -197,13 +197,13 @@ describe('BedrockConverseTextAdapter', () => {
events.push(c)
}
const complete = events.find(
(e): e is Extract<StreamChunk, { type: typeof EventType.CUSTOM }> =>
e.type === EventType.CUSTOM &&
(e): e is Extract<StreamChunk, { type: 'CUSTOM' }> =>
e.type === 'CUSTOM' &&
'name' in e &&
e.name === 'structured-output.complete',
)
expect(complete).toBeDefined()
expect((complete?.value as { object: unknown }).object).toEqual({ n: 5 })
expect(complete?.value.object).toEqual({ n: 5 })
// The forced-tool 'tool_use' stopReason is internal — a clean structured
// run reports 'stop', not 'tool_calls'.
const finished = events.find((e) => e.type === EventType.RUN_FINISHED)
Expand Down
Loading
Loading