Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/openrouter-combined-tools-and-schema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@tanstack/ai-openrouter': minor
---

Add native combined tools + `outputSchema` mode to both OpenRouter text adapters (chat-completions and Responses). When the resolved upstream model supports emitting a schema-constrained final answer alongside tool calls in a single pass, `chat({ outputSchema, tools, stream: true })` now wires the JSON Schema into the same streaming request as the tools and harvests the final-turn JSON, skipping the separate finalization round-trip.
Comment thread
AlemTuzlak marked this conversation as resolved.

Because OpenRouter is a routing layer, capability is keyed per resolved upstream model via the new `OPENROUTER_COMBINED_TOOLS_AND_SCHEMA_MODELS` set, exported from `@tanstack/ai-openrouter/model-meta`, which both adapters consult from `supportsCombinedToolsAndSchema()`. The set is derived from each upstream provider's combined-mode gate (Anthropic 4.5+, Gemini 3.x, OpenAI's strict `json_schema` era, Grok 4.x) rather than the broader catalog `responseFormat` flag, so models that advertise structured output but predate native combined mode stay on the legacy finalization path.
129 changes: 129 additions & 0 deletions docs/adapters/openrouter.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,135 @@ export async function POST(request: Request) {
}
```

## Tools and structured output together

You can pass both `tools` and `outputSchema` on one `chat()` call. For some
upstream models OpenRouter can return the typed object in that same streaming
request, so the engine does not make a second finalization call.

That happens only when **every** model that can receive the request is in
`OPENROUTER_COMBINED_TOOLS_AND_SCHEMA_MODELS`. The set is the upstream models
that already support this combined mode:

- Anthropic Claude 4.5 and later
- Gemini 3.x text models
- Grok 4.x (not the multi-agent variant)
- OpenAI models from the `gpt-4o-2024-08-06` strict JSON Schema era onward
(the unpinned `openai/gpt-4o-mini` alias is included; the dated pin
`openai/gpt-4o-mini-2024-07-18` is not)

If any fallback in `modelOptions.models` is outside that set, OpenRouter keeps
the two-call path. Routing suffixes such as `:nitro` do not change the gate.

Import the set from `@tanstack/ai-openrouter/model-meta` if you need to check a
model before you send:

```typescript
import { OPENROUTER_COMBINED_TOOLS_AND_SCHEMA_MODELS } from "@tanstack/ai-openrouter/model-meta";

OPENROUTER_COMBINED_TOOLS_AND_SCHEMA_MODELS.has("openai/gpt-5.5");
```

Chat Completions (`openRouterText`) and Responses (`openRouterResponsesText`)
both attach the schema on this path. The client does not change: `useChat({
outputSchema })` still reads `partial` and `final`.

Server (Chat Completions):

```typescript
import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai";
import { openRouterText } from "@tanstack/ai-openrouter";
import { z } from "zod";

const getWeather = toolDefinition({
name: "get_weather",
description: "Get the current weather",
inputSchema: z.object({ location: z.string() }),
}).server(async ({ location }) => {
return { temperature: 72, conditions: "sunny", location };
});

const AnswerSchema = z.object({
summary: z.string(),
location: z.string(),
});

export async function POST(request: Request) {
const { messages } = await request.json();

const stream = chat({
adapter: openRouterText("openai/gpt-5.5"),
messages,
tools: [getWeather],
outputSchema: AnswerSchema,
stream: true,
});

return toServerSentEventsResponse(stream);
}
```

Server (Responses). Same `tools` and `outputSchema` as the Chat Completions
example, with `openRouterResponsesText`:

```typescript
import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai";
import { openRouterResponsesText } from "@tanstack/ai-openrouter";
import { z } from "zod";

const getWeather = toolDefinition({
name: "get_weather",
description: "Get the current weather",
inputSchema: z.object({ location: z.string() }),
}).server(async ({ location }) => {
return { temperature: 72, conditions: "sunny", location };
});

const AnswerSchema = z.object({
summary: z.string(),
location: z.string(),
});

export async function POST(request: Request) {
const { messages } = await request.json();

const stream = chat({
adapter: openRouterResponsesText("openai/gpt-5.5"),
messages,
tools: [getWeather],
outputSchema: AnswerSchema,
stream: true,
});

return toServerSentEventsResponse(stream);
}
```

Client:

```tsx
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
import { z } from "zod";

const AnswerSchema = z.object({
summary: z.string(),
location: z.string(),
});

const { sendMessage, partial, final } = useChat({
connection: fetchServerSentEvents("/api/chat"),
outputSchema: AnswerSchema,
});
```

See [Structured Outputs with tools](../structured-outputs/with-tools) for the
event order, and [Middleware](../advanced/middleware) for how
`structuredOutput` phase behaves on this path.

To try this in a browser, run `examples/ts-react-chat` and open
`/generations/openrouter-combined`. The page shows the tool call, the typed
object, and the adapter call counts. `structuredOutputStream` must stay at 0.

## Environment Variables

Set your API key in environment variables:
Expand Down
2 changes: 1 addition & 1 deletion docs/advanced/middleware.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ The context's `phase` field tracks where you are in the lifecycle:
| `modelStream` | While adapter streams chunks | `onChunk`, `onUsage` |
| `beforeTools` | Before tool execution | `onBeforeToolCall` |
| `afterTools` | After tool execution | `onAfterToolCall` |
| `structuredOutput` | During the final structured-output adapter call (when `outputSchema` is set **and** the adapter does not declare `supportsCombinedToolsAndSchema()`). Chunks from `adapter.structuredOutputStream` (or the synthesized non-streaming fallback) flow through `onChunk` with this phase, and `onUsage` fires for the final call's tokens. **Does not fire** for adapters that natively combine tools + schema in one streaming call (modern OpenAI Chat Completions, OpenAI Responses, Claude 4.5+, Gemini 3.x, Grok 4.x familysee issue #605); on that path middleware observes the run through `beforeModel` / `modelStream` as usual. | `onStructuredOutputConfig`, `onConfig`, `onChunk`, `onUsage` |
| `structuredOutput` | During the final structured-output adapter call (when `outputSchema` is set **and** the adapter does not declare `supportsCombinedToolsAndSchema()`). Chunks from `adapter.structuredOutputStream` (or the synthesized non-streaming fallback) flow through `onChunk` with this phase, and `onUsage` fires for the final call's tokens. **Does not fire** for adapters that natively combine tools + schema in one streaming call (modern OpenAI Chat Completions, OpenAI Responses, Claude 4.5+, Gemini 3.x, Grok 4.x family, and OpenRouter when every routed model is in `OPENROUTER_COMBINED_TOOLS_AND_SCHEMA_MODELS`; see issue #605). On that path middleware observes the run through `beforeModel` / `modelStream` as usual. | `onStructuredOutputConfig`, `onConfig`, `onChunk`, `onUsage` |

## Hooks Reference

Expand Down
8 changes: 4 additions & 4 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@
"label": "Overview",
"to": "structured-outputs/overview",
"addedAt": "2026-05-19",
"updatedAt": "2026-08-18"
"updatedAt": "2026-08-19"
},
{
"label": "One-Shot Extraction",
Expand All @@ -360,7 +360,7 @@
"label": "With Tools",
"to": "structured-outputs/with-tools",
"addedAt": "2026-05-19",
"updatedAt": "2026-08-14"
"updatedAt": "2026-08-19"
},
{
"label": "Harness Agents",
Expand Down Expand Up @@ -490,7 +490,7 @@
"label": "Middleware",
"to": "advanced/middleware",
"addedAt": "2026-04-15",
"updatedAt": "2026-07-21"
"updatedAt": "2026-08-19"
},
{
"label": "Built-in Middleware",
Expand Down Expand Up @@ -857,7 +857,7 @@
"label": "OpenRouter Adapter",
"to": "adapters/openrouter",
"addedAt": "2026-04-15",
"updatedAt": "2026-08-13"
"updatedAt": "2026-08-19"
},
{
"label": "Perplexity Search",
Expand Down
8 changes: 6 additions & 2 deletions docs/structured-outputs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,14 @@ exactly once at the end of the entire run.
> - Anthropic Claude 4.5+
> - Gemini 3.x
> - Grok 4.x family
> - OpenRouter, when every routed model is in
> `OPENROUTER_COMBINED_TOOLS_AND_SCHEMA_MODELS` (see the
> [OpenRouter adapter](../adapters/openrouter.md#tools-and-structured-output-together))
>
> **Adapters without native combined-mode support** (Anthropic 4.4-, Gemini
> 2.x, Grok 2/3, Groq, Ollama, OpenRouter) keep the legacy finalization
> path and the `'structuredOutput'` phase fires as before.
> 2.x, Grok 2/3, Groq, Ollama, and OpenRouter models outside that set) keep
> the legacy finalization path and the `'structuredOutput'` phase fires as
> before.

### Observing structured-output chunks

Expand Down
4 changes: 4 additions & 0 deletions docs/structured-outputs/with-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ You want the agent to use tools to gather information, then return a structured

This page covers the combined `outputSchema` + `tools` shape, including the pause/resume points (server-tool approval prompts, client-tool invocations) that can land mid-run before the structured object arrives.

On adapters that support native combined mode (modern OpenAI, Claude 4.5+, Gemini 3.x, Grok 4.x, and [OpenRouter on those same upstream models](../adapters/openrouter.md#tools-and-structured-output-together)), `chat({ tools, outputSchema, stream: true })` is one streaming request. The extra finalization call does not run.

The React chat example has a live OpenRouter page at `/generations/openrouter-combined`.

> **Note:** If you're not yet familiar with how tools work in TanStack AI, read [Tool Architecture](../tools/tool-architecture) and [Server Tools](../tools/server-tools) first. The patterns here build on the regular agent-loop flow — `outputSchema` just adds a final terminal event.

## Non-streaming: tools first, then structured object
Expand Down
13 changes: 13 additions & 0 deletions examples/ts-react-chat/src/components/Header.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,19 @@ export default function Header() {
<span className="font-medium">Structured Output</span>
</Link>

<Link
to="/generations/openrouter-combined"
onClick={() => setIsOpen(false)}
className="flex items-center gap-3 p-3 rounded-lg hover:bg-gray-800 transition-colors mb-1"
activeProps={{
className:
'flex items-center gap-3 p-3 rounded-lg bg-cyan-600 hover:bg-cyan-700 transition-colors mb-1',
}}
>
<Layers size={20} />
<span className="font-medium">OpenRouter Combined</span>
</Link>

<Link
to="/repo-report"
onClick={() => setIsOpen(false)}
Expand Down
Loading
Loading