Skip to content

Commit b9aa0ec

Browse files
tombeckenhamclaude
andcommitted
feat(ai-openrouter): typed per-model durations for video adapter
Wire openRouterVideo onto the shared typed-duration contract (the same one geminiVideo uses): add the sixth BaseVideoAdapter generic (OpenRouterVideoModelDurationByName), narrow `duration` per model from the published `/api/v1/videos/models` metadata, and override availableDurations() / snapDuration() (backed by snapToDurationOption). `duration` is now a compile-time per-model union; the runtime validateVideoDuration backstop stays for JS callers and unknown-meta models. - video-provider-options.ts: OpenRouterVideoModelDurationByName + getVideoDurationOptions (discrete from meta, none when unknown/empty) - adapter: 6th generic, createVideoJob narrowed to per-model size/duration, availableDurations()/snapDuration() overrides - export OpenRouterVideoModelDurationByName from index - tests: 4 introspection cases; existing negative size/duration tests now use @ts-expect-error (proves the union rejects them) while still asserting the runtime throw - docs/media/video-generation.md, docs/adapters/openrouter.md, media-generation SKILL.md: document snapDuration/availableDurations for OpenRouter; bump updatedAt; changeset Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent ba99634 commit b9aa0ec

8 files changed

Lines changed: 138 additions & 9 deletions

File tree

.changeset/openrouter-video-adapter.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@
22
'@tanstack/ai-openrouter': minor
33
---
44

5-
Add `openRouterVideo`, a video generation adapter for OpenRouter's dedicated async API (`POST /api/v1/videos`) — Seedance, Veo 3.1, Wan, Kling, and Sora 2 Pro through one API key. Follows the jobs/polling architecture (`generateVideo()``getVideoJobStatus()`), with per-model `size` / `duration` / provider-option types generated from OpenRouter's `GET /api/v1/videos/models` metadata and validated before submit. Image-conditioned prompts map `metadata.role` onto the wire: `start_frame` / `end_frame``frame_images[]` (`first_frame` / `last_frame`), `reference` / `character``input_references[]`; frame roles are validated against each model's `supported_frame_images`. Completed videos are downloaded server-side and returned as `data:` URLs (OpenRouter's download URLs require the API key), and the gateway-reported cost is surfaced as `usage.cost`.
5+
Add `openRouterVideo`, a video generation adapter for OpenRouter's dedicated async API (`POST /api/v1/videos`) — Seedance, Veo 3.1, Wan, Kling, and Sora 2 Pro through one API key. Follows the jobs/polling architecture (`generateVideo()` → `getVideoJobStatus()`), with per-model `size` / `duration` / provider-option types generated from OpenRouter's `GET /api/v1/videos/models` metadata and validated before submit. `duration` is typed per model on the shared typed-duration contract — the adapter implements `availableDurations()` and `snapDuration(seconds)` (matching the Veo adapter) to enumerate the valid set and coerce raw UI seconds to the closest supported value. Image-conditioned prompts map `metadata.role` onto the wire: `start_frame` / `end_frame` → `frame_images[]` (`first_frame` / `last_frame`), `reference` / `character` → `input_references[]`; frame roles are validated against each model's `supported_frame_images`. Completed videos are downloaded server-side and returned as `data:` URLs (OpenRouter's download URLs require the API key), and the gateway-reported cost is surfaced as `usage.cost`.
66

77
Image adapter fixes from the #624 review: requested `size` is now validated (the `WIDTHxHEIGHT` union previously used a Unicode `×`, so every size except `1024x1024` silently dropped its aspect ratio; unsupported sizes now throw with the supported list), `numberOfImages > 1` throws instead of silently returning one image (verified live: the gateway ignores all count keys in `image_config`), and `image_config.strength` (0.0–1.0 image-to-image influence) is exposed via `modelOptions.strength`.

docs/adapters/openrouter.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -334,6 +334,8 @@ const { jobId } = await generateVideo({
334334
},
335335
],
336336
size: "1280x720",
337+
// `duration` is typed per model from the published metadata; coerce raw
338+
// seconds with adapter.snapDuration() or enumerate via adapter.availableDurations().
337339
duration: 8,
338340
});
339341

docs/media/video-generation.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -807,6 +807,25 @@ const { jobId } = await generateVideo({
807807
})
808808
```
809809

810+
Like the Veo adapter, OpenRouter's `duration` is **typed per model** — each
811+
model narrows `duration` to the whole-second union published in its metadata,
812+
and the adapter implements the same `availableDurations()` / `snapDuration()`
813+
introspection helpers:
814+
815+
```typescript
816+
const adapter = openRouterVideo('bytedance/seedance-2.0')
817+
818+
adapter.availableDurations()
819+
// { kind: 'discrete', values: [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] }
820+
adapter.snapDuration(7.4) // 7 — closest valid duration
821+
822+
await generateVideo({
823+
adapter,
824+
prompt: 'A timelapse of clouds',
825+
duration: adapter.snapDuration(sliderSeconds), // coerce raw UI seconds
826+
})
827+
```
828+
810829
Two OpenRouter-specific behaviors to know about:
811830

812831
- **The completed video arrives as a `data:` URL.** OpenRouter's download

packages/ai-openrouter/src/adapters/video.ts

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,18 @@
11
import { OpenRouter } from '@openrouter/sdk'
22
import { buildBaseUsage, resolveMediaPrompt } from '@tanstack/ai'
3-
import { BaseVideoAdapter } from '@tanstack/ai/adapters'
3+
import { BaseVideoAdapter, snapToDurationOption } from '@tanstack/ai/adapters'
44
import { arrayBufferToBase64 } from '@tanstack/ai-utils'
55
import { getOpenRouterApiKeyFromEnv } from '../utils'
66
import {
7+
getVideoDurationOptions,
78
getVideoModelMeta,
89
validateVideoDuration,
910
validateVideoSize,
1011
} from '../video/video-provider-options'
12+
import type { DurationOptions } from '@tanstack/ai/adapters'
1113
import type {
1214
OpenRouterVideoModel,
15+
OpenRouterVideoModelDurationByName,
1316
OpenRouterVideoModelInputModalitiesByName,
1417
OpenRouterVideoModelProviderOptionsByName,
1518
OpenRouterVideoModelSizeByName,
@@ -235,7 +238,8 @@ export class OpenRouterVideoAdapter<
235238
OpenRouterVideoProviderOptions,
236239
OpenRouterVideoModelProviderOptionsByName,
237240
OpenRouterVideoModelSizeByName,
238-
OpenRouterVideoModelInputModalitiesByName
241+
OpenRouterVideoModelInputModalitiesByName,
242+
OpenRouterVideoModelDurationByName
239243
> {
240244
override readonly kind = 'video' as const
241245
readonly name = 'openrouter' as const
@@ -254,7 +258,11 @@ export class OpenRouterVideoAdapter<
254258
}
255259

256260
async createVideoJob(
257-
options: VideoGenerationOptions<OpenRouterVideoProviderOptions>,
261+
options: VideoGenerationOptions<
262+
OpenRouterVideoProviderOptions,
263+
OpenRouterVideoModelSizeByName[TModel],
264+
OpenRouterVideoModelDurationByName[TModel]
265+
>,
258266
): Promise<VideoJobResult> {
259267
const { size, duration, modelOptions, logger } = options
260268

@@ -319,6 +327,18 @@ export class OpenRouterVideoAdapter<
319327
}
320328
}
321329

330+
override availableDurations(): DurationOptions<
331+
OpenRouterVideoModelDurationByName[TModel]
332+
> {
333+
return getVideoDurationOptions(this.model)
334+
}
335+
336+
override snapDuration(
337+
seconds: number,
338+
): OpenRouterVideoModelDurationByName[TModel] | undefined {
339+
return snapToDurationOption(seconds, this.availableDurations())
340+
}
341+
322342
async getVideoStatus(jobId: string): Promise<VideoStatusResult> {
323343
const response = await this.client.videoGeneration.getGeneration({ jobId })
324344
return {

packages/ai-openrouter/src/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ export type {
6868
OpenRouterVideoModelProviderOptionsByName,
6969
OpenRouterVideoModelSizeByName,
7070
OpenRouterVideoModelInputModalitiesByName,
71+
OpenRouterVideoModelDurationByName,
7172
} from './video/video-provider-options'
7273

7374
// ============================================================================

packages/ai-openrouter/src/video/video-provider-options.ts

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import { OPENROUTER_VIDEO_MODEL_META } from '../model-meta'
2+
import type { DurationOptions } from '@tanstack/ai/adapters'
23
import type { VideoGenerationRequestProvider } from '@openrouter/sdk/models'
34
import type { OPENROUTER_VIDEO_MODELS } from '../model-meta'
45

@@ -123,6 +124,35 @@ export type OpenRouterVideoModelInputModalitiesByName = {
123124
[K in OpenRouterVideoModel]: readonly ['image']
124125
}
125126

127+
/**
128+
* Per-model duration unions (whole seconds, numeric — OpenRouter's
129+
* `/api/v1/videos` `duration` field is a number). Derived from the generated
130+
* model meta; models whose metadata reports `durations: null` (capabilities
131+
* unknown) stay permissive (`number`).
132+
*/
133+
export type OpenRouterVideoModelDurationByName = {
134+
[K in OpenRouterVideoModel]: ElementOf<VideoMeta[K]['durations'], number>
135+
}
136+
137+
/**
138+
* Duration options for a model, backing the adapter's `availableDurations()` /
139+
* `snapDuration()`. OpenRouter publishes durations as a discrete list of whole
140+
* seconds, so known models map to `{ kind: 'discrete' }`. Returns
141+
* `{ kind: 'none' }` when the model is unknown or its meta reports no
142+
* durations — mirroring the permissive runtime behavior of
143+
* {@link validateVideoDuration}.
144+
*/
145+
export function getVideoDurationOptions<TModel extends OpenRouterVideoModel>(
146+
model: TModel,
147+
): DurationOptions<OpenRouterVideoModelDurationByName[TModel]>
148+
export function getVideoDurationOptions(
149+
model: string,
150+
): DurationOptions<number> {
151+
const durations = VIDEO_MODEL_META[model]?.durations
152+
if (!durations || durations.length === 0) return { kind: 'none' }
153+
return { kind: 'discrete', values: durations }
154+
}
155+
126156
/**
127157
* Validate a requested size against the model's supported sizes. No-op when
128158
* the model (or its size list) is unknown — OpenRouter then validates

packages/ai-openrouter/tests/video-adapter.test.ts

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,8 @@ describe('OpenRouter Video Adapter', () => {
117117
adapter.createVideoJob({
118118
model: 'bytedance/seedance-2.0',
119119
prompt: 'A timelapse of clouds',
120+
// @ts-expect-error invalid size — the per-model union rejects it at
121+
// compile time; this still exercises the runtime guard for JS callers.
120122
size: '333x333',
121123
logger: testLogger,
122124
}),
@@ -132,6 +134,8 @@ describe('OpenRouter Video Adapter', () => {
132134
adapter.createVideoJob({
133135
model: 'bytedance/seedance-2.0',
134136
prompt: 'A timelapse of clouds',
137+
// @ts-expect-error invalid duration — the per-model union rejects it
138+
// at compile time; this still exercises the runtime guard.
135139
duration: 99,
136140
logger: testLogger,
137141
}),
@@ -485,4 +489,33 @@ describe('OpenRouter Video Adapter', () => {
485489
expect(mockFetch).not.toHaveBeenCalled()
486490
})
487491
})
492+
493+
describe('typed durations', () => {
494+
it('availableDurations() reports the model discrete duration list', () => {
495+
const adapter = createAdapter() // bytedance/seedance-2.0 → [4..15]
496+
expect(adapter.availableDurations()).toEqual({
497+
kind: 'discrete',
498+
values: [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
499+
})
500+
})
501+
502+
it('snapDuration() returns an exact match unchanged', () => {
503+
const adapter = createAdapter()
504+
expect(adapter.snapDuration(8)).toBe(8)
505+
})
506+
507+
it('snapDuration() snaps to the closest supported value', () => {
508+
const adapter = createAdapter()
509+
// 7.4 → 7 (rounds to nearest), 99 → 15 (clamped to max), 1 → 4 (min)
510+
expect(adapter.snapDuration(7.4)).toBe(7)
511+
expect(adapter.snapDuration(99)).toBe(15)
512+
expect(adapter.snapDuration(1)).toBe(4)
513+
})
514+
515+
it('snapDuration() picks the nearest of a sparse list', () => {
516+
const adapter = createOpenRouterVideo('alibaba/wan-2.6', 'test-key') // [5, 10]
517+
expect(adapter.snapDuration(6)).toBe(5)
518+
expect(adapter.snapDuration(8)).toBe(10)
519+
})
520+
})
488521
})

packages/ai/skills/ai-core/media-generation/SKILL.md

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,10 @@ description: >
44
Image, audio, video, speech (TTS), and transcription generation using
55
activity-specific adapters: generateImage() with openaiImage/geminiImage/byteplusImage,
66
generateAudio() with geminiAudio/falAudio, generateVideo() with async
7-
polling (openaiVideo/geminiVideo/grokVideo/falVideo/byteplusVideo, per-model typed
8-
durations), generateSpeech() with openaiSpeech/byteplusSpeech, generateTranscription()
9-
with openaiTranscription/byteplusTranscription. React hooks: useGenerateImage, useGenerateAudio,
7+
polling (openaiVideo/geminiVideo/grokVideo/falVideo/byteplusVideo/openRouterVideo,
8+
per-model typed durations), generateSpeech() with openaiSpeech/byteplusSpeech,
9+
generateTranscription() with openaiTranscription/byteplusTranscription. React hooks:
10+
useGenerateImage, useGenerateAudio,
1011
useGenerateSpeech, useTranscription, useGenerateVideo.
1112
TanStack Start server function integration with toServerSentEventsResponse.
1213
type: sub-skill
@@ -547,8 +548,9 @@ image-to-video only — needs an `image` prompt part as the starting frame, text
547548
aspect-ratio size template like `'16:9_720p'`, integer durations 1-15s, reports
548549
`usage.unitsBilled` seconds and exact `usage.cost`), `byteplusVideo(...)` (Seedance —
549550
aspect-ratio size template like `'16:9_720p'`, durations 4-15s on the 2.0 family,
550-
4-12s on 1.5-pro, 2-12s on the 1.0-pro models; reads `ARK_API_KEY`), and
551-
`falVideo(...)` (hosted models, see cost tracking below).
551+
4-12s on 1.5-pro, 2-12s on the 1.0-pro models; reads `ARK_API_KEY`),
552+
`openRouterVideo(...)` (OpenRouter's dedicated `POST /api/v1/videos` gateway),
553+
and `falVideo(...)` (hosted models, see cost tracking below).
552554

553555
> **Seedance option applicability is per model and enforced server-side**
554556
> Ark returns a 400 for an inapplicable field rather than ignoring it.
@@ -560,6 +562,28 @@ aspect-ratio size template like `'16:9_720p'`, durations 4-15s on the 2.0 family
560562
> days). Seedance is also reachable via `falVideo``byteplusVideo` is the
561563
> direct-to-BytePlus path.
562564
565+
OpenRouter (`@tanstack/ai-openrouter`, `openRouterVideo`) runs the dedicated
566+
async video API (`POST /api/v1/videos`) and shares the same typed-duration
567+
contract — `duration`, `size`, and provider options are narrowed per model
568+
from OpenRouter's published metadata, with the same `availableDurations()` /
569+
`snapDuration()` helpers:
570+
571+
```typescript
572+
import { openRouterVideo } from '@tanstack/ai-openrouter'
573+
574+
const adapter = openRouterVideo('bytedance/seedance-2.0')
575+
adapter.availableDurations()
576+
// { kind: 'discrete', values: [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] }
577+
adapter.snapDuration(7.4) // 7
578+
579+
const { jobId } = await generateVideo({
580+
adapter,
581+
prompt: 'A timelapse of clouds',
582+
duration: adapter.snapDuration(sliderSeconds),
583+
})
584+
// Completed url is a data: URL; usage.cost carries the real billed cost.
585+
```
586+
563587
Client hook with job tracking:
564588

565589
```tsx

0 commit comments

Comments
 (0)