Skip to content

Commit 27eaac5

Browse files
tombeckenhamclaude
andcommitted
feat(ai-gemini): add Google Veo video adapter on the typed-duration contract (#634)
Builds the Veo adapter directly on the per-model typed-duration VideoAdapter contract from #534: - Uncomment + finalize the five Veo model entries in model-meta.ts and add a GEMINI_VIDEO_MODELS union. - Add video/video-provider-options.ts with GeminiVideoModelDurationByName, GeminiVideoModelSizeByName, and a docs-curated GEMINI_VIDEO_DURATIONS map (the ai-schemas Gemini spec types :predictLongRunning parameters as unknown, so it carries no per-model duration data). - Implement GeminiVideoAdapter over @google/genai generateVideos / getVideosOperation with availableDurations/snapDuration overrides, RAI filter surfacing, and geminiVideo/createGeminiVideo factories. - Wire gemini into the video-gen E2E feature matrix with a Veo predictLongRunning + operations mount in global-setup. - Update video-generation docs and the media-generation skill. - Fix pre-existing no-unnecessary-type-assertion lint errors in the base branch's snap.ts / fal video.ts so the stacked PR gate is green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent a4ed1b6 commit 27eaac5

13 files changed

Lines changed: 1016 additions & 46 deletions

File tree

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
'@tanstack/ai-gemini': minor
3+
---
4+
5+
Add a Google Veo video adapter (`geminiVideo` / `createGeminiVideo`), built
6+
on the per-model typed-duration contract (#534).
7+
8+
Supported models: `veo-3.1-generate-preview`, `veo-3.1-fast-generate-preview`,
9+
`veo-3.0-generate-001`, `veo-3.0-fast-generate-001`, `veo-2.0-generate-001`.
10+
11+
The adapter drives Google's long-running `:predictLongRunning` operation:
12+
`createVideoJob` starts the operation, `getVideoStatus` polls it (mapping
13+
`done`/`error`/Responsible-AI filtering), and `getVideoUrl` extracts the
14+
generated video URI.
15+
16+
Durations are typed per model at compile time and introspectable at runtime:
17+
18+
- `geminiVideo('veo-3.0-generate-001')``duration?: 4 | 6 | 8`
19+
- `geminiVideo('veo-2.0-generate-001')``duration?: 5 | 6 | 8`
20+
- `adapter.availableDurations()``{ kind: 'discrete', values: [4, 6, 8] }`
21+
- `adapter.snapDuration(7)``6`
22+
23+
`size` takes Veo aspect ratios (`'16:9' | '9:16'`); everything else from the
24+
SDK's `GenerateVideosConfig` (e.g. `resolution`, `generateAudio`,
25+
`negativePrompt`) is available through `modelOptions`.
26+
27+
Note: Veo result URLs are served by the Gemini Files API and require the
28+
Google API key to download (`x-goog-api-key` header or `key` query
29+
parameter).

docs/media/video-generation.md

Lines changed: 65 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,13 @@
22
title: Video Generation
33
id: video-generation
44
order: 6
5-
description: "Generate video from text prompts with OpenAI Sora using TanStack AI's experimental generateVideo() jobs/polling API."
5+
description: "Generate video from text prompts with OpenAI Sora or Google Veo using TanStack AI's experimental generateVideo() jobs/polling API."
66
keywords:
77
- tanstack ai
88
- video generation
99
- sora
10+
- veo
11+
- gemini
1012
- generateVideo
1113
- jobs api
1214
- experimental
@@ -36,6 +38,7 @@ TanStack AI provides experimental support for video generation through dedicated
3638

3739
Currently supported:
3840
- **OpenAI**: Sora-2 and Sora-2-Pro models (when available)
41+
- **Google Gemini**: Veo 3.1, Veo 3, and Veo 2 models (via the long-running operations API)
3942

4043
## Basic Usage
4144

@@ -412,6 +415,63 @@ const { jobId } = await generateVideo({
412415
})
413416
```
414417

418+
### Google Veo (Gemini) Model Options
419+
420+
Veo runs on Google's long-running operations API. The adapter starts the
421+
operation, and `getVideoJobStatus` polls it until the video is ready:
422+
423+
```typescript
424+
import { generateVideo, getVideoJobStatus } from '@tanstack/ai'
425+
import { geminiVideo } from '@tanstack/ai-gemini'
426+
427+
const adapter = geminiVideo('veo-3.1-generate-preview')
428+
429+
const { jobId } = await generateVideo({
430+
adapter,
431+
prompt: 'A close-up of a luthier carving a guitar neck',
432+
size: '16:9', // aspect ratio: '16:9' or '9:16'
433+
duration: 8, // typed per model — see below
434+
modelOptions: {
435+
resolution: '1080p', // '720p' (default), '1080p', '4k' (Veo 3.1 only)
436+
negativePrompt: 'cartoon, low quality',
437+
generateAudio: true, // Veo 3+ generates synchronized audio
438+
},
439+
})
440+
```
441+
442+
#### Typed durations
443+
444+
Each Veo model accepts a fixed set of durations, enforced at compile time on
445+
the `duration` option:
446+
447+
| Model | `duration` values (seconds) |
448+
|-------|------------------------------|
449+
| `veo-3.1-generate-preview` | `4`, `6`, `8` |
450+
| `veo-3.1-fast-generate-preview` | `4`, `6`, `8` |
451+
| `veo-3.0-generate-001` | `4`, `6`, `8` |
452+
| `veo-3.0-fast-generate-001` | `4`, `6`, `8` |
453+
| `veo-2.0-generate-001` | `5`, `6`, `8` |
454+
455+
If you have raw seconds (for example from a UI slider), coerce them with
456+
`snapDuration`, or inspect the full set with `availableDurations`:
457+
458+
```typescript
459+
const adapter = geminiVideo('veo-3.0-generate-001')
460+
461+
adapter.availableDurations() // { kind: 'discrete', values: [4, 6, 8] }
462+
adapter.snapDuration(7) // 6 — closest valid duration
463+
464+
await generateVideo({
465+
adapter,
466+
prompt: 'A timelapse of a city skyline at dusk',
467+
duration: adapter.snapDuration(7),
468+
})
469+
```
470+
471+
> **Note:** The video URL returned for Veo jobs is served by the Gemini
472+
> Files API and requires your API key to download (send it as an
473+
> `x-goog-api-key` header or `key` query parameter).
474+
415475
## Response Types
416476

417477
### VideoJobResult (from create)
@@ -497,9 +557,11 @@ Check the [OpenAI documentation](https://platform.openai.com/docs) for current l
497557

498558
## Environment Variables
499559

500-
The video adapter uses the same environment variable as other OpenAI adapters:
560+
The video adapters use the same environment variables as the other adapters
561+
for their provider:
501562

502-
- `OPENAI_API_KEY`: Your OpenAI API key
563+
- `OPENAI_API_KEY`: Your OpenAI API key (Sora)
564+
- `GOOGLE_API_KEY` or `GEMINI_API_KEY`: Your Google API key (Veo)
503565

504566
## Explicit API Keys
505567

packages/typescript/ai-fal/src/adapters/video.ts

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -139,9 +139,7 @@ export class FalVideoAdapter<TModel extends FalModel> extends BaseVideoAdapter<
139139
override snapDuration(
140140
seconds: number,
141141
): FalModelVideoDuration<TModel> | undefined {
142-
return snapToDurationOption(seconds, this.availableDurations()) as
143-
| FalModelVideoDuration<TModel>
144-
| undefined
142+
return snapToDurationOption(seconds, this.availableDurations())
145143
}
146144

147145
async getVideoStatus(jobId: string): Promise<VideoStatusResult> {

0 commit comments

Comments
 (0)