Skip to content

Commit b0dcc1d

Browse files
committed
test(e2e): cover Gemini native image modelOptions on the wire
aimock's handleGemini has no image-response branch and its journal stores a lossy OpenAI-shaped translation that drops safetySettings and generationConfig, so neither fixtures nor /journal can see this path. Mount the endpoint directly instead -- the same escape hatch geminiTTSMount() already uses for the identical {model}:generateContent inlineData shape -- and reject the request if the fields are absent, so a dropped option fails the spec instead of passing silently. Verified revert-proof: with packages/ai-gemini/src reverted to the parent commit, the spec fails with "Missing top-level safetySettings".
1 parent ddc4bda commit b0dcc1d

4 files changed

Lines changed: 320 additions & 0 deletions

File tree

testing/e2e/global-setup.ts

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,13 @@ export default async function globalSetup() {
5555
'/v1beta/models/gemini-3.1-flash-tts-preview:generateContent',
5656
geminiTTSMount(),
5757
)
58+
// Gemini native image generation hits the same generateContent endpoint
59+
// shape, one model id over — see geminiNativeImageMount for why it needs
60+
// a hand-mocked response and a raw-body wire-shape check of its own.
61+
mock.mount(
62+
'/v1beta/models/gemini-2.5-flash-image:generateContent',
63+
geminiNativeImageMount(),
64+
)
5865
// Gemini Veo video generation. aimock 1.29 mocks Gemini's `:predict`
5966
// (Imagen) endpoint but not the long-running `:predictLongRunning` +
6067
// operations-polling pair Veo uses, so mount both here. Non-Veo paths
@@ -265,6 +272,183 @@ function geminiTTSMount(): Mountable {
265272
}
266273
}
267274

275+
/**
276+
* Gemini native image generation hits the standard Gemini generateContent
277+
* endpoint too (POST /v1beta/models/{model}:generateContent) — the same
278+
* shape geminiTTSMount above targets, just with `image/png` inlineData
279+
* instead of PCM audio. aimock's native handleGemini recognizes only
280+
* text / tool-call / text-with-tool-call / audio fixture response shapes:
281+
* isImageResponse (helpers.js) is defined but never imported into
282+
* gemini.js, so there is no image-response branch at all. A fixture shaped
283+
* `{image}`/`{images}` matched against this endpoint falls through every
284+
* isXResponse() check and hits the final fallback — a 500 "Fixture response
285+
* did not match any known type." Native image generation needs a
286+
* hand-mocked response for the same reason TTS does.
287+
*
288+
* There's a second, PR-specific reason this needs its own mount rather than
289+
* a fixture even if one could match: aimock's request-journaling for this
290+
* endpoint goes through geminiToCompletionRequest, which reshapes the raw
291+
* Gemini request into an OpenAI-chat-shaped completionReq carrying only
292+
* {model, messages, stream, temperature, max_tokens, top_p, top_k, tools} —
293+
* it silently drops safetySettings, generationConfig.thinkingConfig, and
294+
* generationConfig.imageConfig before anything is journaled. So GET /journal
295+
* cannot show whether those fields reached the wire, even in principle.
296+
* This mount reads the raw, untranslated body instead (see
297+
* readJsonRequestBody, used the same way by the BytePlus mounts below) and
298+
* validates it directly — the BytePlus mounts' house pattern of turning a
299+
* dropped field into a failing spec rather than a silently green one.
300+
*
301+
* Mounted at the exact model+endpoint path (not the shared '/v1beta/models'
302+
* prefix geminiVeoMount/geminiBatchEmbedMount use below) so it can never
303+
* intercept an unrelated Gemini chat/text generateContent call for a
304+
* different model.
305+
*
306+
* `/api/gemini-native-image-wire` (see that route) drives this with
307+
* `modelOptions: { safetySettings, thinkingConfig }`. Reverting the
308+
* ai-gemini fix that stops dropping modelOptions on the native image path
309+
* removes both from the outgoing `nativeConfig`, so the request this mount
310+
* receives is missing them, this mount answers 400, and the route surfaces
311+
* that as `ok: false` — the companion spec's revert-detection mechanism.
312+
*/
313+
function geminiNativeImageMount(): Mountable {
314+
// 1x1 transparent PNG — just enough for transformGeminiResponse's
315+
// inlineData branch to produce a GeneratedImage. Mirrors FAKE_PCM_BYTES /
316+
// FAKE_MP3_BYTES above: content fidelity isn't under test here.
317+
const PNG_1X1_BASE64 =
318+
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII='
319+
320+
// Field names that only belong on Imagen's GenerateImagesConfig, never on
321+
// generateContent's GenerateContentConfig. The adapter's `nativeConfig`
322+
// picks fields by name specifically so modelOptions carrying both shapes
323+
// (or a future regression back to a wholesale `...modelOptions` spread)
324+
// can't let one through — mirrors the field list documented on
325+
// GeminiImageProviderOptions in image-provider-options.ts, minus `seed`
326+
// (legitimately shared by both configs and always forwarded) and `labels`
327+
// (also a real GenerateContentConfig field name, but the Gemini Developer
328+
// API rejects it outright rather than letting it reach the wire, so it
329+
// can't appear in a body this mount ever sees).
330+
const IMAGEN_ONLY_FIELDS = [
331+
'personGeneration',
332+
'safetyFilterLevel',
333+
'addWatermark',
334+
'language',
335+
'negativePrompt',
336+
'outputMimeType',
337+
'outputCompressionQuality',
338+
'guidanceScale',
339+
'enhancePrompt',
340+
'includeSafetyAttributes',
341+
'includeRaiReason',
342+
'outputGcsUri',
343+
// Imagen's own top-level `aspectRatio` (GenerateImagesConfig) is distinct
344+
// from the native path's nested generationConfig.imageConfig.aspectRatio
345+
// — its presence at either level here would mean the two configs got
346+
// crossed.
347+
'aspectRatio',
348+
]
349+
350+
return {
351+
async handleRequest(
352+
req: http.IncomingMessage,
353+
res: http.ServerResponse,
354+
// Exact-path mount — pathname is "/" for the one path this is
355+
// registered on.
356+
pathname: string,
357+
): Promise<boolean> {
358+
if (pathname !== '/' || req.method !== 'POST') return false
359+
360+
const body = await readJsonRequestBody(req)
361+
if (!body) {
362+
return rejectGeminiImageRequest(res, 'Malformed JSON body.')
363+
}
364+
const generationConfig = asRecord(body.generationConfig)
365+
366+
const leaked = IMAGEN_ONLY_FIELDS.find(
367+
(name) =>
368+
name in body || (generationConfig && name in generationConfig),
369+
)
370+
if (leaked) {
371+
return rejectGeminiImageRequest(
372+
res,
373+
`Imagen-only field "${leaked}" reached generateContent — GenerateImagesConfig and GenerateContentConfig got crossed.`,
374+
)
375+
}
376+
377+
if (
378+
!Array.isArray(body.safetySettings) ||
379+
body.safetySettings.length === 0
380+
) {
381+
return rejectGeminiImageRequest(
382+
res,
383+
'Missing top-level safetySettings (modelOptions.safetySettings did not reach the wire).',
384+
)
385+
}
386+
if (
387+
!generationConfig ||
388+
typeof generationConfig.thinkingConfig !== 'object' ||
389+
generationConfig.thinkingConfig === null
390+
) {
391+
return rejectGeminiImageRequest(
392+
res,
393+
'Missing generationConfig.thinkingConfig (modelOptions.thinkingConfig did not reach the wire).',
394+
)
395+
}
396+
397+
res.statusCode = 200
398+
res.setHeader('Content-Type', 'application/json')
399+
res.end(
400+
JSON.stringify({
401+
candidates: [
402+
{
403+
content: {
404+
role: 'model',
405+
parts: [
406+
{
407+
inlineData: {
408+
mimeType: 'image/png',
409+
data: PNG_1X1_BASE64,
410+
},
411+
},
412+
],
413+
},
414+
finishReason: 'STOP',
415+
index: 0,
416+
},
417+
],
418+
usageMetadata: {
419+
promptTokenCount: 8,
420+
candidatesTokenCount: 1290,
421+
totalTokenCount: 1298,
422+
},
423+
}),
424+
)
425+
return true
426+
},
427+
}
428+
}
429+
430+
/**
431+
* Rejects with Gemini's real MLDev error envelope shape
432+
* (`{ error: { code, message, status } }`) — the same fallback shape
433+
* @google/genai's own `throwErrorIfNotOK` builds for a non-JSON error body,
434+
* so the thrown ApiError's `message` carries the actual validation failure
435+
* (JSON.stringify'd) for debugging, the same way rejectArkRequest /
436+
* rejectVoiceRequest do for BytePlus below.
437+
*/
438+
function rejectGeminiImageRequest(
439+
res: http.ServerResponse,
440+
message: string,
441+
): true {
442+
res.statusCode = 400
443+
res.setHeader('Content-Type', 'application/json')
444+
res.end(
445+
JSON.stringify({
446+
error: { code: 400, message, status: 'INVALID_ARGUMENT' },
447+
}),
448+
)
449+
return true
450+
}
451+
268452
function grokSTTMount(): Mountable {
269453
return {
270454
async handleRequest(

testing/e2e/src/routeTree.gen.ts

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ import { Route as ApiInterruptsTestRouteImport } from './routes/api.interrupts-t
5959
import { Route as ApiImageRouteImport } from './routes/api.image'
6060
import { Route as ApiGenerationPersistenceServerRouteImport } from './routes/api.generation-persistence-server'
6161
import { Route as ApiGenerationPersistenceResumeRouteImport } from './routes/api.generation-persistence-resume'
62+
import { Route as ApiGeminiNativeImageWireRouteImport } from './routes/api.gemini-native-image-wire'
6263
import { Route as ApiForeignInterruptRouteImport } from './routes/api.foreign-interrupt'
6364
import { Route as ApiEmbeddingRouteImport } from './routes/api.embedding'
6465
import { Route as ApiDurableTakeoverRouteImport } from './routes/api.durable-takeover'
@@ -336,6 +337,12 @@ const ApiGenerationPersistenceResumeRoute =
336337
path: '/api/generation-persistence-resume',
337338
getParentRoute: () => rootRouteImport,
338339
} as any)
340+
const ApiGeminiNativeImageWireRoute =
341+
ApiGeminiNativeImageWireRouteImport.update({
342+
id: '/api/gemini-native-image-wire',
343+
path: '/api/gemini-native-image-wire',
344+
getParentRoute: () => rootRouteImport,
345+
} as any)
339346
const ApiForeignInterruptRoute = ApiForeignInterruptRouteImport.update({
340347
id: '/api/foreign-interrupt',
341348
path: '/api/foreign-interrupt',
@@ -453,6 +460,7 @@ export interface FileRoutesByFullPath {
453460
'/api/durable-takeover': typeof ApiDurableTakeoverRoute
454461
'/api/embedding': typeof ApiEmbeddingRoute
455462
'/api/foreign-interrupt': typeof ApiForeignInterruptRoute
463+
'/api/gemini-native-image-wire': typeof ApiGeminiNativeImageWireRoute
456464
'/api/generation-persistence-resume': typeof ApiGenerationPersistenceResumeRoute
457465
'/api/generation-persistence-server': typeof ApiGenerationPersistenceServerRoute
458466
'/api/image': typeof ApiImageRouteWithChildren
@@ -522,6 +530,7 @@ export interface FileRoutesByTo {
522530
'/api/durable-takeover': typeof ApiDurableTakeoverRoute
523531
'/api/embedding': typeof ApiEmbeddingRoute
524532
'/api/foreign-interrupt': typeof ApiForeignInterruptRoute
533+
'/api/gemini-native-image-wire': typeof ApiGeminiNativeImageWireRoute
525534
'/api/generation-persistence-resume': typeof ApiGenerationPersistenceResumeRoute
526535
'/api/generation-persistence-server': typeof ApiGenerationPersistenceServerRoute
527536
'/api/image': typeof ApiImageRouteWithChildren
@@ -592,6 +601,7 @@ export interface FileRoutesById {
592601
'/api/durable-takeover': typeof ApiDurableTakeoverRoute
593602
'/api/embedding': typeof ApiEmbeddingRoute
594603
'/api/foreign-interrupt': typeof ApiForeignInterruptRoute
604+
'/api/gemini-native-image-wire': typeof ApiGeminiNativeImageWireRoute
595605
'/api/generation-persistence-resume': typeof ApiGenerationPersistenceResumeRoute
596606
'/api/generation-persistence-server': typeof ApiGenerationPersistenceServerRoute
597607
'/api/image': typeof ApiImageRouteWithChildren
@@ -663,6 +673,7 @@ export interface FileRouteTypes {
663673
| '/api/durable-takeover'
664674
| '/api/embedding'
665675
| '/api/foreign-interrupt'
676+
| '/api/gemini-native-image-wire'
666677
| '/api/generation-persistence-resume'
667678
| '/api/generation-persistence-server'
668679
| '/api/image'
@@ -732,6 +743,7 @@ export interface FileRouteTypes {
732743
| '/api/durable-takeover'
733744
| '/api/embedding'
734745
| '/api/foreign-interrupt'
746+
| '/api/gemini-native-image-wire'
735747
| '/api/generation-persistence-resume'
736748
| '/api/generation-persistence-server'
737749
| '/api/image'
@@ -801,6 +813,7 @@ export interface FileRouteTypes {
801813
| '/api/durable-takeover'
802814
| '/api/embedding'
803815
| '/api/foreign-interrupt'
816+
| '/api/gemini-native-image-wire'
804817
| '/api/generation-persistence-resume'
805818
| '/api/generation-persistence-server'
806819
| '/api/image'
@@ -871,6 +884,7 @@ export interface RootRouteChildren {
871884
ApiDurableTakeoverRoute: typeof ApiDurableTakeoverRoute
872885
ApiEmbeddingRoute: typeof ApiEmbeddingRoute
873886
ApiForeignInterruptRoute: typeof ApiForeignInterruptRoute
887+
ApiGeminiNativeImageWireRoute: typeof ApiGeminiNativeImageWireRoute
874888
ApiGenerationPersistenceResumeRoute: typeof ApiGenerationPersistenceResumeRoute
875889
ApiGenerationPersistenceServerRoute: typeof ApiGenerationPersistenceServerRoute
876890
ApiImageRoute: typeof ApiImageRouteWithChildren
@@ -1258,6 +1272,13 @@ declare module '@tanstack/react-router' {
12581272
preLoaderRoute: typeof ApiGenerationPersistenceResumeRouteImport
12591273
parentRoute: typeof rootRouteImport
12601274
}
1275+
'/api/gemini-native-image-wire': {
1276+
id: '/api/gemini-native-image-wire'
1277+
path: '/api/gemini-native-image-wire'
1278+
fullPath: '/api/gemini-native-image-wire'
1279+
preLoaderRoute: typeof ApiGeminiNativeImageWireRouteImport
1280+
parentRoute: typeof rootRouteImport
1281+
}
12611282
'/api/foreign-interrupt': {
12621283
id: '/api/foreign-interrupt'
12631284
path: '/api/foreign-interrupt'
@@ -1468,6 +1489,7 @@ const rootRouteChildren: RootRouteChildren = {
14681489
ApiDurableTakeoverRoute: ApiDurableTakeoverRoute,
14691490
ApiEmbeddingRoute: ApiEmbeddingRoute,
14701491
ApiForeignInterruptRoute: ApiForeignInterruptRoute,
1492+
ApiGeminiNativeImageWireRoute: ApiGeminiNativeImageWireRoute,
14711493
ApiGenerationPersistenceResumeRoute: ApiGenerationPersistenceResumeRoute,
14721494
ApiGenerationPersistenceServerRoute: ApiGenerationPersistenceServerRoute,
14731495
ApiImageRoute: ApiImageRouteWithChildren,
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
import { createFileRoute } from '@tanstack/react-router'
2+
import { generateImage } from '@tanstack/ai'
3+
import { createImageAdapter } from '@/lib/media-providers'
4+
5+
/**
6+
* Wire-format verification for Gemini-native `modelOptions` on the image
7+
* generation path (fix/gemini-native-image-model-options).
8+
*
9+
* Before that fix, `GeminiImageAdapter`'s `generateWithGeminiApi` only ever
10+
* forwarded `modelOptions.seed` into the `generateContent` request —
11+
* `safetySettings`, `thinkingConfig`, `imageConfig`, and `systemInstruction`
12+
* were silently dropped even though the adapter's provider-options type
13+
* (`GeminiNativeImageProviderOptions`) already declared them. This route
14+
* drives `generateImage()` against `gemini-2.5-flash-image` with
15+
* `modelOptions: { safetySettings, thinkingConfig }` set, hitting
16+
* `geminiNativeImageMount` in global-setup.ts — a hand-mocked
17+
* `POST /v1beta/models/gemini-2.5-flash-image:generateContent` endpoint that
18+
* reads the raw, untranslated request body (aimock's own journal cannot see
19+
* these fields for this endpoint — see that mount's comment) and rejects
20+
* with 400 unless `safetySettings` is present at the request root and
21+
* `generationConfig.thinkingConfig` is present nested, and rejects unless no
22+
* Imagen-only field (`personGeneration`, `negativePrompt`, a root-level
23+
* `aspectRatio`, …) is present anywhere in the body.
24+
*
25+
* A regression that stops forwarding `modelOptions` on this path — reverting
26+
* to only `seed`, or reverting to a wholesale `...modelOptions` spread that
27+
* lets an Imagen field cross over — makes the mount reject the request, the
28+
* adapter's `client.models.generateContent()` call throws, and this route
29+
* returns `ok: false`. The companion spec asserts `ok: true`.
30+
*/
31+
export const Route = createFileRoute('/api/gemini-native-image-wire')({
32+
server: {
33+
handlers: {
34+
POST: async () => {
35+
const adapter = createImageAdapter('gemini')
36+
37+
try {
38+
const result = await generateImage({
39+
adapter,
40+
prompt: 'a guitar in a music store',
41+
stream: false,
42+
modelOptions: {
43+
safetySettings: [
44+
{
45+
category: 'HARM_CATEGORY_DANGEROUS_CONTENT',
46+
threshold: 'BLOCK_ONLY_HIGH',
47+
},
48+
],
49+
thinkingConfig: { thinkingBudget: 128 },
50+
},
51+
})
52+
return new Response(
53+
JSON.stringify({ ok: true, images: result.images.length }),
54+
{
55+
status: 200,
56+
headers: { 'Content-Type': 'application/json' },
57+
},
58+
)
59+
} catch (error) {
60+
return new Response(
61+
JSON.stringify({
62+
ok: false,
63+
error: error instanceof Error ? error.message : String(error),
64+
}),
65+
{ status: 200, headers: { 'Content-Type': 'application/json' } },
66+
)
67+
}
68+
},
69+
},
70+
},
71+
})

0 commit comments

Comments
 (0)