Skip to content

fix(ai): keep Gemini 3+ requests valid under a trailing user turn (Vertex 400s after tool results) - #4337

Open
sleegme wants to merge 2 commits into
code-yeongyu:mainfrom
sleegme:fix/argon-thought-signature
Open

sleegme wants to merge 2 commits into
code-yeongyu:mainfrom
sleegme:fix/argon-thought-signature

Conversation

@sleegme

@sleegme sleegme commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #3633 (the senpi-side twin of omo#9747). Two Vertex Gemini 3+/4 (incl. gemini-4-argon) request-shape 400s that break long-running sessions on the turn after tool results:

  1. 400 INVALID_ARGUMENT "Requests ending with a model turn are not supported."
  2. 400 INVALID_ARGUMENT "Function call is missing a thought_signature in functionCall parts. Function call default_api:<tool> at position N."

Root cause (both live-measured one variable per request in #3633, plus the real transcript in omo#9747):

  • convertMessages folds a user message that follows tool results into the same user turn after its functionResponse parts: user[functionResponse, text]. Vertex's Gemini 3+ parser rejects exactly that part order (misreporting it as a request ending with a model turn); [text, functionResponse] and [functionResponse] return 200. Trailing user text after tool results is routine in real sessions: steering notes, recalled-memory notices, and the detached/cancelled <system-reminder> notifications of senpi-codemode eval cells (the omo#9747 transcript).
  • The same trailing user message moves the isCurrentTurn boundary (last user message carrying content) past the assistant step that produced the tool results, so the unsigned-toolCall text-replay defense from fix(ai): replay unsigned Gemini 3 tool-call steps as text (Vertex 400) #3055 stops firing for it. Gemini 3+ still strict-validates the request's final model turn, so its unsigned functionCall is sent structured and 400s. This is also the mid-session model-switch case: after switching to Argon, prior-model steps carry no signatures at all.

Fix (minimal; no signature forgery, no copying signatures across parts):

  • Gemini 3+ user turns now keep every functionResponse part at the end of its turn: text/inlineData merged after tool results is ordered ahead of the function responses. Gemini 2 and earlier keep byte-identical output.
  • The unsigned-toolCall text replay now also fires for the request's final model turn (findFinalModelTurnStart: the assistant run immediately before the trailing user/tool-result run), not only for steps after the last user message carrying content. Earlier turns stay structured unsigned (accepted by the live endpoint per Gemini 3+/4 on Vertex: a user message after tool results 400s (model turn / missing thought_signature) #3633), and parallel calls that legitimately carry the signature on the first part only stay structured.

Evidence

Before (new expectations red on unpatched source, each for the intended reason):

  • test/google-shared-role-alternation.test.ts: folded order [functionResponse, text] where [text, functionResponse] is required — the plain follow-up prompt case and the omo#9747 detached-eval reminder with two consecutive results ([fr, fr, text]).
  • test/google-shared-gemini3-unsigned-tool-call.test.ts: an unsigned functionCall part present in the final model turn under a trailing user message (plain trailing note; and a mid-session switch to gemini-4-argon after two foreign-model tool rounds), plus the signed-parallel-call ordering case.
  • Pre-fix runs of the two files: 5 failed | 23 passed (28).

After:

  • The two files: 28 passed (28).
  • Google-focused suites: 150 passed | 7 skipped (157) across 17 files.
  • Full packages/ai suite: 4060 passed | 900 skipped (4960), 0 failed. (A first run had 2 failures in models-entry.test.ts caused by the missing dist/ build artifact in the fresh worktree — ERR_MODULE_NOT_FOUND dist/models.js; after bun run build the file passes 2/2.)
  • packages/coding-agent google/argon suites: 29 passed (vertex-model-garden-extension, prompt-presets-gemini-4-argon).
  • Root bun run check: green (oxfmt, oxlint + ratchet, pinned-deps/ts-imports/install-lock, browser-smoke, entry graphs). Needed NODE_OPTIONS=--max-old-space-size=6144: the monorepo-wide tsc --noEmit exceeds Node's default 2GB heap on this 16GB machine.
  • Hermetic QA (senpi-qa, no live calls): common.mjs --self-check 10/10, cli-smoke.mjs --self-test 8/8, mock-loop.mjs --self-test 47/48 — the one failure (billing-swap) fails identically on pristine origin/main (verified on a clean a854e7a worktree), pre-existing and unrelated to this change.
  • git diff --check clean.

Not reproduced live

Per task scope (no live calls), the two 400s were not re-fired against Vertex from this machine. The asserted request shapes are exactly the ones measured live in #3633's one-variable table and omo#9747's transcript; the fix emits the measured-200 shapes ([text, functionResponse], functionResponse-only, or text-replayed unsigned steps) instead of the measured-400 ones.

Counter-scenarios checked

  1. Gemini 2 (gemini-2.5-pro) keeps the legacy [functionResponse, text] order — gate pinned by test.
  2. Parallel calls signed on the first part only stay structured in the final model turn (no over-broad replay).
  3. Earlier unsigned turns stay structured (per Gemini 3+/4 on Vertex: a user message after tool results 400s (model turn / missing thought_signature) #3633 the live endpoint accepts them).
  4. Signed empty text/thinking blocks, image tool-result routing, tool-call ids, model-switch replay characterization — all unchanged (17-file Google sweep green).

Merge danger

Door: two-way. Blast radius: Google adapters (google-generative-ai, google-vertex), Gemini 3+ requests only. Gemini 2 and non-Gemini models keep byte-identical output; earlier-turn structured replay and first-part-signed parallel calls unchanged.

Copilot AI balanced review requested due to automatic review settings October 26, 2026 05:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

sleegme added a commit to sleegme/senpi that referenced this pull request Oct 26, 2026
@code-yeongyu code-yeongyu added the review-queued Queued for batched Opus review label Oct 26, 2026
@code-yeongyu

Copy link
Copy Markdown
Owner

Thank you for this, @sleegme: the two 400s are exactly the pair we were chasing in #3633, and the live one-variable measurements in the PR body are what made this reviewable.

Our review found one thing to change before we merge, plus two notes.

Please change: in a multi-step tool loop, only the last assistant step counts as the final model turn.

findFinalModelTurnStart (packages/ai/src/api/google-shared.ts:259-265) walks back only over assistant messages that are directly adjacent. For a loop shaped user, A1, toolResult, A2, toolResult, user, only A2 is final, so A1 still goes out as a structured unsigned functionCall. The test at packages/ai/test/google-shared-gemini3-unsigned-tool-call.test.ts:341-372 pins that as safe, citing #3633, but #3633 only measured that an unsigned call in an earlier model turn is accepted when it is NOT in this shape. So the evidence doesn't cover it.

Either of these closes it:

  1. A live one-variable probe of exactly that shape (unsigned A1 in the same loop as A2, with a trailing user note folded into the function-response turn), added to the PR body; or
  2. the safer boundary, which we'd prefer: in findLastUserContentMessageIndex, skip user messages that directly follow a toolResult, since those get folded into a function-response turn anyway. Then A1 and A2 both replay as text, which is what the code already does for loops with no trailing note.

Notes, not blocking:

  • findFinalModelTurnStart reads the message list, while the API sees the converted request. An assistant step that produces no parts is skipped at :400, so when such a step is last, the final model turn actually sent is an earlier one, and it isn't replayed. Narrow, since error and aborted steps are dropped upstream. Computing "final" from the converted turns would close it.
  • The reorder at :488-502 rewrites a user turn that was already sent once, so a steering note now lands before the cached functionResponse parts. That costs a prefix-cache hit. Correctness first, so we accept it; worth a comment in the code.

One more thing, so CI doesn't mislead you: the red senpi-compatibility check on PRs that touch bundled source is expected here. PRs never carry regenerated bundle files; our batch regenerates them and runs those tests. Nothing for you to do about it.

@code-yeongyu code-yeongyu removed the review-queued Queued for batched Opus review label Oct 26, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini 3+/4 on Vertex: a user message after tool results 400s (model turn / missing thought_signature)

3 participants