Skip to content

fix(google): raise on an empty candidate instead of a clean end_turn - #223

Open
gadievron wants to merge 1 commit into
masterfrom
fix/google-empty-content-silent-fn
Open

fix(google): raise on an empty candidate instead of a clean end_turn#223
gadievron wants to merge 1 commit into
masterfrom
fix/google-empty-content-silent-fn

Conversation

@gadievron

@gadievron gadievron commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Root cause

utilities/llm/providers/google.py::_response_to_unified guarded the no-candidates case (prompt blocked → raise, with a comment noting an empty end_turn "would read as a clean (passing) result — for a security tool that would mask a refusal as a non-finding"). But it did not guard a present candidate that carried no usable parts — a thinking-only/blank candidate with a clean STOP finish. That returned CompletionResult(content=[], stop_reason="end_turn"), which the pipeline (and the finding-verifier) reads as a clean, passing result: a silent false-negative for a security tool.

This is the same empty-content silent-FN closed for the OpenAI chat path in #208 and already guarded in the Anthropic adapter (R4-1) and the OpenAI Responses path — google was the remaining gap. Surfaced by an adversarial review of the repo_explorer resilience fix (#222).

Fix

Add the if not content_blocks: raise LLMResponseError(...) guard after the refusal check (more specific signal first) and before stop-reason mapping, mirroring the siblings. A tool-use-only candidate (function_call part, no text) stays valid because content_blocks is non-empty.

Reproduction

A Gemini response with one candidate, finish_reason="STOP", content.parts=[].

Regression tests (tests/test_llm_google_adapter.py)

  • test_present_candidate_with_empty_parts_raises_not_clean_end_turn — RED before, GREEN after.
  • test_tool_use_only_candidate_is_valid_not_empty — control: function-call-only stays valid.
# RED (origin/master): 1 failed, 1 passed
# GREEN (this branch):  6 passed
# Full suite:           2535 passed, 28 skipped, 0 failed

Compatibility

No API/signature change. Behavior change: an empty Gemini candidate now raises LLMResponseError instead of returning an empty end_turn. Text and tool-use responses are unaffected; refusal/blocked candidates still raise first. Consistency fix — brings google in line with the other three provider paths.

Known limitations (latent, not reachable in current config)

An adversarial sweep confirmed the guard raises only on genuinely-empty responses; two latent edges were verified unreachable today (OpenAnt requests a single, text-only candidate — no candidate_count / response_modalities / code_execution config anywhere):

  • A candidate whose only parts are non-text modalities (inline_data / executable_code / code_execution_result) would be treated as empty and raise. Would need part-handling if image/code-exec output is ever enabled.
  • Only candidates[0] is inspected; a multi-candidate response with an empty first candidate would raise. Not reachable while candidate_count defaults to 1.

The error message now names the thinking-model token-budget-exhaustion case (empty parts + MAX_TOKENS) rather than only "filtered/malformed", matching the OpenAI Responses adapter's message.

The Gemini adapter guarded the no-candidates case (prompt blocked) but not a
present candidate that carried no usable parts -- a thinking-only or blank
candidate with a clean STOP finish. It returned CompletionResult(content=[],
stop_reason="end_turn"), which the pipeline reads as a clean, passing result;
for a security verifier an empty end_turn is a silent false-negative.

Add the no-usable-content guard the sibling adapters already have -- the
Anthropic adapter (R4-1), the OpenAI Responses path, and (via #208) the OpenAI
chat path all raise LLMResponseError on empty content. Placed after the
refusal check (the more specific signal) and before stop-reason mapping. A
tool-use-only candidate stays valid (content_blocks is non-empty).

Regression test: a present candidate with empty parts now raises
LLMResponseError (was a clean end_turn); a function_call-only candidate still
resolves as a valid tool_use.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gadievron
gadievron force-pushed the fix/google-empty-content-silent-fn branch from 0e48273 to 17614fa Compare August 8, 2026 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant