fix: Gemini thoughts not correctly accumulated when streaming enabled

[OwenDavisBC]

Fixes #510

[Poggecci]

Hey Owen. Thank you for the contribution!

This PR seems to both resolve the thoughts accumulation issue, but also changes what content we send in our requests to the model. Is there a reason you've coupled these two changes? The original rationale behind stripping thoughts was a naive form of context management (although the relevance of this is fully dependent on the GenAI SDK's handling of the thoughts we provide), but since we've had this behavior since release, a change to it merits its own PR.

Happy to approve if just the accumulation is kept in this PR or if we have some further discussion on the matter.

[Poggecci]

aggregatedThoughtResponse here?

[OwenDavisBC]

@Poggecci thank you for taking a look. The issue with stripping thoughts is that will ultimately lead to empty parts being sent to the gemini API now that we are accumulating thought-only parts. I can add some more processing to remove those from the llm request here

adk-java/core/src/main/java/com/google/adk/models/Gemini.java

Lines 208 to 235 in 7f12064

	public Flowable<LlmResponse> generateContent(LlmRequest llmRequest, boolean stream) {
	llmRequest = GeminiUtil.prepareGenenerateContentRequest(llmRequest, !apiClient.vertexAI());
	GenerateContentConfig config = llmRequest.config().orElse(null);
	String effectiveModelName = llmRequest.model().orElse(model());
	logger.trace("Request Contents: {}", llmRequest.contents());
	logger.trace("Request Config: {}", config);
	if (stream) {
	logger.debug("Sending streaming generateContent request to model {}", effectiveModelName);
	CompletableFuture<ResponseStream<GenerateContentResponse>> streamFuture =
	apiClient.async.models.generateContentStream(
	effectiveModelName, llmRequest.contents(), config);
	return Flowable.defer(
	() ->
	processRawResponses(
	Flowable.fromFuture(streamFuture).flatMapIterable(iterable -> iterable)));
	} else {
	logger.debug("Sending generateContent request to model {}", effectiveModelName);
	return Flowable.fromFuture(
	apiClient
	.async
	.models
	.generateContent(effectiveModelName, llmRequest.contents(), config)
	.thenApplyAsync(LlmResponse::create));
	}
	}

, but that would not match what I see in the python-adk.

In the python-adk if I look at pre-processing before https://github.com/google/adk-python/blob/4a842c5a1334c3ee01406f796651299589fe12ab/src/google/adk/models/google_llm.py#L149-L154

    if stream:
      responses = await self.api_client.aio.models.generate_content_stream(
          model=llm_request.model,
          contents=llm_request.contents,
          config=llm_request.config,
      )

I see nothing removing thoughts - https://github.com/google/adk-python/blob/4a842c5a1334c3ee01406f796651299589fe12ab/src/google/adk/models/google_llm.py#L300-L326.

  async def _preprocess_request(self, llm_request: LlmRequest) -> None:

    if self._api_backend == GoogleLLMVariant.GEMINI_API:
      # Using API key from Google AI Studio to call model doesn't support labels.
      if llm_request.config:
        llm_request.config.labels = None

      if llm_request.contents:
        for content in llm_request.contents:
          if not content.parts:
            continue
          for part in content.parts:
            # Create copies to avoid mutating the original objects
            if part.inline_data:
              part.inline_data = copy.copy(part.inline_data)
              _remove_display_name_if_present(part.inline_data)
            if part.file_data:
              part.file_data = copy.copy(part.file_data)
              _remove_display_name_if_present(part.file_data)

    # Initialize config if needed
    if llm_request.config and llm_request.config.tools:
      # Check if computer use is configured
      for tool in llm_request.config.tools:
        if isinstance(tool, types.Tool) and tool.computer_use:
          llm_request.config.system_instruction = None
          await self._adapt_computer_use_tool(llm_request)