-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Description
Summary
When using ADK with LiteLLM-backed models that emit separate reasoning / "thinking" content (for example via reasoning_content or reasoning fields on messages and deltas), the current LiteLlm adapter in ADK silently discards that reasoning.
As a result, ADK callers cannot access or surface model reasoning even though it is available in the underlying LiteLLM responses.
This behavior has been observed with:
- ADK (
google-adk) 1.19.0 - LiteLLM 1.79.3 (Local models from LM Studio and another model via OpenAI compatible endpoint)
Current Behavior
The ADK side behaves roughly as follows:
LiteLlm:- Aggregates assistant
contentand tool call information from LiteLLM responses. - Does not read or propagate any
reasoning_content/reasoningfields from LiteLLM.
- Aggregates assistant
LlmResponse:- Does not define a field for reasoning, so there is nowhere to store it.
Event:- Inherits from
LlmResponseand is what flows emit to callers. - Since
LlmResponsehas no reasoning field andLiteLlmdoes not populate one,Eventnever exposes reasoning either.
- Inherits from
In practice, this means:
- Downstream consumers using ADK only see final assistant content and tool calls.
- Any reasoning content provided by LiteLLM-backed models is silently dropped and unavailable.
Expected Behavior
For LiteLLM-backed models that emit reasoning fields (e.g. reasoning_content / reasoning):
- ADK's
LiteLlmadapter should:- Map those reasoning fields into an explicit
reasoningfield onLlmResponse. - Do this consistently for both non-streaming and streaming responses.
- Map those reasoning fields into an explicit
LlmResponse/Eventshould:- Expose this
reasoningfield so that callers can choose to display or process it.
- Expose this
For models/providers that do not emit any reasoning fields:
- Behavior should remain unchanged.
- The
reasoningfield should simply be absent /Noneand not appear in serialized output when usingexclude_none=True.
Impact
- ADK currently cannot surface model reasoning even when it is available through LiteLLM.
- This limits the ability of downstream applications to provide transparency/debugging views or UX patterns that distinguish between "thinking" and final answers.
- The problem is purely on the ADK integration side: LiteLLM already exposes the necessary fields.
Proposed Directions
-
Extend
LlmResponsewith an optionalreasoningfield- Add
reasoning: Optional[str] = Nonetogoogle.adk.models.llm_response.LlmResponse. - This becomes the canonical place to store reasoning/thinking content within ADK.
- Because
Eventinherits fromLlmResponse,Eventwill also gain this field automatically.
- Add
-
Map LiteLLM reasoning into
LlmResponse.reasoningin theLiteLlmadapterIn
google.adk.models.lite_llm.LiteLlm:-
Non-streaming path (
_model_response_to_generate_content_response):- After building
LlmResponsefrom the main assistant message, inspectchoices[0].messagefrom the LiteLLMModelResponse. - If the message (whether dict-style or Pydantic-style) contains
reasoning_contentorreasoning, copy that string intollm_response.reasoning.
- After building
-
Streaming path (
generate_content_async(..., stream=True)):- Maintain a
reasoning_textaccumulator alongside the existing text buffer. - For each streaming chunk, inspect
choices[0].deltafrom the LiteLLMModelResponseStream. - If the delta (dict or object) contains
reasoning_contentorreasoning, append it toreasoning_text. - When yielding partial/final
LlmResponseinstances, passreasoning=reasoning_text or None.
- Maintain a
-
-
Rely on existing flow behavior to propagate reasoning to events
-
BaseLlmFlow._finalize_model_response_eventalready mergesLlmResponseintoEventvia something like:model_response_event = Event.model_validate({ **model_response_event.model_dump(exclude_none=True), **llm_response.model_dump(exclude_none=True), })
-
Once
LlmResponse.reasoningis populated, this merge will automatically carry it intoEvent.reasoning. -
No changes should be required in flows or runners.
-
Local patch & verification
I have tested this approach locally by patching ADK 1.19.0 as follows:
-
google.adk.models.llm_response.LlmResponse- Added an optional
reasoning: Optional[str]field to hold model reasoning / thinking content.
- Added an optional
-
google.adk.models.lite_llm.LiteLlm- Non-streaming path: in
_model_response_to_generate_content_response, readreasoning_content/reasoningfromchoices[0].message(supporting both dict- and object-style messages) and assign it tollm_response.reasoningwhen present. - Streaming path: in
generate_content_async(..., stream=True), accumulatereasoning_content/reasoningfromchoices[0].deltaacross chunks into areasoning_textbuffer and pass it asreasoningwhen yielding partial and finalLlmResponseobjects.
- Non-streaming path: in
With this local patch applied, events emitted by ADK now expose a reasoning field whenever the underlying LiteLLM response includes reasoning, and downstream consumers are able to display the model's reasoning traces as expected.
This was verified both in an application consuming ADK and directly via adk web, where the patched ADK shows a reasoning field in the event payload, while the unpatched 1.19.0 build does not.