Make Gemini google_search tool grounding metadata available in model response #2890
Description
Description
When the builtin WebSearchTool is used with Gemini models, the http response includes the groundingMetadata key in candidates[0] (example from their docs)
This is the only way to provide proper citations for results found using the builtin tool as far as I'm aware (seemingly no amount of prompting to tell it to include the sources in the text response makes it listen).
Happy to implement this myself if needed but would like some direction on how it should look (if its even something you'd want to add?)
Pushed a simple commit to my fork that was enough to get what I wanted working locally, but I'm doubtful this would be the right approach:
For more references, here are the types from the google genai lib:
In my version I then use the result by looking for BuiltinToolReturnPart with tool_name == "grounding_metadata" to then do what I want with the sources, e.g. something like
def get_grounding_metadata(msgs: list[ModelMessage]) -> GroundingMetadata | None:
for msg in msgs:
if not isinstance(msg, ModelResponse):
continue
for part in message.parts:
if isinstance(part, BuiltinToolReturnPart) and part.tool_name == "grounding_metadata":
if not isinstance(part, GroundingMetadata):
raise ValueError('grounding_metadata tool part response not an instance of GroundingMetadata')
return part
return NoneReferences
No response