feat(providers/google): Add codeExecution and reasoning support in v5
#6365
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
Vertex now supports extraction of thinking tokens in certain Gemini models.
When the configuration is passed via
providerOptions
, the sdk:include_thoughts
to the providerThe google and google-vertex providers would silently drop user defined tools when used in combination with
useSearchGrounding
. UsinguseSearchGrounding
in combination with user defined tools is not supported. This change throws an error if that happens.In addition, this change also adds support for the Code Execution feature in both the Google Generative AI and Vertex providers.
Summary
includeThoughts
switch to thethinkingConfig
for vertex models.useSearchGrounding
anduseCodeExecution
on the Google Generative AI and thus Vertex providerscodeExecution
tool, and extract theexecutableCode
as a file part, and thecodeExecutionResult
as a text part.Verification
I added and thoroughly tested examples for both providers for both reasoning and code execution. These examples exist for both
stream-text
andgenerate-text
.I have thoroughly tested the reasoning configurations and verified that we can retrieve reasoning tokens for the Vertex provider.
In addition, I discovered that for the
google.generative-ai
provider, you can pass BOTHuseSearchGrounding
anduseCodeExecution
, but NOT in thegoogle.vertex
provider. (See eb4bfc9)Tasks
pnpm changeset
in the project root)pnpm prettier-fix
in the project root)Future Work
Consider how to best handle the
codeExecutionResult
part. Currently, I just add it as atext
part, but maybe that should be a special case. I don't know. To do this, I did change it slightly to incorporate both the outcome and the outputText, as shown herePerhaps that's not the desired effect.
Related Issues
Fixes #6302
Fixes #6219
Closes #6354
Related #6359
Related #6261
Related #6428