Skip to content

feat(providers/google): Add codeExecution and reasoning support in v5 #6365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: v5
Choose a base branch
from

Conversation

Und3rf10w
Copy link
Contributor

@Und3rf10w Und3rf10w commented May 16, 2025

Background

Vertex now supports extraction of thinking tokens in certain Gemini models.

When the configuration is passed via providerOptions, the sdk:

  1. Did not extract reasoning tokens
  2. Did not pass include_thoughts to the provider

The google and google-vertex providers would silently drop user defined tools when used in combination with useSearchGrounding. Using useSearchGrounding in combination with user defined tools is not supported. This change throws an error if that happens.

In addition, this change also adds support for the Code Execution feature in both the Google Generative AI and Vertex providers.

Summary

  • Added extraction logic to google-generative-ai package to parse reasoning tokens.
  • Added a includeThoughts switch to the thinkingConfig for vertex models.

  • Throw an error when tools are used in conjunction with useSearchGrounding and useCodeExecution on the Google Generative AI and thus Vertex providers
  • Added logic to call the google defined codeExecution tool, and extract the executableCode as a file part, and the codeExecutionResult as a text part.

Verification

I added and thoroughly tested examples for both providers for both reasoning and code execution. These examples exist for both stream-text and generate-text.

I have thoroughly tested the reasoning configurations and verified that we can retrieve reasoning tokens for the Vertex provider.

In addition, I discovered that for the google.generative-ai provider, you can pass BOTH useSearchGrounding and useCodeExecution, but NOT in the google.vertex provider. (See eb4bfc9)

Tasks

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • Formatting issues have been fixed (run pnpm prettier-fix in the project root)

Future Work

Consider how to best handle the codeExecutionResult part. Currently, I just add it as a text part, but maybe that should be a special case. I don't know. To do this, I did change it slightly to incorporate both the outcome and the outputText, as shown here

const formattedText = `Execution Result (Outcome: ${part.codeExecutionResult.outcome}):\n${outputText}`;

Perhaps that's not the desired effect.

Related Issues

Fixes #6302
Fixes #6219
Closes #6354
Related #6359
Related #6261
Related #6428

@Und3rf10w Und3rf10w changed the title feat(providers/google): Add codeExecution and reasoning support feat(providers/google): Add codeExecution and reasoning support in v5 May 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant