Add `builtin_tools` to `Agent` #2102

mattbrandman · 2025-06-30T23:53:02Z

Fixes test and merge conflicts for #1722

Closes #840

Co-authored-by: Marcelo Trylesinski <[email protected]>

…dantic#1752) Co-authored-by: Marcelo Trylesinski <[email protected]>

- Added builtin_tools field to ModelRequestParameters - Merged new output_mode and output_object fields from main - Updated test snapshots to include all fields - Resolved import conflicts to include both builtin tools and profiles

…test_google.py

Kludex · 2025-08-07T06:47:01Z

.gitignore

 **.idea/
 .coverage*
 /test_tmp/
+.mcp.json


For curiosity, why MCP servers did you use here?

Kludex · 2025-08-07T09:24:44Z

pydantic_ai_slim/pydantic_ai/models/groq.py

Groq is broken now. 😢

I'm not sure why the VCR test is passing... But this doesn't work anymore.

Kludex · 2025-08-07T09:38:36Z

The only one that returns BuiltinToolCallPart and BuiltinToolReturnPart is Anthropic?

Kludex

We just need to clarify Groq. I've added a lot of tests.

Kludex · 2025-08-07T09:26:33Z

pydantic_ai_slim/pydantic_ai/models/groq.py

+            if isinstance(tool, WebSearchTool):
+                # Groq uses the web_search tool similar to Anthropic
+                # Based on https://console.groq.com/docs/agentic-tooling
+                web_search_tool: chat.ChatCompletionToolParam = {
+                    'type': 'function',
+                    'function': {
+                        'name': 'web_search',
+                        'description': 'Search the web for information',
+                        'parameters': {'type': 'object', 'properties': {}, 'required': []},
+                    },
+                }
+                tools.append(web_search_tool)


Where did you see this being passed?

I don't think we need to pass anything to Groq. If using compoung-beta models, it will use it.

See https://console.groq.com/docs/web-search.

Kludex · 2025-08-07T12:14:14Z

pydantic_ai_slim/pydantic_ai/builtin_tools.py

+    """
+
+    city: str
+    country: str


This check doesn't seem necessary. It seems it works fine with 3 characters.

hyperlint-ai · 2025-08-07T12:31:44Z

PR Change Summary

Added builtin_tools to the Agent, resolving merge conflicts and fixing tests.

Introduced builtin_tools documentation for the Agent
Resolved merge conflicts from a previous pull request
Fixed related tests to ensure functionality

Added Files

docs/api/builtin_tools.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

…h built-in tool

* Add `priority` `service_tier` to `OpenAIModelSettings` and respect it in `OpenAIResponsesModel` (pydantic#2368) * Add an example of using RunContext to pass data among tools (pydantic#2316) Co-authored-by: Douwe Maan <[email protected]> * Rename gemini-2.5-flash-lite-preview-06-17 to gemini-2.5-flash-lite as it's out of preview (pydantic#2387) * Fix toggleable toolset example so toolset state is not shared across agent runs (pydantic#2396) * Support custom thinking tags specified on the model profile (pydantic#2364) Co-authored-by: jescudero <[email protected]> Co-authored-by: Douwe Maan <[email protected]> * Add convenience functions to handle AG-UI requests with request-specific deps (pydantic#2397) * docs: add missing optional packages in `install.md` (pydantic#2412) * Include default values in tool arguments JSON schema (pydantic#2418) * Fix "test_download_item_no_content_type test fails on macOS" (pydantic#2404) * Allow string format, pattern and others in OpenAI strict JSON mode (pydantic#2420) * Let more `BaseModel`s use OpenAI strict JSON mode by defaulting to `additionalProperties=False` (pydantic#2419) * BREAKING CHANGE: Change type of 'source' field on EvaluationResult (pydantic#2388) Co-authored-by: Douwe Maan <[email protected]> * Fix ImageUrl, VideoUrl, AudioUrl and DocumentUrl not being serializable (pydantic#2422) * BREAKING CHANGE: Support printing reasons in the console output for pydantic-evals (pydantic#2163) * Document performance implications of async vs sync tools (pydantic#2298) Co-authored-by: Douwe Maan <[email protected]> * Mention that tools become toolset internally (pydantic#2395) Co-authored-by: Douwe Maan <[email protected]> * Fix tests for Logfire>=3.22.0 (pydantic#2346) * tests: speed up the test suite (pydantic#2414) * google: add more information about schema on union (pydantic#2426) * typo in output docs (pydantic#2427) * Deprecate `GeminiModel` in favor of `GoogleModel` (pydantic#2416) * Use `httpx` on `GoogleProvider` (pydantic#2438) * Remove older deprecated models and add new model of Anthropic (pydantic#2435) * Remove `next()` method from `Graph` (pydantic#2440) * BREAKING CHANGE: Remove `data` from `FinalResult` (pydantic#2443) * BREAKING CHANGE: Remove `get_data` and `validate_structured_result` from `StreamedRunResult` (pydantic#2445) * docs: add `griffe_warnings_deprecated` (pydantic#2444) * BREAKING CHANGE: Remove `format_as_xml` module (pydantic#2446) * BREAKING CHANGE: Remove `result_type` parameter and similar from `Agent` (pydantic#2441) * Deprecate `GoogleGLAProvider` and `GoogleVertexProvider` (pydantic#2450) * BREAKING CHANGE: drop 4 months old deprecation warnings (pydantic#2451) * Automatically use OpenAI strict mode for strict-compatible native output types (pydantic#2447) * Make `InlineDefsJsonSchemaTransformer` public (pydantic#2455) * Send `ThinkingPart`s back to Anthropic used through Bedrock (pydantic#2454) * Bump boto3 to support `AWS_BEARER_TOKEN_BEDROCK` API key env var (pydantic#2456) * Add new Heroku models (pydantic#2459) * Add `builtin_tools` to `Agent` (pydantic#2102) Co-authored-by: Marcelo Trylesinski <[email protected]> Co-authored-by: Douwe Maan <[email protected]> * Bump mcp-run-python (pydantic#2470) * Remove fail_under from top-level coverage config so <100% html-coverage step doesn't end CI run (pydantic#2475) * Add AbstractAgent, WrapperAgent, Agent.event_stream_handler, Toolset.id, Agent.override(tools=...) in preparation for Temporal (pydantic#2458) * Let toolsets be built dynamically based on run context (pydantic#2366) Co-authored-by: Douwe Maan <[email protected]> * Add ToolsetFunc to API docs (fix CI) (pydantic#2486) * tests: change time of evals example (pydantic#2501) * ci: remove html and xml reports (pydantic#2491) * fix: Add gpt-5 models to reasoning model detection for temperature parameter handling (pydantic#2483) Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Douwe Maan <[email protected]> Co-authored-by: Marcelo Trylesinski <[email protected]> * History processor replaces message history (pydantic#2324) Co-authored-by: Marcelo Trylesinski <[email protected]> * ci: split test suite (pydantic#2436) Co-authored-by: Douwe Maan <[email protected]> * ci: use the right install command (pydantic#2506) * Update config.yaml (pydantic#2514) * Skip testing flaky evals example (pydantic#2518) * Fix error when parsing usage details for video without audio track in Google models (pydantic#2507) * Make OpenAIResponsesModelSettings.openai_builtin_tools work again (pydantic#2520) * Let Agent be run in a Temporal workflow by moving model requests, tool calls, and MCP to Temporal activities (pydantic#2225) * Install only dev in CI (pydantic#2523) * Improve CLAUDE.md (pydantic#2524) * Add best practices regarding to coverage to CLAUDE.md (pydantic#2527) * Add support for `"openai-responses"` model inference string (pydantic#2528) Co-authored-by: Claude <[email protected]> * docs: Confident AI (pydantic#2529) * chore: mention what to do with the documentation when deprecating a class (pydantic#2530) * chore: drop hyperlint (pydantic#2531) * ci: improve matrix readability (pydantic#2532) * Add pip to dev deps for PyCharm (pydantic#2533) Co-authored-by: Marcelo Trylesinski <[email protected]> * Add genai-prices to dev deps and a basic test (pydantic#2537) * Add `--durations=100` to all pytest calls in CI (pydantic#2534) * Cleanup snapshot in test_evaluate_async_logfire (pydantic#2538) * Make some minor tweaks to the temporal docs (pydantic#2522) Co-authored-by: Douwe Maan <[email protected]> * Add new OpenAI GPT-5 models (pydantic#2503) * Fix `FallbackModel` to respect each model's model settings (pydantic#2540) * Add support for OpenAI verbosity parameter in Responses API (pydantic#2493) Co-authored-by: Claude <[email protected]> Co-authored-by: Douwe Maan <[email protected]> * Add `UsageLimits.count_tokens_before_request` using Gemini `count_tokens` API (pydantic#2137) Co-authored-by: Douwe Maan <[email protected]> * chore: Fix uv.lock (pydantic#2546) * Stop calling MCP server `get_tools` ahead of `agent run` span (pydantic#2545) * Disable instrumentation by default in tests (pydantic#2535) Co-authored-by: Marcelo Trylesinski <[email protected]> * Only wrap necessary parts of type aliases in forward annotations (pydantic#2548) * Remove anthropic-beta default header set in `AnthropicModel` (pydantic#2544) Co-authored-by: Marcelo Trylesinski <[email protected]> * docs: Clarify why AG-UI example links are on localhost (pydantic#2549) * chore: Fix path to agent class in CLAUDE.md (pydantic#2550) * Ignore leading whitespace when streaming from Qwen or DeepSeek (pydantic#2554) * Ask model to try again if it produced a response without text or tool calls, only thinking (pydantic#2556) Co-authored-by: Douwe Maan <[email protected]> * chore: Improve Temporal test to check trace as tree instead of list (pydantic#2559) * Fix: Forward max_uses parameter to Anthropic WebSearchTool (pydantic#2561) * Let message history end on ModelResponse and execute pending tool calls (pydantic#2562) * Fix type issues * skip tests requiring API keys * add `google-genai` dependency * add other provider deps * add pragma: no cover for untested logic --------- Co-authored-by: akenar <[email protected]> Co-authored-by: Tony Woland <[email protected]> Co-authored-by: Douwe Maan <[email protected]> Co-authored-by: Yi-Chen Lin <[email protected]> Co-authored-by: José I. Escudero <[email protected]> Co-authored-by: jescudero <[email protected]> Co-authored-by: Marcelo Trylesinski <[email protected]> Co-authored-by: William Easton <[email protected]> Co-authored-by: David Montague <[email protected]> Co-authored-by: Guillermo <[email protected]> Co-authored-by: Hamza Farhan <[email protected]> Co-authored-by: Mohamed Amine Zghal <[email protected]> Co-authored-by: Yinon Ehrlich <[email protected]> Co-authored-by: Matthew Brandman <[email protected]> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Douwe Maan <[email protected]> Co-authored-by: Alex Enrique <[email protected]> Co-authored-by: Jerry Yan <[email protected]> Co-authored-by: Claude <[email protected]> Co-authored-by: Mayank <[email protected]> Co-authored-by: Alex Hall <[email protected]> Co-authored-by: Jerry Lin <[email protected]> Co-authored-by: Raymond Xu <[email protected]> Co-authored-by: kauabh <[email protected]> Co-authored-by: Victorien <[email protected]> Co-authored-by: Ethan Brooks <[email protected]> Co-authored-by: eballesteros <[email protected]>

mys1erious · 2025-09-09T15:56:34Z

@Kludex Hi, is there a way to extract grounding metadata from the agent with gemini google search like described in here ?
https://ai.google.dev/gemini-api/docs/google-search "Understanding the Grounding Response" section

DouweM · 2025-09-09T17:11:18Z

@mys1erious Not currently, that'll be implemented in #2657. I'll put that on my list to look into wrapping up later this month.

Kludex and others added 30 commits May 14, 2025 11:00

Add builtin_tools to Agent

57e568b

make AbstractBuiltinTool serializable

97ab44b

Add more work on it

e3dda9d

Merge remote-tracking branch 'origin/main' into add-builtin-tools

3ad6d38

Add builtin tools

0b43f65

merge

fa7fd11

add more built-in-tools

32324fa

Fix test

f33e568

Add support on Groq

13d7433

Add support for Google

ac85205

Add support for MCP's Streamable HTTP transport (pydantic#1716)

c93633f

Timeout for initializing MCP client (pydantic#1833)

3a8b640

Require mcp 1.9.0+ (pydantic#1840)

360de87

Don't send empty messages to Anthropic (pydantic#1027)

cb4e539

Co-authored-by: Marcelo Trylesinski <[email protected]>

Add vendor_id and finish_reason to Gemini/Google model responses (p…

4e3769a

…ydantic#1800)

Fix units of sse_read_timeout timedelta (pydantic#1843)

ebb536f

Support functions as output_type, as well as lists of functions and o…

c8bb611

…ther types (pydantic#1785)

Enhance Gemini usage tracking to collect comprehensive token data (py…

6bcc1a8

…dantic#1752) Co-authored-by: Marcelo Trylesinski <[email protected]>

more

97ff651

merge

1d47e1e

merge

5f89444

merge

9512987

Pass tests

800a71a

Fix remaining merge conflict markers in openai.py, anthropic.py, and …

bc298d6

…test_google.py

add extra google

46c06c2

fix formatting

3496567

fix codespell

c193059

fixing types

427dec2

fixing types in gemini

866ad21

mattbrandman added 2 commits August 6, 2025 12:41

addressing comments

38b5b7d

Merge upstream/main into builtin-tool branch - resolved conflicts

359b9e1

mattbrandman requested a review from DouweM August 6, 2025 16:56

mattbrandman added 2 commits August 6, 2025 13:25

Fix failing tests after merge - update snapshots for builtin tools

bd6e5f7

fixing types

69267ca

Kludex reviewed Aug 7, 2025

View reviewed changes

.gitignore

**.idea/

.coverage*

/test_tmp/

.mcp.json

Copy link

Member

Kludex Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For curiosity, why MCP servers did you use here?

Kludex added 3 commits August 7, 2025 08:49

Merge remote-tracking branch 'origin/main' into builtin-tool

1a4a0f3

Remove deprecated stuff from yesterday

c9d0d18

Merge branch 'main' into builtin-tool

a631229

Kludex reviewed Aug 7, 2025

View reviewed changes

Simplify cohere

78fe612

Kludex added 4 commits August 7, 2025 11:45

refactor a bit

a673c90

Handle Google builtin tools

df2ad47

Add test for unsupported builtin tools

b4add90

Add 100% coverage

dec1ba3

Kludex reviewed Aug 7, 2025

View reviewed changes

Kludex added 2 commits August 7, 2025 14:24

fix linting issue

e134d43

Add API documentation for builtin tools

ef4ab6a

Kludex and others added 4 commits August 7, 2025 14:58

Add documentation

bb98ccf

pass docs tests

739f142

Add Groq model profile for compound models that always have web searc…

cc7ab39

…h built-in tool

add pragma

5534bf4

Kludex merged commit 1a22d68 into pydantic:main Aug 7, 2025
19 checks passed

This was referenced Aug 12, 2025

Make OpenAIResponsesModelSettings.openai_builtin_tools work again #2520

Merged

Error when using GPT-5 with output_type and built-in tools because tool_choice=required is not supported #2488

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `builtin_tools` to `Agent` #2102

Add `builtin_tools` to `Agent` #2102

Uh oh!

mattbrandman commented Jun 30, 2025 •

edited by DouweM

Loading

Uh oh!

Kludex Aug 7, 2025

Uh oh!

Kludex Aug 7, 2025

Uh oh!

Kludex commented Aug 7, 2025

Uh oh!

Kludex left a comment

Uh oh!

Kludex Aug 7, 2025

Uh oh!

Kludex Aug 7, 2025

Uh oh!

hyperlint-ai bot commented Aug 7, 2025

Uh oh!

Uh oh!

mys1erious commented Sep 9, 2025

Uh oh!

DouweM commented Sep 9, 2025

Uh oh!

Uh oh!

Add builtin_tools to Agent #2102

Add builtin_tools to Agent #2102

Uh oh!

Conversation

mattbrandman commented Jun 30, 2025 • edited by DouweM Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kludex Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex commented Aug 7, 2025

Uh oh!

Kludex left a comment

Choose a reason for hiding this comment

Uh oh!

Kludex Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

Kludex Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

hyperlint-ai bot commented Aug 7, 2025

PR Change Summary

Uh oh!

Uh oh!

mys1erious commented Sep 9, 2025

Uh oh!

DouweM commented Sep 9, 2025

Uh oh!

Uh oh!

Add `builtin_tools` to `Agent` #2102

Add `builtin_tools` to `Agent` #2102

mattbrandman commented Jun 30, 2025 •

edited by DouweM

Loading