Allow setting `tool_choice`

[moritzwilksch]

Description

This might also be a bug / unintended behavior?

Request: Force non-tool-call although tools are defined

I'd like to be able to create an OpenAI generation with tool_choice='none' such that - even though tools are defined - the model is forced to respond with a non-tool-call. This can be advantageous for cache hit rates as opposed to setting function_tools=[] for the generation in agentic loops.

Output mode?

My intuition would be I'd be able to set output_mode = "text" since there exists an output_mode = "text_or_tool" which implies that "text" means "text, not tool". However, this is not the case. Is this intentional or an inconsistency / bug?

Example

# %%
from dotenv import load_dotenv
from pydantic import BaseModel
from pydantic_ai.direct import model_request_sync
from pydantic_ai.messages import ModelRequest, SystemPromptPart, UserPromptPart
from pydantic_ai.models import ModelRequestParameters
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.tools import ToolDefinition

load_dotenv()

model = OpenAIChatModel("gpt-4.1-mini")
request = ModelRequest(
    [
        SystemPromptPart(
            "Before answering any weather-related question use the `get_current_weather` tool."
        ),
        UserPromptPart("Hows the temp in Berlin?"),
    ]
)


class Params(BaseModel):
    location: str
    date: str


response = model_request_sync(
    messages=[request],
    model=model,
    model_request_parameters=ModelRequestParameters(
        output_mode="text",  # how do I emulate tool_choice='none'?
        function_tools=[
            ToolDefinition(
                name="get_current_weather",
                description="Get the current weather in a given location",
                parameters_json_schema=Params.model_json_schema(),
            )
        ],
    ),
)
print(response)

ModelResponse(
    parts=[
        ToolCallPart(
            tool_name="get_current_weather",
            args='{"location":"Berlin","date":"2024-06-05"}',
            tool_call_id="call_P9Ln6s7Np1LnlYxZlrH6HGvL",
        )
    ],
    usage=RequestUsage(
        input_tokens=73,
        output_tokens=24,
        details={
            "accepted_prediction_tokens": 0,
            "audio_tokens": 0,
            "reasoning_tokens": 0,
            "rejected_prediction_tokens": 0,
        },
    ),
    model_name="gpt-4.1-mini-2025-04-14",
    timestamp=datetime.datetime(2025, 9, 4, 9, 34, 1, tzinfo=TzInfo(UTC)),
    provider_name="openai",
    provider_response_id="chatcmpl-CC0dtBsakxMEVJGvnPnL6OVIGBrkq",
)

References

No response

[moritzwilksch]

It appears that the literal 'none' is never used for tool_choice:

rg --no-ignore 'tool_choice' --type py .pixi/envs/default/lib/python3.13/site-packages/pydantic_ai/

.pixi/envs/default/lib/python3.13/site-packages/pydantic_ai/models/openai.py
387:            tool_choice: Literal['none', 'required', 'auto'] | None = None
390:            and OpenAIModelProfile.from_profile(self.profile).openai_supports_tool_choice_required
392:            tool_choice = 'required'
394:            tool_choice = 'auto'
427:                tool_choice=tool_choice or NOT_GIVEN,
891:            tool_choice: Literal['none', 'required', 'auto'] | None = None
893:            tool_choice = 'required'
895:            tool_choice = 'auto'
936:                tool_choice=tool_choice or NOT_GIVEN,

[DouweM]

My intuition would be I'd be able to set output_mode = "text" since there exists an output_mode = "text_or_tool" which implies that "text" means "text, not tool". However, this is not the case. Is this intentional or an inconsistency / bug?

@moritzwilksch Output modes do not affect function tool calls, just output tool calls: text_or_tool means that the model can produce final agent run output by generating text or calling an output tool, and text means it can only generate text as final output. But in both cases (and with the other output modes), the model is also allowed to call regular function tools in the process of getting to its final output.

---

In any case, the feature request makes sense. You may already be able to get this to work by using ModelSettings(extra_body={'tool_choice': 'none'}), but I agree we should support it natively.

The easiest option would be to add a tool_calls=False field to ModelSettings, similar to the existing parallel_tool_calls. That would map to tool_choice='none' if there are no output tools, or tool_choice={'type': 'allowed_tools': {'mode': 'required'/'auto', 'tools': [<output tool defs>]} if there are, to make sure the field only affects regular function tools while output tools can still be called. The other model APIs support something analogous.

It would also be good to design this in such a way that it also supports forcing the use of any tool or specific tool(s) (#1820). So maybe we do end up with a tool_choice field, although it wouldn't map directly to the API param, as we'd need to have it only affect function tools while still allowing output tools.

Perhaps it would make sense to have a separate output_tool_choice to allow that selection to be tweaked as well, although it's not clear what would happen if the model is not allowed to generate text and you set output_tool_choice and tool_choice both to 'none'. Would we just get TextParts and then force the model to do another request as the agent run can't finish yet? This may have some relation to #2793 as well.

#1820 also requests being able to tweak this on a per-model-request basis, so we may want to add a hook to Agent and the run methods that lets the ModelSettings be processed similar to how history_processors processes the list[ModelMessage], but we can consider that separately.

@moritzwilksch Would you be interested in contributing something like that ModelSettings.tool_choice?

[moritzwilksch]

Thanks for the thorough response! I don't think I quite understand the usage of output_tools yet and didn't find any clarifying docs/comments. Could you explain what it is for? Again my intuition (based on the name) would've been a constraint on which tools the model may use, but that doesn't seem to be the case. TBF, I am not using the Agent interface at all, just the direct mode for its nice context representation.

It would also be good to design this in such a way that it also supports forcing the use of any tool or specific tool(s)

I definitely agree, tool_choice should allow users to select a subset of defined tools as available to the model. I'm currently migrating some raw requests code to pydantic_ai and am using this feature of the OpenAI API.

@moritzwilksch Would you be interested in contributing something like that ModelSettings.tool_choice?

I'll def use that feature and am willing to contribute it if it is not too tightly coupled to the Agent API. Since I don't use the higher-level API it'd probably take me too long to to work in this part of the codebase. Not too attached to it though, if it's a quick fix you can make, go for it.
What's your assessment, would most of these changes be closer to the low-level API if we just start with supporting a ModelSettings.tool_choice: list[Something?]

[DouweM]

Thanks for the thorough response! I don't think I quite understand the usage of output_tools yet and didn't find any clarifying docs/comments. Could you explain what it is for?

See https://ai.pydantic.dev/output/#tool-output. Using that mode for structured output will result in output_mode='tool'/'text_or_tool' plus a list of output_tools on ModelRequestParameters.

ModelRequestParameters.function_tools are the tools the user has registered themselves. So tool_choice should only affect function tools, while leaving the output tools available to the model (unless the user specifically requested for the model to not be able to generate final output, which would be odd).

What's your assessment, would most of these changes be closer to the low-level API if we just start with supporting a ModelSettings.tool_choice: list[Something]?

Yep I expect this to live completely on ModelSettings, not Agent. I imagine it'd be Literal['none', 'required'] | list[str] | None, where None means auto, and list[str] means "one of these tools". The trickiest bit would be implementing this in all the model classes to properly map to their values, and to make sure output tools are unaffected.

[moritzwilksch]

i see, thanks. I'm not sure i'll have the capacity to work on such a PR at least for the next week or so, so depending on your timeline it might make more sense for you folks to implement this feature?

[moritzwilksch]

Prototyped something in #2815, along those lines? Quite finicky to get the format and branching right 😅 This is how I'd use it:

# %%
from dotenv import load_dotenv
from pydantic import BaseModel

from pydantic_ai.direct import model_request_sync
from pydantic_ai.messages import ModelRequest, SystemPromptPart, UserPromptPart
from pydantic_ai.models import ModelRequestParameters
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.settings import ModelSettings
from pydantic_ai.tools import ToolDefinition

load_dotenv()

model = OpenAIChatModel('gpt-4.1-mini')
# model = OpenAIResponsesModel("gpt-4.1-mini")
request = ModelRequest(
    [
        SystemPromptPart('Before answering any weather-related question use the `get_current_weather` tool.'),
        UserPromptPart('Hows the temp in Berlin?'),
    ]
)


class Params(BaseModel):
    location: str
    date: str


# with patch_to_print(model):
response = model_request_sync(
    messages=[request],
    model=model,
    # API: responses
    # model_settings=ModelSettings(extra_body={"tool_choice": {"type": "allowed_tools", "mode": "required", "tools": [{ "type": "function", "name": "greet_user" }]}}),
    # API: completions
    # model_settings=ModelSettings(
    #     extra_body={
    #         'tool_choice': {
    #             'type': 'allowed_tools',
    #             'allowed_tools': {
    #                 'mode': 'required',
    #                 'tools': [
    #                     {'type': 'function', 'function': {'name': 'greet_user'}},
    #                 ],
    #             },
    #         }
    #     }
    # ),
    model_settings=ModelSettings(
        tool_choice=["greet_user"]
    ),
    model_request_parameters=ModelRequestParameters(
        output_mode='text',
        function_tools=[
            ToolDefinition(
                name='get_current_weather',
                description='Get the current weather in a given location',
                parameters_json_schema=Params.model_json_schema(),
            ),
            ToolDefinition(
                name='greet_user',
                description='Greet the user with a friendly message',
            ),
        ],
    ),
)
print(response)

# ModelResponse(
#     parts=[ToolCallPart(tool_name='greet_user', args='{}', tool_call_id='call_UD9CJCyWcpfxgPaFhrKMz30p')],
#     usage=RequestUsage(
#         input_tokens=100,
#         output_tokens=1,
#         details={
#             'accepted_prediction_tokens': 0,
#             'audio_tokens': 0,
#             'reasoning_tokens': 0,
#             'rejected_prediction_tokens': 0,
#         },
#     ),
#     model_name='gpt-4.1-mini-2025-04-14',
#     timestamp=datetime.datetime(2025, 9, 5, 9, 24, 44, tzinfo=TzInfo(UTC)),
#     provider_name='openai',
#     provider_response_id='chatcmpl-CCMyShs6aI0jPKoQQRXJMMMrj7wq8',
# )

[DouweM]

@moritzwilksch Thanks for filing that, it doesn't take the output tool stuff into account yet but as you mentioned that'll likely be easier for us to look into. I can't promise exactly when I'll get to this, but I've put it on my list for the near future.