Skip to content

Add direct public API #1599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
May 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/api/direct.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# `pydantic_ai.direct`

::: pydantic_ai.direct
160 changes: 160 additions & 0 deletions docs/direct.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# Direct Model Requests

The `direct` module provides low-level methods for making imperative requests to LLMs where the only abstraction is input and output schema translation, enabling you to use all models with the same API.

These methods are thin wrappers around the [`Model`][pydantic_ai.models.Model] implementations, offering a simpler interface when you don't need the full functionality of an [`Agent`][pydantic_ai.Agent].

The following functions are available:

- [`model_request`][pydantic_ai.direct.model_request]: Make a non-streamed async request to a model
- [`model_request_sync`][pydantic_ai.direct.model_request_sync]: Make a non-streamed synchronous request to a model
- [`model_request_stream`][pydantic_ai.direct.model_request_stream]: Make a streamed async request to a model

## Basic Example

Here's a simple example demonstrating how to use the direct API to make a basic request:

```python title="direct_basic.py"
from pydantic_ai.direct import model_request_sync
from pydantic_ai.messages import ModelRequest

# Make a synchronous request to the model
model_response = model_request_sync(
'anthropic:claude-3-5-haiku-latest',
[ModelRequest.user_text_prompt('What is the capital of France?')]
)

print(model_response.parts[0].content)
#> Paris
print(model_response.usage)
"""
Usage(requests=1, request_tokens=56, response_tokens=1, total_tokens=57, details=None)
"""
```

_(This example is complete, it can be run "as is")_

## Advanced Example with Tool Calling

You can also use the direct API to work with function/tool calling.

Even here we can use Pydantic to generate the JSON schema for the tool:

```python
from pydantic import BaseModel
from typing_extensions import Literal

from pydantic_ai.direct import model_request
from pydantic_ai.messages import ModelRequest
from pydantic_ai.models import ModelRequestParameters
from pydantic_ai.tools import ToolDefinition


class Divide(BaseModel):
"""Divide two numbers."""

numerator: float
denominator: float
on_inf: Literal['error', 'infinity'] = 'infinity'


async def main():
# Make a request to the model with tool access
model_response = await model_request(
'openai:gpt-4.1-nano',
[ModelRequest.user_text_prompt('What is 123 / 456?')],
model_request_parameters=ModelRequestParameters(
function_tools=[
ToolDefinition(
name=Divide.__name__.lower(),
description=Divide.__doc__ or '',
parameters_json_schema=Divide.model_json_schema(),
)
],
allow_text_output=True, # Allow model to either use tools or respond directly
),
)
print(model_response)
"""
ModelResponse(
parts=[
ToolCallPart(
tool_name='divide',
args={'numerator': '123', 'denominator': '456'},
tool_call_id='pyd_ai_2e0e396768a14fe482df90a29a78dc7b',
part_kind='tool-call',
)
],
usage=Usage(
requests=1,
request_tokens=55,
response_tokens=7,
total_tokens=62,
details=None,
),
model_name='gpt-4.1-nano',
timestamp=datetime.datetime(...),
kind='response',
)
"""
```

_(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_

## When to Use the direct API vs Agent

The direct API is ideal when:

1. You need more direct control over model interactions
2. You want to implement custom behavior around model requests
3. You're building your own abstractions on top of model interactions

For most application use cases, the higher-level [`Agent`][pydantic_ai.Agent] API provides a more convenient interface with additional features such as built-in tool execution, retrying, structured output parsing, and more.

## OpenTelemetry or Logfire Instrumentation

As with [agents][pydantic_ai.Agent], you can enable OpenTelemetry/Logfire instrumentation with just a few extra lines

```python {title="direct_instrumented.py" hl_lines="1 6 7"}
import logfire

from pydantic_ai.direct import model_request_sync
from pydantic_ai.messages import ModelRequest

logfire.configure()
logfire.instrument_pydantic_ai()

# Make a synchronous request to the model
model_response = model_request_sync(
'anthropic:claude-3-5-haiku-latest',
[ModelRequest.user_text_prompt('What is the capital of France?')],
)

print(model_response.parts[0].content)
#> Paris
```

_(This example is complete, it can be run "as is")_

You can also enable OpenTelemetry on a per call basis:

```python {title="direct_instrumented.py" hl_lines="1 6 12"}
import logfire

from pydantic_ai.direct import model_request_sync
from pydantic_ai.messages import ModelRequest

logfire.configure()

# Make a synchronous request to the model
model_response = model_request_sync(
'anthropic:claude-3-5-haiku-latest',
[ModelRequest.user_text_prompt('What is the capital of France?')],
instrument=True
)

print(model_response.parts[0].content)
#> Paris
```

See [Debugging and Monitoring](logfire.md) for more details, including how to instrument with plain OpenTelemetry without Logfire.
2 changes: 2 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ nav:
- graph.md
- evals.md
- input.md
- direct.md
- MCP:
- mcp/index.md
- mcp/client.md
Expand Down Expand Up @@ -68,6 +69,7 @@ nav:
- api/usage.md
- api/mcp.md
- api/format_as_xml.md
- api/direct.md
- api/models/base.md
- api/models/openai.md
- api/models/anthropic.md
Expand Down
14 changes: 4 additions & 10 deletions pydantic_ai_slim/pydantic_ai/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
result,
usage as _usage,
)
from .models.instrumented import InstrumentationSettings, InstrumentedModel
from .models.instrumented import InstrumentationSettings, InstrumentedModel, instrument_model
from .result import FinalResult, OutputDataT, StreamedRunResult, ToolOutput
from .settings import ModelSettings, merge_model_settings
from .tools import (
Expand Down Expand Up @@ -108,7 +108,7 @@ class Agent(Generic[AgentDepsT, OutputDataT]):
model: models.Model | models.KnownModelName | str | None
"""The default model configured for this agent.

We allow str here since the actual list of allowed models changes frequently.
We allow `str` here since the actual list of allowed models changes frequently.
"""

name: str | None
Expand Down Expand Up @@ -233,7 +233,7 @@ def __init__(

Args:
model: The default model to use for this agent, if not provide,
you must provide the model when calling it. We allow str here since the actual list of allowed models changes frequently.
you must provide the model when calling it. We allow `str` here since the actual list of allowed models changes frequently.
output_type: The type of the output data, used to validate the data returned by the model,
defaults to `str`.
instructions: Instructions to use for this agent, you can also register instructions via a function with
Expand Down Expand Up @@ -1582,13 +1582,7 @@ def _get_model(self, model: models.Model | models.KnownModelName | str | None) -
if instrument is None:
instrument = self._instrument_default

if instrument and not isinstance(model_, InstrumentedModel):
if instrument is True:
instrument = InstrumentationSettings()

model_ = InstrumentedModel(model_, instrument)

return model_
return instrument_model(model_, instrument)

def _get_deps(self: Agent[T, OutputDataT], deps: T) -> T:
"""Get deps for a run.
Expand Down
Loading