Support overriding agent instructions #2926
Conversation
| @@ -370,6 +370,14 @@ def __init__( | |||
| _utils.Option[Sequence[Tool[AgentDepsT] | ToolFuncEither[AgentDepsT, ...]]] | |||
| ] = ContextVar('_override_tools', default=None) | |||
|
|
|||
| # Prompt overrides (experimental) | |||
| self._override_instructions: ContextVar[_utils.Option[str | None]] = ContextVar( | |||
There was a problem hiding this comment.
Could we support lists and functions as well, like on Agent.__init__?
pydantic-ai/pydantic_ai_slim/pydantic_ai/agent/__init__.py
Lines 166 to 169 in a2883d2
| @@ -370,6 +370,14 @@ def __init__( | |||
| _utils.Option[Sequence[Tool[AgentDepsT] | ToolFuncEither[AgentDepsT, ...]]] | |||
| ] = ContextVar('_override_tools', default=None) | |||
|
|
|||
| # Prompt overrides (experimental) | |||
There was a problem hiding this comment.
Reminder to remove this comment before we merge
| self._override_instructions: ContextVar[_utils.Option[str | None]] = ContextVar( | ||
| '_override_instructions', default=None | ||
| ) | ||
| self._override_system_prompts: ContextVar[_utils.Option[tuple[str, ...]]] = ContextVar( |
There was a problem hiding this comment.
Do we need this? Instructions seem to be the more modern version of system prompts, so unless you have a specific reason to want both, I'd rather only have instructions.
| if override_instructions := self._override_instructions.get(): | ||
| base_text = override_instructions.value | ||
| parts = [base_text] | ||
| else: | ||
| parts = [ | ||
| self._instructions, | ||
| *[await func.run(run_context) for func in self._instructions_functions], | ||
| ] |
There was a problem hiding this comment.
Let's do this in a new Agent._get_instructions() function, like we have _get_model, _get_deps, etc.
| return None | ||
| return '\n\n'.join(parts).strip() | ||
| return '\n\n'.join(parts_to_format).strip() |
There was a problem hiding this comment.
The existing logic filters out empty strings as well, why do we now only filter Nones? Note also that if we do want this change, we can just change parts instead of having both parts and parts_to_format :)
| else: | ||
| system_prompts = self._system_prompts | ||
|
|
||
| if override_instructions := self._override_instructions.get(): |
There was a problem hiding this comment.
Could we move this to some top-level method, like the other override context var usages?
| instructions_functions=self._instructions_functions, | ||
| system_prompts=self._system_prompts, | ||
| instructions=instructions_for_node, | ||
| instructions_functions=instructions_functions_for_node, |
There was a problem hiding this comment.
These 2 fields are not actually used anywhere, but I suppose we can't remove them or stop setting them because UserPromptNode is public (exposed via pydantic_ai.agent) and people may be using them...
| @@ -764,6 +793,31 @@ def override( | |||
| if tools_token is not None: | |||
| self._override_tools.reset(tools_token) | |||
|
|
|||
| @contextmanager | |||
| def override_prompts( | |||
There was a problem hiding this comment.
Can we add this to the existing override method instead?
There was a problem hiding this comment.
Yup, I realized this yesterday too!
I just started working on a library: https://github.com/mwildehahn/pydantic-ai-gepa to integrate https://github.com/gepa-ai/gepa into pydantic-ai. This would provide similar functionality to https://dspy.ai/ where you can provide a signature and then let an LLM handle constructing the prompt.
This is very experimental and I just started on this yesterday, but I wanted to propose a small extension to pydantic-ai that would allow us to override system_prompt and instructions in the same way we can override toolsets etc. With that minimal surface area, I can hook in something like pydantic-ai-gepa to optimize the prompts and then run those optimized prompts.
LMK if there are better ways to override the system prompts on demand.