Introduce dspy.Reasoning to capture native reasoning from reasoning models #8986
Conversation
| is_last_chunk=self.stream_end, | ||
| ) | ||
|
|
||
| try: |
There was a problem hiding this comment.
nit: maybe add comment why this logic should come after native response handling?
There was a problem hiding this comment.
good call, done!
| origin = get_origin(annotation) | ||
| args = get_args(annotation) | ||
| if origin is None: | ||
| if annotation is Reasoning: |
There was a problem hiding this comment.
Is there any way to implement the conversion more generically? Ideally this information should reside in Reasoning.
There was a problem hiding this comment.
that's a good question.
I did think about the same thing, but changing the __name__ in dspy.Reasoning could lead to confusion, because essentially it's just a type, but from the perspective of DSPy Adapter, it is treated as string. So I kept the logic inside adapter utils.
| field_type = field_info.annotation | ||
|
|
||
| if get_dspy_field_type(field_info) == "input" or field_type is str: | ||
| if get_dspy_field_type(field_info) == "input" or field_type is str or field_type is Reasoning: |
There was a problem hiding this comment.
same as above, let me know your thought!
| return signature | ||
|
|
||
| if "gpt-5" in lm.model and lm.model_type == "chat": | ||
| # There is a caveat of Litellm as 1.79.0 that when using the chat completion API on GPT-5 family models, |
There was a problem hiding this comment.
Is there any github issue for this on LiteLLM?
There was a problem hiding this comment.
good call! added
dspy/adapters/types/reasoning.py
Outdated
| lm: "LM", | ||
| lm_kwargs: dict[str, Any], | ||
| ) -> type["Signature"]: | ||
| reasoning_effort = "unspecified" |
There was a problem hiding this comment.
Is unspecified a valid reasoning effort?
There was a problem hiding this comment.
the code was a bit misleading, refactored
dspy/adapters/types/reasoning.py
Outdated
| The reasoning content (str) if available, None otherwise. | ||
| """ | ||
| try: | ||
| if hasattr(chunk, "choices") and chunk.choices: |
There was a problem hiding this comment.
nit:
| if hasattr(chunk, "choices") and chunk.choices: | |
| if choices := getattr(chunk, "choices"): |
3 participants
The reasoning models' response contains the reasoning content, and this PR introduce
dspy.Reasoningso that the module output captures the native reasoning, instead of letting LM regenerate one.Usage:
For non-reasoning model, the above code will treat
dspy.Reasoningfield as a string field, which prompts the model to explicitly generate the reasoning.There is a caveat of GPT-5 family models though - when using litellm chat completion + GPT-5 family models, the response doesn't contain the reasoning content. So temporarily we are treating GPT-5 family models as non-reasoning models.