Alui/traced

[luiarthur]

@ndebard Let's chat about this draft regarding #201 on 3/19.

This draft PR implements TracedChatOpenAI and TracedChatOllama. Their function is to wrap .invoke (of ChatOpenAI and ChatOllama) in logic to save text to/from the LLM, and metadata from LLM calls if available. In particular, reasoning summaries are saved when present.

import tempfile
from pathlib import Path

from ursa.agents import ExecutionAgent
from ursa.util.traced import TracedChatOpenAI

# OpenAI model with reasoning abilities
llm = TracedChatOpenAI(
    model="gpt-5-nano", reasoning={"effort": "low", "summary": "auto"}
)

# NOTE: Ollama model with reasoning abilities
# from ursa.util.traced import TracedChatOpenAI
# llm = "ollama-nemotron-nano": TracedChatOllama(
#   model="nemotron-3-nano:4b", reasoning=True
# )

# NOTE: Ollama Model without reasoning abilities
# llm TracedChatOllama(model="nemotron-mini:4b")

executor = ExecutionAgent(llm=llm)
executor.invoke(
    "Write a python script to print the first 10 positive integer."
)
# Save messages to json. Omit indent arg for minified json
llm.save_messages(Path("messages.json"), indent=2)

[luiarthur]

TODO

• [ ] Add a planning agent demo
• Add other LLM metadata to messages.json
  • context windows
  • reasoning level
  • other typical metadata that is easily obtained

[luiarthur]

@mikegros Could you review this PR?

The only existing source code that was changed is src/ursa/workflows/plan_execute_workflow.py where I added a None default for workspace as it currently isn't used.

The other changes are new files. So this PR introduces new abilities to log messages and does not introduce breaking changes.

[mikegros]

I'll try to review before our meeting. Otherwise, later today.

[mikegros]

Since this is connected to the LLM, not a specific agent, how are the messages sorted here? It would be nice to make it clear to a user in the documentation how that works. Maybe just with a small comment.

[luiarthur]

"sorted" as in what order the messages appear?

[mikegros]

This appends at each LLM call, which is really smooth, but I am a little worried that the message log could get messy in multiagent settings. Specifically, when we instantiate one LLM object and then hand it to multiple agents (or even agents that can be used as tools of other agents).

I think this will lead to conversation histories that are mixed into each other. It might be nice to have some meta_data tag for what agent did the call.

This might be a "future work" effort instead of something to merge in here, but I wanted to bring it up just in case.

[luiarthur]

Implementing code to add agent info will indeed be a heavier lift and could easily break existing code. Could we revisit after this PR?