This repository was archived by the owner on Oct 23, 2025. It is now read-only.
This repository was archived by the owner on Oct 23, 2025. It is now read-only.
Memory Mechanism #4
Description
Enhancement Description
Provide a summary mechanism to provide more optimal conversatioanl context for the agents.
Current Behavior
With:
"""
Receives a response + message (context), observes if a tool call is requested, executes the given
tool call, feeds the output back to the model, then recursively calls back this procedure.
If no tool call is needed it will return the response object.
"""
messages.append(response.choices[0].message.dict()) # Adding to conversation context the tool call request . NOTE: All this much context should probably not be included. To be researched```
we are appending the whole raw context, which looks like this:
```{'role': 'developer', 'content': 'Do some basic arithmetic with the provided tools'}
==============================
{'role': 'user', 'content': [{'type': 'text', 'text': 'Add 4 to 3. Then multiply the result of this operation to 4, then stop'}]}
==============================
{'role': 'developer', 'content': '\n You must use the provided tools.\n - When a tool is relevant, **immediately call the tool without any conversational filler or "thinking out loud" text.**\n - If you have successfully gathered all necessary information from tool calls to answer the user\'s request, provide the final answer directly.\n - If something is not sufficiently clear, ask for clarifications.\n - If you are needing a tool but you dont have access to it, you have to sttop and specify that you need it.\n - If there is any logical error in the request, express it, then stop.\n - At each step reorganize your planning based on prior conversational context. Elaborate on how you readjust your approach.\n '}
==============================
{'content': '', 'refusal': None, 'role': 'assistant', 'annotations': None, 'audio': None, 'function_call': None, 'tool_calls': [{'id': 'tool_0_addition', 'function': {'arguments': '{"y":3,"x":4}', 'name': 'addition'}, 'type': 'function', 'index': 0}], 'reasoning': "**Analyzing the Sequence**\n\nI've mapped out the initial steps: sum 4 and 3 using the `addition` tool. The output is then the input for a `multiplication` tool, multiplying that sum by 4. I'm focusing on the precise order and tool selection.\n\n\n**Planning the Tool Usage**\n\nI'm now detailing the tool calls. The `addition` tool is straightforward, with inputs x=4 and y=3. The multiplication call will use the addition result as its first input, and 4 as the second. I've added a check to confirm that the addition result feeds correctly into multiplication, ensuring the final output is the correct answer.\n\n\n", 'reasoning_details': [{'type': 'reasoning.text', 'text': "**Analyzing the Sequence**\n\nI've mapped out the initial steps: sum 4 and 3 using the `addition` tool. The output is then the input for a `multiplication` tool, multiplying that sum by 4. I'm focusing on the precise order and tool selection.\n\n\n**Planning the Tool Usage**\n\nI'm now detailing the tool calls. The `addition` tool is straightforward, with inputs x=4 and y=3. The multiplication call will use the addition result as its first input, and 4 as the second. I've added a check to confirm that the addition result feeds correctly into multiplication, ensuring the final output is the correct answer.\n\n\n", 'format': 'unknown'}]}
==============================
{'role': 'tool', 'tool_call_id': 'tool_0_addition', 'name': 'addition', 'output': '{"result": 7}'}
==============================
{'content': '', 'refusal': None, 'role': 'assistant', 'annotations': None, 'audio': None, 'function_call': None, 'tool_calls': [{'id': 'tool_0_multiplication', 'function': {'arguments': '{"y":4,"x":7}', 'name': 'multiplication'}, 'type': 'function', 'index': 0}], 'reasoning': "**Contemplating Operations**\n\nI'm currently pondering the best approach. My initial assessment indicates that both available tools, addition and multiplication, could be relevant, but in what order? I see the need to add 4 to 3 first. I will need to determine the proper tool use based on the inputs the user is requesting to be operated on.\n\n\n**Mapping Tool Use**\n\nI've refined the plan, mapping the operations to the tools. Addition will handle 4 and 3, generating 7. This result, now a defined value, becomes the first parameter for multiplication, paired with 4. The next step is generating the `multiplication` call.\n\n\n", 'reasoning_details': [{'type': 'reasoning.text', 'text': "**Contemplating Operations**\n\nI'm currently pondering the best approach. My initial assessment indicates that both available tools, addition and multiplication, could be relevant, but in what order? I see the need to add 4 to 3 first. I will need to determine the proper tool use based on the inputs the user is requesting to be operated on.\n\n\n**Mapping Tool Use**\n\nI've refined the plan, mapping the operations to the tools. Addition will handle 4 and 3, generating 7. This result, now a defined value, becomes the first parameter for multiplication, paired with 4. The next step is generating the `multiplication` call.\n\n\n", 'format': 'unknown'}]}
==============================
{'role': 'tool', 'tool_call_id': 'tool_0_multiplication', 'name': 'multiplication', 'output': '{"result": 28}'}
==============================```
This is suboptimal both in token usage + offering meaningful contex.
### Proposed Improvement
I suggest we create an option for an AI agent that inspects the whole context window and keeps only relevant information, eliminating redundant phrasing and convoluted explanations to generate concise, brief, minimal messages.
### Benefits
- Increased in model performance due to a better context
- Decrease in model usage
### Affected Components
- `open_router.py`'s `__complete_tool_calling_cycle` would include at the very start the new AI agent
### Code Example
```python
async def __complete_tool_calling_cycle(self, response: ChatCompletion, messages: List[dict[str, str]]):
"""
Receives a response + message (context), observes if a tool call is requested, executes the given
tool call, feeds the output back to the model, then recursively calls back this procedure.
If no tool call is needed it will return the response object.
"""
renewed_messages = MemoryOptimzer().summarize(messages)
tool_calls = response.choices[0].message.tool_calls
logger.debug(f"(🔧) Tool calls ({len(tool_calls) if tool_calls else 0} tools requested): {tool_calls}")
under_max_limit_of_interactions_reached = self.number_of_interactions < self.interactions_limit
if tool_calls and under_max_limit_of_interactions_reached:
self.number_of_interactions += 1
with add_context_to_log(interacion_number=self.number_of_interactions):
sync_futures = []
async_tasks = []
tool_call_info_map = {}
with concurrent.futures.ProcessPoolExecutor() as executor:
for tool_call in tool_calls:
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
tool_call_id = tool_call.id
procedure = self.toolkit.tools.get(function_name)
if not procedure:
logger.warning(f"Tool '{function_name}' requested by LLM but not found in toolkit.")
renewed_messages.append({
"tool_call_id": tool_call_id,
"output": f"Error: Tool '{function_name}' not found.",
"name": function_name
})
continue
executable_method = procedure.get_executable()
if procedure.is_coroutine:
task = asyncio.create_task(executable_method(**function_args))
async_tasks.append(task)
tool_call_info_map[task] = (function_name, tool_call_id)
else:
future = executor.submit(executable_method, **function_args)
sync_futures.append(future)
tool_call_info_map[future] = (function_name, tool_call.id)
messages_with_tool_results = await self.extract_results_tools(messages=renewed_messages,
tool_call_info_map=tool_call_info_map,
sync_futures=sync_futures,
async_tasks=async_tasks)
response = await self.__generate_completition(
messages=messages_with_tool_results,
tools=self.toolkit.schematize() if self.toolkit else None,
)
self.__log_response(response)
return await self.__complete_tool_calling_cycle(response=response, messages=renewed_messages)
Enhancement Category
Performance
Impact Level
Medium - Noticeable improvement
Migration Considerations
None.
Additional Context
No response