feat: add minimalist mode for low-resource hardware (Pi 5)#610
Closed
watrworld wants to merge 1 commit intoHKUDS:mainfrom
Closed
feat: add minimalist mode for low-resource hardware (Pi 5)#610watrworld wants to merge 1 commit intoHKUDS:mainfrom
watrworld wants to merge 1 commit intoHKUDS:mainfrom
Conversation
Collaborator
|
Hi, thank you for your contribution! This PR has merge conflicts with the main branch that have remained unresolved for an extended period. To keep the repository healthy, we're closing stale PRs with conflicts. If you're still interested in this feature, please feel free to reopen with the conflicts resolved. We'd be happy to review a fresh PR! |
Collaborator
|
Because this issue has been inactive for a long time, I will close it. If there are any other problems, please feel free to open a new issue. |
WTHDonghai
pushed a commit
to WTHDonghai/nanobot
that referenced
this pull request
Mar 22, 2026
…ction (HKUDS#610) commit_async() called extract_long_term_memories(messages=...) without passing user, session_id, or ctx. Because the compressor returns early when ctx is None, async commits always produced memories_extracted=0. The sync commit() path already passes all three parameters correctly. This aligns the async path to match. Regression from the COW pattern revert (HKUDS#584) which dropped these arguments from the async call. Fixes HKUDS#602 Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces a Minimalist Mode to allow Nanobot to run efficiently on low-resource hardware like the Raspberry Pi 5.
The Problem:
When using small local models (e.g., 1B or 3B parameters) via Ollama, the overhead of injecting full JSON tool definitions into the system prompt causes:
Extreme Latency: Response times exceeding 5–6 minutes.
Model Hallucinations: Small models often "hallucinate" tool calls or output raw JSON instead of answering the user.
The Solution:
Added a minimalist configuration flag. When set to true:
The agent skips fetching and sending tool definitions to the LLM.
The system prompt is streamlined for basic chat/reasoning.
Performance Gain: On a Raspberry Pi 5, response time dropped from ~6 minutes to < 2 seconds.
Changes:
nanobot/config/schema.py: Added minimalist boolean to AgentDefaults.
nanobot/agent/context.py: Modified build_system_prompt to respect the flag.
nanobot/agent/loop.py: Logic to pass tools=None to the provider when minimalist mode is active.
nanobot/cli/commands.py: Wired the config flag into the Agent Loop initialization.