Skip to content

Major upgrade: persistent memory, Firefox default, voice config, system control#9

Open
nlaaroussi wants to merge 2 commits into
ethanplusai:mainfrom
nlaaroussi:main
Open

Major upgrade: persistent memory, Firefox default, voice config, system control#9
nlaaroussi wants to merge 2 commits into
ethanplusai:mainfrom
nlaaroussi:main

Conversation

@nlaaroussi
Copy link
Copy Markdown

Summary

  • Persistent 3-tier conversation memory — every exchange logged to SQLite, rolling session summaries compact every 5 messages, Haiku generates a full session summary on disconnect, and past session context is injected into every LLM call on reconnect. Jarvis now remembers across sessions without manual prompting.
  • Memory extraction — after every exchange, Haiku extracts facts, preferences, decisions and project details and stores them as searchable memories ranked by importance.
  • Edge TTS migration — Fish Audio replaced with edge-tts (free, no API key required). Default voice: en-GB-RyanNeural. Configurable via EDGE_TTS_VOICE in .env.
  • Firefox as default browser — all [ACTION:BROWSE] actions route to Firefox. Falls back to Chrome only if explicitly requested.
  • Universal "pull up" command"pull up [anything]" opens Firefox with a search. Bare app names (1–2 words, no articles) still switch the app. Works for any query: weather, search, research, navigation.
  • Weather location fixed — corrected from St. Petersburg FL (°F) to Leicester, UK (°C).
  • Calendar and Mail no longer auto-launch — both apps are now passive; Jarvis uses them only if already open.
  • Model config externalisedJARVIS_FAST_MODEL and JARVIS_SMART_MODEL env vars allow model swapping without touching code.
  • Local LLM support — fast voice responses routed through LM Studio (LOCAL_LLM_MODEL) with automatic Anthropic fallback.
  • system_control.py — new module for AppleScript-based macOS automation: app switching, volume, keyboard shortcuts, window management.
  • Planner blueprints — plans serialise to .jarvis_blueprint.json for sub-agent consumption.
  • Work mode timeout raised from 5 → 15 minutes to support complex builds.
  • HANDBOOK.md — operations and configuration guide covering all commands, .env variables, file map, and troubleshooting.

Test plan

  • Start server — confirm no startup errors and API key loads from .env
  • Speak a command — confirm Edge TTS voice responds in en-GB-RyanNeural
  • Say "pull up the weather forecast for London" — confirm Firefox opens with weather search
  • Say "pull up Spotify" — confirm app switches, does not open browser
  • Say "what's on my calendar today" — confirm works only if Calendar is already open
  • Close browser tab — check data/jarvis.db sessions table for a new summary entry
  • Reconnect — confirm Jarvis references the previous session in context
  • Verify LOCAL_LLM_MODEL fallback by stopping LM Studio mid-session

🤖 Generated with Claude Code

Najeem Laaroussi and others added 2 commits April 23, 2026 19:00
…em control

Core changes:
- Persistent 3-tier conversation memory: per-message SQLite logging, rolling
  session summaries (every 5 messages), end-of-session Haiku compaction, and
  cross-session context injected into every LLM call on reconnect
- Memory extraction: Haiku extracts facts/preferences/decisions after every
  exchange and stores them as searchable memories
- Edge TTS migration: replaced Fish Audio with edge-tts (free, no API key).
  Voice defaults to en-GB-RyanNeural
- Firefox set as default browser across all browse actions
- "pull up" is now a universal Firefox command — any descriptive phrase opens
  Firefox search; bare app names (1-2 words, no articles) still switch the app
- Weather location corrected from St. Petersburg FL to Leicester UK (°C)
- Calendar and Mail no longer auto-launch on startup — only used if already open
- Model config externalised: JARVIS_FAST_MODEL and JARVIS_SMART_MODEL env vars
- Local LLM support via LM Studio (LOCAL_LLM_MODEL) with Anthropic fallback
- system_control.py: AppleScript-based macOS system automation (app switching,
  volume, keyboard shortcuts, window management)
- dispatch_registry.py: blueprint column, increased response storage to 10k chars
- work_mode.py: Claude Code session timeout raised from 5 to 15 minutes
- planner.py: plans serialise to JSON blueprints for sub-agent reading
- HANDBOOK.md: operations and configuration guide

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SIGTERM/SIGINT handlers registered in lifespan — on pkill or Ctrl+C,
  all active WebSocket sessions generate a Haiku summary and persist to
  SQLite before the process exits. Previously a hard kill discarded the
  final session summary.
- Global _active_sessions registry tracks live sessions with buffer,
  client, and rolling summary refs. Deregistered on clean disconnect so
  the signal handler only touches sessions that didn't close gracefully.
- Rolling summary synced to registry after each mid-session compaction.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant