Rebuild Reachy voice and motion pipeline

.env.example

@@ -2,18 +2,26 @@
 # Give your OpenClaw AI agent a physical robot body!
 
 # ==============================================================================
-# REQUIRED: OpenAI API Key
+# OPTIONAL: OpenAI API Key (legacy cloud vision fallback only)
 # ==============================================================================
 # Get your key at: https://platform.openai.com/api-keys
-# Requires Realtime API access
 OPENAI_API_KEY=sk-your-openai-key
 
+# ==============================================================================
+# REQUIRED: Always-hot local speech-to-text
+# ==============================================================================
+# OpenAI-compatible /v1/audio/transcriptions service on xeon-serv.
+STT_BASE_URL=http://speech-host.local:8890/v1
+STT_MODEL=distil-large-v3
+STT_LANGUAGE=en
+# STT_API_KEY=optional-local-bearer-token
+
 # ==============================================================================
 # REQUIRED: OpenClaw Gateway
 # ==============================================================================
 # The URL where your OpenClaw gateway is running
 # If running on the same machine as the robot, use the host machine's IP
-OPENCLAW_GATEWAY_URL=http://192.168.1.100:18789
+OPENCLAW_GATEWAY_URL=http://openclaw-host.local:18789
 
 # Your OpenClaw gateway authentication token
 # Find this in ~/.openclaw/openclaw.json under gateway.token
@@ -23,22 +31,64 @@ OPENCLAW_TOKEN=your-gateway-token
 OPENCLAW_AGENT_ID=main
 
 # Session key for conversation context - IMPORTANT!
-# Use "main" (default) to share context with WhatsApp and other DM channels
-# This allows the robot to be aware of all your conversations
-OPENCLAW_SESSION_KEY=main
+# Keep the robot in a dedicated session while using the same OpenClaw agent.
+OPENCLAW_SESSION_KEY=reachy
+
+# Reachy daemon. These settings let ClawBody run on xeon-serv while using
+# Reachy's remote WebRTC media and motion connection.
+ROBOT_HOST=reachy-mini.local
+ROBOT_PORT=8000
+ROBOT_CONNECTION_MODE=network
 
 # ==============================================================================
-# OPTIONAL: Voice Settings
+# REQUIRED: Voice Settings
 # ==============================================================================
-# OpenAI Realtime voice (alloy, echo, fable, onyx, nova, shimmer, cedar)
-OPENAI_VOICE=cedar
+# Chatterbox is the default local provider. Its service uses a named voice
+# profile installed on xeon-serv rather than receiving a secret/reference file
+# on every request.
+TTS_PROVIDER=chatterbox
+CHATTERBOX_URL=http://speech-host.local:8890/v1/audio/speech
+CHATTERBOX_VOICE=default
+# Software playback gain before Reachy's hardware/app volume. Start at 1.0;
+# values above 1.0 boost quiet voices with clipping protection.
+AUDIO_OUTPUT_GAIN=1.0
 
-# OpenAI model for Realtime API
-OPENAI_MODEL=gpt-4o-realtime-preview-2024-12-17
+# If both values are present, ElevenLabs is the automatic Chatterbox fallback.
+# Set TTS_PROVIDER=elevenlabs to make it primary.
+# ELEVENLABS_API_KEY=...
+# ELEVENLABS_VOICE_ID=...
+# ELEVENLABS_MODEL_ID=eleven_flash_v2_5
+
+# Local VAD tuning
+VAD_RMS_THRESHOLD=0.022
+VAD_ACTIVATION_MS=160
+VAD_PREFIX_MS=250
+VAD_SILENCE_MS=650
+VAD_MIN_SPEECH_MS=300
+VAD_MAX_SPEECH_SECONDS=20
+VAD_REQUIRE_HARDWARE_SPEECH=true
+
+# Optional wake phrase gate. When enabled, only transcripts beginning with the
+# configured phrase are sent to OpenClaw; the phrase is stripped from commands.
+ENABLE_WAKE_PHRASE=false
+WAKE_PHRASE=Hey Asmo
+
+# Quiet mail-attention indicator and head double-tap playback.
+ENABLE_ATTENTION_QUEUE=true
+ATTENTION_API_URL=http://192.168.1.238:18790
+ATTENTION_POLL_SECONDS=60
 
 # ==============================================================================
 # OPTIONAL: Features
 # ==============================================================================
 # Enable/disable features (true/false)
 ENABLE_CAMERA=true
 ENABLE_OPENCLAW_TOOLS=true
+ENABLE_FACE_TRACKING=true
+HEAD_TRACKER_TYPE=daemon
+FACE_TRACKING_WEIGHT=0.7
+ENABLE_SOUND_TRACKING=true
+
+# Dedicated body-session latency. "off" is the instant conversational mode;
+# it does not change reasoning settings for Discord, cron, or other sessions.
+OPENCLAW_THINKING=off

.gitignore

@@ -2,6 +2,7 @@
 
 # Environment and secrets
 .env
+.private/
 *.env.local
 
 # Python

EMOTIONS.md

@@ -0,0 +1,87 @@
+# Reachy emotion map
+
+ClawBody exposes the 81 expressive recordings from Pollen's Reachy Mini
+emotions dataset. Exact numbered names always select that recording; a family
+name or conversational alias selects the first (restrained default) variant.
+
+The four additional dataset recordings are utilities, not emotions:
+`mini-deep-sleep`, `toc-toc-toc`, `waiting`, and `wake-mini-up`.
+
+## Positive and social
+
+- amazed: `amazed1` (astonished, impressed, wow)
+- attentive: `attentive1`, `attentive2` (alert, focused, listening)
+- calming: `calming1` (soothing, reassuring)
+- cheerful: `cheerful1` (happy, upbeat)
+- come: `come1` (come here, beckon, invite)
+- curious: `curious1`
+- enthusiastic: `enthusiastic1`, `enthusiastic2` (excited, eager)
+- grateful: `grateful1` (thankful)
+- helpful: `helpful1`, `helpful2` (assist)
+- laughing: `laughing1`, `laughing2` (laugh, amused)
+- loving: `loving1` (love, affectionate)
+- proud: `proud1`, `proud2`, `proud3`
+- relief: `relief1`, `relief2` (relieved)
+- serenity: `serenity1` (peaceful)
+- success: `success1`, `success2` (victory, celebrating)
+- understanding: `understanding1`, `understanding2` (got it, comprehend)
+- welcoming: `welcoming1`, `welcoming2` (welcome, greet)
+- yes: `yes1`, `yes_sad1` (agree, affirmative; “sad yes” selects `yes_sad1`)
+
+## Thinking and uncertainty
+
+- anxiety: `anxiety1` (anxious, nervous, worried)
+- confused: `confused1` (puzzled)
+- incomprehensible: `incomprehensible2` (baffled, don't understand)
+- inquiring: `inquiring1`, `inquiring2`, `inquiring3` (questioning)
+- lost: `lost1`
+- oops: `oops1`, `oops2` (mistake)
+- surprised: `surprised1`, `surprised2` (startled)
+- thoughtful: `thoughtful1`, `thoughtful2` (thinking, contemplative)
+- uncertain: `uncertain1` (unsure, doubtful)
+
+## Low energy and vulnerable
+
+- boredom: `boredom1`, `boredom2` (bored)
+- downcast: `downcast1` (dejected)
+- exhausted: `exhausted1` (wiped out)
+- lonely: `lonely1`
+- resigned: `resigned1` (give up, accept defeat)
+- sad: `sad1`, `sad2` (unhappy)
+- shy: `shy1` (bashful)
+- sleep: `sleep1` (sleepy)
+- tired: `tired1`
+- uncomfortable: `uncomfortable1` (awkward, uneasy)
+
+## Negative and defensive
+
+- contempt: `contempt1` (disdain, dismissive)
+- disgusted: `disgusted1` (grossed out)
+- displeased: `displeased1`, `displeased2` (dissatisfied)
+- fear: `fear1` (afraid)
+- frustrated: `frustrated1`
+- furious: `furious1` (livid)
+- go away: `go_away1` (shoo, leave)
+- impatient: `impatient1`, `impatient2`
+- indifferent: `indifferent1` (meh, apathetic)
+- irritated: `irritated1`, `irritated2` (annoyed)
+- no: `no1`, `no_excited1`, `no_sad1` (decline, refuse; “excited no” and “sad no” select their exact variants)
+- rage: `rage1` (enraged)
+- reprimand: `reprimand1`, `reprimand2`, `reprimand3` (scold, admonish)
+- scared: `scared1` (frightened)
+
+## Theatrical and movement reactions
+
+- dance: `dance1`, `dance2`, `dance3` (legacy emotion-dataset motions; explicit dance requests use the separate official dance library)
+- dying: `dying1` (dramatic death)
+- electric: `electric1` (energized)
+
+## Selection rules
+
+- Exact names win: “express proud three” selects `proud3`.
+- Family names choose the first installed variant: “act proud” selects `proud1`.
+- Common aliases resolve to a family: “show victory” selects `success1`.
+- Authored emotion moves get exclusive body control; face tracking, sound tracking,
+  and speech wobble pause until the move completes.
+- New upstream recordings appear as unmapped until intentionally assigned to a
+  family, preventing silent or nonsensical gesture selection.

NATIVE_APP_REVIEW.md

@@ -0,0 +1,67 @@
+# Native Reachy App Contract Review
+
+Reviewed against the installed Reachy Mini 1.9 SDK and its bundled
+`reachy-mini-app-assistant` validator.
+
+## Confirmed contract
+
+- The project name is `clawbody`; the validator derives the package from the
+  entry point and therefore expects class `ReachyMiniOpenclaw` in
+  `reachy_mini_openclaw.main`.
+- The native class must explicitly inherit `reachy_mini.ReachyMiniApp`.
+- The entry point must be:
+
+  ```toml
+  [project.entry-points."reachy_mini_apps"]
+  clawbody = "reachy_mini_openclaw.main:ReachyMiniOpenclaw"
+  ```
+
+- The SDK owns the `ReachyMini` context, creates a `threading.Event`, and calls
+  `run(reachy_mini, stop_event)` through `ReachyMiniApp.wrapped_run()`.
+- The app must stop when that event is set and must not close/disconnect the
+  SDK-owned robot itself.
+- The existing README frontmatter, root `index.html`, root `style.css`, and src
+  package layout satisfy the validator's static metadata/layout requirements.
+- The official validator finishes by installing the project into an isolated
+  venv, verifying the entry point, and uninstalling it.
+
+## Current defects to correct
+
+1. `ClawBodyApp` neither has the validator-required name nor inherits
+   `ReachyMiniApp`. The current validator fails before installation with the
+   expected entry point `reachy_mini_openclaw.main:ReachyMiniOpenclaw`.
+2. The native wrapper catches and suppresses every exception. That prevents the
+   app manager from observing a failed app and undermines watchdog/restart
+   behavior. Cleanup should occur in `finally`, then the failure must propagate.
+3. The external stop event is only polled by the record/play loops. The
+   conversation handler remains in the `asyncio.gather()` set, so a dashboard
+   stop can leave `run()` waiting indefinitely. A stop watcher must cancel all
+   app tasks promptly.
+4. `ClawBodyCore.stop()` is synchronous but tries to run the asynchronous
+   gateway disconnect with `run_until_complete()` on the current event loop.
+   When called from the native wrapper after `run_until_complete()`, this is
+   fragile; when called while a loop is active, it is invalid. Async shutdown
+   needs a single owning loop and an idempotent lifecycle.
+5. Configuration validation calls `sys.exit(1)` inside an embeddable app class.
+   Native lifecycle errors should raise typed exceptions so the SDK can record
+   `app.error` and manage failure correctly.
+6. Native camera tracking currently builds its daemon URL from
+   `ROBOT_HOST`/`ROBOT_PORT`. On the Wireless body, the SDK-owned robot is local
+   and the daemon-side HTTP API should use localhost (or an explicit native
+   daemon URL), not the remote workstation default.
+7. The package deliberately depends on remote xeon STT/TTS and the existing
+   OpenClaw gateway. Startup must tolerate those services being temporarily
+   unavailable and reconnect with bounded backoff rather than exiting or
+   pretending the app is healthy.
+
+## Required implementation shape
+
+- Introduce `ReachyMiniOpenclaw(ReachyMiniApp)` with the exact entry point.
+- Keep `ClawBodyCore` reusable for CLI/simulator operation.
+- Give the core an idempotent async shutdown path and an explicit external-stop
+  watcher.
+- Let fatal native exceptions propagate after cleanup.
+- Add contract tests for inheritance, entry-point loading, stop behavior,
+  cleanup idempotency, and reconnect/backoff behavior.
+- Preserve the existing `main` agent and `agent:main:reachy` session. Native
+  packaging changes transport/lifecycle only; it must not create Asmo(deux).

NATIVE_DEPLOY_TODO.md

@@ -0,0 +1,80 @@
+# Reachy Native Deployment TODO
+
+Last checked: 2026-07-22
+
+Goal: package and deploy ClawBody as a native Reachy Mini app that starts and recovers without the laptop, while continuing to use the existing OpenClaw `main` agent and `agent:main:reachy` session. Do not create a second Asmo agent.
+
+## Work queue
+
+- [x] **Consolidate reviewer findings and inspect the current native app contract**
+  - Capture the actionable Claude review findings in the repo.
+  - Verify the expected `ReachyMiniApp` class, lifecycle hooks, metadata, and runtime assumptions against the installed Reachy 1.9 tooling.
+  - Exit condition: findings are recorded and the exact contract changes are identified.
+  - Evidence: `NATIVE_APP_REVIEW.md`; installed Reachy 1.9 validator reproduces the entry-point/class failure and documents the required lifecycle corrections.
+
+- [x] **Correct the native Reachy app entry point and lifecycle**
+  - Fix the app class/entry-point contract.
+  - Ensure startup, stop, cleanup, and robot-resource ownership are safe and idempotent.
+  - Preserve the existing xeon STT/TTS and `agent:main:reachy` architecture.
+  - Exit condition: native entry point imports and initializes under the official tooling.
+  - Evidence: `ReachyMiniOpenclaw(ReachyMiniApp)` is the package entry point; external stop events cancel all app tasks, shutdown is async/idempotent, native failures propagate, and native camera tracking uses localhost.
+
+- [x] **Add reconnect and watchdog resilience**
+  - Recover from Reachy daemon, WebRTC, xeon voice service, and OpenClaw gateway interruptions without a laptop.
+  - Use bounded backoff and clean shutdown; do not create restart storms.
+  - Exit condition: automated tests cover disconnect/reconnect and process restart behavior.
+  - Evidence: gateway reconnect, transient STT/TTS retry, microphone/speaker recovery backoff, and native failure propagation to the Reachy supervisor are covered by the 33-test suite.
+
+- [x] **Make native configuration and secrets deployment-safe**
+  - Remove LAN-specific or workstation-only assumptions from the package.
+  - Keep secrets out of git and Hugging Face history.
+  - Document the minimum runtime configuration for Reachy, xeon STT/TTS, and the existing OpenClaw gateway/session.
+  - Exit condition: a clean install can be configured without editing source.
+  - Evidence: service endpoints are explicit required configuration, deployment environment overrides local `.env`, native configuration is documented in `README.md`, and secrets remain environment-only.
+
+- [x] **Run local validation and final code review**
+  - Run the complete test suite.
+  - Run `reachy-mini-app-assistant check` (or the current official equivalent).
+  - Review the final diff for safety, secret leakage, lifecycle correctness, and accidental creation of an Asmo(deux).
+  - Exit condition: validation passes with no unresolved critical findings.
+  - Evidence: UV-managed CPython 3.13.12; 35 tests pass; the Reachy 1.9 app assistant passes its clean install, entry-point, and uninstall checks; package builds succeed; artifact and workspace scans found no credential material; final review fixed executor-thread leakage, un-awaited CLI/Gradio shutdown, and blocking async startup sleeps.
+
+- [x] **Package and publish a private Hugging Face Space**
+  - Use the Hugging Face credentials in 1Password vault `Asmo`, item `3nm3rm35entwmzoc64bznvdbma` at runtime only.
+  - Create/update the private native Reachy app Space and push the validated package.
+  - Never print or persist the token in logs, repo files, shell history, or memory.
+  - Exit condition: the private Space exists and exposes the expected app revision.
+  - Evidence: the scoped write token was read from 1Password at runtime only; the official Reachy 1.9 publish check passed; private Space `asmolebot/clawbodylocal` was created and verified at revision `dc3d1ae0c2d4386ed186476edc6efb388d08c97d` with the expected native app files and no secret-like filenames. The original `asmolebot/clawbody` identifier resolves to the renamed Space through the authenticated Hub API.
+
+- [ ] **Install and configure the app on Reachy Mini**
+  - HOLD: do not begin this item until Chris explicitly signs off on the final deployment.
+  - Install through the Reachy app API/tooling.
+  - Configure it as the startup app if supported by the installed Reachy version.
+  - Keep deployment reversible and record the prior app/startup state.
+  - Exit condition: the app launches on Reachy without the laptop console.
+
+- [ ] **Run laptop-independent end-to-end acceptance tests**
+  - Verify microphone -> xeon STT -> existing OpenClaw Asmo session -> xeon TTS -> speaker.
+  - Verify camera/face/sound tracking, representative emotions, and at least one safe dance.
+  - Restart the app/daemon and confirm recovery.
+  - Exit condition: a complete spoken turn and motion test pass while the laptop is closed/disconnected.
+
+- [ ] **Commit, push, and update the upstream draft PR**
+  - Commit only reviewed source/docs/config changes; exclude private deployment state and credentials.
+  - Push the existing branch and update the draft PR with native-deployment validation results.
+  - Exit condition: clean worktree, pushed commit, and PR reflects the deployed revision.
+
+- [ ] **Begin the separate body-latency/bootstrap optimization**
+  - Only after native deployment is stable, slim the Reachy session bootstrap/tool surface and add first-token/first-audio timing.
+  - Do not create a second agent personality or duplicate long-term memory.
+  - Exit condition: a measured optimization plan is recorded from the native deployment baseline.
+
+## Heartbeat rules
+
+- Advance at most one unchecked item per heartbeat.
+- Read current repo/runtime state before acting; do not redo completed work.
+- Mark an item complete only when its exit condition is evidenced.
+- Continue safe local implementation and validation autonomously.
+- Publishing/staging the private Space is authorized. Installing, launching, or changing startup state on Reachy requires Chris's explicit final sign-off.
+- Report only completion, a new blocker, deployment/recovery state, or a request that genuinely needs Chris.
+- If no material state changed, stay quiet.