From f40a8234c0c7364e838d409109decacda548cfaa Mon Sep 17 00:00:00 2001 From: Tig Date: Tue, 28 Jul 2026 14:55:54 -0600 Subject: [PATCH 1/3] docs: prepare silico to leverage esprec when ready Add host knowledge, lexicon, and first-ship pointers so agents can use tig/esprec for ESP screen capture without redefining GCU sim as QEMU or replacing operator product face confirm. No hard dependency yet. --- AGENTS.md | 18 +++++- BEDSIDE.md | 3 +- silico/doctor.py | 2 +- silico/knowledge/INDEX.md | 1 + silico/knowledge/README.md | 1 + silico/knowledge/esprec.md | 113 ++++++++++++++++++++++++++++++++++ silico/plates/gcu-c/AGENTS.md | 2 + specs/lexicon.md | 16 ++++- specs/silicov1.md | 1 + 9 files changed, 151 insertions(+), 6 deletions(-) create mode 100644 silico/knowledge/esprec.md diff --git a/AGENTS.md b/AGENTS.md index 4e67542..0d9644a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -83,7 +83,7 @@ Context is finite. **Do not** open every manners file into the active window. | Operator language (first prompt orient, first-use term defs, big-step why/where) | **silico AGENTS** + [lexicon](specs/lexicon.md) | Not bedside tenets; domain on top of contract | | COM / first-flash (UF2 **or** esptool) / board identity / metal deploy | **silico BEDSIDE.md** + `silico/knowledge/first-flash.md` + CLI | AGENTS first-ship metal sections may point here; avoid full restatement | | GCU `bedside.toml` pin (sibling paths to silico vendor) | **plate** ships it; GCU keeps it | Do not leave GCU without a pin and then invent a prose path | -| Board/host capability notes (audio, bridges, …) | **silico/knowledge/** (self-improving) | Product must not soft-fork; agents **append** host truths here | +| Board/host capability notes (audio, bridges, screened capture via esprec when ready, …) | **silico/knowledge/** (self-improving) | Product must not soft-fork; agents **append** host truths here; open [esprec.md](silico/knowledge/esprec.md) only for ESP display capture | | Eval rubric / fixtures | **tig/bedside** `eval/` | Run CLI; do not paste rubric into context | | Product domain (idle, vehicle, tunes-as-product, …) | **GCU repo** | Never silico or bedside | @@ -851,6 +851,19 @@ Deploy verify + `FW_VERSION` match prove the **host wrote this build**. They do 3. If product docs name a product face (e.g. M5 front-panel / side LEDs, speaker riff) and the plate only toggles a generic/dev-board pin (e.g. XIAO GPIO LED that is not the product face): **that is HW confusion, not first ship done.** Resolve it with the operator — prefer **`silico board-profile`** pin packs (e.g. `m5go`: side strip **15**, speaker **25**) and `silico board-profile seed` into `firmware/defaults.py` **after** operator confirm of the map; also use parts/spec/board knowledge. Host-test, redeploy, re-confirm observe. Filing a GitHub issue is a tracker, not acceptance. 4. Product face gaps that block observe are **in-scope first-ship metal work**. Silico exists to help the operator through this. Do not label the session “on the metal” / “first ship complete” while an open issue still says the product face is unproven. +##### Screened ESP UIs — esprec (when ready) + +When the product face is a **display** on ESP32-class hardware, agents may use **esprec** ([tig/esprec](https://github.com/tig/esprec) — tuirec for device screens) for **PNG/GIF capture** over the same USB serial path **after** that tool is ready (host CLI + on-device component + upstream unit/QEMU gates green). Detail and readiness bar: [silico/knowledge/esprec.md](silico/knowledge/esprec.md). + +| Do | Do not | +|----|--------| +| Open `esprec.md` only when the GCU has a screen face and you need agent eyes | Invent a private capture stack in silico or the GCU “because esprec is coming” | +| Use capture as **extra** evidence after deploy + app running | Replace **operator** product face confirm on first ship with “I viewed a PNG” | +| Keep GCU **host gate** as pytest/CTest + plate CI | Force QEMU jobs onto every GCU because esprec’s CI uses QEMU | +| Treat esprec as **external tooling** until a second GCU forces a pin | Redefine GCU **sim** (`sim/` HAL plant) as “run under QEMU” | + +Until esprec is ready: operator see/hear remains the metal bar for screens (and LEDs/audio). + ##### GPIO / pin / product face ambiguity → **stop and ask** (mandatory) When you notice (or should notice) that the **pin, LED, speaker, or product face “good”** in firmware does not clearly match the **product board / spec**, you **must not** quietly assume, monologue an “Honesty” paragraph, or only open a GitHub issue. Never call this bare “face.” @@ -891,7 +904,8 @@ bedside ask --id clarify-product-face \ 6. Soft-reset so **main.py runs as the app** (deploy verify uses REPL and parks the loop). If the soft-reset itself will start sound/motion, announce that **before** the reset. If raw REPL fails: product `repl` door or boot window (Ctrl-C may be data). 7. **Operator-observable check:** document the **product face** “good”; confirm with the operator from the bench. If product face ≠ plate generic pin (or mapping is unclear): **clarify with the operator first** (structured ask), then fix, redeploy, re-confirm observe — do not stop at version match or issue-only. 8. Optional: `silico monitor --port COMx --duration 10`. -9. Document `install/` leave-behind (update-path one-liner + product face “good”: LEDs/audio/etc.). +9. Optional (screened ESP, **when esprec is ready**): snapshot/record via esprec for agent eyes — still confirm product face with the operator on first ship ([knowledge/esprec.md](silico/knowledge/esprec.md)). +10. Document `install/` leave-behind (update-path one-liner + product face “good”: LEDs/audio/etc.). Non-Python deploy assets (e.g. audio riffs) may appear in `[deploy].core`; host hygiene skips them as copy-only. diff --git a/BEDSIDE.md b/BEDSIDE.md index cf71891..b881b64 100644 --- a/BEDSIDE.md +++ b/BEDSIDE.md @@ -10,6 +10,7 @@ This file is **not** a fork of the Bedside tenets. | ESP32 USB duplex / console lockout | `silico/knowledge/esp32-usb-serial.md` | | ESP32 audio / DAC / PWM silence | `silico/knowledge/esp32-audio.md` | | SPI IPS color / partial blit | `silico/knowledge/esp32-lcd-ips.md` | +| ESP screen capture for agents (when ready) | `silico/knowledge/esprec.md` ([tig/esprec](https://github.com/tig/esprec)) | | Large binary asset deploy | `silico/knowledge/deploy-assets.md` | | M5GO / Core v2.7 power button | `silico/knowledge/m5go-power.md` | | This file | metal-host glossary only — do **not** also reload the full contract if AGENTS already pinned it | @@ -35,7 +36,7 @@ Host tools and first-ship **stage order** live in **AGENTS.md**. Metal-specific 5. Deploy overwrite only after `bedside ask --id confirm-deploy` (or host UI same contract). 6. **Before write/reset:** clearly tell the operator what the board may do after boot (tones, LEDs, motion, duration) — especially audio products. Permission to overwrite is not a license to startle. 7. After `--verify`, **soft-reset again** so the product boot entry runs (verify parks the app loop). -8. **Operator-observable good:** confirm the human can **see or hear** the documented **product face** for **this** product board (not only `FW_VERSION` over REPL). Always say **product face**, never bare “face.” Plate generic LED on the wrong pin is not acceptance. +8. **Operator-observable good:** confirm the human can **see or hear** the documented **product face** for **this** product board (not only `FW_VERSION` over REPL). Always say **product face**, never bare “face.” Plate generic LED on the wrong pin is not acceptance. Screened ESP: when **esprec** is ready, agents may capture PNG/GIF for evidence — that does **not** replace this operator confirm (`silico/knowledge/esprec.md`). 9. **Pin / product face mismatch → ask first:** if GPIO, LED, or audio that makes up the product face is unclear vs product docs/board, **stop and clarify with the operator** (`bedside ask` / host picker) before assuming, filing-only, or advancing stages. Then fix → redeploy → re-confirm observe. 10. App updates after first-flash: no re-teaching UF2/esptool. diff --git a/silico/doctor.py b/silico/doctor.py index c722b1c..f2fd4a7 100644 --- a/silico/doctor.py +++ b/silico/doctor.py @@ -215,7 +215,7 @@ def run_doctor(*, root: Path | None = None) -> DoctorReport: # Point agents at growing host knowledge (board caps, audio, first-flash). lines.append( - "Host knowledge: silico/knowledge/ (ESP32 audio, first-flash notes). " + "Host knowledge: silico/knowledge/ (ESP32 audio, first-flash, esprec when ready). " "When first ship friction is board/host-generic, add a note there (Make it better)." ) diff --git a/silico/knowledge/INDEX.md b/silico/knowledge/INDEX.md index 9d4d847..f98353f 100644 --- a/silico/knowledge/INDEX.md +++ b/silico/knowledge/INDEX.md @@ -6,6 +6,7 @@ | esp32-usb-serial | ESP32 USB-UART duplex ladder, lockout recovery, UART0 footgun | [esp32-usb-serial.md](esp32-usb-serial.md) | | esp32-audio | ESP32 DAC lifecycle, soft-park silence, smooth PCM / music; IDF dac_continuous queue/drain | [esp32-audio.md](esp32-audio.md) | | esp32-lcd-ips | SPI IPS color pack (R/B, INVON), partial blit | [esp32-lcd-ips.md](esp32-lcd-ips.md) | +| esprec | Agent screen capture (PNG/GIF) for ESP displays — when tig/esprec is ready | [esprec.md](esprec.md) | | m5-core | M5GO/Core face pins, buttons, MPU6886 WHO_AM_I + temp formula | [m5-core.md](m5-core.md) | | deploy-assets | Large binary asset deploy + size verify | [deploy-assets.md](deploy-assets.md) | | m5go-power | M5GO / Core v2.7 red button on/off (double-click off) | [m5go-power.md](m5go-power.md) | diff --git a/silico/knowledge/README.md b/silico/knowledge/README.md index a378f14..7da1fe2 100644 --- a/silico/knowledge/README.md +++ b/silico/knowledge/README.md @@ -10,6 +10,7 @@ Product domain (idle control, drone songs, vehicle acceptance) stays in the **GC |------|------| | ESP32 / DAC / speaker / PWM tone / sample playback | [esp32-audio.md](esp32-audio.md) | | SPI IPS color / INVON / partial paint | [esp32-lcd-ips.md](esp32-lcd-ips.md) | +| Screen capture for agents (PNG/GIF) when tig/esprec is ready | [esprec.md](esprec.md) | | M5GO / Core face pins, buttons, MPU6886 temp | [m5-core.md](m5-core.md) | | Large binary assets (PCM, images) deploy verify | [deploy-assets.md](deploy-assets.md) | | M5GO / Core v2.7 power button | [m5go-power.md](m5go-power.md) | diff --git a/silico/knowledge/esprec.md b/silico/knowledge/esprec.md new file mode 100644 index 0000000..647ba19 --- /dev/null +++ b/silico/knowledge/esprec.md @@ -0,0 +1,113 @@ +# esprec — agent eyes on ESP displays (when ready) + +**Status:** companion tooling is **not** shipped as a silico package feature. +Upstream: [tig/esprec](https://github.com/tig/esprec) (requirements / CI shape; +implementation landing later). Open this file only when the GCU has a **screened +ESP32-class** product face and you need capture, not for LED-only or audio-only +first ship. + +esprec is the [tuirec](https://github.com/tui-cs/tuirec) equivalent for **device +screens**: on command, on-device capture → USB serial → host **PNG** (snapshot) +or **GIF** (short record) so agents can **see** the panel without a camera. + +## Relationship to silico (do not soft-fork) + +| Silico concept | Role | esprec role | +|----------------|------|-------------| +| **product face** | Operator **see/hear** acceptance for first ship | Optional **agent** evidence of the screen face — does **not** replace operator confirm on first ship | +| **sim** / host plant | GCU `sim/` HAL double + pytest/CTest | **Not** esprec. Do not rename GCU sim to “QEMU” | +| **host gate** | Named product gate (`pytest` / `silico gate`) | GCU CI stays this; do **not** force every GCU to run QEMU | +| **QEMU gate** | — | Lives in **esprec’s** CI ([specs/ci.md](https://github.com/tig/esprec/blob/main/specs/ci.md) via [tobozo/esp32-qemu-sim](https://github.com/tobozo/esp32-qemu-sim)): unit → QEMU → optional metal. Proves esprec’s capture path, not every GCU plant | +| **metal** | Real USB + operator-observable face | Real panel still confirms color/timing quirks QEMU cannot prove | + +**Apps stay apps.** Product domain UI stays in the GCU. esprec is **tooling** +(firmware component + host CLI). Pin or vendor only when a GCU needs it — +prefer treating it as an external tool until a second GCU forces a silico pin +(**Extract, then open**). + +## When ready (readiness bar) + +Treat esprec as **usable by agents** only when **all** of these are true: + +1. Upstream ships a host CLI (or library) agents can run non-interactively with + stable flags and exit codes. +2. On-device component has a documented integration surface (raw framebuffer + and/or LVGL snapshot path). +3. Upstream **unit** gate is green without metal. +4. Upstream **QEMU** gate is green (example firmware + capture assertion), or + metal capture is proven and documented if QEMU is temporarily unavailable. +5. `esprec --help` / agent-guide (or equivalent) exists so agents do not read + source to discover the capture command. + +Until then: **do not** invent a parallel capture stack in silico or the GCU +“because esprec is coming.” Use operator observe + product docs. File friction +on [tig/esprec](https://github.com/tig/esprec) if the gap blocks UI work. + +## Agent recipe (after ready) + +**Where we are:** Stage D (hello metal) or UI domain work after deploy. Host +gate is green; board talks; product face includes a **screen**. + +**Why:** Agents cannot trust “UI looks right in source.” A PNG/GIF of the live +buffer is evidence; the operator still owns first-ship product face judgment. + +```text +# After confirmed deploy + app running (soft-reset if verify parked the loop): +# 1. Confirm esprec is installed / on PATH (or GCU-documented path). +# 2. Snapshot (illustrative — use real flags from esprec help when shipped): +esprec snapshot --port COMx --out .silico/esprec/face.png +# 3. Read the PNG (vision / multimodal) and compare to product face “good.” +# 4. For multi-step UI checks, bounded record → GIF (finite frames/duration). +``` + +Rules: + +1. **Announce** before capture if the product UI will change brightness/content + in a surprising way (same spirit as surprising metal effects). +2. Prefer **snapshot** for “what is on screen now?” Prefer **bounded GIF** for + boot/navigation sequences — never unbounded stream that can fill disk. +3. Store captures under **gitignored** paths (e.g. `.silico/esprec/`) unless the + product explicitly wants checked-in docs stills. +4. **Do not** claim metal acceptance from PNG alone on first ship — still ask + the operator whether the documented product face is true on the bench. +5. **Do not** add esprec QEMU jobs to every GCU `ci.yml` by default. Rely on + esprec’s own CI for component honesty; GCU may optionally add a snapshot + step later when the product needs regression visuals. + +## Integration surface (GCU) + +When adopting esprec in a screened GCU: + +1. Link the component per esprec’s firmware-api (when published); keep capture + polite (bounded RAM; product UI continues after capture). +2. Coexist with product serial logging — structured capture must be recoverable + amid log noise. +3. Choose **raw framebuffer** vs **LVGL snapshot** explicitly in product docs / + HAL; do not silently guess. +4. Host path: document the one-liner in product `install/` or `scripts/` only + after the tool is ready (same commands as CI when the GCU opts into capture). +5. Panel path still needs color/partial-paint host knowledge when relevant: + [esp32-lcd-ips.md](esp32-lcd-ips.md), [esp32s3-amoled-1.8.md](esp32s3-amoled-1.8.md). + +## What not to do + +- Treat esprec as silico’s default **sim** (HAL plant) or replace `sim/hal_double`. +- Require QEMU on cloud CI for every MicroPython or LED-only GCU. +- Claim first ship complete from agent-viewed PNG while the operator never + confirmed product face. +- Embed product UI logic into silico or into esprec. +- Build a private camera-on-desk folklore path when esprec is ready and the GCU + already embeds the component. + +## Compound + +If the path is rough (serial framing, QEMU serial attach, LVGL major drift): +prefer a durable fix or issue on **tig/esprec**; promote a silico knowledge note +here only when the truth is host/board-generic beyond the tool itself. + +## Spec map (upstream) + +| Spec | Scope | +|------|--------| +| [esprec specs/spec.md](https://github.com/tig/esprec/blob/main/specs/spec.md) | Product requirements | +| [esprec specs/ci.md](https://github.com/tig/esprec/blob/main/specs/ci.md) | Unit → QEMU → optional metal | diff --git a/silico/plates/gcu-c/AGENTS.md b/silico/plates/gcu-c/AGENTS.md index ba1af78..5534572 100644 --- a/silico/plates/gcu-c/AGENTS.md +++ b/silico/plates/gcu-c/AGENTS.md @@ -64,6 +64,8 @@ silico deploy --port COMx --yes --verify ESP-IDF must be installed (`idf.py` or `IDF_PATH`). First flash and update flash are the same image path. +After deploy, **operator-confirm product face** on the bench (see silico root Stage D1). If this GCU’s face is a **screen**, open silico `knowledge/esprec.md` when **esprec** is ready for optional PNG/GIF agent capture — not a substitute for operator confirm, and not a reason to add QEMU to this GCU’s host gate by default. + ## HAL seam Portable domain under `include/` + `src/` must not include freertos / esp_* / driver headers. diff --git a/specs/lexicon.md b/specs/lexicon.md index d2ec94a..f91bc0e 100644 --- a/specs/lexicon.md +++ b/specs/lexicon.md @@ -80,7 +80,9 @@ The **human-observable product indication** on the [GCU](#gcu) after the app is Always say the full term **product face**. Never shorten to bare “face.” On first use in a session, define it (see Silico `AGENTS.md` operator language). First-ship [metal](#metal) acceptance requires the operator to confirm the product face, not only a version string. -See also: [metal](#metal), [host gate](#host-gate), [Help the operator](#help-the-operator), [first ship](#first-ship). +For **screened** ESP32-class products, agents may later use [esprec](#esprec) (when ready) as optional eyes on the panel; that does **not** replace operator product face confirm on first ship. + +See also: [metal](#metal), [host gate](#host-gate), [Help the operator](#help-the-operator), [first ship](#first-ship), [esprec](#esprec). ### first ship @@ -149,7 +151,17 @@ See also: [version identity](#version-identity). Also **host plant** / **closed-loop plant.** [Host](#host)-only simulation of the product world used for regression without a board. Never deployed to the device. Complements [metal](#metal); does not replace [host gate](#host-gate) or real USB on first ship. -See also: [HAL](#hal), [host-first](#host-first). +Default GCU shape: `sim/` HAL doubles + host tests (pytest or CTest) — **not** full SoC emulation. Companion tools (e.g. [esprec](#esprec) QEMU gates for *their* firmware path) may use QEMU; that does **not** redefine Silico **sim** for every GCU. + +See also: [HAL](#hal), [host-first](#host-first), [esprec](#esprec). + +### esprec + +**External companion tooling** ([tig/esprec](https://github.com/tig/esprec)): on-device screen capture over USB serial to host **PNG** / **GIF** so agents can see ESP display UIs (tuirec for panels). **Not** a GCU and **not** the Silico package. When ready: optional agent evidence for screened product faces. Until ready: do not invent a parallel capture stack in the spine. + +Host knowledge: [silico/knowledge/esprec.md](../silico/knowledge/esprec.md). Does not replace [product face](#product-face) operator confirm or GCU [sim](#sim). + +See also: [product face](#product-face), [agent-first CLI](#agent-first-cli), [ESP32-class](#esp32-class). --- diff --git a/specs/silicov1.md b/specs/silicov1.md index f825222..6b386d9 100644 --- a/specs/silicov1.md +++ b/specs/silicov1.md @@ -317,6 +317,7 @@ Until then, Quilan owns modem, credentials, and uplink protocol in private app c 8. Minimal package layout so `pip install` of a tag works (make-PR-true; pre-alpha until then). 9. CLI verbs (spec now; see tui-cs/cli design discussion) and first ship rehearsal harness. 10. MicroPython host-sim stack ([issue #3](https://github.com/tig/silico/issues/3)); integrity for beta ([issue #4](https://github.com/tig/silico/issues/4)). +11. **esprec** ([tig/esprec](https://github.com/tig/esprec)): when ready, agent screen capture for screened ESP GCUs. Spine holds knowledge + first-ship pointers only until a second GCU forces a pin/CLI verb (**Extract, then open**). Do not redefine GCU **sim** as QEMU; esprec’s own CI may use QEMU for its firmware path. ## 16. Acceptance for silico v1 From 43c8383f1110a8b881545b7d29a01b000b2fb9f0 Mon Sep 17 00:00:00 2001 From: Tig Date: Tue, 28 Jul 2026 15:01:12 -0600 Subject: [PATCH 2/3] docs(esprec): drop broken AMOLED knowledge link Codex P2: esp32s3-amoled-1.8.md is not on main; keep only in-tree panel pointers. --- silico/knowledge/esprec.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/silico/knowledge/esprec.md b/silico/knowledge/esprec.md index 647ba19..b3fa4e5 100644 --- a/silico/knowledge/esprec.md +++ b/silico/knowledge/esprec.md @@ -87,7 +87,8 @@ When adopting esprec in a screened GCU: 4. Host path: document the one-liner in product `install/` or `scripts/` only after the tool is ready (same commands as CI when the GCU opts into capture). 5. Panel path still needs color/partial-paint host knowledge when relevant: - [esp32-lcd-ips.md](esp32-lcd-ips.md), [esp32s3-amoled-1.8.md](esp32s3-amoled-1.8.md). + [esp32-lcd-ips.md](esp32-lcd-ips.md) (and any other **in-tree** panel topics under + `silico/knowledge/` for the board class — do not link unmerged topic files). ## What not to do From d2e497a18e6848d5d4f58c53c271490f4f6543e9 Mon Sep 17 00:00:00 2001 From: Tig Date: Tue, 28 Jul 2026 15:20:33 -0600 Subject: [PATCH 3/3] =?UTF-8?q?docs(esprec):=20mark=20ready=20=E2=80=94=20?= =?UTF-8?q?real=20CLI=20recipe=20and=20metal=20path?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update knowledge/AGENTS/lexicon for implemented tig/esprec; keep sim/QEMU boundaries and operator product-face confirm. --- AGENTS.md | 12 ++-- silico/doctor.py | 2 +- silico/knowledge/esprec.md | 134 +++++++++++++++++-------------------- specs/lexicon.md | 2 +- specs/silicov1.md | 2 +- 5 files changed, 69 insertions(+), 83 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0d9644a..6e66d3b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -851,19 +851,17 @@ Deploy verify + `FW_VERSION` match prove the **host wrote this build**. They do 3. If product docs name a product face (e.g. M5 front-panel / side LEDs, speaker riff) and the plate only toggles a generic/dev-board pin (e.g. XIAO GPIO LED that is not the product face): **that is HW confusion, not first ship done.** Resolve it with the operator — prefer **`silico board-profile`** pin packs (e.g. `m5go`: side strip **15**, speaker **25**) and `silico board-profile seed` into `firmware/defaults.py` **after** operator confirm of the map; also use parts/spec/board knowledge. Host-test, redeploy, re-confirm observe. Filing a GitHub issue is a tracker, not acceptance. 4. Product face gaps that block observe are **in-scope first-ship metal work**. Silico exists to help the operator through this. Do not label the session “on the metal” / “first ship complete” while an open issue still says the product face is unproven. -##### Screened ESP UIs — esprec (when ready) +##### Screened ESP UIs — esprec (**ready**) -When the product face is a **display** on ESP32-class hardware, agents may use **esprec** ([tig/esprec](https://github.com/tig/esprec) — tuirec for device screens) for **PNG/GIF capture** over the same USB serial path **after** that tool is ready (host CLI + on-device component + upstream unit/QEMU gates green). Detail and readiness bar: [silico/knowledge/esprec.md](silico/knowledge/esprec.md). +When the product face is a **display** on ESP32-class hardware, agents use **esprec** ([tig/esprec](https://github.com/tig/esprec) — tuirec *analogue* for device screens) for **PNG/GIF capture** over USB serial: `pip install -e ../esprec`, then `esprec snapshot --port COMx -o face.png`. Detail: [silico/knowledge/esprec.md](silico/knowledge/esprec.md). | Do | Do not | |----|--------| -| Open `esprec.md` only when the GCU has a screen face and you need agent eyes | Invent a private capture stack in silico or the GCU “because esprec is coming” | +| Open `esprec.md` only when the GCU has a screen face and you need agent eyes | Invent a private capture stack when esprec already covers the wire | | Use capture as **extra** evidence after deploy + app running | Replace **operator** product face confirm on first ship with “I viewed a PNG” | -| Keep GCU **host gate** as pytest/CTest + plate CI | Force QEMU jobs onto every GCU because esprec’s CI uses QEMU | +| Keep GCU **host gate** as pytest/CTest + plate CI | Force QEMU jobs onto every GCU because esprec may use QEMU in *its* CI | | Treat esprec as **external tooling** until a second GCU forces a pin | Redefine GCU **sim** (`sim/` HAL plant) as “run under QEMU” | -Until esprec is ready: operator see/hear remains the metal bar for screens (and LEDs/audio). - ##### GPIO / pin / product face ambiguity → **stop and ask** (mandatory) When you notice (or should notice) that the **pin, LED, speaker, or product face “good”** in firmware does not clearly match the **product board / spec**, you **must not** quietly assume, monologue an “Honesty” paragraph, or only open a GitHub issue. Never call this bare “face.” @@ -904,7 +902,7 @@ bedside ask --id clarify-product-face \ 6. Soft-reset so **main.py runs as the app** (deploy verify uses REPL and parks the loop). If the soft-reset itself will start sound/motion, announce that **before** the reset. If raw REPL fails: product `repl` door or boot window (Ctrl-C may be data). 7. **Operator-observable check:** document the **product face** “good”; confirm with the operator from the bench. If product face ≠ plate generic pin (or mapping is unclear): **clarify with the operator first** (structured ask), then fix, redeploy, re-confirm observe — do not stop at version match or issue-only. 8. Optional: `silico monitor --port COMx --duration 10`. -9. Optional (screened ESP, **when esprec is ready**): snapshot/record via esprec for agent eyes — still confirm product face with the operator on first ship ([knowledge/esprec.md](silico/knowledge/esprec.md)). +9. Optional (screened ESP): `esprec snapshot --port COMx -o …` for agent eyes — still confirm product face with the operator on first ship ([knowledge/esprec.md](silico/knowledge/esprec.md)). 10. Document `install/` leave-behind (update-path one-liner + product face “good”: LEDs/audio/etc.). Non-Python deploy assets (e.g. audio riffs) may appear in `[deploy].core`; host hygiene skips them as copy-only. diff --git a/silico/doctor.py b/silico/doctor.py index f2fd4a7..17840b4 100644 --- a/silico/doctor.py +++ b/silico/doctor.py @@ -215,7 +215,7 @@ def run_doctor(*, root: Path | None = None) -> DoctorReport: # Point agents at growing host knowledge (board caps, audio, first-flash). lines.append( - "Host knowledge: silico/knowledge/ (ESP32 audio, first-flash, esprec when ready). " + "Host knowledge: silico/knowledge/ (ESP32 audio, first-flash, esprec for screens). " "When first ship friction is board/host-generic, add a note there (Make it better)." ) diff --git a/silico/knowledge/esprec.md b/silico/knowledge/esprec.md index b3fa4e5..42bc454 100644 --- a/silico/knowledge/esprec.md +++ b/silico/knowledge/esprec.md @@ -1,14 +1,14 @@ -# esprec — agent eyes on ESP displays (when ready) +# esprec — agent eyes on ESP displays (**ready**) -**Status:** companion tooling is **not** shipped as a silico package feature. -Upstream: [tig/esprec](https://github.com/tig/esprec) (requirements / CI shape; -implementation landing later). Open this file only when the GCU has a **screened -ESP32-class** product face and you need capture, not for LED-only or audio-only +**Status:** **implemented** as external tooling ([tig/esprec](https://github.com/tig/esprec)). +Not a silico package feature. Open this file only when the GCU has a **screened +ESP32-class** product face and you need capture — not for LED-only or audio-only first ship. -esprec is the [tuirec](https://github.com/tui-cs/tuirec) equivalent for **device -screens**: on command, on-device capture → USB serial → host **PNG** (snapshot) -or **GIF** (short record) so agents can **see** the panel without a camera. +esprec is the [tuirec](https://github.com/tui-cs/tuirec) *analogue* for **device +screens** (mission kinship, not a port of tuirec’s PTY/cast pipeline): on +command, on-device capture → USB serial → host **PNG** (snapshot) or **GIF** +(keyframe or continuous sequence). ## Relationship to silico (do not soft-fork) @@ -17,33 +17,45 @@ or **GIF** (short record) so agents can **see** the panel without a camera. | **product face** | Operator **see/hear** acceptance for first ship | Optional **agent** evidence of the screen face — does **not** replace operator confirm on first ship | | **sim** / host plant | GCU `sim/` HAL double + pytest/CTest | **Not** esprec. Do not rename GCU sim to “QEMU” | | **host gate** | Named product gate (`pytest` / `silico gate`) | GCU CI stays this; do **not** force every GCU to run QEMU | -| **QEMU gate** | — | Lives in **esprec’s** CI ([specs/ci.md](https://github.com/tig/esprec/blob/main/specs/ci.md) via [tobozo/esp32-qemu-sim](https://github.com/tobozo/esp32-qemu-sim)): unit → QEMU → optional metal. Proves esprec’s capture path, not every GCU plant | -| **metal** | Real USB + operator-observable face | Real panel still confirms color/timing quirks QEMU cannot prove | +| **QEMU gate** | — | esprec’s own CI ladder (unit required; QEMU example may be follow-up). Proves esprec’s capture path, not every GCU plant | +| **metal** | Real USB + operator-observable face | Real panel still confirms color/timing quirks | **Apps stay apps.** Product domain UI stays in the GCU. esprec is **tooling** -(firmware component + host CLI). Pin or vendor only when a GCU needs it — -prefer treating it as an external tool until a second GCU forces a silico pin -(**Extract, then open**). +(firmware component + host CLI). Prefer sibling clone + `pip install -e` until +a second GCU forces a silico pin (**Extract, then open**). -## When ready (readiness bar) +## Readiness (current) -Treat esprec as **usable by agents** only when **all** of these are true: +| Bar | State | +|-----|--------| +| Host CLI (`esprec snapshot` / `record` / `agent-guide`) | **yes** | +| On-device component (`component/esprec`, `esprec_emit_rgb565_spi_be`) | **yes** | +| Unit gate `python -m pytest -q` (protocol integrity, PNG/GIF, fake device) | **yes** | +| QEMU CI example | **follow-up** if env lacks it — metal + unit are honest for firmware path | +| Agent guide | `esprec agent-guide` | -1. Upstream ships a host CLI (or library) agents can run non-interactively with - stable flags and exit codes. -2. On-device component has a documented integration surface (raw framebuffer - and/or LVGL snapshot path). -3. Upstream **unit** gate is green without metal. -4. Upstream **QEMU** gate is green (example firmware + capture assertion), or - metal capture is proven and documented if QEMU is temporarily unavailable. -5. `esprec --help` / agent-guide (or equivalent) exists so agents do not read - source to discover the capture command. +## Install / invoke -Until then: **do not** invent a parallel capture stack in silico or the GCU -“because esprec is coming.” Use operator observe + product docs. File friction -on [tig/esprec](https://github.com/tig/esprec) if the gap blocks UI work. +```text +# sibling layout (typical): …/tig/esprec next to …/tig/ +python -m pip install -e "../esprec[dev]" +esprec agent-guide +esprec snapshot --fake -o face.png # offline +esprec snapshot --port COMx -o face.png # metal still +esprec record --port COMx --frames 5 --hz 2 -o clip.gif +# named unit gate: +python -m pytest -q # inside the esprec checkout +``` + +Device commands: `esprec shot` or `shot` (alias). Wire: **ESPREC1** header + +base64 raster + end line; CRC covers **metadata + raster** (fail closed on +truncate / header tamper). Legacy `SHOT` (pixels-only CRC) still decodes. + +**Serial open:** esprec sets DTR/RTS low before open so ESP auto-reset does not +reboot mid-session (black unpainted shadow). Prefer **one open session** for +btn inject + multiple snaps (see esprec `scripts/xuss_c_screen_scenario.py`). -## Agent recipe (after ready) +## Agent recipe **Where we are:** Stage D (hello metal) or UI domain work after deploy. Host gate is green; board talks; product face includes a **screen**. @@ -53,62 +65,38 @@ buffer is evidence; the operator still owns first-ship product face judgment. ```text # After confirmed deploy + app running (soft-reset if verify parked the loop): -# 1. Confirm esprec is installed / on PATH (or GCU-documented path). -# 2. Snapshot (illustrative — use real flags from esprec help when shipped): -esprec snapshot --port COMx --out .silico/esprec/face.png -# 3. Read the PNG (vision / multimodal) and compare to product face “good.” -# 4. For multi-step UI checks, bounded record → GIF (finite frames/duration). +esprec snapshot --port COMx --command shot -o .silico/esprec/face.png +# Read the PNG (vision) vs product face “good.” +# Multi-step: keep one serial session; settle after each product action; then snap. ``` Rules: -1. **Announce** before capture if the product UI will change brightness/content - in a surprising way (same spirit as surprising metal effects). +1. **Announce** before capture if UI will change brightness/content surprisingly. 2. Prefer **snapshot** for “what is on screen now?” Prefer **bounded GIF** for - boot/navigation sequences — never unbounded stream that can fill disk. -3. Store captures under **gitignored** paths (e.g. `.silico/esprec/`) unless the - product explicitly wants checked-in docs stills. -4. **Do not** claim metal acceptance from PNG alone on first ship — still ask - the operator whether the documented product face is true on the bench. -5. **Do not** add esprec QEMU jobs to every GCU `ci.yml` by default. Rely on - esprec’s own CI for component honesty; GCU may optionally add a snapshot - step later when the product needs regression visuals. + sequences — never unbounded streams. +3. Store under **gitignored** paths (e.g. `.silico/esprec/`) unless product docs want stills. +4. **Do not** claim metal acceptance from PNG alone on first ship — still ask the operator. +5. **Do not** add esprec QEMU jobs to every GCU `ci.yml` by default. +6. If PNG disagrees with the glass: **pipeline first** (integrity, packing, cooked serial), not product folklore. ## Integration surface (GCU) -When adopting esprec in a screened GCU: - -1. Link the component per esprec’s firmware-api (when published); keep capture - polite (bounded RAM; product UI continues after capture). -2. Coexist with product serial logging — structured capture must be recoverable - amid log noise. -3. Choose **raw framebuffer** vs **LVGL snapshot** explicitly in product docs / - HAL; do not silently guess. -4. Host path: document the one-liner in product `install/` or `scripts/` only - after the tool is ready (same commands as CI when the GCU opts into capture). -5. Panel path still needs color/partial-paint host knowledge when relevant: - [esp32-lcd-ips.md](esp32-lcd-ips.md) (and any other **in-tree** panel topics under - `silico/knowledge/` for the board class — do not link unmerged topic files). +1. EXTRA_COMPONENT_DIRS → `esprec/component` (sibling clone) or vendor `component/esprec`. +2. Maintain a full-frame **shadow** RGB565 (`spi_be` packing as on panel DMA). +3. On `shot` / `esprec shot`: hush logs, call `esprec_emit_rgb565_spi_be(shadow, w, h)`. +4. Document host one-liner in product `install/` when capture is part of the update path. +5. Panel color/partial paint: [esp32-lcd-ips.md](esp32-lcd-ips.md) when relevant. ## What not to do -- Treat esprec as silico’s default **sim** (HAL plant) or replace `sim/hal_double`. -- Require QEMU on cloud CI for every MicroPython or LED-only GCU. -- Claim first ship complete from agent-viewed PNG while the operator never - confirmed product face. -- Embed product UI logic into silico or into esprec. -- Build a private camera-on-desk folklore path when esprec is ready and the GCU - already embeds the component. +- Treat esprec as silico’s default **sim** (HAL plant). +- Require QEMU on cloud CI for every GCU. +- Claim first ship complete from agent-viewed PNG without operator product face confirm. +- Embed product UI logic into silico or esprec. +- Reopen serial with default DTR/RTS between every snap (resets ESP, black frames). ## Compound -If the path is rough (serial framing, QEMU serial attach, LVGL major drift): -prefer a durable fix or issue on **tig/esprec**; promote a silico knowledge note -here only when the truth is host/board-generic beyond the tool itself. - -## Spec map (upstream) - -| Spec | Scope | -|------|--------| -| [esprec specs/spec.md](https://github.com/tig/esprec/blob/main/specs/spec.md) | Product requirements | -| [esprec specs/ci.md](https://github.com/tig/esprec/blob/main/specs/ci.md) | Unit → QEMU → optional metal | +Protocol/CRC/serial friction → fix **tig/esprec**. Board-generic panel notes → +this knowledge tree. Product domain → GCU. diff --git a/specs/lexicon.md b/specs/lexicon.md index f91bc0e..1f918af 100644 --- a/specs/lexicon.md +++ b/specs/lexicon.md @@ -157,7 +157,7 @@ See also: [HAL](#hal), [host-first](#host-first), [esprec](#esprec). ### esprec -**External companion tooling** ([tig/esprec](https://github.com/tig/esprec)): on-device screen capture over USB serial to host **PNG** / **GIF** so agents can see ESP display UIs (tuirec for panels). **Not** a GCU and **not** the Silico package. When ready: optional agent evidence for screened product faces. Until ready: do not invent a parallel capture stack in the spine. +**External companion tooling** ([tig/esprec](https://github.com/tig/esprec)): on-device screen capture over USB serial to host **PNG** / **GIF** so agents can see ESP display UIs (tuirec *analogue* for panels). **Not** a GCU and **not** the Silico package. **Ready** for agent use via `esprec snapshot|record` + on-device component; optional agent evidence for screened product faces. Host knowledge: [silico/knowledge/esprec.md](../silico/knowledge/esprec.md). Does not replace [product face](#product-face) operator confirm or GCU [sim](#sim). diff --git a/specs/silicov1.md b/specs/silicov1.md index 6b386d9..b6f6f0f 100644 --- a/specs/silicov1.md +++ b/specs/silicov1.md @@ -317,7 +317,7 @@ Until then, Quilan owns modem, credentials, and uplink protocol in private app c 8. Minimal package layout so `pip install` of a tag works (make-PR-true; pre-alpha until then). 9. CLI verbs (spec now; see tui-cs/cli design discussion) and first ship rehearsal harness. 10. MicroPython host-sim stack ([issue #3](https://github.com/tig/silico/issues/3)); integrity for beta ([issue #4](https://github.com/tig/silico/issues/4)). -11. **esprec** ([tig/esprec](https://github.com/tig/esprec)): when ready, agent screen capture for screened ESP GCUs. Spine holds knowledge + first-ship pointers only until a second GCU forces a pin/CLI verb (**Extract, then open**). Do not redefine GCU **sim** as QEMU; esprec’s own CI may use QEMU for its firmware path. +11. **esprec** ([tig/esprec](https://github.com/tig/esprec)): agent screen capture for screened ESP GCUs is **implemented** externally; spine holds knowledge + first-ship pointers. Pin/CLI verb only if a second GCU forces it (**Extract, then open**). Do not redefine GCU **sim** as QEMU. ## 16. Acceptance for silico v1