diff --git a/AGENTS.md b/AGENTS.md index 4e67542..6e66d3b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -83,7 +83,7 @@ Context is finite. **Do not** open every manners file into the active window. | Operator language (first prompt orient, first-use term defs, big-step why/where) | **silico AGENTS** + [lexicon](specs/lexicon.md) | Not bedside tenets; domain on top of contract | | COM / first-flash (UF2 **or** esptool) / board identity / metal deploy | **silico BEDSIDE.md** + `silico/knowledge/first-flash.md` + CLI | AGENTS first-ship metal sections may point here; avoid full restatement | | GCU `bedside.toml` pin (sibling paths to silico vendor) | **plate** ships it; GCU keeps it | Do not leave GCU without a pin and then invent a prose path | -| Board/host capability notes (audio, bridges, …) | **silico/knowledge/** (self-improving) | Product must not soft-fork; agents **append** host truths here | +| Board/host capability notes (audio, bridges, screened capture via esprec when ready, …) | **silico/knowledge/** (self-improving) | Product must not soft-fork; agents **append** host truths here; open [esprec.md](silico/knowledge/esprec.md) only for ESP display capture | | Eval rubric / fixtures | **tig/bedside** `eval/` | Run CLI; do not paste rubric into context | | Product domain (idle, vehicle, tunes-as-product, …) | **GCU repo** | Never silico or bedside | @@ -851,6 +851,17 @@ Deploy verify + `FW_VERSION` match prove the **host wrote this build**. They do 3. If product docs name a product face (e.g. M5 front-panel / side LEDs, speaker riff) and the plate only toggles a generic/dev-board pin (e.g. XIAO GPIO LED that is not the product face): **that is HW confusion, not first ship done.** Resolve it with the operator — prefer **`silico board-profile`** pin packs (e.g. `m5go`: side strip **15**, speaker **25**) and `silico board-profile seed` into `firmware/defaults.py` **after** operator confirm of the map; also use parts/spec/board knowledge. Host-test, redeploy, re-confirm observe. Filing a GitHub issue is a tracker, not acceptance. 4. Product face gaps that block observe are **in-scope first-ship metal work**. Silico exists to help the operator through this. Do not label the session “on the metal” / “first ship complete” while an open issue still says the product face is unproven. +##### Screened ESP UIs — esprec (**ready**) + +When the product face is a **display** on ESP32-class hardware, agents use **esprec** ([tig/esprec](https://github.com/tig/esprec) — tuirec *analogue* for device screens) for **PNG/GIF capture** over USB serial: `pip install -e ../esprec`, then `esprec snapshot --port COMx -o face.png`. Detail: [silico/knowledge/esprec.md](silico/knowledge/esprec.md). + +| Do | Do not | +|----|--------| +| Open `esprec.md` only when the GCU has a screen face and you need agent eyes | Invent a private capture stack when esprec already covers the wire | +| Use capture as **extra** evidence after deploy + app running | Replace **operator** product face confirm on first ship with “I viewed a PNG” | +| Keep GCU **host gate** as pytest/CTest + plate CI | Force QEMU jobs onto every GCU because esprec may use QEMU in *its* CI | +| Treat esprec as **external tooling** until a second GCU forces a pin | Redefine GCU **sim** (`sim/` HAL plant) as “run under QEMU” | + ##### GPIO / pin / product face ambiguity → **stop and ask** (mandatory) When you notice (or should notice) that the **pin, LED, speaker, or product face “good”** in firmware does not clearly match the **product board / spec**, you **must not** quietly assume, monologue an “Honesty” paragraph, or only open a GitHub issue. Never call this bare “face.” @@ -891,7 +902,8 @@ bedside ask --id clarify-product-face \ 6. Soft-reset so **main.py runs as the app** (deploy verify uses REPL and parks the loop). If the soft-reset itself will start sound/motion, announce that **before** the reset. If raw REPL fails: product `repl` door or boot window (Ctrl-C may be data). 7. **Operator-observable check:** document the **product face** “good”; confirm with the operator from the bench. If product face ≠ plate generic pin (or mapping is unclear): **clarify with the operator first** (structured ask), then fix, redeploy, re-confirm observe — do not stop at version match or issue-only. 8. Optional: `silico monitor --port COMx --duration 10`. -9. Document `install/` leave-behind (update-path one-liner + product face “good”: LEDs/audio/etc.). +9. Optional (screened ESP): `esprec snapshot --port COMx -o …` for agent eyes — still confirm product face with the operator on first ship ([knowledge/esprec.md](silico/knowledge/esprec.md)). +10. Document `install/` leave-behind (update-path one-liner + product face “good”: LEDs/audio/etc.). Non-Python deploy assets (e.g. audio riffs) may appear in `[deploy].core`; host hygiene skips them as copy-only. diff --git a/BEDSIDE.md b/BEDSIDE.md index cf71891..b881b64 100644 --- a/BEDSIDE.md +++ b/BEDSIDE.md @@ -10,6 +10,7 @@ This file is **not** a fork of the Bedside tenets. | ESP32 USB duplex / console lockout | `silico/knowledge/esp32-usb-serial.md` | | ESP32 audio / DAC / PWM silence | `silico/knowledge/esp32-audio.md` | | SPI IPS color / partial blit | `silico/knowledge/esp32-lcd-ips.md` | +| ESP screen capture for agents (when ready) | `silico/knowledge/esprec.md` ([tig/esprec](https://github.com/tig/esprec)) | | Large binary asset deploy | `silico/knowledge/deploy-assets.md` | | M5GO / Core v2.7 power button | `silico/knowledge/m5go-power.md` | | This file | metal-host glossary only — do **not** also reload the full contract if AGENTS already pinned it | @@ -35,7 +36,7 @@ Host tools and first-ship **stage order** live in **AGENTS.md**. Metal-specific 5. Deploy overwrite only after `bedside ask --id confirm-deploy` (or host UI same contract). 6. **Before write/reset:** clearly tell the operator what the board may do after boot (tones, LEDs, motion, duration) — especially audio products. Permission to overwrite is not a license to startle. 7. After `--verify`, **soft-reset again** so the product boot entry runs (verify parks the app loop). -8. **Operator-observable good:** confirm the human can **see or hear** the documented **product face** for **this** product board (not only `FW_VERSION` over REPL). Always say **product face**, never bare “face.” Plate generic LED on the wrong pin is not acceptance. +8. **Operator-observable good:** confirm the human can **see or hear** the documented **product face** for **this** product board (not only `FW_VERSION` over REPL). Always say **product face**, never bare “face.” Plate generic LED on the wrong pin is not acceptance. Screened ESP: when **esprec** is ready, agents may capture PNG/GIF for evidence — that does **not** replace this operator confirm (`silico/knowledge/esprec.md`). 9. **Pin / product face mismatch → ask first:** if GPIO, LED, or audio that makes up the product face is unclear vs product docs/board, **stop and clarify with the operator** (`bedside ask` / host picker) before assuming, filing-only, or advancing stages. Then fix → redeploy → re-confirm observe. 10. App updates after first-flash: no re-teaching UF2/esptool. diff --git a/silico/doctor.py b/silico/doctor.py index c722b1c..17840b4 100644 --- a/silico/doctor.py +++ b/silico/doctor.py @@ -215,7 +215,7 @@ def run_doctor(*, root: Path | None = None) -> DoctorReport: # Point agents at growing host knowledge (board caps, audio, first-flash). lines.append( - "Host knowledge: silico/knowledge/ (ESP32 audio, first-flash notes). " + "Host knowledge: silico/knowledge/ (ESP32 audio, first-flash, esprec for screens). " "When first ship friction is board/host-generic, add a note there (Make it better)." ) diff --git a/silico/knowledge/INDEX.md b/silico/knowledge/INDEX.md index 9d4d847..f98353f 100644 --- a/silico/knowledge/INDEX.md +++ b/silico/knowledge/INDEX.md @@ -6,6 +6,7 @@ | esp32-usb-serial | ESP32 USB-UART duplex ladder, lockout recovery, UART0 footgun | [esp32-usb-serial.md](esp32-usb-serial.md) | | esp32-audio | ESP32 DAC lifecycle, soft-park silence, smooth PCM / music; IDF dac_continuous queue/drain | [esp32-audio.md](esp32-audio.md) | | esp32-lcd-ips | SPI IPS color pack (R/B, INVON), partial blit | [esp32-lcd-ips.md](esp32-lcd-ips.md) | +| esprec | Agent screen capture (PNG/GIF) for ESP displays — when tig/esprec is ready | [esprec.md](esprec.md) | | m5-core | M5GO/Core face pins, buttons, MPU6886 WHO_AM_I + temp formula | [m5-core.md](m5-core.md) | | deploy-assets | Large binary asset deploy + size verify | [deploy-assets.md](deploy-assets.md) | | m5go-power | M5GO / Core v2.7 red button on/off (double-click off) | [m5go-power.md](m5go-power.md) | diff --git a/silico/knowledge/README.md b/silico/knowledge/README.md index a378f14..7da1fe2 100644 --- a/silico/knowledge/README.md +++ b/silico/knowledge/README.md @@ -10,6 +10,7 @@ Product domain (idle control, drone songs, vehicle acceptance) stays in the **GC |------|------| | ESP32 / DAC / speaker / PWM tone / sample playback | [esp32-audio.md](esp32-audio.md) | | SPI IPS color / INVON / partial paint | [esp32-lcd-ips.md](esp32-lcd-ips.md) | +| Screen capture for agents (PNG/GIF) when tig/esprec is ready | [esprec.md](esprec.md) | | M5GO / Core face pins, buttons, MPU6886 temp | [m5-core.md](m5-core.md) | | Large binary assets (PCM, images) deploy verify | [deploy-assets.md](deploy-assets.md) | | M5GO / Core v2.7 power button | [m5go-power.md](m5go-power.md) | diff --git a/silico/knowledge/esprec.md b/silico/knowledge/esprec.md new file mode 100644 index 0000000..42bc454 --- /dev/null +++ b/silico/knowledge/esprec.md @@ -0,0 +1,102 @@ +# esprec — agent eyes on ESP displays (**ready**) + +**Status:** **implemented** as external tooling ([tig/esprec](https://github.com/tig/esprec)). +Not a silico package feature. Open this file only when the GCU has a **screened +ESP32-class** product face and you need capture — not for LED-only or audio-only +first ship. + +esprec is the [tuirec](https://github.com/tui-cs/tuirec) *analogue* for **device +screens** (mission kinship, not a port of tuirec’s PTY/cast pipeline): on +command, on-device capture → USB serial → host **PNG** (snapshot) or **GIF** +(keyframe or continuous sequence). + +## Relationship to silico (do not soft-fork) + +| Silico concept | Role | esprec role | +|----------------|------|-------------| +| **product face** | Operator **see/hear** acceptance for first ship | Optional **agent** evidence of the screen face — does **not** replace operator confirm on first ship | +| **sim** / host plant | GCU `sim/` HAL double + pytest/CTest | **Not** esprec. Do not rename GCU sim to “QEMU” | +| **host gate** | Named product gate (`pytest` / `silico gate`) | GCU CI stays this; do **not** force every GCU to run QEMU | +| **QEMU gate** | — | esprec’s own CI ladder (unit required; QEMU example may be follow-up). Proves esprec’s capture path, not every GCU plant | +| **metal** | Real USB + operator-observable face | Real panel still confirms color/timing quirks | + +**Apps stay apps.** Product domain UI stays in the GCU. esprec is **tooling** +(firmware component + host CLI). Prefer sibling clone + `pip install -e` until +a second GCU forces a silico pin (**Extract, then open**). + +## Readiness (current) + +| Bar | State | +|-----|--------| +| Host CLI (`esprec snapshot` / `record` / `agent-guide`) | **yes** | +| On-device component (`component/esprec`, `esprec_emit_rgb565_spi_be`) | **yes** | +| Unit gate `python -m pytest -q` (protocol integrity, PNG/GIF, fake device) | **yes** | +| QEMU CI example | **follow-up** if env lacks it — metal + unit are honest for firmware path | +| Agent guide | `esprec agent-guide` | + +## Install / invoke + +```text +# sibling layout (typical): …/tig/esprec next to …/tig/ +python -m pip install -e "../esprec[dev]" +esprec agent-guide +esprec snapshot --fake -o face.png # offline +esprec snapshot --port COMx -o face.png # metal still +esprec record --port COMx --frames 5 --hz 2 -o clip.gif +# named unit gate: +python -m pytest -q # inside the esprec checkout +``` + +Device commands: `esprec shot` or `shot` (alias). Wire: **ESPREC1** header + +base64 raster + end line; CRC covers **metadata + raster** (fail closed on +truncate / header tamper). Legacy `SHOT` (pixels-only CRC) still decodes. + +**Serial open:** esprec sets DTR/RTS low before open so ESP auto-reset does not +reboot mid-session (black unpainted shadow). Prefer **one open session** for +btn inject + multiple snaps (see esprec `scripts/xuss_c_screen_scenario.py`). + +## Agent recipe + +**Where we are:** Stage D (hello metal) or UI domain work after deploy. Host +gate is green; board talks; product face includes a **screen**. + +**Why:** Agents cannot trust “UI looks right in source.” A PNG/GIF of the live +buffer is evidence; the operator still owns first-ship product face judgment. + +```text +# After confirmed deploy + app running (soft-reset if verify parked the loop): +esprec snapshot --port COMx --command shot -o .silico/esprec/face.png +# Read the PNG (vision) vs product face “good.” +# Multi-step: keep one serial session; settle after each product action; then snap. +``` + +Rules: + +1. **Announce** before capture if UI will change brightness/content surprisingly. +2. Prefer **snapshot** for “what is on screen now?” Prefer **bounded GIF** for + sequences — never unbounded streams. +3. Store under **gitignored** paths (e.g. `.silico/esprec/`) unless product docs want stills. +4. **Do not** claim metal acceptance from PNG alone on first ship — still ask the operator. +5. **Do not** add esprec QEMU jobs to every GCU `ci.yml` by default. +6. If PNG disagrees with the glass: **pipeline first** (integrity, packing, cooked serial), not product folklore. + +## Integration surface (GCU) + +1. EXTRA_COMPONENT_DIRS → `esprec/component` (sibling clone) or vendor `component/esprec`. +2. Maintain a full-frame **shadow** RGB565 (`spi_be` packing as on panel DMA). +3. On `shot` / `esprec shot`: hush logs, call `esprec_emit_rgb565_spi_be(shadow, w, h)`. +4. Document host one-liner in product `install/` when capture is part of the update path. +5. Panel color/partial paint: [esp32-lcd-ips.md](esp32-lcd-ips.md) when relevant. + +## What not to do + +- Treat esprec as silico’s default **sim** (HAL plant). +- Require QEMU on cloud CI for every GCU. +- Claim first ship complete from agent-viewed PNG without operator product face confirm. +- Embed product UI logic into silico or esprec. +- Reopen serial with default DTR/RTS between every snap (resets ESP, black frames). + +## Compound + +Protocol/CRC/serial friction → fix **tig/esprec**. Board-generic panel notes → +this knowledge tree. Product domain → GCU. diff --git a/silico/plates/gcu-c/AGENTS.md b/silico/plates/gcu-c/AGENTS.md index ba1af78..5534572 100644 --- a/silico/plates/gcu-c/AGENTS.md +++ b/silico/plates/gcu-c/AGENTS.md @@ -64,6 +64,8 @@ silico deploy --port COMx --yes --verify ESP-IDF must be installed (`idf.py` or `IDF_PATH`). First flash and update flash are the same image path. +After deploy, **operator-confirm product face** on the bench (see silico root Stage D1). If this GCU’s face is a **screen**, open silico `knowledge/esprec.md` when **esprec** is ready for optional PNG/GIF agent capture — not a substitute for operator confirm, and not a reason to add QEMU to this GCU’s host gate by default. + ## HAL seam Portable domain under `include/` + `src/` must not include freertos / esp_* / driver headers. diff --git a/specs/lexicon.md b/specs/lexicon.md index d2ec94a..1f918af 100644 --- a/specs/lexicon.md +++ b/specs/lexicon.md @@ -80,7 +80,9 @@ The **human-observable product indication** on the [GCU](#gcu) after the app is Always say the full term **product face**. Never shorten to bare “face.” On first use in a session, define it (see Silico `AGENTS.md` operator language). First-ship [metal](#metal) acceptance requires the operator to confirm the product face, not only a version string. -See also: [metal](#metal), [host gate](#host-gate), [Help the operator](#help-the-operator), [first ship](#first-ship). +For **screened** ESP32-class products, agents may later use [esprec](#esprec) (when ready) as optional eyes on the panel; that does **not** replace operator product face confirm on first ship. + +See also: [metal](#metal), [host gate](#host-gate), [Help the operator](#help-the-operator), [first ship](#first-ship), [esprec](#esprec). ### first ship @@ -149,7 +151,17 @@ See also: [version identity](#version-identity). Also **host plant** / **closed-loop plant.** [Host](#host)-only simulation of the product world used for regression without a board. Never deployed to the device. Complements [metal](#metal); does not replace [host gate](#host-gate) or real USB on first ship. -See also: [HAL](#hal), [host-first](#host-first). +Default GCU shape: `sim/` HAL doubles + host tests (pytest or CTest) — **not** full SoC emulation. Companion tools (e.g. [esprec](#esprec) QEMU gates for *their* firmware path) may use QEMU; that does **not** redefine Silico **sim** for every GCU. + +See also: [HAL](#hal), [host-first](#host-first), [esprec](#esprec). + +### esprec + +**External companion tooling** ([tig/esprec](https://github.com/tig/esprec)): on-device screen capture over USB serial to host **PNG** / **GIF** so agents can see ESP display UIs (tuirec *analogue* for panels). **Not** a GCU and **not** the Silico package. **Ready** for agent use via `esprec snapshot|record` + on-device component; optional agent evidence for screened product faces. + +Host knowledge: [silico/knowledge/esprec.md](../silico/knowledge/esprec.md). Does not replace [product face](#product-face) operator confirm or GCU [sim](#sim). + +See also: [product face](#product-face), [agent-first CLI](#agent-first-cli), [ESP32-class](#esp32-class). --- diff --git a/specs/silicov1.md b/specs/silicov1.md index f825222..b6f6f0f 100644 --- a/specs/silicov1.md +++ b/specs/silicov1.md @@ -317,6 +317,7 @@ Until then, Quilan owns modem, credentials, and uplink protocol in private app c 8. Minimal package layout so `pip install` of a tag works (make-PR-true; pre-alpha until then). 9. CLI verbs (spec now; see tui-cs/cli design discussion) and first ship rehearsal harness. 10. MicroPython host-sim stack ([issue #3](https://github.com/tig/silico/issues/3)); integrity for beta ([issue #4](https://github.com/tig/silico/issues/4)). +11. **esprec** ([tig/esprec](https://github.com/tig/esprec)): agent screen capture for screened ESP GCUs is **implemented** externally; spine holds knowledge + first-ship pointers. Pin/CLI verb only if a second GCU forces it (**Extract, then open**). Do not redefine GCU **sim** as QEMU. ## 16. Acceptance for silico v1