Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Context is finite. **Do not** open every manners file into the active window.
| Operator language (first prompt orient, first-use term defs, big-step why/where) | **silico AGENTS** + [lexicon](specs/lexicon.md) | Not bedside tenets; domain on top of contract |
| COM / first-flash (UF2 **or** esptool) / board identity / metal deploy | **silico BEDSIDE.md** + `silico/knowledge/first-flash.md` + CLI | AGENTS first-ship metal sections may point here; avoid full restatement |
| GCU `bedside.toml` pin (sibling paths to silico vendor) | **plate** ships it; GCU keeps it | Do not leave GCU without a pin and then invent a prose path |
| Board/host capability notes (audio, bridges, …) | **silico/knowledge/** (self-improving) | Product must not soft-fork; agents **append** host truths here |
| Board/host capability notes (audio, bridges, screened capture via esprec when ready, …) | **silico/knowledge/** (self-improving) | Product must not soft-fork; agents **append** host truths here; open [esprec.md](silico/knowledge/esprec.md) only for ESP display capture |
| Eval rubric / fixtures | **tig/bedside** `eval/` | Run CLI; do not paste rubric into context |
| Product domain (idle, vehicle, tunes-as-product, …) | **GCU repo** | Never silico or bedside |

Expand Down Expand Up @@ -851,6 +851,17 @@ Deploy verify + `FW_VERSION` match prove the **host wrote this build**. They do
3. If product docs name a product face (e.g. M5 front-panel / side LEDs, speaker riff) and the plate only toggles a generic/dev-board pin (e.g. XIAO GPIO LED that is not the product face): **that is HW confusion, not first ship done.** Resolve it with the operator — prefer **`silico board-profile`** pin packs (e.g. `m5go`: side strip **15**, speaker **25**) and `silico board-profile seed` into `firmware/defaults.py` **after** operator confirm of the map; also use parts/spec/board knowledge. Host-test, redeploy, re-confirm observe. Filing a GitHub issue is a tracker, not acceptance.
4. Product face gaps that block observe are **in-scope first-ship metal work**. Silico exists to help the operator through this. Do not label the session “on the metal” / “first ship complete” while an open issue still says the product face is unproven.

##### Screened ESP UIs — esprec (**ready**)

When the product face is a **display** on ESP32-class hardware, agents use **esprec** ([tig/esprec](https://github.com/tig/esprec) — tuirec *analogue* for device screens) for **PNG/GIF capture** over USB serial: `pip install -e ../esprec`, then `esprec snapshot --port COMx -o face.png`. Detail: [silico/knowledge/esprec.md](silico/knowledge/esprec.md).

| Do | Do not |
|----|--------|
| Open `esprec.md` only when the GCU has a screen face and you need agent eyes | Invent a private capture stack when esprec already covers the wire |
| Use capture as **extra** evidence after deploy + app running | Replace **operator** product face confirm on first ship with “I viewed a PNG” |
| Keep GCU **host gate** as pytest/CTest + plate CI | Force QEMU jobs onto every GCU because esprec may use QEMU in *its* CI |
| Treat esprec as **external tooling** until a second GCU forces a pin | Redefine GCU **sim** (`sim/` HAL plant) as “run under QEMU” |

##### GPIO / pin / product face ambiguity → **stop and ask** (mandatory)

When you notice (or should notice) that the **pin, LED, speaker, or product face “good”** in firmware does not clearly match the **product board / spec**, you **must not** quietly assume, monologue an “Honesty” paragraph, or only open a GitHub issue. Never call this bare “face.”
Expand Down Expand Up @@ -891,7 +902,8 @@ bedside ask --id clarify-product-face \
6. Soft-reset so **main.py runs as the app** (deploy verify uses REPL and parks the loop). If the soft-reset itself will start sound/motion, announce that **before** the reset. If raw REPL fails: product `repl` door or boot window (Ctrl-C may be data).
7. **Operator-observable check:** document the **product face** “good”; confirm with the operator from the bench. If product face ≠ plate generic pin (or mapping is unclear): **clarify with the operator first** (structured ask), then fix, redeploy, re-confirm observe — do not stop at version match or issue-only.
8. Optional: `silico monitor --port COMx --duration 10`.
9. Document `install/` leave-behind (update-path one-liner + product face “good”: LEDs/audio/etc.).
9. Optional (screened ESP): `esprec snapshot --port COMx -o …` for agent eyes — still confirm product face with the operator on first ship ([knowledge/esprec.md](silico/knowledge/esprec.md)).
10. Document `install/` leave-behind (update-path one-liner + product face “good”: LEDs/audio/etc.).

Non-Python deploy assets (e.g. audio riffs) may appear in `[deploy].core`; host hygiene skips them as copy-only.

Expand Down
3 changes: 2 additions & 1 deletion BEDSIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ This file is **not** a fork of the Bedside tenets.
| ESP32 USB duplex / console lockout | `silico/knowledge/esp32-usb-serial.md` |
| ESP32 audio / DAC / PWM silence | `silico/knowledge/esp32-audio.md` |
| SPI IPS color / partial blit | `silico/knowledge/esp32-lcd-ips.md` |
| ESP screen capture for agents (when ready) | `silico/knowledge/esprec.md` ([tig/esprec](https://github.com/tig/esprec)) |
| Large binary asset deploy | `silico/knowledge/deploy-assets.md` |
| M5GO / Core v2.7 power button | `silico/knowledge/m5go-power.md` |
| This file | metal-host glossary only — do **not** also reload the full contract if AGENTS already pinned it |
Expand All @@ -35,7 +36,7 @@ Host tools and first-ship **stage order** live in **AGENTS.md**. Metal-specific
5. Deploy overwrite only after `bedside ask --id confirm-deploy` (or host UI same contract).
6. **Before write/reset:** clearly tell the operator what the board may do after boot (tones, LEDs, motion, duration) — especially audio products. Permission to overwrite is not a license to startle.
7. After `--verify`, **soft-reset again** so the product boot entry runs (verify parks the app loop).
8. **Operator-observable good:** confirm the human can **see or hear** the documented **product face** for **this** product board (not only `FW_VERSION` over REPL). Always say **product face**, never bare “face.” Plate generic LED on the wrong pin is not acceptance.
8. **Operator-observable good:** confirm the human can **see or hear** the documented **product face** for **this** product board (not only `FW_VERSION` over REPL). Always say **product face**, never bare “face.” Plate generic LED on the wrong pin is not acceptance. Screened ESP: when **esprec** is ready, agents may capture PNG/GIF for evidence — that does **not** replace this operator confirm (`silico/knowledge/esprec.md`).
9. **Pin / product face mismatch → ask first:** if GPIO, LED, or audio that makes up the product face is unclear vs product docs/board, **stop and clarify with the operator** (`bedside ask` / host picker) before assuming, filing-only, or advancing stages. Then fix → redeploy → re-confirm observe.
10. App updates after first-flash: no re-teaching UF2/esptool.

Expand Down
2 changes: 1 addition & 1 deletion silico/doctor.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ def run_doctor(*, root: Path | None = None) -> DoctorReport:

# Point agents at growing host knowledge (board caps, audio, first-flash).
lines.append(
"Host knowledge: silico/knowledge/ (ESP32 audio, first-flash notes). "
"Host knowledge: silico/knowledge/ (ESP32 audio, first-flash, esprec for screens). "
"When first ship friction is board/host-generic, add a note there (Make it better)."
)

Expand Down
1 change: 1 addition & 0 deletions silico/knowledge/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
| esp32-usb-serial | ESP32 USB-UART duplex ladder, lockout recovery, UART0 footgun | [esp32-usb-serial.md](esp32-usb-serial.md) |
| esp32-audio | ESP32 DAC lifecycle, soft-park silence, smooth PCM / music; IDF dac_continuous queue/drain | [esp32-audio.md](esp32-audio.md) |
| esp32-lcd-ips | SPI IPS color pack (R/B, INVON), partial blit | [esp32-lcd-ips.md](esp32-lcd-ips.md) |
| esprec | Agent screen capture (PNG/GIF) for ESP displays — when tig/esprec is ready | [esprec.md](esprec.md) |
| m5-core | M5GO/Core face pins, buttons, MPU6886 WHO_AM_I + temp formula | [m5-core.md](m5-core.md) |
| deploy-assets | Large binary asset deploy + size verify | [deploy-assets.md](deploy-assets.md) |
| m5go-power | M5GO / Core v2.7 red button on/off (double-click off) | [m5go-power.md](m5go-power.md) |
Expand Down
1 change: 1 addition & 0 deletions silico/knowledge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Product domain (idle control, drone songs, vehicle acceptance) stays in the **GC
|------|------|
| ESP32 / DAC / speaker / PWM tone / sample playback | [esp32-audio.md](esp32-audio.md) |
| SPI IPS color / INVON / partial paint | [esp32-lcd-ips.md](esp32-lcd-ips.md) |
| Screen capture for agents (PNG/GIF) when tig/esprec is ready | [esprec.md](esprec.md) |
| M5GO / Core face pins, buttons, MPU6886 temp | [m5-core.md](m5-core.md) |
| Large binary assets (PCM, images) deploy verify | [deploy-assets.md](deploy-assets.md) |
| M5GO / Core v2.7 power button | [m5go-power.md](m5go-power.md) |
Expand Down
102 changes: 102 additions & 0 deletions silico/knowledge/esprec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# esprec — agent eyes on ESP displays (**ready**)

**Status:** **implemented** as external tooling ([tig/esprec](https://github.com/tig/esprec)).
Not a silico package feature. Open this file only when the GCU has a **screened
ESP32-class** product face and you need capture — not for LED-only or audio-only
first ship.

esprec is the [tuirec](https://github.com/tui-cs/tuirec) *analogue* for **device
screens** (mission kinship, not a port of tuirec’s PTY/cast pipeline): on
command, on-device capture → USB serial → host **PNG** (snapshot) or **GIF**
(keyframe or continuous sequence).

## Relationship to silico (do not soft-fork)

| Silico concept | Role | esprec role |
|----------------|------|-------------|
| **product face** | Operator **see/hear** acceptance for first ship | Optional **agent** evidence of the screen face — does **not** replace operator confirm on first ship |
| **sim** / host plant | GCU `sim/` HAL double + pytest/CTest | **Not** esprec. Do not rename GCU sim to “QEMU” |
| **host gate** | Named product gate (`pytest` / `silico gate`) | GCU CI stays this; do **not** force every GCU to run QEMU |
| **QEMU gate** | — | esprec’s own CI ladder (unit required; QEMU example may be follow-up). Proves esprec’s capture path, not every GCU plant |
| **metal** | Real USB + operator-observable face | Real panel still confirms color/timing quirks |

**Apps stay apps.** Product domain UI stays in the GCU. esprec is **tooling**
(firmware component + host CLI). Prefer sibling clone + `pip install -e` until
a second GCU forces a silico pin (**Extract, then open**).

## Readiness (current)

| Bar | State |
|-----|--------|
| Host CLI (`esprec snapshot` / `record` / `agent-guide`) | **yes** |
| On-device component (`component/esprec`, `esprec_emit_rgb565_spi_be`) | **yes** |
| Unit gate `python -m pytest -q` (protocol integrity, PNG/GIF, fake device) | **yes** |
| QEMU CI example | **follow-up** if env lacks it — metal + unit are honest for firmware path |
| Agent guide | `esprec agent-guide` |

## Install / invoke

```text
# sibling layout (typical): …/tig/esprec next to …/tig/<gcu>
python -m pip install -e "../esprec[dev]"
esprec agent-guide
esprec snapshot --fake -o face.png # offline
esprec snapshot --port COMx -o face.png # metal still
esprec record --port COMx --frames 5 --hz 2 -o clip.gif
# named unit gate:
python -m pytest -q # inside the esprec checkout
```

Device commands: `esprec shot` or `shot` (alias). Wire: **ESPREC1** header +
base64 raster + end line; CRC covers **metadata + raster** (fail closed on
truncate / header tamper). Legacy `SHOT` (pixels-only CRC) still decodes.

**Serial open:** esprec sets DTR/RTS low before open so ESP auto-reset does not
reboot mid-session (black unpainted shadow). Prefer **one open session** for
btn inject + multiple snaps (see esprec `scripts/xuss_c_screen_scenario.py`).

## Agent recipe

**Where we are:** Stage D (hello metal) or UI domain work after deploy. Host
gate is green; board talks; product face includes a **screen**.

**Why:** Agents cannot trust “UI looks right in source.” A PNG/GIF of the live
buffer is evidence; the operator still owns first-ship product face judgment.

```text
# After confirmed deploy + app running (soft-reset if verify parked the loop):
esprec snapshot --port COMx --command shot -o .silico/esprec/face.png
# Read the PNG (vision) vs product face “good.”
# Multi-step: keep one serial session; settle after each product action; then snap.
```

Rules:

1. **Announce** before capture if UI will change brightness/content surprisingly.
2. Prefer **snapshot** for “what is on screen now?” Prefer **bounded GIF** for
sequences — never unbounded streams.
3. Store under **gitignored** paths (e.g. `.silico/esprec/`) unless product docs want stills.
4. **Do not** claim metal acceptance from PNG alone on first ship — still ask the operator.
5. **Do not** add esprec QEMU jobs to every GCU `ci.yml` by default.
6. If PNG disagrees with the glass: **pipeline first** (integrity, packing, cooked serial), not product folklore.

## Integration surface (GCU)

1. EXTRA_COMPONENT_DIRS → `esprec/component` (sibling clone) or vendor `component/esprec`.
2. Maintain a full-frame **shadow** RGB565 (`spi_be` packing as on panel DMA).
3. On `shot` / `esprec shot`: hush logs, call `esprec_emit_rgb565_spi_be(shadow, w, h)`.
4. Document host one-liner in product `install/` when capture is part of the update path.
5. Panel color/partial paint: [esp32-lcd-ips.md](esp32-lcd-ips.md) when relevant.

## What not to do

- Treat esprec as silico’s default **sim** (HAL plant).
- Require QEMU on cloud CI for every GCU.
- Claim first ship complete from agent-viewed PNG without operator product face confirm.
- Embed product UI logic into silico or esprec.
- Reopen serial with default DTR/RTS between every snap (resets ESP, black frames).

## Compound

Protocol/CRC/serial friction → fix **tig/esprec**. Board-generic panel notes →
this knowledge tree. Product domain → GCU.
2 changes: 2 additions & 0 deletions silico/plates/gcu-c/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ silico deploy --port COMx --yes --verify

ESP-IDF must be installed (`idf.py` or `IDF_PATH`). First flash and update flash are the same image path.

After deploy, **operator-confirm product face** on the bench (see silico root Stage D1). If this GCU’s face is a **screen**, open silico `knowledge/esprec.md` when **esprec** is ready for optional PNG/GIF agent capture — not a substitute for operator confirm, and not a reason to add QEMU to this GCU’s host gate by default.

## HAL seam

Portable domain under `include/` + `src/` must not include freertos / esp_* / driver headers.
Expand Down
16 changes: 14 additions & 2 deletions specs/lexicon.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,9 @@ The **human-observable product indication** on the [GCU](#gcu) after the app is

Always say the full term **product face**. Never shorten to bare “face.” On first use in a session, define it (see Silico `AGENTS.md` operator language). First-ship [metal](#metal) acceptance requires the operator to confirm the product face, not only a version string.

See also: [metal](#metal), [host gate](#host-gate), [Help the operator](#help-the-operator), [first ship](#first-ship).
For **screened** ESP32-class products, agents may later use [esprec](#esprec) (when ready) as optional eyes on the panel; that does **not** replace operator product face confirm on first ship.

See also: [metal](#metal), [host gate](#host-gate), [Help the operator](#help-the-operator), [first ship](#first-ship), [esprec](#esprec).

### first ship

Expand Down Expand Up @@ -149,7 +151,17 @@ See also: [version identity](#version-identity).

Also **host plant** / **closed-loop plant.** [Host](#host)-only simulation of the product world used for regression without a board. Never deployed to the device. Complements [metal](#metal); does not replace [host gate](#host-gate) or real USB on first ship.

See also: [HAL](#hal), [host-first](#host-first).
Default GCU shape: `sim/` HAL doubles + host tests (pytest or CTest) — **not** full SoC emulation. Companion tools (e.g. [esprec](#esprec) QEMU gates for *their* firmware path) may use QEMU; that does **not** redefine Silico **sim** for every GCU.

See also: [HAL](#hal), [host-first](#host-first), [esprec](#esprec).

### esprec

**External companion tooling** ([tig/esprec](https://github.com/tig/esprec)): on-device screen capture over USB serial to host **PNG** / **GIF** so agents can see ESP display UIs (tuirec *analogue* for panels). **Not** a GCU and **not** the Silico package. **Ready** for agent use via `esprec snapshot|record` + on-device component; optional agent evidence for screened product faces.

Host knowledge: [silico/knowledge/esprec.md](../silico/knowledge/esprec.md). Does not replace [product face](#product-face) operator confirm or GCU [sim](#sim).

See also: [product face](#product-face), [agent-first CLI](#agent-first-cli), [ESP32-class](#esp32-class).

---

Expand Down
1 change: 1 addition & 0 deletions specs/silicov1.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,7 @@ Until then, Quilan owns modem, credentials, and uplink protocol in private app c
8. Minimal package layout so `pip install` of a tag works (make-PR-true; pre-alpha until then).
9. CLI verbs (spec now; see tui-cs/cli design discussion) and first ship rehearsal harness.
10. MicroPython host-sim stack ([issue #3](https://github.com/tig/silico/issues/3)); integrity for beta ([issue #4](https://github.com/tig/silico/issues/4)).
11. **esprec** ([tig/esprec](https://github.com/tig/esprec)): agent screen capture for screened ESP GCUs is **implemented** externally; spine holds knowledge + first-ship pointers. Pin/CLI verb only if a second GCU forces it (**Extract, then open**). Do not redefine GCU **sim** as QEMU.

## 16. Acceptance for silico v1

Expand Down
Loading