Skip to content

Commit 0d78aba

Browse files
Staacksclaude
andcommitted
lab: a crashing ble suite still writes its report
The report is written at the end of the run, so an exception took the whole run's evidence with it. A MacBook pass on 2026-08-28 reached its eighth scenario, raised on a missing mpremote and left nothing at all - not even the seven scenarios that had already run against the phone, which were the ones worth reading. An hour of bench time producing no record of what the phone did. The crash is now caught, printed with its traceback, and recorded as a finding per device, so the run finishes and writes what it has. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 6a1ce4e commit 0d78aba

2 files changed

Lines changed: 23 additions & 3 deletions

File tree

tools/lab/README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -111,8 +111,10 @@ one is missing - a host that was never set up looks nothing like a board
111111
fault, but discovered per scenario it arrives as two failed flashes and a
112112
board declared dead. On the MacBook on 2026-08-28 a missing `mpremote`
113113
was worse still: the run raised on the first MicroPython scenario, and
114-
the operator watched an idle phone until the run ended before seeing
115-
why. pyserial is deliberately NOT in requirements.txt: the docs build and app CI install
114+
the operator watched an idle phone until the run ended before seeing why
115+
- and because the report is written only at the end, the seven scenarios
116+
that HAD already run against the phone were lost with it. A crashing
117+
suite now still produces a report, with the exception as its finding. pyserial is deliberately NOT in requirements.txt: the docs build and app CI install
116118
that file and never open a serial port. It is needed only for the three
117119
scenarios where the data goes phone -> board and the board's own printout
118120
is the evidence.

tools/lab/run.py

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -473,7 +473,25 @@ def main():
473473
# orchestration and only hands back the per-device results.
474474
if suite == "ble":
475475
from lab import ble
476-
for dev_id, r in ble.run_suite(devices, args).items():
476+
try:
477+
per_device = ble.run_suite(devices, args)
478+
except Exception as e:
479+
# A crashed suite must still produce a report. The
480+
# write happens at the end of this function, so an
481+
# exception used to take the whole run's evidence with
482+
# it: on 2026-08-28 a MacBook pass reached its eighth
483+
# scenario, raised on a missing mpremote, and left
484+
# nothing at all - not even the seven scenarios that
485+
# had already run against the phone, which were the
486+
# ones worth reading.
487+
import traceback
488+
traceback.print_exc()
489+
per_device = {
490+
dev_id: {"passed": False, "warnings": [],
491+
"findings": [f"the ble suite crashed: "
492+
f"{type(e).__name__}: {e}"]}
493+
for dev_id, _entry, _dev in devices}
494+
for dev_id, r in per_device.items():
477495
report["devices"][dev_id]["ble"] = r
478496
state = "ok" if r.get("passed") else "FAIL"
479497
print(f"== ble @ {dev_id}: {state}"

0 commit comments

Comments
 (0)