Skip to content

Commit eefc7da

Browse files
Staacksclaude
Staacks
and
claude
committed
lab: fixes from the first full BLE pass, which found more in the harness than in the app
Ten scenarios against one phone, and most of what failed was the driver: * the Nano 33 BLE re-enumerates on every upload and does not reliably come back on the node it left, so EVERY distractor flash failed with "No device found on ttyACM1" while the board sat there working. The port is now resolved through `arduino-cli board list`, which identifies boards by USB id, with one retry for the re-enumeration window. * `arduino-cli upload` does not take --build-property and answers with a usage dump; the properties belong on compile alone. * adb joins argv into one shell command, so `-e bleDevice phyphox device` became two arguments and am instrument printed usage instead of running. Every scenario whose board advertises a name with a space - four of ten - could never have connected. Quoted now. * killing the local adb does not stop the instrumentation on the phone, so a timed-out scenario left the app running and answering the API, and every later scenario "connected" to a phone still holding the wrong experiment. The app is force-stopped and the transfer file deleted before each connect. That last one had teeth: four captures went into the corpus byte-identical, all of them a leftover file naming a device called "phyphox-sim" that is not on this bench. Deleted. A capture is now refused unless the experiment names the board the scenario asked for. Captures are also no longer filed blindly: a capture the spec check flags is staged in fixtures/ble/captured/ with the finding instead of landing in corpus/valid, where it would break the next docs build. Two are waiting there over `facor="1"` - a typo for `factor` that both libraries emit (phyphox-arduino src/view_elements/value.cpp:58 and phyphox-micropython phyphoxBLE/experiment.py:575, the same line copied). Harmless, since the value equals the attribute's default and both parsers ignore unknown attributes, but it is real output that a release must keep loading, which is what corpus/invalid + `parser: accepts` is for. Filing them there is a classification call and stays with the maintainer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent c69a816 commit eefc7da

5 files changed

Lines changed: 298 additions & 10 deletions

File tree

‎corpus/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,12 @@ test suite as well is planned.
2525
emits — the whole point is that they are what a released app must keep
2626
loading. Note what they legitimately contain: no `xmlns`, a format version
2727
well behind the apps (MicroPython declares 1.10), and containers sized `0`.
28+
A capture the spec check flags does **not** land here: it is staged in
29+
`fixtures/ble/captured/` with the finding, because a file that the apps
30+
load but the validators reject belongs in `invalid/` with
31+
`parser: accepts`, and that classification is a decision rather than
32+
something a lab run should make on its own. Two Arduino captures are
33+
waiting there now over a `facor="1"` typo in the library.
2834
- `generated/` — hand-written fixtures for the spec surface that no collected
2935
file exercises either: the inverse and hyperbolic trigonometry modules,
3036
`gcd`/`lcm`, `butterworth`, the `events` block of saved states, `appleBan`,

‎fixtures/ble/captured/README.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Staged BLE captures
2+
3+
Experiment XML taken off a phone that received it from a board
4+
(`tools/lab/run.py --suites ble --capture-ble-xml`), held here because the
5+
spec check flags it and where it belongs is a decision, not a default.
6+
7+
`corpus/valid/` means two things at once: the file validates against
8+
`spec/`, and both apps load it. A library capture can fail the first while
9+
passing the second, and dropping one into `corpus/valid/` anyway would
10+
break the docs build on the next commit.
11+
12+
What is here now, and why:
13+
14+
- `arduino-getSensorDataFromSmartphone.phyphox`
15+
- `arduino-getSystemAndEventTime.phyphox`
16+
17+
Both carry `facor="1"` on their `<value>` view elements — a typo for
18+
`factor`, emitted by the library itself (`phyphox-arduino`
19+
`src/view_elements/value.cpp:58` and, identically, `phyphox-micropython`
20+
`phyphoxBLE/experiment.py:575`). It is harmless: the value written is the
21+
attribute's own default, and both parsers ignore unknown attributes, so
22+
nothing misbehaves. But it is real output that released apps must keep
23+
loading, which is precisely what `corpus/invalid/` with `parser: accepts`
24+
records.
25+
26+
Moving them there is a classification decision and the maintainer's call —
27+
and it is worth making after the libraries are next touched, since a fixed
28+
library would make these captures clean and put them in `corpus/valid/`
29+
instead.

‎fixtures/ble/captured/arduino-getSensorDataFromSmartphone.phyphox‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
<phyphox version="1.15">
2+
<title>Get Accelerometer Data</title>
3+
<category>Arduino Experiments</category>
4+
<description>Send smartphone accelerometer data to an arduino/esp32</description>
5+
<data-containers>
6+
<container size="0" static="false">CH0</container>
7+
<container size="0" static="false">CB1</container>
8+
<container size="0" static="false">CB2</container>
9+
<container size="0" static="false">CB3</container>
10+
<container size="0" static="false">CB4</container>
11+
<container size="0" static="false">CB5</container>
12+
<container size="0" static="false">CH1</container>
13+
<container size="0" static="false">CH2</container>
14+
<container size="0" static="false">CH3</container>
15+
<container size="0" static="false">CH4</container>
16+
<container size="0" static="false">CH5</container>
17+
</data-containers>
18+
<input>
19+
<bluetooth name="phyphox-Arduino" id="phyphoxBLE" mode="notification" subscribeOnStart="false">
20+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="0">CH1</output>
21+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="4">CH2</output>
22+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="8">CH3</output>
23+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="12">CH4</output>
24+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="16">CH5</output>
25+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" extra="time">CH0</output>
26+
</bluetooth>
27+
<sensor type="accelerometer" rate="80" average="true">
28+
<output component="x">CB1</output>
29+
<output component="y">CB2</output>
30+
<output component="z">CB3</output>
31+
</sensor>
32+
</input>
33+
<output>
34+
<bluetooth id="phyphoxBLE" name="phyphox-Arduino">
35+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian">CB1</input>
36+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="4">CB2</input>
37+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="8">CB3</input>
38+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="12">CB4</input>
39+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="16">CB5</input>
40+
</bluetooth>
41+
</output>
42+
<analysis sleep="0" onUserInput="false"></analysis>
43+
<views>
44+
<view label="FirstView">
45+
<info label="Acc data is sent to smartphone ">
46+
</info>
47+
</view>
48+
<view label="SENSOR RAW DATA">
49+
<info label="accelerometer ">
50+
</info>
51+
<value label="x" facor="1">
52+
<input>CB1</input>
53+
</value>
54+
<value label="y" facor="1">
55+
<input>CB2</input>
56+
</value>
57+
<value label="z" facor="1">
58+
<input>CB3</input>
59+
</value>
60+
</view>
61+
</views>
62+
<export>
63+
<set name="mySet">
64+
<data name="myData0">CH0</data>
65+
<data name="myData1">CH1</data>
66+
<data name="myData2">CH2</data>
67+
<data name="myData3">CH3</data>
68+
<data name="myData4">CH4</data>
69+
<data name="myData5">CH5</data>
70+
</set>
71+
</export>
72+
</phyphox>

‎fixtures/ble/captured/arduino-getSystemAndEventTime.phyphox‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
<phyphox version="1.15">
2+
<title>Arduino-Experiment</title>
3+
<category>Arduino Experiments</category>
4+
<description>An experiment created with the phyphox BLE library for Arduino-compatible micro controllers.</description>
5+
<data-containers>
6+
<container size="0" static="false">CH0</container>
7+
<container size="0" static="false">CB1</container>
8+
<container size="0" static="false">CB2</container>
9+
<container size="0" static="false">CB3</container>
10+
<container size="0" static="false">CB4</container>
11+
<container size="0" static="false">CB5</container>
12+
<container size="0" static="false">CH1</container>
13+
<container size="0" static="false">CH2</container>
14+
<container size="0" static="false">CH3</container>
15+
<container size="0" static="false">CH4</container>
16+
<container size="0" static="false">CH5</container>
17+
</data-containers>
18+
<input>
19+
<bluetooth name="phyphox-Arduino" id="phyphoxBLE" mode="notification" subscribeOnStart="false">
20+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="0">CH1</output>
21+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="4">CH2</output>
22+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="8">CH3</output>
23+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="12">CH4</output>
24+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="16">CH5</output>
25+
<output char="cddf1002-30f7-4671-8b43-5e40ba53514a" extra="time">CH0</output>
26+
</bluetooth>
27+
</input>
28+
<output>
29+
<bluetooth id="phyphoxBLE" name="phyphox-Arduino">
30+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian">CB1</input>
31+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="4">CB2</input>
32+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="8">CB3</input>
33+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="12">CB4</input>
34+
<input char="cddf1003-30f7-4671-8b43-5e40ba53514a" conversion="float32LittleEndian" offset="16">CB5</input>
35+
</bluetooth>
36+
</output>
37+
<analysis sleep="0" onUserInput="false"></analysis>
38+
<views>
39+
<view label="label">
40+
<graph label="myLabel" labelX="label x" labelY="label y">
41+
<input axis="x">CH0</input>
42+
<input axis="y">CH1</input>
43+
</graph>
44+
<value label="myLabel" facor="1">
45+
<input>CH1</input>
46+
</value>
47+
</view>
48+
</views>
49+
<export>
50+
<set name="mySet">
51+
<data name="myData0">CH0</data>
52+
<data name="myData1">CH1</data>
53+
<data name="myData2">CH2</data>
54+
<data name="myData3">CH3</data>
55+
<data name="myData4">CH4</data>
56+
<data name="myData5">CH5</data>
57+
</set>
58+
</export>
59+
</phyphox>

‎tools/lab/ble.py‎

Lines changed: 132 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@
4141
import json
4242
import os
4343
import re
44+
import shlex
4445
import subprocess
4546
import sys
4647
import time
@@ -102,11 +103,21 @@ def flash(scenario, board, cfg, args):
102103
timeout=600)
103104
if r.returncode != 0:
104105
return False, f"compile failed: {(r.stderr or r.stdout)[-300:]}"
105-
r = sh(["arduino-cli", "upload", "-p", port, "--fqbn", fqbn] + props
106-
+ [sketch], timeout=300)
107-
if r.returncode != 0:
108-
return False, f"upload failed: {(r.stderr or r.stdout)[-300:]}"
109-
return True, "flashed"
106+
for attempt in (1, 2):
107+
live = resolve_arduino_port(fqbn, port)
108+
# No props here: `upload` does not take --build-property and
109+
# answers a usage dump if given one. It uploads what `compile`
110+
# just built, so the properties are already baked in.
111+
r = sh(["arduino-cli", "upload", "-p", live, "--fqbn", fqbn,
112+
sketch], timeout=300)
113+
if r.returncode == 0:
114+
return True, ("flashed" if live == port
115+
else f"flashed (on {live}, not {port})")
116+
# One retry, because the failure to beat is a board caught
117+
# mid-re-enumeration: it is absent for a second and then back.
118+
if attempt == 1:
119+
time.sleep(5)
120+
return False, f"upload failed: {(r.stderr or r.stdout)[-300:]}"
110121

111122
# MicroPython: the ESP32 is shared with the Arduino scenarios, whose
112123
# uploads overwrite the whole flash, so the firmware cannot be a
@@ -136,6 +147,33 @@ def flash(scenario, board, cfg, args):
136147
return True, "copied and reset"
137148

138149

150+
def resolve_arduino_port(fqbn, configured):
151+
"""Where that board actually is right now, not where lab.yml said.
152+
153+
A Nano 33 BLE re-enumerates on every upload (1200 bps touch into the
154+
bootloader and back), and it does not reliably come back on the node
155+
it left: a full pass on 2026-08-27 failed EVERY distractor flash with
156+
"No device found on ttyACM1" while the board sat there working, and
157+
it was on ttyACM1 again by the time the run ended. So ask arduino-cli,
158+
which identifies boards by USB id and reports the FQBN per port, and
159+
fall back to what was configured when it cannot say (the ESP32's
160+
CP2102 is a generic bridge and matches no board).
161+
"""
162+
r = sh(["arduino-cli", "board", "list", "--format", "json"], timeout=60)
163+
if r.returncode != 0:
164+
return configured
165+
try:
166+
doc = json.loads(r.stdout or "{}")
167+
except ValueError:
168+
return configured
169+
base = fqbn.split(":")[:3]
170+
for entry in doc.get("detected_ports") or []:
171+
for board in entry.get("matching_boards") or []:
172+
if (board.get("fqbn") or "").split(":")[:3] == base:
173+
return (entry.get("port") or {}).get("address") or configured
174+
return configured
175+
176+
139177
def _esptool():
140178
"""esptool ships as `esptool` (v5) or `esptool.py` (v4) - accept both."""
141179
import shutil
@@ -213,6 +251,9 @@ def pick_distractor(scenario, cfg, boards_available):
213251
# ------------------------------------------------------- the phone-side step
214252

215253
RELEASE_PROP = "debug.phyphox.labRelease"
254+
ANDROID_PACKAGE = "de.rwth_aachen.phyphox"
255+
# Where Android leaves the experiment a device just sent it.
256+
TRANSFER_FILE = "files/temp_bt/bt.phyphox"
216257

217258

218259
def connect_phone(dev, scenario, args):
@@ -235,12 +276,25 @@ def connect_phone(dev, scenario, args):
235276
"""
236277
name = scenario.get("device_name") or ""
237278
if dev.platform == "android":
279+
# Start from nothing on the phone. Both of these were learned from
280+
# one bad full pass: a previous scenario's app instance survived a
281+
# killed instrumentation and kept answering the API, so every
282+
# later scenario "connected" to a phone still holding the wrong
283+
# experiment - and the leftover transfer file was captured four
284+
# times under four different names.
285+
sh(dev.adb + ["shell", "am", "force-stop", ANDROID_PACKAGE])
286+
sh(dev.adb + ["shell", "run-as", ANDROID_PACKAGE,
287+
"rm", "-f", TRANSFER_FILE])
238288
sh(dev.adb + ["shell", "setprop", RELEASE_PROP, "0"])
239289
proc = subprocess.Popen(
240290
dev.adb + ["shell", "am", "instrument", "-w",
241291
"-e", "class",
242292
"de.rwth_aachen.phyphox.BleCompatConnectTest",
243-
"-e", "bleDevice", name,
293+
# Quoted: adb joins argv into ONE shell command, so
294+
# a name with a space ("phyphox device", "My
295+
# Device") becomes two arguments and am instrument
296+
# answers with a usage dump instead of running.
297+
"-e", "bleDevice", shlex.quote(name),
244298
"-e", "holdForHost", "true",
245299
"de.rwth_aachen.phyphox.test/"
246300
"androidx.test.runner.AndroidJUnitRunner"],
@@ -249,6 +303,10 @@ def connect_phone(dev, scenario, args):
249303
# there is no other one while the test is still running.
250304
if wait_api(dev.base, args.connect_timeout) is None:
251305
proc.kill()
306+
# Killing the local adb does NOT stop the instrumentation on
307+
# the phone; without this it keeps the app alive and the next
308+
# scenario talks to it.
309+
sh(dev.adb + ["shell", "am", "force-stop", ANDROID_PACKAGE])
252310
out = (proc.communicate()[0] or "")[-300:]
253311
return False, ("the phone did not reach a loaded experiment "
254312
"within the connect timeout: " + out), None
@@ -491,9 +549,48 @@ def assert_scenario(dev, scenario, baseline, args, board_port):
491549
# ------------------------------------------------------------ XML capture
492550

493551
CAPTURE_DIR = os.path.join(ROOT, "corpus", "valid", "ble-libraries")
552+
# Where a capture waits when it cannot go straight into corpus/valid.
553+
CAPTURE_STAGING = os.path.join(ROOT, "fixtures", "ble", "captured")
494554
MACISH = re.compile(r"\b[0-9A-Fa-f]{2}(:[0-9A-Fa-f]{2}){5}\b")
495555

496556

557+
def spec_findings(path):
558+
"""What tools/validate_experiments.py says about one file.
559+
560+
corpus/valid means "validates cleanly AND both apps load it", and a
561+
capture cannot be assumed to do the first: the libraries emit
562+
`facor="1"` for `factor` on every <value> element (Arduino
563+
src/view_elements/value.cpp:58, MicroPython phyphoxBLE/
564+
experiment.py:575 - the same typo, copied). The apps ignore unknown
565+
attributes and load it fine, so that belongs in corpus/invalid with
566+
`parser: accepts`, which is a classification decision and therefore
567+
the maintainer's. A flagged capture is staged instead of filed, and
568+
the run says so - dropping it into corpus/valid would break the docs
569+
build on the next commit.
570+
"""
571+
sys.path.insert(0, os.path.join(ROOT, "tools"))
572+
try:
573+
import validate_experiments as ve
574+
except ImportError as e:
575+
return [f"could not run the spec check: {e}"]
576+
import xml.etree.ElementTree as ET
577+
spec, common, slots, components = ve.load_spec()
578+
rep = ve.Report()
579+
name = os.path.basename(path)
580+
root = ve.normalize_namespace(ET.parse(path).getroot())
581+
for child in root:
582+
ve.check_element(child, "phyphox", spec, common, slots, components,
583+
rep, "", name)
584+
for attr, value in root.attrib.items():
585+
if spec.get((None, "phyphox"), {"attrs": {}})["attrs"].get(attr) is None:
586+
rep.add("unknown attribute", name, f'<phyphox>: {attr}="{value}"')
587+
ve.check_slots(root, slots, components, rep, name)
588+
ve.check_root_once(root, rep, name)
589+
return [f"{kind}: {where}"
590+
for kind, entries in rep.items.items()
591+
for _fn, where in entries][:5]
592+
593+
497594
def capture_xml(dev, scenario):
498595
"""Pull the experiment the board just served, for the T0 half.
499596
@@ -510,25 +607,50 @@ def capture_xml(dev, scenario):
510607
"""
511608
if dev.platform != "android":
512609
return None, "capture runs on Android only (the file is the board's)"
513-
r = sh(dev.adb + ["shell", "run-as", "de.rwth_aachen.phyphox",
514-
"cat", "files/temp_bt/bt.phyphox"], timeout=30)
610+
r = sh(dev.adb + ["shell", "run-as", ANDROID_PACKAGE,
611+
"cat", TRANSFER_FILE], timeout=30)
515612
if r.returncode != 0 or not (r.stdout or "").lstrip().startswith("<"):
516613
return None, ("no transferred experiment on the phone - "
517614
f"{(r.stderr or r.stdout or '')[-120:]}")
518615
xml = r.stdout
616+
# Prove it came from THIS board. connect_phone deletes the file first,
617+
# so a leftover should be impossible - but a bad full pass on
618+
# 2026-08-27 wrote one stale file into the corpus under four different
619+
# scenario names, and a fixture that silently describes the wrong
620+
# device is worse than no fixture.
621+
wanted = scenario.get("device_name")
622+
m = re.search(r'<bluetooth\b[^>]*\bname="([^"]*)"', xml)
623+
if wanted and m and m.group(1) != wanted:
624+
return None, (f"the transferred experiment names {m.group(1)!r}, not "
625+
f"{wanted!r} - this is not that board's file")
519626
# The corpus is public. Nothing seen so far carries one - the
520627
# libraries identify their device by name - but a captured file is
521628
# not hand-written, so check rather than trust.
522629
if MACISH.search(xml):
523630
return None, ("the capture contains something MAC-shaped; sanitize "
524631
"it by hand before it goes into the public corpus")
632+
stem = f"{scenario['library']}-{scenario['example']}.phyphox"
633+
# Write first, then judge: the spec check needs a file, and where the
634+
# file belongs depends on what it says.
635+
os.makedirs(CAPTURE_STAGING, exist_ok=True)
636+
staged = os.path.join(CAPTURE_STAGING, stem)
637+
with open(staged, "w", encoding="utf-8") as f:
638+
f.write(xml)
639+
findings = spec_findings(staged)
640+
if findings:
641+
return staged, ("staged, NOT filed - the spec check flags it ("
642+
+ "; ".join(findings) + "). If the apps load it "
643+
"anyway it belongs in corpus/invalid with "
644+
"`parser: accepts`, which is a call to make by "
645+
"hand")
646+
525647
os.makedirs(CAPTURE_DIR, exist_ok=True)
526-
path = os.path.join(
527-
CAPTURE_DIR, f"{scenario['library']}-{scenario['example']}.phyphox")
648+
path = os.path.join(CAPTURE_DIR, stem)
528649
old = None
529650
if os.path.exists(path):
530651
with open(path, encoding="utf-8") as f:
531652
old = f.read()
653+
os.remove(staged)
532654
if old == xml:
533655
return path, "unchanged"
534656
with open(path, "w", encoding="utf-8") as f:

0 commit comments

Comments
 (0)