Skip to content

Commit 48b660b

Browse files
Staacksclaude
andcommitted
Phase 5 docs side: the device-lab driver
tools/lab/: run.py (split-host orchestrator - per-host JSON results, --merge, a fixture file server with the FQDN-lookup fix), device handles for adb-driven Android (forward+reverse, both debug.phyphox switches, permission pre-grants, media volume) and devicectl/pymobiledevice3-driven iOS hardware (explicitly UNVERIFIED until the first MacBook run), and the four T2 suites carrying their matrix tags: manifest-driven sensor plausibility with a --record-manifest bootstrap that captures buffer names and /meta sensors per device, the audio loopback over the new fixtures/audio/loopback.phyphox (speaker to microphone, FFT peak corrected by the achieved rate), the full experiment matrix through t1_experiments per device, and the release language gate against built artifacts - verified against a real debug APK, where it flags exactly the four testing locales and nothing else (aapt spells the default resources '--_--' and Chinese zh-CN/zh-TW; the map now knows). check_test_matrix accepts tags from this repo's tools/ for rows a single host-side driver implements for both platforms. The four device-* rows are planned; the first real lab run flips them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 08c79d6 commit 48b660b

8 files changed

Lines changed: 847 additions & 0 deletions

File tree

fixtures/audio/loopback.phyphox

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
<phyphox xmlns="http://phyphox.org/xml" version="1.17" locale="en">
2+
<title>fixture: audio loopback</title>
3+
<category>Audio fixtures</category>
4+
<description>Self-contained loopback for the device lab (row device-audio): the speaker plays a 1 kHz sine while the microphone records; the analysis reports the dominant frequency and its magnitude. The frequency axis assumes 48 kHz - the lab driver corrects the expectation by the achievedrate buffer where the system chose another rate (iOS). Covers audio input, audio output and the fft in one automated test.</description>
5+
<data-containers>
6+
<container size="2048">recording</container>
7+
<container size="1">achievedrate</container>
8+
<container size="2048">fre</container>
9+
<container size="2048">fim</container>
10+
<container size="2048">mag</container>
11+
<container size="1024">magh</container>
12+
<container size="1024">freqs</container>
13+
<container size="1">peakfreq</container>
14+
<container size="1">level</container>
15+
</data-containers>
16+
<input>
17+
<audio rate="48000">
18+
<output component="out">recording</output>
19+
<output component="rate">achievedrate</output>
20+
</audio>
21+
</input>
22+
<output>
23+
<audio rate="48000" loop="true">
24+
<tone waveform="sine">
25+
<input parameter="frequency" type="value">1000</input>
26+
<input parameter="amplitude" type="value">1</input>
27+
<input parameter="duration" type="value">1</input>
28+
</tone>
29+
</audio>
30+
</output>
31+
<analysis requireFill="recording" requireFillThreshold="2048">
32+
<fft>
33+
<input as="re">recording</input>
34+
<output as="re">fre</output>
35+
<output as="im">fim</output>
36+
</fft>
37+
<formula formula="sqrt([1_]^2+[2_]^2)">
38+
<input>fre</input>
39+
<input>fim</input>
40+
<output>mag</output>
41+
</formula>
42+
<subrange>
43+
<input as="from" type="value">1</input>
44+
<input as="length" type="value">1023</input>
45+
<input>mag</input>
46+
<output>magh</output>
47+
</subrange>
48+
<ramp>
49+
<input as="start" type="value">23.4375</input>
50+
<input as="stop" type="value">23976.5625</input>
51+
<input as="length" type="value">1023</input>
52+
<output>freqs</output>
53+
</ramp>
54+
<max>
55+
<input as="x" keep="true">freqs</input>
56+
<input as="y">magh</input>
57+
<output as="max">level</output>
58+
<output as="position">peakfreq</output>
59+
</max>
60+
</analysis>
61+
<views>
62+
<view label="Loopback">
63+
<value label="peak frequency" unit="Hz" precision="1"><input>peakfreq</input></value>
64+
<value label="level" precision="4"><input>level</input></value>
65+
<value label="achieved rate" unit="Hz" precision="0"><input>achievedrate</input></value>
66+
</view>
67+
</views>
68+
</phyphox>

test-matrix.yml

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -261,3 +261,49 @@ tests:
261261
render without truncation/overlap regressions in every enabled
262262
language (snapshot or layout-assertion based, the platform's
263263
choice).
264+
265+
- id: device-sensors
266+
area: hardware
267+
tier: T2
268+
platforms: [android, ios]
269+
status: planned
270+
description: >-
271+
Per-device sensor truth on the seven lab phones, manifest-driven
272+
(tools/lab/devices/): liveness, plausibility windows (|accel| at
273+
rest, gyro near zero, earth-field magnitude, pressure range, light
274+
positive), achieved rate vs expectation, graceful GPS no-fix
275+
indoors. Implemented once host-side in tools/lab/ - the tag there
276+
counts for both platforms.
277+
278+
- id: device-audio
279+
area: hardware
280+
tier: T2
281+
platforms: [android, ios]
282+
status: planned
283+
description: >-
284+
The self-contained audio loopback (fixtures/audio/loopback.phyphox)
285+
per device: speaker plays 1 kHz while the microphone records, the
286+
FFT peak matches (corrected by the achieved rate), level above the
287+
floor. Covers audio input, output and analysis in one test.
288+
289+
- id: device-experiments
290+
area: hardware
291+
tier: T2
292+
platforms: [android, ios]
293+
status: planned
294+
description: >-
295+
The full shipped experiment matrix on every lab device through
296+
tools/t1_experiments.py: open via phyphox://asset=, run, stop, all
297+
six export formats validated. Bluetooth experiments load-phase
298+
only (the phase-6 board lab owns their data plane).
299+
300+
- id: device-languages
301+
area: translations
302+
tier: T2
303+
platforms: [android, ios]
304+
status: planned
305+
description: >-
306+
The release language gate: the BUILT artifact's locale set (aapt
307+
dump badging / the .lproj set) must equal languages.yml exactly -
308+
a missing language or a testing locale left in FAILS. Runs when
309+
the lab host names the release artifacts.

tools/check_test_matrix.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,11 @@
5050
"ios": os.path.normpath(os.path.join(ROOT, "..", "phyphox-ios")),
5151
}
5252

53+
# The T2 device-lab rows are implemented once, host-side, in this
54+
# repository's tools/lab/ - one driver serves both platforms. Tags found
55+
# there count for EVERY platform a row names.
56+
SHARED_TOOLS = os.path.join(ROOT, "tools")
57+
5358

5459
def load_matrix():
5560
with open(MATRIX, encoding="utf-8") as f:
@@ -112,6 +117,12 @@ def check(verbose=True):
112117
if not os.path.isdir(repo):
113118
continue
114119
found[platform] = scan_tags(repo)
120+
if found and os.path.isdir(SHARED_TOOLS):
121+
shared = scan_tags(SHARED_TOOLS)
122+
for platform in found:
123+
for tag, files in shared.items():
124+
found[platform].setdefault(tag, []).extend(
125+
"phyphox-docs/" + f for f in files)
115126

116127
for platform, tags in found.items():
117128
for tag, files in sorted(tags.items()):

tools/lab/README.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# The device lab (T2)
2+
3+
Host-side driver for the hardware tier: the full experiment matrix,
4+
per-device sensor plausibility, the audio loopback and the release
5+
language check, on the seven lab phones - split-host capable, one entry
6+
point, JSON per host, merged report.
7+
8+
cp lab.yml.example lab.yml # local: serials, host address, artifacts
9+
python3 tools/lab/run.py --config tools/lab/lab.yml --host macbook
10+
python3 tools/lab/run.py --merge lab-results/
11+
12+
The split: the MacBook can run everything; or the Linux machine runs the
13+
Android devices (`--host linuxbox`) while the MacBook runs iOS - each
14+
writes `<host>.json` into the shared results directory and `--merge`
15+
combines them. Phones stay in developer mode and are unlocked once per
16+
run; media volume for the audio suite is set automatically on Android
17+
and by hand on iOS (part of the unlock-once ritual).
18+
19+
## Suites and their matrix rows
20+
21+
- `sensors` (`device-sensors`): per-device manifest driven - liveness,
22+
plausibility (|accel| at rest, gyro near zero, earth-field magnitude,
23+
pressure range, light positive), achieved rate vs expectation, and the
24+
graceful GPS no-fix path indoors. A real fix is T3.
25+
- `audio` (`device-audio`): fixtures/audio/loopback.phyphox - speaker to
26+
microphone, FFT peak at the played 1 kHz (corrected by the achieved
27+
rate), level above the floor. The fixture is served from this checkout
28+
over http (adb reverse on Android, the host's LAN address for iOS).
29+
- `experiments` (`device-experiments`): the full shipped matrix through
30+
tools/t1_experiments.py per device - open, run, stop, all six export
31+
formats validated. Bluetooth experiments stay load-phase only (their
32+
data plane is the phase-6 board lab).
33+
- `languages` (`device-languages`): runs when the host entry names built
34+
artifacts - the apk's `aapt dump badging` locales / the ipa's .lproj
35+
set against languages.yml. This is the release gate that makes a
36+
forgotten testing locale or missing language impossible; mismatch
37+
FAILS.
38+
39+
## Per-device sensor manifests
40+
41+
`devices/<id>.yml`, committed - the expected truth per lab phone. First
42+
time on a device:
43+
44+
python3 tools/lab/run.py --config lab.yml --host X --record-manifest pixel-9-pro
45+
46+
writes `devices/pixel-9-pro.skeleton.yml` with the observed buffer names
47+
per core experiment and the /meta sensor list (Android). Hand-finish it:
48+
which sensors this device HAS, per sensor the experiment, value/time
49+
buffer names, a plausibility kind (magnitude9.81, near0, earthfield,
50+
pressure, positive) and the expected rate. A sensor absent from the
51+
manifest is not tested on that device - the older phones are in the lab
52+
precisely because their sensor sets differ.
53+
54+
## Honesty notes
55+
56+
- The iOS device paths (devicectl launch, pymobiledevice3 forward) were
57+
written on the Linux machine and are UNVERIFIED until the first
58+
MacBook run; whatever needed fixing is a finding for the docs session.
59+
- The suites carry the phyphox-test tags in suites.py - one driver
60+
serves both platforms, so the matrix checker accepts tags from this
61+
repository for the T2 rows.

tools/lab/device.py

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
"""Device handles for the lab driver (run.py).
2+
3+
AndroidDevice drives a phone over adb by serial: port forward, the two
4+
debug.phyphox.* switches, asset launches, media volume for the audio
5+
loopback. IOSDevice drives an iPhone/iPad from a macOS host via devicectl
6+
(launch with the -phyphox* arguments) and a user-provided port forward
7+
(iproxy or pymobiledevice3 usbmux forward - started by run.py when the
8+
lab.yml entry names a local port). The iOS paths were written on the
9+
Linux machine and are UNVERIFIED until the first MacBook run - anything
10+
that needed fixing there is a finding for the docs session.
11+
"""
12+
13+
import subprocess
14+
import time
15+
import urllib.parse
16+
import urllib.request
17+
18+
ANDROID_BUNDLE = "de.rwth_aachen.phyphox"
19+
IOS_BUNDLE = "de.rwth-aachen.physics.phyphox"
20+
21+
22+
def sh(cmd, timeout=30):
23+
"""One visible retry on timeout, then a failed stand-in (the
24+
t1_experiments convention - a wedged tool fails one step, not the
25+
run)."""
26+
for attempt in (1, 2):
27+
try:
28+
r = subprocess.run(cmd, capture_output=True, text=True,
29+
timeout=timeout)
30+
if attempt == 2:
31+
print(f" ~ retried after a timeout: {' '.join(cmd[:4])} ...")
32+
return r
33+
except subprocess.TimeoutExpired:
34+
if attempt == 2:
35+
print(f" ~ command timed out twice: {' '.join(cmd[:4])} ...")
36+
37+
class T:
38+
returncode, stdout, stderr = -1, "", "timed out"
39+
return T()
40+
41+
42+
def api(base, path, timeout=5):
43+
try:
44+
with urllib.request.urlopen(base + path, timeout=timeout) as r:
45+
return r.status, r.read()
46+
except urllib.error.HTTPError as e:
47+
return e.code, e.read()
48+
except Exception as e:
49+
return None, str(e).encode()
50+
51+
52+
def wait_api(base, seconds, probe_timeout=2):
53+
t0 = time.time()
54+
while time.time() - t0 < seconds:
55+
status, _ = api(base, "/config", timeout=probe_timeout)
56+
if status == 200:
57+
return time.time() - t0
58+
time.sleep(0.5)
59+
return None
60+
61+
62+
class AndroidDevice:
63+
platform = "android"
64+
65+
def __init__(self, serial, port):
66+
self.serial, self.port = serial, port
67+
self.base = f"http://127.0.0.1:{port}"
68+
self.adb = ["adb", "-s", serial]
69+
70+
def prepare(self, fixture_port=None):
71+
sh(self.adb + ["forward", f"tcp:{self.port}", "tcp:8080"])
72+
if fixture_port:
73+
# the phone reaches the host's fixture server at 127.0.0.1
74+
sh(self.adb + ["reverse", f"tcp:{fixture_port}",
75+
f"tcp:{fixture_port}"])
76+
for prop in ("remote", "autoConfirm"):
77+
sh(self.adb + ["shell", "setprop", f"debug.phyphox.{prop}", "1"])
78+
for perm in ("RECORD_AUDIO", "CAMERA", "ACCESS_FINE_LOCATION",
79+
"ACCESS_COARSE_LOCATION"):
80+
sh(self.adb + ["shell", "pm", "grant", ANDROID_BUNDLE,
81+
f"android.permission.{perm}"])
82+
83+
def cleanup(self):
84+
for prop in ("remote", "autoConfirm"):
85+
sh(self.adb + ["shell", "setprop", f"debug.phyphox.{prop}", "''"])
86+
87+
def launch(self, asset_path):
88+
self.stop_app()
89+
url = "phyphox://asset=" + urllib.parse.quote(asset_path, safe="")
90+
r = sh(self.adb + ["shell", "am", "start", "-W", "-a",
91+
"android.intent.action.VIEW", "-d", url])
92+
return r.returncode == 0 and "Error" not in r.stdout
93+
94+
def stop_app(self):
95+
sh(self.adb + ["shell", "am", "force-stop", ANDROID_BUNDLE])
96+
97+
def fixture_host(self):
98+
return "127.0.0.1" # via adb reverse
99+
100+
def open_url(self, phyphox_url):
101+
"""Open a phyphox:// URL (e.g. a fixture served by run.py)."""
102+
self.stop_app()
103+
r = sh(self.adb + ["shell", "am", "start", "-W", "-a",
104+
"android.intent.action.VIEW", "-d", phyphox_url])
105+
return r.returncode == 0 and "Error" not in r.stdout
106+
107+
def set_media_volume_max(self):
108+
sh(self.adb + ["shell", "cmd", "media_session", "volume",
109+
"--stream", "3", "--set", "15"])
110+
sh(self.adb + ["shell", "media", "volume", "--stream", "3",
111+
"--set", "15"])
112+
113+
114+
class IOSDevice:
115+
platform = "ios"
116+
117+
def __init__(self, udid, port):
118+
self.udid, self.port = udid, port
119+
self.base = f"http://127.0.0.1:{port}"
120+
self._forward = None
121+
122+
def prepare(self, fixture_port=None):
123+
self.fixture_port = fixture_port
124+
# forward local port -> device port 80 (the app's server on
125+
# hardware). pymobiledevice3 is the portable choice; iproxy works
126+
# too. UNVERIFIED on hardware.
127+
self._forward = subprocess.Popen(
128+
["pymobiledevice3", "usbmux", "forward", str(self.port), "80",
129+
"--serial", self.udid],
130+
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
131+
time.sleep(1)
132+
133+
def cleanup(self):
134+
if self._forward:
135+
self._forward.terminate()
136+
137+
def launch(self, asset_path):
138+
url = "phyphox://asset=" + urllib.parse.quote(asset_path, safe="")
139+
r = sh(["xcrun", "devicectl", "device", "process", "launch",
140+
"--terminate-existing", "--device", self.udid, IOS_BUNDLE,
141+
"-phyphoxUrl", url, "-phyphoxRemote",
142+
"-phyphoxRemotePort", "80", "-phyphoxAutoConfirm"],
143+
timeout=60)
144+
return r.returncode == 0
145+
146+
def stop_app(self):
147+
pass # --terminate-existing on launch
148+
149+
def fixture_host(self):
150+
# an iOS device reaches the host over the LAN; run.py passes the
151+
# host address from lab.yml (host_ip)
152+
return getattr(self, "host_ip", None) or "HOST-IP-UNSET"
153+
154+
def open_url(self, phyphox_url):
155+
r = sh(["xcrun", "devicectl", "device", "process", "launch",
156+
"--terminate-existing", "--device", self.udid, IOS_BUNDLE,
157+
"-phyphoxUrl", phyphox_url, "-phyphoxRemote",
158+
"-phyphoxRemotePort", "80", "-phyphoxAutoConfirm"],
159+
timeout=60)
160+
return r.returncode == 0
161+
162+
def set_media_volume_max(self):
163+
pass # no CLI path; the unlock-once checklist sets the volume

tools/lab/lab.yml.example

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Device-lab assignment - copy to lab.yml (local, NEVER committed: it
2+
# holds serials and host addresses) and adjust. Each host runs
3+
# python3 tools/lab/run.py --config lab.yml --host <name>
4+
# and produces one JSON; --merge combines them.
5+
6+
hosts:
7+
macbook: # the default full set
8+
host_ip: 192.168.2.50 # how iOS devices reach this host's fixture server
9+
aapt: aapt # or the full path into Android build-tools
10+
artifacts: # optional: enables the languages release check
11+
android: /path/to/phyphox-release.apk
12+
ios: /path/to/phyphox.ipa
13+
devices:
14+
iphone-14-pro: {platform: ios, serial: "UDID-HERE", port: 8081}
15+
iphone-8: {platform: ios, serial: "UDID-HERE", port: 8082}
16+
ipad-pro: {platform: ios, serial: "UDID-HERE", port: 8083}
17+
pixel-9-pro: {platform: android, serial: "SERIAL", port: 8091}
18+
pixel-3: {platform: android, serial: "SERIAL", port: 8092}
19+
nexus-5x: {platform: android, serial: "SERIAL", port: 8093}
20+
galaxy-a3: {platform: android, serial: "SERIAL", port: 8094}
21+
22+
linuxbox: # the split: Android (+ boards, phase 6) here,
23+
devices: # iOS stays with the MacBook
24+
pixel-9-pro: {platform: android, serial: "SERIAL", port: 8091}
25+
pixel-3: {platform: android, serial: "SERIAL", port: 8092}
26+
nexus-5x: {platform: android, serial: "SERIAL", port: 8093}
27+
galaxy-a3: {platform: android, serial: "SERIAL", port: 8094}

0 commit comments

Comments
 (0)