Skip to content

dotbot: push float32 LH2 calibrations with a site gate, reframe, and capture from the button - #296

Merged
geonnave merged 47 commits into
DotBots:mainfrom
geonnave:lh2-phase3
Sep 23, 2026
Merged

geonnave merged 47 commits into
DotBots:mainfrom
geonnave:lh2-phase3

Conversation

@geonnave

@geonnave geonnave commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

The PyDotBot piece of the LH2 calibration rework: the host sends calibrations as float32 with their site and id, checks what each robot reports before pushing, can re-express a calibration in another site's frame without recapturing, and can take calibration points from the robot's own button.

Lines added Files
Tests +1464 12 incl. a 94-line wire fixture shared with swarmit
Source and the rest +1358 25
Total +2822 (-454) 37

Changes since review. The default validity rectangle, for a calibration that declares none, goes from 4 x 4.5 m to 10 x 10 m, with the shared wire fixture, matching swarmit. The console's calibration step card now follows captures that arrive from a robot's button: it reads the outstanding point off the session's points, as the map does, adopts the pressing robot as the capturing device, and moves to save and push after the last point instead of showing "Step NaN", which came from the session notification dropping outstanding: null.

What changes

  • Float32 on the wire. The swarmit calibration message (84 bytes per station: float32 matrix in mm, valid_mm, a 16-byte site name, an 8-byte id) and the bare-metal homography payload carry float32 matrices. The int32 x 1000 path is removed. dotbot/tests/lh2_wire_fixture.py pins the bytes, and swarmit's packer test carries the same fixture, so the two packers cannot drift apart silently. Breaking: only the float32 firmware of the companion swarmit, DotBot-libs and DotBot-firmware PRs reads these messages.
  • Cable flash writes the float32 config page. dotbot device flash-swarmit-sandbox writes the 632-byte page under swarmit's new config magic; the gateway keeps mari's magic and its 12-byte page, so the magic is chosen per role. Breaking: the option is --lh2-calibration (no -l, no --calibration), matching the controller's option name.
  • A push is gated on device info. A push first reads device info and refuses robots on firmware older than this host expects (device info below v2), naming them and the reflash command, robots that did not answer, and robots reporting another site name unless --site-changed is given. Nothing tries to push to older firmware anyway. calibrate-lh2 push sends unicast to exactly the robots the gate checked, and collect --push to the robots whose captures built the calibration, plus --device. From the console a push goes to the selected robots, or to the whole swarm when none is selected, as flash and start/stop do; the push route takes an optional devices list. After a push it waits up to 30 s for each pushed robot to reset, rejoin and report, and lists those still on the old calibration id, or not heard at all, as the worklist. This replaces the controller's stale_devices, which could only see the per-station bitmask and not which calibration a robot holds.
  • dotbot swarm calibrate-lh2 reframe re-expresses a saved calibration in another site's frame without a capture: --site <name> --shift x,y [--rotate deg] turns every placement's points about the first placement's first point and shifts them (the --shift help now says so), re-solves every station from the samples stored in the file, and writes a new file with a new id. This is how a set of calibrations moves from a per-arena frame to a whole-floor frame, and how a roughly measured offset between the two is refined later.
  • Button capture in calibrate-lh2 collect. While collecting, a press on any robot running the calibrate sandbox app arrives as log events ([0xCB][press:5 | chunk:3][up to 13 records], each capture sent three times). The host deduplicates by (device, press, chunk), assembles each press into 25 reads per station, reports a gap in the press counter, stores the result as the pending point, and gives up a press still incomplete after 20 s, above the paced app's spread for four stations (12.7 s on huge at the earlier half-budget pacing, about half that on the uplink interval). A copy is recognised by its content, so a chunk unlike the one stored starts a new press instead of merging into a stale one, and the late copies of a press given up on stay quiet. The console listens for presses from the moment a session starts, without a capture request first, and stores them under the session lock. A capture missing a station an earlier point saw is stored, as an Enter capture is. collect takes button captures with or without --device: its swarmit client covers the whole swarm, since swarmit drops log events from devices outside a client's own list. The Enter-key path is unchanged and stays bound to --device. calibrate is added to the swarm flash catalogue.
  • Each read's two counts are ordered before averaging. The two sweeps of a read can arrive in either order; averaging them as received mixed the two sweeps whenever one read was swapped, pulling both means toward the middle.
  • Console push target. The Localization tab has a Selection | Stale toggle, remembered per browser, next to its push button: Selection follows the rule above, Stale pushes only to the robots whose device info reports another calibration id or none (robots that cannot report one are listed apart, since the gate would refuse them), and the button always names what it does ("Push to 3 selected", "Push to all 12", "Push to 2 stale"), disabled with the reason shown when nobody is stale.
  • The fake swarmit server used by the console's dev mode answers device info v2.

Merge order

  1. DotBot-libs: bsp: carry LH2 homographies as float32 and fix the LH2 decoder and timer re-arm DotBot-libs#31
  2. swarmit: device: carry float32 LH2 calibrations with a site, and expose raw counts swarmit#165
  3. DotBot-firmware: apps-sandbox: move to the float32 bootloader, add dotbot-next and a button calibration app DotBot-firmware#424
  4. PyDotBot: this PR

The push gate reads swarmit's device-info v2 fields (lh2_site_name, lh2_calibration_id), so this needs a swarmit release that carries the swarmit PR; the swarmit >= 0.9.0 floor in pyproject.toml should move to that release when it exists. No other gate: these can merge when reviewed.

Still to do in this PR after review: the docs sweep (the calibrate-lh2 CLI page for reframe and --site-changed, and the flash option rename).

Validation

pytest dotbot/tests: 759 passed, 6 skipped; one test fails only because the port it binds was held by a controller running on the same machine. pre-commit run --all-files clean. In dotbot/console-web: typecheck, lint and 418 tests pass.

On one DotBot v3 with one lighthouse, on the companion firmware: 12 cable flashes with --lh2-calibration each gave a live position within 1-3 s; device info reported v2 with the file's site name and id; a push of a file naming another site was refused and landed with --site-changed; the worklist printed; a solve outside valid_mm was dropped. collect received button presses at the four corners of a 906 x 963 mm rectangle and saved a calibration with 25 reads per point, a spread of at most 5 counts and a 4-point residual of 2e-5 mm, even when only one of each chunk's three copies arrived. A button capture matched a READY-path capture at the same spot to under 1 count.

Re-run on 2026-09-22 after the review fixes: a console session opened with no robot and no capture request stored a simulated button press from calibrate as its first point and the next three presses as points 1-3, then dropped a fifth press with every point captured.

Re-run after the final review round, 2026-09-22 13:55-14:19, on the same tethered robot and gateway (schedule tiny): two cable flashes of the Debug bootloader and network core with a calibration each reported a fresh boot 2-3 s after the flash tool returned, device info v2 carrying the file's site and id. A dotbot build with a receive counter got 400 of 400 move_raw packets exactly once (100 at 4/s, then 300 at about 20/s), with STATUS flowing and no mari rejoin. A 15-minute window on dotbot had no reset and no NODE_LEFT, swarm status had updated within the last second at each of 58 samples, and 99.6% of the 2/s adverts arrived (largest gap 1.27 s). Two calibration pushes, one to the whole swarm and one to the robot by name, each left an empty worklist within 6 s. The pushes ran through the same SessionDriver.push the console route calls, with devices empty and then naming the robot. pytest dotbot/tests: 778 passed, the port-bound simulator test failing only because a controller held its port; console-web 422 tests, build and lint pass. On the final head, console-web typecheck, lint and 438 tests pass.

Final firmware run, 2026-09-22 17:38-18:19, on the same bench: the cable flash with --lh2-calibration from this branch wrote the float32 config page and the robot reported device info v2 with the file's site and id, and swarm status, swarm info, swarm flash and swarm start/stop from this branch drove the stress, calibrate and dotbot apps on the swarmit PR's final bootloader and network core. The firmware results are in the swarmit PR.

Not covered on hardware: reframe (tests only), more than one station, and a push to a robot still on the old firmware, which is exercised in tests against a faked device-info reply.

Breaking change: the calibration messages and the bare-metal homography
payload carry float32 matrices, and the swarmit message grows to 84
bytes with valid_mm, the site name and the calibration id. Only the
float32 firmware of the same swarmit and DotBot-libs wave reads them.

AI-assisted: Claude Opus 5
Breaking change: flash-swarmit-sandbox takes --lh2-calibration (no -l,
no --calibration). The sandbox page is the 632-byte float32
swarmit_config_t under swarmit's new config magic, which must match the
netcore's; the gateway keeps mari's magic and its 12-byte page.

AI-assisted: Claude Opus 5
A push, from the CLI or the console, now reads device info first and
refuses robots on device info below v2 (int32 firmware), robots that
did not answer, and robots reporting another site unless --site-changed.
Needs swarmit's DeviceInfo with lh2_site_name and lh2_calibration_id.

AI-assisted: Claude Opus 5
The chunk layout mirrors DotBot-firmware's apps-sandbox/calibrate app,
[0xCB][press:5 | chunk:3][up to 13 nine-byte records] with a short last
chunk; the two change together.

AI-assisted: Claude Opus 5
AI-assisted: Claude Opus 5
@codecov

codecov Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.07032% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.72%. Comparing base (237bd0e) to head (8813b4b).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
dotbot/cli/swarm_lh2.py 83.33% 18 Missing ⚠️
dotbot/console-web/src/LocalizationPanel.tsx 90.69% 8 Missing ⚠️
dotbot/console-web/src/CalibrationLayer.tsx 33.33% 4 Missing ⚠️
dotbot/console-web/src/api.ts 0.00% 2 Missing ⚠️
dotbot/console-web/dev/fake_swarmit_server.py 80.00% 1 Missing ⚠️
dotbot/console-web/src/App.tsx 80.00% 1 Missing ⚠️
dotbot/console-web/src/pushTarget.ts 97.29% 1 Missing ⚠️
dotbot/controller.py 87.50% 1 Missing ⚠️
dotbot/tests/test_calibration_ota.py 99.25% 1 Missing ⚠️
dotbot/tests/test_calibration_push.py 99.20% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #296      +/-   ##
==========================================
+ Coverage   82.77%   83.72%   +0.94%     
==========================================
  Files         197      201       +4     
  Lines       23548    24369     +821     
  Branches     1628     1683      +55     
==========================================
+ Hits        19493    20404     +911     
+ Misses       4048     3958      -90     
  Partials        7        7              
Flag Coverage Δ
console 73.51% <91.16%> (+1.22%) ⬆️
frontend 97.80% <ø> (ø)
python 85.16% <97.20%> (+1.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dotbot/cli/_swarm_flash.py 100.00% <ø> (ø)
dotbot/cli/device.py 85.43% <ø> (ø)
dotbot/console-web/src/StepCard.tsx 71.49% <100.00%> (+12.56%) ⬆️
dotbot/console-web/src/TestbedRail.tsx 47.78% <100.00%> (+0.11%) ⬆️
dotbot/console-web/src/calibration.ts 100.00% <100.00%> (ø)
dotbot/console-web/src/localization.ts 100.00% <100.00%> (ø)
dotbot/console-web/src/types.ts 100.00% <ø> (ø)
dotbot/console-web/src/useCalibration.ts 88.57% <100.00%> (+20.05%) ⬆️
dotbot/models.py 100.00% <100.00%> (ø)
dotbot/protocol.py 100.00% <100.00%> (ø)
... and 19 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

geonnave and others added 14 commits September 22, 2026 10:01
A pushed robot resets, and its rejoin moves the device-info generation
that swarmit already refreshes by one broadcast, so the worklist reads
status and sends nothing. A push checked for named robots stays unicast
to exactly those.

AI-assisted: Claude Opus 5
swarmit's tests/lh2_wire_fixture.py carries the same defaults fixture, so
a file without [site] or [validity] packs to the same bytes from either
repo.

AI-assisted: Claude Opus 5
swarmit's in-process controller drops every log event from a device
outside its settings' devices, so a client built for one robot never
saw a button capture from another. Captures name their robot, and
every call that targets robots passes the addresses explicitly.

AI-assisted: Claude Opus 5
collect --push sends to the robots whose captures built the calibration
(and --device) rather than to the fleet, because a push resets every
robot that takes it.

AI-assisted: Claude Opus 5
A console push with no robot chosen no longer goes to the whole swarm;
like collect --push, it goes to the capturing robots and the chosen one.

AI-assisted: Claude Opus 5
notify_clients drops every None, so a complete session reached the
console without `outstanding` and the step card read it as a step
number. The session is now sent with its nulls, as the REST routes do.

AI-assisted: Claude Opus 5.5
@geonnave
geonnave merged commit 8f7a992 into DotBots:main Sep 23, 2026
16 checks passed
@geonnave
geonnave deleted the lh2-phase3 branch September 23, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant