Skip to content

Commit 9dd882c

Browse files
K2GO-394 docs(maps): record why the maps role is not patched (dash-node download)
Add ADR-K2GO-394 and a "Notable non-patches" note in the upstream-patches README so the deliberate absence of a maps download patch is discoverable, not mistaken for an oversight. On the K2Go device path dash-node pre-downloads the base-map pmtiles and the stock maps role skips those downloads via `creates:`; the role reads exactly as upstream ships it. An earlier is_proot gate + assert broke the CI rootfs bake -- is_proot is True for the Android tiers there too, but no dash-node runs -- so the role stays stock and the device-vs-upstream divergence is documented instead of patched.
1 parent 1b56601 commit 9dd882c

2 files changed

Lines changed: 131 additions & 0 deletions

File tree

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# ADR-K2GO-394 -- Base-map downloads run through dash-node, not the maps role
2+
3+
## Status
4+
5+
Accepted (2026-09).
6+
7+
## Context
8+
9+
The maps SETUP downloads large whole-world base-map pmtiles (vector, satellite,
10+
terrain -- gigabytes at high zoom). Upstream's maps role downloads them in-proot
11+
through `roles/maps/tasks/download_large_file.yml` (an `aria2c` over a `.meta4`
12+
metalink).
13+
14+
On Android the role runs in a proot with a mobile radio. An in-proot `aria2c`
15+
that the app drove over JSON-RPC wedged at zero connections on a full network
16+
drop and never recovered (K2GO-394): it did not exit, so nothing could retry it,
17+
and the SETUP hung. We need the base-map download to be resilient (survive a
18+
radio drop) and controllable (pause / resume / live progress) -- the same
19+
properties ZIM and Kolibri downloads already have.
20+
21+
The box already has an engine for exactly this: **dash-node**, the in-server
22+
durable job engine (ADR-4832, ADR-4893). It downloads with `aria2 --continue`
23+
plus an outer reconnect loop, reports "Reconnecting n/5", and survives a client
24+
disconnect. ZIM, Books and Kolibri downloads all run through it (LIVE-REST class,
25+
ADR-5061); the app POSTs and polls, the box owns the download.
26+
27+
## Decision
28+
29+
**Route the base-map download through dash-node, ahead of the maps runrole, and
30+
do NOT patch the maps role.**
31+
32+
1. **dash-node** gains a `basemaps` job (see the dash-node CHANGELOG, 1.3.2 /
33+
1.3.3): given a bare pmtiles file name it composes the mirror URL and
34+
downloads into the maps serve dir (`/library/www/maps`) with the proven kiwix
35+
reconnect mechanism.
36+
2. **The app** orchestrates it, the same way it drives ZIM (`RestContentClient`):
37+
for the maps module it resolves the selected layers to file names from the
38+
catalog, POSTs `{ids:[...]}` to dash-node, shows the live download bar, and
39+
only when the download finishes does it run the maps runrole. The download and
40+
the runrole are two sequential phases, not one.
41+
3. **Ansible (the maps role)** stays STOCK. Because dash-node has already placed
42+
each pmtiles at its `dest_path`, the role's native `creates: dest_path` (and
43+
the `.meta4` size-probe's own existence check) SKIP those downloads. The role
44+
only post-processes (symlinks, `maps-config.js`). Nothing in the role changes.
45+
46+
So three parts share the work -- ansible post-processes, dash-node downloads, the
47+
app orchestrates -- mirroring the live-download pattern the rest of the app uses.
48+
49+
## Why the role is NOT patched
50+
51+
An earlier version patched `download_large_file.yml`: it gated the download
52+
`when: not is_proot` and asserted dash-node pre-placement `when: is_proot`. This
53+
**broke the CI rootfs bake**.
54+
55+
`is_proot` is `True` for the Android tiers (`vars/local_vars_android_*.yml`), and
56+
the bake builds those tiers. But the bake has **no dash-node and no app** -- it
57+
builds the rootfs image and is meant to download the base maps itself. With the
58+
patch, the bake skipped the downloads and the `is_proot` assert failed:
59+
60+
```
61+
TASK [maps : Fail if dash-node did not place maps.black-component.js on proot]
62+
fatal: assertion 'proot_basemap.stat.exists' failed
63+
```
64+
65+
`is_proot` does not distinguish "a live device with dash-node" from "a CI bake
66+
without it". Both are proot. So no `is_proot` condition -- inline or in a separate
67+
task file -- is correct here.
68+
69+
The native `creates:` skip needs no condition and is right in both contexts:
70+
71+
- **Device:** the app pre-places the selected pmtiles through dash-node, so
72+
`creates:` skips them; the role post-processes.
73+
- **Bake:** nothing is pre-placed, so the role downloads everything itself over
74+
the CI runner's stable network, exactly as upstream intends.
75+
76+
Patching an upstream role only to say "we do not run this step here" would also
77+
be a carry with no upstream value -- the opposite of the upstream-first policy in
78+
`tools/upstream-patches/README.md`.
79+
80+
## The search tarball stays in-proot
81+
82+
The maps role also downloads the static-search database through the same task
83+
with `expand_archive=true` (a `.tar.gz` it extracts). dash-node downloads files,
84+
not archives -- it does not extract -- so search is **not** delegated. It
85+
downloads AND extracts in-proot as upstream does. It is small (~16 MB), so the
86+
in-proot download's exposure to a radio drop is short; a drop there fails the
87+
role and the install's existing Retry re-runs it.
88+
89+
## Consequences
90+
91+
- The maps role reads exactly as upstream ships it. This divergence -- that on
92+
the K2Go device path the base maps arrive from dash-node, not from the role --
93+
is invisible in the role itself, so it is recorded here and pointed to from
94+
`tools/upstream-patches/README.md` (which is where a maintainer looks and finds
95+
no maps patch).
96+
- Anything the app does NOT delegate (the search tarball, the small map JS
97+
components, or a selected layer the catalog cannot resolve) downloads in-proot
98+
with the stock role behavior. The big, selected pmtiles -- the ones worth many
99+
gigabytes and the reason this ticket exists -- are the ones delegated, so the
100+
wedge-prone case is covered.
101+
- The three-part split (ansible + dash-node + app) is more moving parts than a
102+
lone role, but it is the same pattern ZIM/Books/Kolibri already use, so it is
103+
not new machinery -- just a new content type on it.
104+
105+
## References
106+
107+
- ADR-4832 (live content channel / single proot dash-node core)
108+
- ADR-4893 (download execution and user control)
109+
- ADR-5061 (LIVE-REST vs STOPPED-proot operation model)
110+
- dash-node CHANGELOG: 1.3.2 (basemaps runner), 1.3.3 (file-id + URL composition)
111+
- `tools/upstream-patches/README.md` (why there is no maps download patch)
112+
- Jira: K2GO-394

‎tools/upstream-patches/README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,3 +77,22 @@ When an upstream PR merges and ships in the pinned `iiab/iiab` commit, its patch
7777
no-op (reverse-dry-run skip). At the next maintenance pass, read each patch's
7878
`Upstream-Status`, delete the ones marked merged, and bump the pinned commit. Keep the set
7979
small.
80+
81+
## Notable non-patches
82+
83+
Sometimes the right carry is **no patch at all** -- recorded here so a deliberate absence is
84+
not mistaken for an oversight.
85+
86+
- **Maps base-map download (K2GO-394) -- no patch, on purpose.** Upstream's maps role
87+
downloads the base-map pmtiles in-proot through `roles/maps/tasks/download_large_file.yml`.
88+
On the K2Go device path, dash-node (the in-server durable job engine) pre-downloads them --
89+
app-driven, resilient, resumable -- into the maps serve dir BEFORE the runrole, so the role's
90+
native `creates: dest_path` skips those downloads and it only post-processes. The role reads
91+
exactly as upstream ships it. We deliberately do NOT patch it: an earlier `is_proot`
92+
gate + assert broke the CI rootfs bake, where `is_proot` is `True` for the Android tiers too
93+
(`vars/local_vars_android_*.yml`) but no dash-node runs -- so the bake must download the base
94+
maps itself, which the stock role does. `is_proot` cannot tell "device with dash-node" from
95+
"CI bake without it"; `creates:` needs no such flag and is correct in both. Rationale:
96+
`controller/docs/ADR-K2GO-394-maps-download-via-dashnode.md`. (The search tarball is the one
97+
map file NOT delegated -- dash-node does not extract archives -- so it still downloads and
98+
extracts in-proot as upstream does.)

0 commit comments

Comments
 (0)