Skip to content

Commit 870066f

Browse files
K2GO-377 chore(rootfs): name the built artifacts k2go_, and sweep the old ones once
The rootfs images carried the iiab-oa_ prefix from the upstream project. They are the product's own artifacts, so they take its name; the stable pointers keep theirs, since latest_<tier>_<arch>.meta4/.installed never carried a product prefix. The app is unaffected: it asks for the pointer and takes the file name and URL from inside the metalink, so it never builds the artifact name itself. The publish loop and the retention prune move to k2go_ together — a prune left filtering the old prefix would let the new artifacts pile up at ~1.7 GB each while still eating the old ones. A transitional sweep clears the pre-rename artifacts in full instead of ageing them out seven deep, and is marked for removal before merge: run the workflow from this branch to migrate the bucket, confirm no iiab-oa_ keys remain, then delete the block so it does not outlive its purpose. ROOTFS_PUBLIC_BASE comes along from K2GO-90: on main it still defaults to a path-based route that was never created, so a migration build run without it would republish metalinks pointing at 404.
1 parent dde0540 commit 870066f

2 files changed

Lines changed: 37 additions & 9 deletions

File tree

‎.github/workflows/bake-rootfs.yml‎

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,13 @@ on:
2626
env:
2727
BUCKET_NAME: k2go-rootfs
2828
R2_ACCOUNT_ID: ${{ vars.CLOUDFLARE_ACCOUNT_ID }}
29-
# Public base that goes into the .meta4 <url>. Path-based route
30-
# k2go-download.appdevforall.org/rootfs/* -> k2go-rootfs.
31-
ROOTFS_PUBLIC_BASE: ${{ vars.ROOTFS_PUBLIC_BASE || 'https://k2go-download.appdevforall.org/rootfs' }}
29+
# Public base that goes into the .meta4 <url> — where the APK actually downloads the
30+
# rootfs FROM, so it has to name the bucket that holds it. The two buckets are separate
31+
# and so are their names: k2go-download.appdevforall.org serves the APK repo only.
32+
# K2GO-90: this defaulted to k2go-download.appdevforall.org/rootfs, a path-based route that
33+
# was never created. The metalinks published with it resolved to 404 — and since
34+
# --reset-mirrors leaves a single source, there was nothing to fall back to.
35+
ROOTFS_PUBLIC_BASE: ${{ vars.ROOTFS_PUBLIC_BASE || 'https://pub-d64c885cef6c42db8c7925144d73d0ee.r2.dev' }}
3236
# Key prefix inside the bucket. Empty when the route strips /rootfs/ (keys at root);
3337
# set to 'rootfs/' if the route keeps the prefix.
3438
R2_KEY_PREFIX: ${{ vars.ROOTFS_KEY_PREFIX }}
@@ -133,7 +137,7 @@ jobs:
133137
134138
# 1) Big artifacts FIRST (data before any pointer that resolves to them).
135139
shopt -s nullglob
136-
for f in iiab-oa_*.tar.gz; do
140+
for f in k2go_*.tar.gz; do
137141
echo ">> upload artifact $f"
138142
aws s3 cp "$f" "s3://${BUCKET_NAME}/${PFX}${f}" --endpoint-url "$ENDPOINT"
139143
done
@@ -157,12 +161,12 @@ jobs:
157161
158162
# 4) Prune: keep only the newest $KEEP batches per (tier,arch); delete older + sidecars.
159163
# Only the heavy dated tarballs accumulate (they carry <date>_<sha>); logs and the
160-
# TSV overwrite in place, so they never pile up. The prefix filter matches iiab-oa_*
164+
# TSV overwrite in place, so they never pile up. The prefix filter matches k2go_*
161165
# only, so proot-distro-v*/ is never touched.
162166
for meta in latest_*_"${ARCH}".meta4; do
163167
base="${meta%.meta4}"; te="${base#latest_}"; tier="${te%_${ARCH}}"
164168
mapfile -t keys < <(aws s3api list-objects-v2 --bucket "$BUCKET_NAME" \
165-
--prefix "${PFX}iiab-oa_" --endpoint-url "$ENDPOINT" \
169+
--prefix "${PFX}k2go_" --endpoint-url "$ENDPOINT" \
166170
--query "sort_by(Contents,&LastModified)[?contains(Key,'_${tier}_') && ends_with(Key,'_${ARCH}.tar.gz')].Key" \
167171
--output text | tr '\t' '\n' | sed '/^$/d')
168172
n=${#keys[@]}
@@ -174,4 +178,26 @@ jobs:
174178
done
175179
done
176180
fi
181+
182+
# K2GO-377 (TRANSITIONAL — remove before merging). The retention above keeps the
183+
# newest $KEEP k2go_ batches; the pre-rename iiab-oa_ artifacts have no pointer left
184+
# once this tier republished, so they go in full rather than ageing out $KEEP deep at
185+
# ~1.7 GB each. It sits INSIDE this loop on purpose: it only ever touches the tier and
186+
# arch this job just republished, so a tier that failed to build keeps the artifact its
187+
# metalink still names.
188+
#
189+
# Removal is gated on evidence, not on this run finishing. Delete the block only when
190+
# aws s3api list-objects-v2 --bucket "$BUCKET_NAME" --prefix "iiab-oa_" \
191+
# --endpoint-url "$ENDPOINT" --query "length(Contents)" --output text
192+
# reports None/0 — otherwise the leftovers outlive the only code that would clear them.
193+
mapfile -t legacy < <(aws s3api list-objects-v2 --bucket "$BUCKET_NAME" \
194+
--prefix "${PFX}iiab-oa_" --endpoint-url "$ENDPOINT" \
195+
--query "Contents[?contains(Key,'_${tier}_') && ends_with(Key,'_${ARCH}.tar.gz')].Key" \
196+
--output text | tr '\t' '\n' | sed '/^$/d')
197+
for key in "${legacy[@]}"; do
198+
echo ">> sweep legacy $key (+ sidecars)"
199+
for ext in '' .meta4 .sha256 .installed .torrent; do
200+
aws s3 rm "s3://${BUCKET_NAME}/${key}${ext}" --endpoint-url "$ENDPOINT" || true
201+
done
202+
done
177203
done

‎tools/rootfs-builder/build-iiab-rootfs.sh‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@
2222
# branch (git ls-remote, then the REST API). --iiab-commit overrides everything.
2323
#
2424
# Output:
25-
# - iiab-oa_<date>_<tier>_<iiab-sha>_<arch>.tar.gz (top-level = installed-rootfs/iiab/)
26-
# - iiab-oa_<...>.tar.gz.meta4 / .torrent (per-artifact, mkmetalink)
25+
# - k2go_<date>_<tier>_<iiab-sha>_<arch>.tar.gz (top-level = installed-rootfs/iiab/)
26+
# - k2go_<...>.tar.gz.meta4 / .torrent (per-artifact, mkmetalink)
2727
# - latest_<tier>_<arch>.meta4 (STABLE pointer the APK requests; a copy of the
2828
# per-artifact .meta4. Inside it, <url> mirrors point at the full-named tarball,
2929
# so this stable name always resolves to the newest build.)
@@ -744,7 +744,9 @@ fi
744744
[[ -n "$IIAB_SHA" ]] || die "Could not determine the iiab/iiab commit id. Pass --iiab-commit <sha>."
745745
log "iiab/iiab id: ${IIAB_SHA} [source: ${IIAB_SRC}]"
746746

747-
ARTIFACT="iiab-oa_${STAMP}_${TIER_NAME}_${IIAB_SHA}_${ARCH}.tar.gz"
747+
# K2GO-377: the artifacts carry the product name. The stable pointers do not change —
748+
# latest_<tier>_<arch>.meta4/.installed never carried a product prefix.
749+
ARTIFACT="k2go_${STAMP}_${TIER_NAME}_${IIAB_SHA}_${ARCH}.tar.gz"
748750
META4="latest_${TIER_NAME}_${ARCH}.meta4"
749751
INSTALLED="latest_${TIER_NAME}_${ARCH}.installed" # ADFA-5110: stable uncompressed-size sidecar
750752

0 commit comments

Comments
 (0)