Skip to content

Add local HTTPS staging for Arch Linux ARM mirrors - #3

Open
ryanrhughes wants to merge 4 commits into
masterfrom
feat/archlinuxarm-http-mirror
Open

Add local HTTPS staging for Arch Linux ARM mirrors#3
ryanrhughes wants to merge 4 commits into
masterfrom
feat/archlinuxarm-http-mirror

Conversation

@ryanrhughes

Copy link
Copy Markdown
Contributor

Add a standalone Python 3.9+ command to stage Arch Linux ARM repositories locally over HTTPS. It defaults to aarch64 core, extra, alarm, and aur, using repository databases as package manifests without third-party Python dependencies.

The command verifies package sizes and SHA-256 on downloads and incremental runs, preserves package signatures, and publishes metadata only after all selected packages verify and upstream databases/signatures remain unchanged. It includes stage locking, disk-space checks, bounded concurrent downloads, optional pruning, JSON status reports, and per-repository freshness markers.

Malformed databases and unrecognized .files-only entries fail by default. Explicit recovery options can retain verified packages from an all-NUL record's exact .files counterpart or permit a named orphan, but report degraded status (exit 3), withhold metadata/freshness publication, and skip pruning. Preserving signatures does not itself authenticate them; that remains a pacman/keyring validation step.

Validation:

  • Full local aarch64 inventory: core 311, extra 12,895, alarm 75, aur 12 — 13,293 packages, 50.59 GiB of package data and approximately 51 GiB on disk. The staging filesystem started with 963 GiB free.
  • Observed a rolling upstream update during the first pull: metadata publication was withheld and completed packages were reused on rerun. The latest full recovery run verified all 13,293 packages with zero downloads and correctly exited degraded.
  • A normal core/alarm/aur run verified all 398 packages with zero downloads and updated only those repositories' freshness markers.
  • All 22 integration tests passed locally on Python 3.9 and 3.14, including an isolated pacman consumer test. Python 3.9, Python 3.14, and Arch/pacman CI jobs passed at 9f0bf38.

Remaining upstream blockers: Florida's extra.db still has an all-NUL findnewest-0.3-4/desc; its readable .files counterpart allows package verification but does not repair pacman metadata. extra.files also references qemu-system-cris-9.1.2-1-aarch64.pkg.tar.xz, absent from .db and returning HTTP 404. A normal full sync remains blocked pending upstream correction; the staged tree is not certified ready to serve.

This adds ARM staging, tests, CI, documentation, and ignore rules only. Existing x86_64 and T2 mirror workflows, production services, and R2 uploads are unchanged. Downloaded packages and local reports are Git-ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant