What runs inside unkernel: the base initramfs that the kernel bundle
(unkernel.wasm) boots, the distro images it mounts under an overlay root
(with their index distros.json), and the guest programs. The npm package
carries the bundle; the images are release assets that the SDK's
fetchDistro() and the CLI download (.agents/js.md). Behaviour and
invariants: .agents/kernel/guest.md (the images) and
.agents/data-disk.md (the overlay root on the data disk). test/guest/
is unrelated: kwemu's own test programs.
guest/build.sh base # vendor/build/initramfs/base.cpio (+ base-test.cpio)
guest/build.sh distro [NAME...] # vendor/build/distro/NAME.{erofs,ext4,id}; default: all; then index
guest/build.sh index # vendor/build/distro/distros.json for the images there
guest/build.sh list # the distros and their versions
kernel/build.sh base # base, then the bundle vendor/build/bundle/unkernel.wasm
kernel/build.sh distro [NAME...] # the same as guest/build.sh distro| Path | What |
|---|---|
build.sh |
the driver; it finds the distros as distro/*/build.sh |
lib/ |
bash, sourced only: env.sh (paths, cc_guest; sources kernel/tools/env.sh), stage.sh (distro_unpack, install_tree, install_common, install_user, install_readme), image.sh (distro_images: the erofs and ext4 images) |
common/ |
what every distro image has from a source of its own: README (the user's /home/user/README, rendered per distro by install_readme: @KEY@ values, @if DISTRO ... @end blocks) |
tools/ |
host tools: mkcpio.py (cpio archives), mktree.py (an image's tar stream and id), erofs-utils.sh (pinned mkfs.erofs, with erofs-utils-patches/), erofs-trace.py (the files a traced boot read), distros-json.mjs (the index), vm-run.ts (run a command in an image at build time); pins.sh (erofs-utils, lz4, zstd) |
src/ |
the guest programs in the base's kw layer: kwtty, kw9p, kwvsock, kwshm, kwwasi, kwcmd; glimmer, which every distro image gets (install_common, lib/stage.sh); src/test/: kwtest, kwbench, abitest (base-test.cpio) |
base/ |
build.sh, pins.sh (static busybox), initramfs/ (the cpio root: /init, the base shell), rootfs/ (the kw layer, /kw/layer: on top of every distro's root) |
distro/NAME/ |
build.sh, pins.sh, rootfs/ (files added to the image); Debian: tools/ (apt-lz4) |
distros.json (written by guest/build.sh distro and index, and
refreshed by kernel/build.sh base, kernel, smp, lower and
datadisk) indexes the images of the output directory for the base bundle
(the one of KW_BASE_OUT, else vendor/build/bundle/unkernel.wasm); a
release publishes it with the images and the bundle:
{ "abi": 2, "base": "base-<hash>",
"distros": [{ "name": "alpine", "file": "alpine.erofs", "size": 4317184, "sha256": "<hex>", "image": "alpine-3.24.4-<hash>" }] }One entry per distro: NAME.erofs, or NAME.ext4 without one; Alpine
first, then by name. abi is the bundle's KW_ABI_VERSION, base its
base id (/kw/layer/usr/lib/kw-base), image the image id from NAME.id.
initramfs/,base/rootfs/anddistro/*/rootfs/are mirror trees: each file at its installed path and name.install_treecopies one into a staged root without the file's leading header (an optional#!line, then the comment block with the SPDX line, ended by an empty line;#or//comments), so images carry none; a file without comment syntax keeps its header inNAME.license, which is not installed. Mode: 755 with the x bit, else 644 (umask 022); anything else is set by the build script. No heredocs for image content.- What unkernel adds to an image is named
kw-*or lives under/usr/lib/kw/, so the kw layer never shadows a distro's file. The kw layer is RAM-resident: keep it small. Its scripts run the base's busybox (#!/run/kw-lower/bin/busybox sh), so they work on any distro, and only gain features (a distro image may meet a newer base). - Build scripts are
build.sh, pinspins.sh; pins live next to their owner (kernel/build.shkeeps the kernel's and the toolchain's). - Builds are reproducible: an identical rebuild gives byte-identical cpio archives and images, with the same ids.
- An erofs image starts with the distro header (
lib/image.shdistro_hint), 128 bytes at offset 0, before the superblock at 1024:KWDISTRO, version 1, header length 128, then u64 image size and u64prefetch(bytes from the start a boot reads; the image size: fetch it whole; 0: no hint), the distro name (32 bytes) and the image id (64), NUL-padded, little-endian (js/core/distro.ts). The SDK fetches[0, prefetch)ahead when it reads the image over HTTP. The image is laid out so that a boot's reads are one piece at its front: all inodes and directories, then the data of the files a boot to the prompt and a first exec reads (found by booting an uncompressed image of the same tree,tools/vm-run.ts --trace, the same set in every boot so far), without fragments, then the rest (lib/image.sh, the patched mkfs.erofs's--sort=tar --nofrag-first --MZ-front). An image of at mostKW_PREFETCH_WHOLE_MAXbytes (16 MiB) is fetched whole; for a larger oneprefetchis what a boot of the final image reads of that piece, rounded up to 1 MiB. Both need the base bundle and node, like Debian's apt cache; without them the image is not ordered, and a larger one getsprefetch0.
KW_BUILD (output root, default vendor/build), KW_BASE_OUT (the
base's cpio files and, with kernel/build.sh base, unkernel.wasm; not
copied to web/kernel/), KW_DISTRO_OUT (images and staging trees),
KW_DISTRO_FORMATS (erofs ext4), KW_EROFS_COMPRESS, KW_EROFS_OPTS,
KW_EXT4_SLACK, KW_RELEASE=1 (base.cpio without the test programs),
NAME_TARBALL=file (another upstream tarball), KW_DEBIAN_APT_CACHE=0,
KW_DISTRO_HINT=0 (no distro header, no boot layout), KW_DISTRO_TRACE=1
(boot the image here and record distro/NAME/boot-files again: what a
boot reads, laid out first in the erofs image), KW_PREFETCH_WHOLE_MAX. A build beside the
main outputs:
KW_BASE_OUT=/tmp/b kernel/build.sh base
KW_BASE_OUT=/tmp/b KW_DISTRO_OUT=/tmp/d guest/build.sh distro
node js/cli.ts --kernel /tmp/b/unkernel.wasm --distro /tmp/d/alpine.erofs
UNKERNEL_TEST_BASE_KERNEL=/tmp/b/unkernel.wasm UNKERNEL_TEST_DISTRO_DIR=/tmp/d pnpm test:distro
UNKERNEL_TEST_KERNEL=/tmp/b/unkernel.wasm UNKERNEL_TEST_DISTRO_DIR=/tmp/d pnpm test:apidistro/NAME/pins.sh:NAME_VERSION,NAME_URL,NAME_SHA256,NAME_TARBALL_NAME(NAME upper-case), the upstream root file system tarball for arm64 ([a-z0-9]+for the name).distro/NAME/build.sh(executable): sourcelib/env.sh,lib/stage.sh,lib/image.shand the pins;distro_unpack NAME VAR(the tree in$stage,/usr/lib/kw-distrowritten);install_tree "$guest/distro/NAME/rootfs" "$stage"; end withdistro_images NAME "$stage" "$NAME_TARBALL" "NAME-$NAME_VERSION", which writes/usr/lib/kw-imageand prints the image id last. It getsKW_DISTROandKW_DISTRO_DIRfrom the driver.distro/NAME/rootfs/: at least an executable/sbin/init(or what it links to). The base has mounted/dev,/proc,/sys,/run,/tmpand/data; init runs/usr/lib/kw/rc,/usr/lib/kw/welcomeand/usr/lib/kw/net, keeps/usr/bin/kwtty loginon hvc0 (the console's shell, as the image's user) and/usr/bin/kwtty serverunning, and at poweroff stops every process and runs/usr/lib/kw/shutdownbefore reboot(2) (Debian'skw-initis a small example;poweroffmust work for the user, throughsudo -n). The build gives the image sudo (or asudocommand) without a password for the user,install_user(the account, in the distro's audio, video and input groups) andinstall_readme(its values forguest/common/README), passes"${user_owner[@]}"todistro_images, and makes the distro's interactive non-login shells source/usr/lib/kw/shrc. The image must not carry the kw layer's files.guest/build.sh distro NAME, then boot it withnode js/cli.ts --distro NAME;test/distro.test.tslists the distros it runs. Once a release carries it (distros.json lists every image built),fetchDistro("NAME")andnpx unkernel --distro NAMEfind it.