Skip to content

[MicroVM] Update Nanvix to v0.15.4#362

Merged
ppenna merged 3 commits into
microsoft:mainfrom
nanvix:user/ppenna/update-nanvix
May 21, 2026
Merged

[MicroVM] Update Nanvix to v0.15.4#362
ppenna merged 3 commits into
microsoft:mainfrom
nanvix:user/ppenna/update-nanvix

Conversation

@ppenna

@ppenna ppenna commented May 20, 2026

Copy link
Copy Markdown
Contributor

📖 Description

Updates the MicroVM (NanVix) backend to Nanvix 0.15.4 and switches the runner to NanVix's warm-start snapshot protocol, dropping the explicit guest Python command line in favor of a multi-binary python3.initrd. Cold-boot latency on the hot path drops by restoring a pre-generated WHP snapshot instead of re-booting the guest on every execution. Also expands the microvm_staging unit-test coverage to lock in the new single-file bootstrap layout and pre-existing path-rewriting / staging behavior.

Binary layout

  • Replace the flat python3.12 sidecar with a single python3.initrd multi-binary archive (daemons + CPython) loaded by NanVix at warm start.
  • Move kernel.elf into a bin/ subdirectory next to wxc-exec.exe so nanvixd can locate it via -bin-dir as it now expects.
  • Add a snapshots/ subdirectory holding the WHP snapshot pair (kernel.vmem + kernel.whp.cbor).
  • Refresh versions.json / checksums.json for the new release artifacts.

Build-time changes (nanvix_binaries/build.rs)

  • Extend the download/extract path to populate the new bin/ subdir and verify its checksums alongside the flat binaries.
  • After download, pre-generate the WHP snapshots locally via a one-shot nanvixd cold boot so even the first runtime execution is warm.
  • Remove the legacy partial-binary cleanup since the new layout no longer ships those files.

Runtime changes (wxc_common/src/nanvix_runner.rs)

  • Introduce ResolvedPaths and resolve_snapshot_home() that picks $NANVIX_HOME → OS-local data path (%LOCALAPPDATA%\nanvix or ~/.local/share/nanvix) → <exe> as a fallback.
  • If no snapshot is present (fresh install or env override), generate one on demand via cold boot; subsequent runs restore from the snapshot.
  • Spawn nanvixd with -snapshot snapshots/kernel.whp.cbor -bin-dir <exe>/bin -ramfs … -mount … -- python3.initrd, dropping the old python3.12 -S -B …; PYTHONHOME=/sysroot guest cmdline since warm-start auto-executes the initrd entry point.
  • Pipe nanvixd stderr and drain it before wait() to surface guest diagnostics on failure without blocking the child on a full pipe.

Staging changes (wxc_common/src/microvm_staging.rs)

  • Drop the separate .mxc-bootstrap.py + .mxc-script.py pair. The warm-start protocol executes a single bootstrap.py automatically, so the staging layer now writes one file consisting of a small sys.argv preamble followed by the (path-rewritten) user script.

Shared helpers (nanvix_common)

  • Add BIN_SUBDIR, SNAPSHOTS_SUBDIR, BIN_SUBDIR_FILES, and SNAPSHOT_FILES constants describing the new on-disk layout.
  • Add generate_snapshot() (shared by the build script and runtime fallback) and copy_artifacts_to_target() (used by wxc/build.rs to stage flat binaries, bin/, and snapshots/ next to the output exe).
  • Add wxc_commonnanvix_common dependency (Windows-only) and simplify wxc/build.rs to delegate to the shared copy helper.

New microvm_staging tests

  • Path validation: reject .. traversal components in staged paths.
  • Mixed RW + RO staging lands files at the expected guest-relative locations.
  • Staging overhead heuristic: near-zero for empty stagings, 30-second cap honored for oversized inputs.
  • preserved_path() returns None when nothing was preserved.
  • Host→guest path conversion strips trailing slashes and lowercases the drive letter.
  • Script rewriting handles escaped backslash literals (C:\\Users\\…) and matches overlapping prefixes longest-first.
  • build_guest_path() format check for both rw and ro categories.
  • Empty user script produces a bootstrap consisting solely of the preamble.
  • RW copy-back round-trips modified deep files and newly created files back to the host.
  • sweep_orphaned_staging_dirs() is a no-op on a non-existent root.
  • Concurrent stagings under the same root get unique directory names.

🔗 References

Related docs: docs/nanvix-microvm/nanvix.md, docs/nanvix-microvm/nanvix-integration-plan.md.

🔍 Validation

  • cargo build --release --features microvm on x86_64-pc-windows-msvc — fetches the 0.15.4 artifacts, verifies checksums, generates the WHP snapshot at build time, and stages bin/ + snapshots/ next to wxc-exec.exe.
  • cargo test -p wxc_common — new and existing microvm_staging unit tests pass.
  • Manual smoke: examples/01_hello_world.json and a multi-file RW/RO config execute end-to-end on the warm-start path; cold-boot fallback exercised by deleting snapshots/ and rerunning.
  • Verified $NANVIX_HOME override and the %LOCALAPPDATA%\nanvix default both resolve correctly and trigger on-demand snapshot generation when missing.

✅ Checklist

📋 Issue Type

  • Bug fix
  • Feature
  • Task
Microsoft Reviewers: Open in CodeFlow

@ppenna ppenna requested review from esaurez and huzaifa-d May 20, 2026 21:47
@ppenna ppenna self-assigned this May 20, 2026
Copilot AI review requested due to automatic review settings May 20, 2026 21:47
@ppenna ppenna changed the title [MicroVM] Update Nanvix to Version **0.15.4** [MicroVM] Update Nanvix to v0.15.4 May 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Windows MicroVM (NanVix) backend to NanVix 0.15.4, switching execution to NanVix’s warm-start snapshot protocol and updating the on-disk artifact layout and staging layer accordingly.

Changes:

  • Update NanVix artifact layout to use python3.initrd, add bin/ (for kernel.elf) and snapshots/, and refresh pinned versions/checksums.
  • Generate WHP snapshots at build time (and on-demand at runtime if missing) and invoke nanvixd using -snapshot ... -bin-dir ... -- python3.initrd.
  • Simplify MicroVM staging to produce a single bootstrap.py (preamble + rewritten user script) and expand unit tests.
Show a summary per file
File Description
src/wxc/build.rs Switch to shared helper for copying NanVix artifacts (flat + bin/ + snapshots/) next to wxc-exec.exe.
src/wxc_common/src/nanvix_runner.rs Add snapshot-home resolution, warm-start invocation, and stderr piping/draining.
src/wxc_common/src/microvm_staging.rs Replace two-file staging (.mxc-*) with single bootstrap.py; add extensive new tests.
src/wxc_common/Cargo.toml Add nanvix_common dependency (Windows-only) for shared constants/helpers.
src/nanvix_common/src/lib.rs Introduce new layout constants, snapshot generation helper, and artifact copy helper.
src/nanvix_binaries/build.rs Update download/extract for new layout and generate snapshots at build time.
src/nanvix_binaries/versions.json Bump NanVix release tag and update expected binary list.
src/nanvix_binaries/checksums.json Refresh SHA256 hashes (including kernel.elf).
src/Cargo.lock Add nanvix_common to the lockfile dependency graph.

Copilot's findings

  • Files reviewed: 8/9 changed files
  • Comments generated: 6

Comment thread src/wxc_common/src/nanvix_runner.rs Outdated
Comment thread src/wxc_common/src/nanvix_runner.rs Outdated
Comment thread src/nanvix_common/src/lib.rs
Comment thread src/wxc_common/src/microvm_staging.rs Outdated
Comment thread src/wxc_common/src/nanvix_runner.rs
Comment thread src/wxc_common/src/nanvix_runner.rs Outdated
@ppenna ppenna changed the title [MicroVM] Update Nanvix to v0.15.4 [MicroVM] Update Nanvix to Version v0.15.4 May 20, 2026
@ppenna ppenna requested a review from goiri May 20, 2026 22:20
@ppenna ppenna marked this pull request as draft May 20, 2026 22:21
@ppenna ppenna force-pushed the user/ppenna/update-nanvix branch from 3d77113 to a9fdd73 Compare May 20, 2026 23:39
@ppenna ppenna changed the title [MicroVM] Update Nanvix to Version v0.15.4 [MicroVM] Update Nanvix to v0.15.4 May 21, 2026
@ppenna ppenna force-pushed the user/ppenna/update-nanvix branch from 520de4f to caa3025 Compare May 21, 2026 14:40
@ppenna ppenna marked this pull request as ready for review May 21, 2026 17:32
Comment thread src/wxc_common/Cargo.toml Outdated
Comment thread src/wxc_common/src/nanvix_runner.rs
@ppenna ppenna requested a review from huzaifa-d May 21, 2026 19:16
ppenna added 3 commits May 21, 2026 12:49
Updates the Nanvix toolchain from 0.12.529 to 0.15.4 and switches the
runner to Nanvix's warm-start snapshot protocol, dropping the explicit
guest Python command line in favor of a multi-binary initrd.

Binary layout changes:
- Replace flat `python3.12` sidecar with a single `python3.initrd`
  multi-binary archive (daemons + CPython) loaded by Nanvix at warm start.
- Move `kernel.elf` into a `bin/` subdirectory next to wxc-exec.exe so
  that nanvixd can locate it via `-bin-dir` as it now expects.
- Add a `snapshots/` subdirectory holding the WHP snapshot pair
  (`kernel.vmem` + `kernel.whp.cbor`) used to restore VM state directly
  instead of cold-booting on every execution.
- Refresh `versions.json` / `checksums.json` for the new release artifacts.

Build-time changes (`nanvix_binaries/build.rs`):
- Extend the download/extract path to populate the new `bin/` subdir and
  verify its checksums alongside the flat binaries.
- After download, pre-generate the WHP snapshots locally via a one-shot
  nanvixd cold boot so even the first runtime execution is warm.
- Remove the legacy partial-binary cleanup since the new layout no longer
  ships those files.

Runtime changes (`wxc_common/src/nanvix_runner.rs`):
- Introduce a `ResolvedPaths` struct and `resolve_snapshot_home()` that
  picks `$NANVIX_HOME` → OS-local data path (`%LOCALAPPDATA%\nanvix` or
  `~/.local/share/nanvix`) → `<exe>` as a fallback.
- If no snapshot is present (e.g. fresh install or env override), generate
  one on demand via cold boot; subsequent runs restore from the snapshot.
- Spawn nanvixd with `-snapshot snapshots/kernel.whp.cbor -bin-dir
  <exe>/bin -ramfs … -mount … -- python3.initrd`, dropping the old
  `python3.12 -S -B …;PYTHONHOME=/sysroot` guest cmdline since the
  warm-start protocol auto-executes the initrd entry point.
- Pipe nanvixd stderr and drain it before `wait()` to surface guest
  diagnostics on failure without blocking the child on a full pipe.

Staging changes (`wxc_common/src/microvm_staging.rs`):
- Drop the separate `.mxc-bootstrap.py` + `.mxc-script.py` pair. The
  warm-start protocol executes a single `bootstrap.py` automatically, so
  the staging layer now writes one file consisting of a small `sys.argv`
  preamble followed by the (path-rewritten) user script.
- Update tests for the single-file layout.

Shared helpers (`nanvix_common`):
- Add `BIN_SUBDIR`, `SNAPSHOTS_SUBDIR`, `BIN_SUBDIR_FILES`, and
  `SNAPSHOT_FILES` constants describing the new on-disk layout.
- Add `generate_snapshot()` (shared by the build script and runtime
  fallback) and `copy_artifacts_to_target()` (used by `wxc/build.rs` to
  stage flat binaries, `bin/`, and `snapshots/` next to the output exe).
- Add `wxc_common` → `nanvix_common` dependency (Windows-only) and
  simplify `wxc/build.rs` to delegate to the shared copy helper.
Adds additional unit tests for the microvm staging layer to lock in
existing behavior and guard against regressions. No production code
changes.

New tests in `wxc_common::microvm_staging`:
- Path validation: reject `..` traversal components in staged paths
  (`staging_rejects_path_with_parent_dir_component`).
- Mixed RW + RO staging: both subdirectories are created and file
  contents land at the expected guest-relative locations
  (`staging_mixed_rw_and_ro_paths`).
- Staging overhead heuristic: near-zero overhead for empty stagings and
  the 30-second cap is honored for oversized inputs
  (`staging_overhead_ms_scales_with_size`,
  `staging_overhead_ms_capped_at_30s`).
- `preserved_path()` returns `None` when nothing was preserved.
- Host→guest path conversion: trailing slashes are stripped and the
  drive letter is lowercased
  (`host_path_to_guest_relative_handles_trailing_slash`,
  `host_path_to_guest_relative_lowercase_drive`).
- Script rewriting: escaped backslash literals (`C:\\Users\\…`) are
  rewritten, and overlapping prefixes are matched longest-first so a
  shorter mapping cannot partially shadow a longer one
  (`rewrite_paths_handles_escaped_backslashes`,
  `rewrite_paths_longer_prefix_first`).
- `build_guest_path()` format check for both `rw` and `ro` categories.
- Empty user script produces a bootstrap consisting solely of the
  preamble (`staging_empty_script`).
- RW copy-back round-trips modified deep files and newly created files
  back to the host (`staging_nested_directory_rw_copyback`).
- `sweep_orphaned_staging_dirs()` is a no-op on a non-existent root.
- Concurrent stagings under the same root get unique directory names
  (`staging_dir_has_unique_names`).
The "Verify MicroVM binaries" step in the MicroVM E2E workflow still
asserted the pre-0.15.4 on-disk layout and would fail (or silently miss
files) after a309530, which switched
the runner to Nanvix's warm-start snapshot protocol and reorganized the
staged artifacts.

Align the required-file list with the new layout described by
`nanvix_common` (`REQUIRED_BINARIES`, `BIN_SUBDIR_FILES`, `SNAPSHOT_FILES`):

- Drop the flat `python3.12` sidecar; replace with `python3.initrd`
  (the multi-binary initrd loaded by Nanvix at warm start).
- Move `kernel.elf` under the new `bin\` subdirectory that nanvixd
  locates via `-bin-dir`.
- Add the WHP snapshot pair under `snapshots\` (`kernel.vmem` and
  `kernel.whp.cbor`) that `nanvix_binaries/build.rs` now pre-generates
  so the first runtime execution is warm.

Also adjust the `Get-ChildItem -Include` listing to use leaf names
(since `-Include` matches against the file name, not a subpath) and
print `FullName` so the subdirectory placement is visible in CI logs.

No behavior change for the test run itself; this only fixes the
verification gate that guards it.
@ppenna ppenna force-pushed the user/ppenna/update-nanvix branch from ac55ac9 to 9ad8ba0 Compare May 21, 2026 19:55
@ppenna ppenna merged commit 67e012c into microsoft:main May 21, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants