Skip to content

Add Hyperlight containment backend (Hyperlight + Unikraft micro-VM)#276

Merged
MGudgin merged 8 commits into
microsoft:mainfrom
danbugs:pyhl-backend
May 12, 2026
Merged

Add Hyperlight containment backend (Hyperlight + Unikraft micro-VM)#276
MGudgin merged 8 commits into
microsoft:mainfrom
danbugs:pyhl-backend

Conversation

@danbugs

@danbugs danbugs commented May 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a new "containment": "hyperlight" backend that runs Python workloads inside Hyperlight micro-VMs powered by Unikraft unikernels. The guest boots from a pre-warmed CPython 3.12 snapshot (~30 ms on KVM, ~140 ms on WHP) with ~20 pre-imported packages (numpy, pandas, pydantic, etc.) and 17 MB per-VM memory.

  • In-process executionHyperlightScriptRunner links hyperlight-unikraft-host directly.
  • Cross-platform — same code path on Linux (KVM) and Windows (WHP).
  • Host filesystem accessreadonlyPaths/readwritePaths mapped into the guest via Hyperlight preopens.
  • --setup-hyperlight CLI flag on both wxc-exec and lxc-exec to pull and warm the snapshot from GHCR.
  • E2E tests and CIhyperlight-e2e.yml workflow with hello, pandas, and hostfs test cases (gated on WHP).
  • Gated behind --experimental.

Changes

  • wxc_common: add ContainmentBackend::Hyperlight variant, config parser arm, and hyperlight_runner module (policy validation, home resolution, GHCR setup, runtime management, host mounts)
  • wxc-exec / lxc-exec: Hyperlight dispatch arm + --setup-hyperlight / --force flags
  • Test configs: hyperlight_hello.json, hyperlight_pandas.json, hyperlight_fs.json
  • E2E: test_hyperlight_suite with hello, pandas, and hostfs cases; CI workflow using crane for Docker-free GHCR image download
  • Build scripts: --with-hyperlight flag for build.sh and build.bat
  • Design doc: docs/hyperlight-integration-plan.md

@danbugs danbugs force-pushed the pyhl-backend branch 2 times, most recently from c2880f1 to 84de8c6 Compare May 11, 2026 22:50
@danbugs danbugs closed this May 11, 2026
@danbugs danbugs reopened this May 11, 2026
@danbugs danbugs force-pushed the pyhl-backend branch 3 times, most recently from a2aeadb to 8390ee8 Compare May 12, 2026 05:41
Comment thread docs/pyhl-integration-plan.md Outdated
Comment thread src/lxc/src/main.rs Outdated
Comment thread src/lxc/src/main.rs Outdated
Comment thread docs/hyperlight-integration-plan.md
@danbugs danbugs force-pushed the pyhl-backend branch 4 times, most recently from 627257d to 9419788 Compare May 12, 2026 18:07
danbugs added 6 commits May 12, 2026 18:17
Add ContainmentBackend::Hyperlight enum variant and config_parser
dispatch. Gate the hyperlight-unikraft-host dependency behind an
optional 'hyperlight' Cargo feature (x86_64 only).

Signed-off-by: danbugs <danilochiarlone@gmail.com>
Embedded Hyperlight+Unikraft micro-VM runner that executes Python code
via snapshot restore. Supports --setup-hyperlight for eager GHCR pull,
lazy auto-install on first use, and host filesystem mounts via Preopen.

Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
@danbugs danbugs force-pushed the pyhl-backend branch 3 times, most recently from fffea2d to 5deda47 Compare May 12, 2026 18:39
danbugs added 2 commits May 12, 2026 18:59
Add test_hyperlight_suite to wxc_e2e_tests with hello and pandas
cases. Add hyperlight-e2e.yml workflow that builds with the hyperlight
feature, sets up the snapshot, and runs the suite (gated on WHP).

Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
@danbugs danbugs changed the title Add pyhl containment backend (Hyperlight + Unikraft micro-VM) Add Hyperlight containment backend (Hyperlight + Unikraft micro-VM) May 12, 2026
@MGudgin MGudgin merged commit a1225b8 into microsoft:main May 12, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants