Skip to content

Add Incus Container Support as an Alternative to LXD#47

Open
tejashgawasibm wants to merge 4 commits into
IBM:mainfrom
tejashgawasibm:feat/incus
Open

Add Incus Container Support as an Alternative to LXD#47
tejashgawasibm wants to merge 4 commits into
IBM:mainfrom
tejashgawasibm:feat/incus

Conversation

@tejashgawasibm

Copy link
Copy Markdown
Member

This PR adds Incus container support as an alternative to LXD for building GitHub Actions runner images. The implementation includes Incus installation, LVM-backed storage support, architecture-specific preseed configurations, and compatibility fixes required for distrobuilder-based Ubuntu images.

What's Changed

Incus Support

  • Add Incus installation scripts for Ubuntu and CentOS
  • Add incus_container.sh build workflow with LXD-compatible behavior
  • Add Incus preseed configurations for ppc64le, s390x, and x86_64 architectures
  • Integrate Incus Container option into run.sh
  • Add LVM-backed storage support for Incus image builds

Compatibility & Reliability Fixes

  • Fix distrobuilder image compatibility (APT sources format and networking configuration)
  • Fix Maven download reliability with archive.apache.org fallback
  • Fix script execution order dependency (install git before runner package configuration)
  • Add graceful error handling for Apache and Ansible installation on ppc64le
  • Add runner test failure workaround for ppc64le architecture

Testing

Validated on CentOS Stream 10 (ppc64le) with Incus v7.1.

  • Built Ubuntu 24.04 runner images using Incus containers
  • Validated both minimal and complete image generation workflows
  • Verified LVM-backed storage pool configuration and image publishing
  • Tested distrobuilder-based Ubuntu 24.04 image compatibility
  • Successfully registered self-hosted runners
  • Executed GitHub Actions workflows end-to-end
  • Verified image export and reuse workflow

This enables building and validating GitHub Actions runner images using Incus while maintaining compatibility with existing LXD-based workflows.

@tejashgawasibm tejashgawasibm force-pushed the feat/incus branch 2 times, most recently from 8f30d93 to 517af9a Compare June 26, 2026 07:10
Add Incus container build support as an alternative to LXD, maintaining
API compatibility while supporting the community-led Incus fork.

Core Components:
- Incus installation scripts for Ubuntu and CentOS (builds from source)
- incus_container.sh build script with LXD-compatible workflow
- Preseed configurations for ppc64le, s390x, x86_64 architectures
- Flexible version configuration via environment variables
  (default: INCUS_VERSION=v7.1.0, RAFT_VERSION=v0.22.1)
- Configurable LVM storage with loop device support
- Idempotent initialization for safe re-runs

Features:
- Multi-architecture support (ppc64le, s390x, x86_64)
- Automatic Incus installation if not present
- LVM thin provisioning with configurable pool size
- Network isolation with dedicated bridge (incusbr0)
- Compatible with distrobuilder-generated images

Tested on:
- Host: CentOS Stream 10 (ppc64le)
- Incus: v7.1
- Container: Ubuntu 24.04 LTS

Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
…un.sh

Integration:
- Add Incus Container option to run.sh interactive menu
- Add Incus-specific command-line flags (--incus-debug, --skip-incus-*)
- Add Incus variables to setup_vars.sh for build customization

Distrobuilder Compatibility Fixes:
- Support both deb822 (ubuntu.sources) and classic (sources.list) APT formats
- Remove manual netplan configuration (distrobuilder images pre-configured)
- Add software-properties-common to vital packages for add-apt-repository
- Fix script execution order: install git before runner-package build

Architecture-Specific Fixes (ppc64le):
- Add graceful error handling for Apache installation (not available on ppc64le)
- Add non-fatal error handling for Ansible/pipx package installations
- Add runner test failure workaround (tests fail but binary works)
- Add defensive checks for optional tools (yarn, npm, needrestart)

Build Reliability Improvements:
- Add Maven download fallback to archive.apache.org
- Add tar export cleanup to prevent read-only file conflicts
- Replace Incus preseed with idempotent imperative commands
- Fix group membership to conditionally add docker/incus groups

These fixes ensure compatibility with distrobuilder-generated images
and handle architecture-specific issues on ppc64le, s390x, and x86_64.

Tested on:
- Host: CentOS Stream 10 (ppc64le)
- Container: Ubuntu 24.04 (complete setup)
- Incus: v7.1

Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
@tejashgawasibm tejashgawasibm force-pushed the feat/incus branch 3 times, most recently from ee3e6dc to 73a4756 Compare June 26, 2026 11:21
@mtarsel mtarsel self-assigned this Jun 26, 2026
Comment thread images/centos/scripts/build/install-incus.sh Outdated
Comment thread images/centos/scripts/build/install-incus.sh Outdated
Comment thread images/ubuntu/scripts/build/install-incus.sh Outdated
Comment thread scripts/incus_container.sh
@rahulssv-ibm

rahulssv-ibm commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

@tejashgawasibm, we also have to validate the changes made in https://github.com/actions/runner-images/tree/main/images/ubuntu/scripts/build shell scripts and try to sync those with our scripts. Both the repos should be on same page.

Or we can have separate PR to sync the upstream changes. You can refer this PR
ppc64le/gaplib#39

Add comprehensive Incus container/VM support with the following features:

- Hybrid preseed/manual initialization approach
  * Try preseed first on fresh installations (no storage/network)
  * Fallback to manual configuration for partial states or preseed failures
  * Maintains idempotency for re-runs and partial configurations

- Address PR review suggestions:
  * Reduce LVM storage size from 180GiB to 100GiB for boot volume compatibility
  * Use helper functions (install_dpkgs, install_dnfpkgs, update_dpkgs) for package management
  * Add image type filter to incus_container.sh to distinguish containers from VMs

- Sync with upstream actions/runner-images:
  * Pin sbt to 1.x versions to avoid compatibility issues

- Code quality improvements:
  * Replace 'cat file | command' with 'command < file' (ShellCheck SC2002)
  * Fix network existence check logic

Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
@tejashgawasibm tejashgawasibm force-pushed the feat/incus branch 2 times, most recently from 5da53a7 to 0416ea0 Compare July 1, 2026 15:26
…tainer

- Auto-select image import method based on host architecture
- Use Incus image server for x86_64/aarch64 (fast, ~30s)
- Use Distrobuilder for ppc64le/s390x (custom build, ~10-15min)
- Simplify the Ubuntu image import workflow

Helper Scripts:
- import_ubuntu_base_images.sh: Main wrapper with auto-detection
  * Detects architecture and routes to appropriate method
  * Can be sourced by incus_container.sh, incus_vm.sh, etc.
  * Reusable across different Incus deployment types

- import-incus-ubuntu-image.sh: Incus image server import
  * For x86_64/aarch64 architectures

- build-distrobuilder-image.sh: Custom image builder
  * For ppc64le/s390x architectures

Signed-off-by: tejashgawasibm <Tejash.Gawas@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants