Problem Description
On Linux hosts, running just build-assets can fail or cause subsequent permission issues due to three distinct bugs in the build chain:
- Missing
npx dependency: The builder backend uses cdxgen to generate the CycloneDX OBOM. It defaults to using npx to run it, which fails if Node/npm is installed without npx (common on some Debian/Ubuntu setups), even though pnpm (which is a project prerequisite) is available.
- Root-owned assets: When building the EROFS rootfs,
mkfs.erofs runs inside a Docker container. Because the output directory is mounted, the generated rootfs.erofs file is owned by root:root on Linux hosts. This prevents non-root host processes (like just smoke / create_hash_assets.py) from creating hardlinks to it, resulting in PermissionError: Operation not permitted.
- ReadOnly Binary Overwrite: Native agent cross-compilation fails with
Permission Denied when trying to overwrite existing read-only (555) binaries in target/linux-agent/.
Steps to Reproduce
- Ensure
npx is not installed, but pnpm is.
- Run
just build-assets code x86_64. It will fail with FileNotFoundError: [Errno 2] No such file or directory: 'npx'.
- (If
npx is present or bypassed) Run just build-assets code x86_64 as non-root on Linux.
- Run
just smoke. It will fail with PermissionError: [Errno 1] Operation not permitted when trying to link rootfs.erofs.
Proposed Solution / Fixes Applied
We have implemented the following fixes in src/capsem/builder/docker.py:
- cdxgen Fallback: Modified
_cdxgen_command to check for npx availability and fallback to pnpm dlx @cyclonedx/cdxgen@latest if npx is missing.
- Restore Ownership: Added a
chown step inside the EROFS builder container to change the ownership of the output file back to the host user's UID:GID when running on Linux.
- Unlink before copy: Updated
cross_compile_agent to unlink the destination binary before copying to avoid permission errors when overwriting read-only files.
diff --git a/src/capsem/builder/docker.py b/src/capsem/builder/docker.py
index ea948bb7..abf418d7 100644
--- a/src/capsem/builder/docker.py
+++ b/src/capsem/builder/docker.py
@@ -458,6 +458,10 @@ def create_erofs(
level_flag = f",level={compression_level}" if compression_level else ""
mkdir_output = "" if out_dir == "." else f"mkdir -p /assets/{out_dir} && "
+ chown_cmd = ""
+ if sys.platform != "darwin" and hasattr(os, "getuid") and hasattr(os, "getgid"):
+ chown_cmd = f" && chown {os.getuid()}:{os.getgid()} /assets/{out_rel}"
+
run_cmd([
runtime, "run", "--rm",
"-v", f"{common_dir}:/assets",
@@ -467,10 +471,11 @@ def create_erofs(
f"DEBIAN_FRONTEND=noninteractive apt-get install -y erofs-utils && "
f"mkdir /rootfs && {mkdir_output}tar xf /assets/{tar_rel} -C /rootfs && "
f"mkfs.erofs -Enosbcrc -z{compression}{level_flag}{cluster_flag} "
- f"/assets/{out_rel} /rootfs",
+ f"/assets/{out_rel} /rootfs{chown_cmd}",
])
+
def erofs_utils_image_for(compression: str) -> str:
"""Return the container image used to create an EROFS image."""
if compression == "zstd":
@@ -633,7 +638,10 @@ def cross_compile_agent(
if not src.exists():
raise RuntimeError(f"Expected binary not found: {src}")
dst = output_dir / binary
+ if dst.exists():
+ dst.unlink()
shutil.copy2(str(src), str(dst))
+
if dst.stat().st_size == 0:
raise RuntimeError(f"Binary is empty: {dst}")
copied.append(dst)
@@ -717,11 +725,22 @@ def _cdxgen_command() -> list[str]:
default uses npm's package runner so the rootfs build does not depend on a
globally installed binary.
"""
- configured = os.environ.get("CAPSEM_CDXGEN_CMD", "npx --yes @cyclonedx/cdxgen@latest")
- command = shlex.split(configured)
- if not command:
- raise RuntimeError("CAPSEM_CDXGEN_CMD must not be empty")
- return command
+ configured = os.environ.get("CAPSEM_CDXGEN_CMD")
+ if configured:
+ command = shlex.split(configured)
+ if not command:
+ raise RuntimeError("CAPSEM_CDXGEN_CMD must not be empty")
+ return command
+
+ if shutil.which("npx"):
+ return ["npx", "--yes", "@cyclonedx/cdxgen@latest"]
+ elif shutil.which("pnpm"):
+ return ["pnpm", "dlx", "@cyclonedx/cdxgen@latest"]
+ else:
+ raise RuntimeError(
+ "Neither 'npx' nor 'pnpm' found in PATH. Cannot run cdxgen to generate OBOM."
+ )
+
def _validate_cyclonedx_obom(path: Path) -> None:
Problem Description
On Linux hosts, running
just build-assetscan fail or cause subsequent permission issues due to three distinct bugs in the build chain:npxdependency: The builder backend usescdxgento generate the CycloneDX OBOM. It defaults to usingnpxto run it, which fails if Node/npm is installed withoutnpx(common on some Debian/Ubuntu setups), even thoughpnpm(which is a project prerequisite) is available.mkfs.erofsruns inside a Docker container. Because the output directory is mounted, the generatedrootfs.erofsfile is owned byroot:rooton Linux hosts. This prevents non-root host processes (likejust smoke/create_hash_assets.py) from creating hardlinks to it, resulting inPermissionError: Operation not permitted.Permission Deniedwhen trying to overwrite existing read-only (555) binaries intarget/linux-agent/.Steps to Reproduce
npxis not installed, butpnpmis.just build-assets code x86_64. It will fail withFileNotFoundError: [Errno 2] No such file or directory: 'npx'.npxis present or bypassed) Runjust build-assets code x86_64as non-root on Linux.just smoke. It will fail withPermissionError: [Errno 1] Operation not permittedwhen trying to linkrootfs.erofs.Proposed Solution / Fixes Applied
We have implemented the following fixes in
src/capsem/builder/docker.py:_cdxgen_commandto check fornpxavailability and fallback topnpm dlx @cyclonedx/cdxgen@latestifnpxis missing.chownstep inside the EROFS builder container to change the ownership of the output file back to the host user's UID:GID when running on Linux.cross_compile_agentto unlink the destination binary before copying to avoid permission errors when overwriting read-only files.