Skip to content

Outer builder container: agent-server + rootless podman image - #3

Closed
alexanderkreidich wants to merge 7 commits into
mainfrom
builder-container
Closed

Outer builder container: agent-server + rootless podman image#3
alexanderkreidich wants to merge 7 commits into
mainfrom
builder-container

Conversation

@alexanderkreidich

@alexanderkreidich alexanderkreidich commented Jun 10, 2026

Copy link
Copy Markdown

Summary

  • Implements items 1, 2 and 4 of builder-container-architecture.md's "What Needs to Be Built": the outer container Dockerfile (Ubuntu 24.04 + NodeSource 22 + rootless podman with fuse-overlayfs, non-root builder user with subuid/subgid ranges), the canonical spawn contract docker/builder/run.sh, and the builder system prompt AGENTS.builder.md baked as an image template and installed at ${WORKSPACE_DIR}/.pi-global/AGENTS.md on startup.
  • run.sh runs the container without --privileged: --device /dev/fuse --device /dev/net/tun --cap-add SYS_ADMIN --security-opt seccomp/apparmor/systempaths=unconfined, named volumes for /workspace and podman storage, ports 4001 + 3000-3010, provider keys via -e. Each flag is documented in-line with the failure it fixes; a tailored seccomp profile is a named hardening follow-up.
  • docker/builder/verify.sh is the acceptance suite; design + verification record live in docs/superpowers/specs/2026-06-10-outer-builder-container-design.md. Architecture doc and README updated.

Test Plan

Verified on a clean Ubuntu noble arm64 VM (OrbStack) with Docker 29 — all checks pass:

  • image builds from a clean checkout
  • /v1/healthz, idempotent POST /v1/projects, session create
  • nested rootless podman (podman run alpine inside the container)
  • inner app (nginx) published on :3000 reachable from the host — the full host → outer → inner port chain
  • project registry survives an outer-container restart (volume-backed)
  • no provider credentials visible in inner-container env
  • live builder-agent turn (litellm/openai/gpt-5.5): the agent created index.html and itself ran podman run -d --name demo-live-app -p 3000:80 nginx:alpine; the page is reachable from the VM and from the macOS host (http://appx-builder-vm.orb.local:3000)

Additional verification for the prompt-path fix:

  • npm run build
  • npm test
  • npx biome check . --error-on-warnings
  • docker build -f docker/builder/Dockerfile -t appx-agent-server-builder-prompt-test .
  • smoke-run built image and verify /workspace/.pi-global/AGENTS.md exists and matches /usr/local/share/appx-builder/AGENTS.md

🤖 Generated with Claude Code

sasha and others added 5 commits June 10, 2026 16:22
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements items 1, 2 and 4 of builder-container-architecture.md's
'What Needs to Be Built': the Dockerfile, the canonical run script
(spawn contract for orchestrators), and the builder system prompt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verified on Ubuntu noble (arm64): --cap-add SYS_ADMIN for newuidmap,
--device /dev/net/tun for slirp4netns, systempaths=unconfined for crun
sysctl writes. All verify.sh acceptance checks pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alexanderkreidich

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95d5e0e02b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docker/builder/Dockerfile Outdated
sasha and others added 2 commits June 10, 2026 17:45
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Builder images copied AGENTS.builder.md under /home/builder/.pi/agent, but ProjectRegistry passes /workspace/.pi-global as the runtime agentDir. New projects without their own .pi/AGENTS.md therefore could start without the podman, port, and secret-handling instructions required by the builder container.

Bake the prompt as an image template and have the entrypoint install it into WORKSPACE_DIR/.pi-global/AGENTS.md when the mounted workspace does not already provide one. Update the Docker docs to describe the actual lookup path and the project-local override behavior.
@neuromaxer

Copy link
Copy Markdown
Contributor

Thanks for the suggestion, took a few things from it into the main. Closing this PR in favor of what landed on main as it's achieving the same thing without giving CAP_SYS_ADMIN to the builder container making it much safer for running agentic flows. See the fix below

The issue. This PR runs the builder container with --cap-add SYS_ADMIN, --device /dev/fuse, and seccomp/apparmor/systempaths=unconfined. CAP_SYS_ADMIN is effectively "near-root" — it's the single most powerful Linux capability and hands the builder container a large chunk of host-admin privileges. Combined with unconfined seccomp, that widens the blast radius a lot for a container whose whole job is to run untrusted agent-generated code.

The reason those flags were added is real, though: nested rootless podman needs mount(2) and a few namespace-setup syscalls that Docker's defaults block, so the quick fix is "grant more privilege."

The solution we found in the spike. Instead of granting privilege, we removed the reasons the privilege seemed necessary, one at a time (all documented in container/SPIKE-FINDINGS.md):

  • No CAP_SYS_ADMIN — the real blocker was newuidmap/newgidmap running setuid-root and tripping a kernel capability check. Shipping them with file capabilities instead (the way Fedora/podman-stable do) keeps the process at uid 1000 and makes the check pass without any added cap.
  • No /dev/fuse — we switched to native rootless overlay, which needs no fuse device and is ~2.2× faster on our kernel.
  • No seccomp=unconfined — we ship a tailored seccomp-builder.json that's podman's stock profile with only the specific namespace syscalls (mount, sethostname, setns, …) ungated. It's strictly tighter than unconfined, and the genuinely dangerous syscalls (bpf, perf_event_open, etc.) stay denied.

Net result: the same nested-podman capability, but the outer container runs unprivileged as uid 1000 with a hardened syscall profile — no SYS_ADMIN, no /dev/fuse, no unconfined seccomp.

@neuromaxer neuromaxer closed this Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants