Skip to content

[LXC] Inbound default-deny: install window, namespace identity, and workload escape #897

Description

The inbound (INPUT) default-deny chain: the window before it exists, how the target namespace is identified, and what the workload can do to it once inside.

This consolidates 5 separately filed issues so the backlog carries one entry per area instead of one per finding. Every original report is reproduced below in full, unedited. The originals are closed and point here.

Folded from Status Title
#850 Open LXC inbound: close the start-to-install fail-open window and make the ingress reset atomic
#853 Open LXC ingress: pin the container network namespace instead of re-resolving a recyclable PID
#854 Open LXC inbound default-deny is removable by the workload: CAP_NET_ADMIN is never dropped
#859 Fixed in #849, closes on merge LXC state-aware firewall mode never installs the inbound default-deny INPUT chain
#905 Open LXC start leaves a live unfiltered-inbound container if killed between start and ingress, and a retry is refused

Rows marked Fixed in #849 are already addressed by that pull request and are reproduced here only so the record stays complete. Read them as history once #849 merges, not as work remaining.


LXC inbound: close the start-to-install fail-open window and make the ingress reset atomic

Originally #850.

Follow-up from #836. Two reviewer findings on that PR were confirmed real and deliberately deferred, because both need a design change validated on a live LXC host rather than a late edit to a PR already at +2707. They are contained in one another, so they should be fixed by the same change.

1. Inbound is unfiltered between container start and ingress install

container.start() runs at src/backends/lxc/common/src/lxc_runner.rs:182-190. wait_for_network follows at :205 -- its return value is discarded, so a timeout falls through rather than aborting -- then veth discovery and the egress apply at :211-236, and only then the ingress install at :250-310. For that whole interval the container's INPUT policy is unchanged, so a reused container's services, or a new container's init services, are reachable despite a requested default-deny.

Two things bound the severity today: the workload runs via lxc-attach only after the ingress install returns, so no sandboxed code executes during the interval, and the exposure is therefore to external traffic only.

The naive fix does not work. Ingress cannot simply move ahead of wait_for_network, because the IPv6 classification needs the namespace to have come up. The shape that does work is an early unconditional deny, with family-specific rules completed once the namespace has settled.

2. reset_family unhooks a leftover policy before the replacement exists

In src/backends/lxc/common/src/network_ingress.rs: resetting a leftover hooked chain from a crashed run removes every INPUT hook, then recreates and re-hooks. On a still-running reused container that is a fail-open interval. It does not affect a fresh install, where there is no prior hook to remove, and the doc comment has been corrected to say so rather than claiming no window exists at all.

The fix is an atomic swap: build the replacement chain, then repoint INPUT. iptables-restore is the mechanism, and adopting it means rewriting the install path away from incremental commands -- which is also what the per-resource ownership tracking and its teardown depend on.

Why they belong to one change

(2) is strictly contained inside (1). An early-deny design that holds INPUT at deny across the whole startup window removes the exposure in (2) as well, so fixing them separately would build the same machinery twice.

Validation

Neither is credible without a live host. tests/scripts/run_lxc_inbound_deny_test.sh exists and asserts namespace containment, and it has now been executed against real LXC containers (Ubuntu 24.04 under WSL 2, LXC 5.0.3, iptables 1.8.10 nf_tables). An earlier version of this issue said no LXC host was available and that WSL could not provide one; that was wrong, and it was never checked. That run surfaced a separate defect in the merged code, fixed in #863. It still does not prove this issue fixed. The test that would actually prove this fixed is traffic-level: start listeners inside the container, probe them from the host over IPv4 and IPv6, both during and after the startup window.

Source: the review threads on #836 at lxc_runner.rs:256 and network_ingress.rs:630.


LXC ingress: pin the container network namespace instead of re-resolving a recyclable PID

Originally #853.

Raised in review of #836 (#836 (comment)).

Problem

The LXC ingress path addresses the container network namespace by numeric PID, and re-resolves that PID for every privileged command instead of pinning the namespace once.

  • lxc_runner.rs takes container.init_pid() and passes the raw u32 to IngressManager::new(&container_name, pid).
  • network_ingress.rs builds every command as nsenter -t <pid> -n <argv>, so the PID is resolved again per command.
  • signal_cleanup.rs:93-98 stores the same raw PID in the watchdog slot, and signal_cleanup.rs:214-215 uses it at signal time via IngressManager::force_cleanup(&name, pid, ..).

A PID is a recyclable name for a process, not a stable handle on its namespace. If container init exits and the PID is reallocated, nsenter -t <pid> -n enters whatever process now holds that number -- commonly a host process, and therefore the host network namespace.

Severity, split by operation

These are not equally exposed, and the split matters for prioritization:

  • Install is the dangerous operation but has the narrow window. It creates an MXCI-<container> chain and hooks it into INPUT with a terminal DROP. Doing that in the host namespace would firewall the host. The window is only the multi-command install, and reuse requires PID wraparound -- PID_MAX_LIMIT is 2^22 (4194304) on 64-bit Linux, so this needs ~4M PIDs allocated within seconds. Hosts with a lowered /proc/sys/kernel/pid_max shrink that requirement.
  • Cleanup has the wide window but is far less dangerous. The watchdog holds the PID for the container's whole lifetime, so reuse there is entirely plausible. However it only issues targeted deletes against the uniquely-named MXCI-<container> chain, so entering a foreign namespace produces "no chain by that name" errors rather than damage.

So the catastrophic case is unlikely and the likely case is benign -- but the catastrophic case is host-wide inbound DROP, which is not a risk worth carrying on a probabilistic argument.

Proposed fix

Pin the namespace with an open handle rather than a number: open /proc/<pid>/ns/net once at IngressManager construction and use that handle for install, cleanup, and watchdog state. An open namespace file descriptor keeps referring to the namespace it was opened on, so PID reuse cannot redirect it.

Explicitly rejected as a substitute: re-checking the PID's identity immediately before each command. That narrows the window without closing it -- the check and the exec are still separate -- and a partial guard here would read as a solved problem.

Why not in #836

The change touches three subsystems (install, teardown, watchdog signal handling) and depends on file-descriptor inheritance semantics across the nsenter exec, which cannot be exercised on the Windows development host or in CI -- neither runs LXC. It wants its own PR with a Linux host to validate against.

Validation

Needs a live LXC host: start a container, capture the init PID, confirm the ingress chain lands in the container namespace and not the host's, and confirm that a handle opened at construction still resolves after the original PID is gone.


LXC inbound default-deny is removable by the workload: CAP_NET_ADMIN is never dropped

Originally #854.

Summary

The inbound default-deny chain added in #836 is installed inside the container's own network namespace, and the sandboxed workload retains CAP_NET_ADMIN in that namespace. A workload that wants to can run iptables -F MXCI-<name> (or delete the chain outright) and restore full inbound reachability.

This does not make the feature useless — it still closes external reachability for any workload that does not deliberately tear it down, including services the workload itself starts — but it does mean inbound default-deny is not a containment boundary against the sandboxed code, and it should not be relied on as one.

Evidence

  1. The chain is installed in the container netns, via nsenter -t <init-pid> -n iptables ...network_ingress.rs, nsenter_command.
  2. MXC creates containers with the stock download template and no capability configuration:
    • lxc_bindings.rs:227-235lxc-create -t download -- -d <dist> -r <release> -a <arch>.
    • The only config key MXC ever writes is lxc.mount.entry (filesystem_mounts.rs:239,250,304). A repo-wide search for lxc.cap / lxc.seccomp / lxc.apparmor / lxc.init.uid returns only two path-injection test fixtures in filesystem_mounts.rs:341-342.
  3. LXC's defaults therefore apply, and they leave net_admin in place:
    • lxc/lxc, config/templates/common.conf.in: lxc.cap.drop = mac_admin mac_override sys_time sys_module sys_rawionet_admin is not listed.
    • lxc/lxc, config/templates/userns.conf.in: # Start with a full set of capabilities in user namespaces. followed by empty lxc.cap.drop = and lxc.cap.keep =.
  4. The workload runs as container root: attach_run (lxc_bindings.rs:337-368) invokes lxc-attach with only build_attach_args (env/cwd/command layering). No -u / -g, no capability or credential flags.

Contrast with egress

Egress is not exposed this way. Its chains live on the host and are hooked into FORWARD by the container's host-side veth (network_iptables.rs:1142-1148, -I FORWARD -i <veth> -j <chain>), which the workload cannot reach. The asymmetry is inherent to where each chain is installed, not an oversight in one of them.

Why ingress was put in the container netns anyway

Host-side FORWARD filtering only sees traffic routed through the host to the container. Traffic originating on the host itself and addressed to the container does not traverse FORWARD, so a purely host-side inbound chain would miss it. Enforcing in the container netns catches every inbound path regardless of origin. That is a real reason for the current design, and any replacement has to keep that coverage.

Options (not yet evaluated)

  • Drop net_admin for the container (lxc.cap.drop), or attach the workload as a non-root uid. Cheapest, but may break workloads that legitimately configure their own networking.
  • Move inbound enforcement to the host and add explicit coverage for host-origin traffic, so it matches the egress trust model.
  • Keep the current mechanism and document it as best-effort exposure reduction rather than a boundary. This is what Enforce inbound default-deny for LXC containers #836 does today.

Scope

#836 documents the limitation truthfully. Choosing and implementing a tamper-proof mechanism is this issue.

Found in review of #836 by copilot-pull-request-reviewer.


LXC state-aware firewall mode never installs the inbound default-deny INPUT chain

Originally #859. Fixed in #849; closes on merge.

Found while reviewing #849. This is a fail-open gap: a state-aware container accepts inbound connections that the one-shot path would drop.

The asymmetry

One-shot installs ingress. src/backends/lxc/common/src/lxc_runner.rs:20 imports IngressManager; :247 computes use_firewall; :265 constructs IngressManager::new(&container_name, pid); :296 fails closed when the PID is unavailable in firewall mode. network_ingress.rs:8 states inbound traffic is dropped by default, :39 gates on ContainerPolicy::allow_local_network, and :50-51 builds the container's own INPUT chain.

State-aware never does. state_aware.rs contains no reference to IngressManager, network_ingress, or Ingress at all. apply_network_policy installs only the host-side egress FORWARD chain via NetworkIptablesManager (state_aware.rs:351).

Since allowLocalNetwork defaults to false, a container started through the state-aware path with an accepted firewall policy can still listen for host and LAN connections.

Note on the rollback step

signal_cleanup.rs has a RollbackStep::RemoveIngress, which might look like the state-aware path handles ingress. It does not. State-aware start uses set_active_network_only (state_aware.rs:532) which yields SignalRollback::NetworkOnly, and that plan is only StopContainer + RemoveFirewall (signal_cleanup.rs:107-113). RemoveIngress appears only in the DestroyContainer plan (:114-119), reached from the one-shot path that does install ingress. The rollback is correctly paired; the apply side is what is missing.

Suggested direction

Install inbound rules with IngressManager after start on the state-aware path, and roll the start back if it fails -- matching the one-shot fail-closed posture. Alternatively, reject firewall mode on the state-aware path until it is supported, rather than reporting it as enforced.


LXC start leaves a live unfiltered-inbound container if killed between start and ingress, and a retry is refused

A kill between container.start() and apply_ingress_policy in src/backends/lxc/common/src/state_aware.rs leaves the container running with no inbound policy, and a retry is rejected as already started.


Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions