Make bash sandbox network policy configurable (issue #1397) - #1398
Merged
Conversation
Behavioral tests added ahead of implementation for issue #1397: - PermissionConfig.Network validation (allow/deny/empty valid, anything else rejected) and normalizePermissionConfig defaulting to allow. - tools.NetworkPolicy context plumbing, seatbeltProfile (darwin) and bwrap args (linux) gated by network policy instead of always denying network for workspace/local scope, and CheckSandboxCommand's local-scope heuristic gated the same way. - Runner first-turn permissions notice includes "network=allow|deny" and, when denied, the "Outbound network is blocked..." warning sentence. - CLI --sandbox/--network flags populate a "permissions" object on the run-create request body; omitted when neither flag is set. - A live, real-network integration test that curls https://proxy.golang.org through the actual seatbelt sandbox under both policies. None of this compiles yet: the tests reference NetworkPolicy, its constants, WithNetworkPolicy, SandboxExecResult.NetworkPolicy, the new seatbeltProfile/CheckSandboxCommand signatures, and PermissionConfig.Network, none of which exist before the next commit. Existing tests were also updated in place to assert the new default (network allowed unless denied) where they previously asserted the old always-deny behavior for SandboxScopeLocal, and to account for the new permissions-notice tail message. go vet output (internal/harness, internal/harness/tools): # go-agent-harness/internal/harness/tools [go-agent-harness/internal/harness/tools.test] internal/harness/tools/sandbox_darwin_test.go:19:50: undefined: NetworkPolicyDeny internal/harness/tools/sandbox_darwin_test.go:19:50: too many arguments in call to seatbeltProfile have (SandboxScope, string, unknown type) want (SandboxScope, string) internal/harness/tools/sandbox_darwin_test.go:24:49: undefined: NetworkPolicyAllow internal/harness/tools/sandbox_darwin_test.go:40:9: undefined: WithNetworkPolicy internal/harness/tools/sandbox_darwin_test.go:46:9: res.NetworkPolicy undefined (type SandboxExecResult has no field or method NetworkPolicy) # go-agent-harness/internal/harness vet: internal/harness/permission_config_test.go:80:66: unknown field Network in struct literal of type PermissionConfig go test output (cmd/harnesscli, does not depend on new symbols so it compiles and fails at runtime instead): === RUN TestRunParsesSandboxAndNetworkFlagsIntoPermissions main_permissions_test.go:61: expected exit code 0, got 1 --- FAIL: TestRunParsesSandboxAndNetworkFlagsIntoPermissions (0.00s) === RUN TestRunOmitsPermissionsWhenNoSandboxOrNetworkFlagSet --- PASS: TestRunOmitsPermissionsWhenNoSandboxOrNetworkFlagSet (0.00s) FAIL These tests will pass after the implementation in the next commit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
Implementation for tests added in fdf22b5. Summary (issue #1397): workspace/local sandbox scopes now allow outbound network by default. A new PermissionConfig.Network axis ("" / "allow" / "deny") lets a caller explicitly deny it; the bash sandbox enforces the resolved policy at the OS level (seatbelt on darwin, bubblewrap on linux), the model is told the policy on every turn including the first, and the CLI can set it via flags. - internal/harness/types.go: NetworkPolicy type + PermissionConfig.Network field (json "network,omitempty"); ValidatePermissionConfig rejects anything other than "", "allow", "deny"; DefaultPermissionConfig sets Network: NetworkPolicyAllow explicitly. - internal/harness/tools/types.go: mirrored tools.NetworkPolicy type, ContextKeyNetworkPolicy, WithNetworkPolicy, NetworkPolicyFromContext — same import-cycle-avoidance pattern as SandboxScope. - internal/harness/tools/sandbox.go: SandboxExecResult gains a NetworkPolicy field so bash tool output reports the applied policy; CheckSandboxCommand takes a NetworkPolicy parameter and only applies the SandboxScopeLocal curl/wget/nc/netcat/telnet heuristic when it is deny. - internal/harness/tools/sandbox_darwin.go: seatbeltProfile takes the network policy and emits an explicit "(allow network*)" for allow/default or "(deny network*)" for deny — under seatbelt's "(deny default)", merely omitting a deny rule still leaves network denied, so the allow case needs its own explicit rule. Confirmed with a real curl against https://proxy.golang.org via /usr/bin/sandbox-exec: exit 0 (HTTP 200) under allow, exit 6 (could not resolve host) under deny. - internal/harness/tools/sandbox_linux.go: --unshare-net is only added to the bwrap invocation when network is deny. - internal/harness/tools/sandbox_other.go: reports the resolved NetworkPolicy on SandboxExecResult for parity on unimplemented platforms; behavior otherwise unchanged (degrades to heuristic/fail-closed as before). - internal/harness/tools/bash_manager.go: JobManager gets SetNetworkPolicy/networkPolicyForContext (mirroring SandboxScope), wired into runForeground/runBackground; bash tool results gain a "sandbox_network" field. - internal/harness/runner.go: runStepEngine/newStepEngine/stepEngine thread the resolved tools.NetworkPolicy alongside SandboxScope from effectivePermissions down to the per-tool-call context (htools.WithNetworkPolicy); normalizePermissionConfig defaults an empty Network to allow; permissionsNoticeLines() renders "Permissions for this run: sandbox=%s, approval=%s, network=%s." plus, when denied, "Outbound network is blocked for this run: dependency installs will fail; report the blocker instead of substituting a different design." — reused by both the existing continuation-changed notice and the new per-turn notice below. - internal/harness/runner_step_engine.go / clone.go: buildTurnMessages gains a permissionsNotice tail parameter, computed once from the run's effective sandbox/approval/network and re-appended to every turn's wire message list (never persisted into conversation history, unlike the continuation-changed notice), so the model is told its network policy starting on turn one — not only when a continuation's permissions change. - cmd/harnesscli/main.go: --sandbox (workspace|local|unrestricted) and --network (allow|deny) flags populate runCreateRequest.Permissions (*harness.PermissionConfig, json "permissions,omitempty") only when either flag is set; both flags unset omits "permissions" entirely so the server's own defaults apply. Existing tests were updated where they asserted the previous always-deny default for SandboxScopeLocal, or fixed-position tail-message assumptions that the new per-turn permissions notice message shifts by one (working memory / observational memory / trusted-origin system-prompt capture tests) — see the red commit for the full list and reasoning. go vet output (internal/harness, internal/harness/tools, cmd/harnesscli): clean, no output. go test -race output (internal/harness/..., cmd/harnesscli): ok go-agent-harness/internal/harness 8.362s ok go-agent-harness/internal/harness/tools 18.140s ok go-agent-harness/internal/harness/tools/core (cached) ok go-agent-harness/internal/harness/tools/deferred (cached) ok go-agent-harness/internal/harness/tools/descriptions (cached) ok go-agent-harness/internal/harness/tools/recipe (cached) ok go-agent-harness/internal/harness/tools/script (cached) ok go-agent-harness/cmd/harnesscli 6.861s Behavioral tests covered: PermissionConfig.Network validation, darwin seatbelt/linux bwrap network gating, first-turn permissions notice (allow and deny), CLI --sandbox/--network flags. Files changed: cmd/harnesscli/main.go, internal/harness/clone.go, internal/harness/runner.go, internal/harness/runner_step_engine.go, internal/harness/tools/bash_manager.go, internal/harness/tools/sandbox.go, internal/harness/tools/sandbox_darwin.go, internal/harness/tools/sandbox_linux.go, internal/harness/tools/sandbox_other.go, internal/harness/tools/types.go, internal/harness/types.go Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
…olicy Regression tests added that would fail if the change in cdbc73d is reverted: - TestRedTeam_SandboxNetwork_DefaultPermissionsAllowCurl (internal/harness/runner_redteam_test.go): drives a real bash tool call through the full runner -> step engine -> tool context path with NO explicit Permissions on the RunRequest, and asserts curl is NOT rejected as a sandbox violation. This exercises the end-to-end plumbing (not just the unit-level seatbeltProfile/CheckSandboxCommand functions already covered by the red commit), so it would catch a regression where the runner stopped threading the resolved network policy from effectivePermissions into the tool execution context, or where normalizePermissionConfig stopped defaulting Network to allow. - TestJobManagerRunForegroundReportsSandboxNetworkInResult (internal/harness/tools/sandbox_test.go): asserts the bash tool result map's "sandbox_network" field matches the network policy actually applied, for both allow and deny, under a real OS-level sandbox. Catches a regression where SandboxExecResult.NetworkPolicy stopped being threaded into the JobManager result map. Full test suite output (internal/harness/..., cmd/harnesscli, -race): ok go-agent-harness/internal/harness 7.945s ok go-agent-harness/internal/harness/tools 18.497s ok go-agent-harness/internal/harness/tools/core 2.917s ok go-agent-harness/internal/harness/tools/deferred 12.696s ok go-agent-harness/internal/harness/tools/descriptions 1.628s ok go-agent-harness/internal/harness/tools/recipe 2.520s ok go-agent-harness/internal/harness/tools/script 4.397s ok go-agent-harness/cmd/harnesscli 7.709s (2498 individual `--- PASS` lines across both packages under -v, 0 FAIL) go vet ./internal/harness/... ./cmd/harnesscli/...: clean, no output. Regression scenarios covered: - Default permissions (no Permissions field at all) allow outbound network for a real bash tool call end to end. - The bash tool result surfaces the actually-applied network policy for both allow and deny. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
Updates every doc page that stated bash network was unconditionally denied for workspace/local sandbox scope, or claimed the wrong default sandbox scope (unrestricted instead of workspace), to match the implementation in cdbc73d: - website/docs/concepts/tools-and-permissions.md: rewrites the permission model section for three axes (sandbox, network, approval), adds a Network policy tabbed section, and calls out the security-relevant default change explicitly. - website/docs/reference/tools-catalog.md: corrects the sandbox-scope default (workspace, not unrestricted) and adds the network-policy table. - website/docs/reference/glossary.md: corrects the sandbox-scope default and documents the network axis. - website/docs/reference/http-routes.md: corrects the RunRequest example body's default permissions and field notes. - website/docs/server/http-api-guide.md: adds "network" to the permissions JSON examples, corrects the omitted-permissions default, and documents the network field/default in the field table. - website/docs/reference/cli-flags.md, website/docs/cli/harnesscli.md: document the new --sandbox/--network flags. - docs/logs/engineering-log.md: new entry describing the change, the seatbeltProfile "(deny default)" gotcha (an omitted deny rule does not become an allow — an explicit "(allow network*)" is required), and calling out the default change as security-relevant. website/docs/server/harnessd.md and docs/runbooks/benchmark-smoke.md were checked (per the task) but contain no bash-sandbox-network statements to update; docs/runbooks/profile-authoring.md's "network-denied profile" reference is a different mechanism (the allow_net_access profile capability gating ActionFetch/ActionDownload) and is out of scope. Docs-only change; no rebuild required. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1397
Summary
The bash sandbox previously denied outbound network unconditionally for
SandboxScopeWorkspace/SandboxScopeLocal, with no way for a caller to opt back in. This adds a thirdPermissionConfigaxis,network(""/"allow"/"deny", default"allow"), enforced at the OS level (seatbelt on macOS, bubblewrap on Linux), surfaced to the model in its first-turn permissions notice, exposed viaharnesscli --sandbox/--networkflags, and reported back on bash tool output (sandbox_network).Security-relevant default change: workspace/local sandbox scopes now allow outbound network by default. Callers that need the old always-deny behavior must set
permissions.network: "deny"explicitly.Scope and issue reconciliation
Followed the coordinator's 2026-09-06 design-decision comment on #1397 exactly:
harness.PermissionConfig.Network(jsonnetwork,omitempty),""/"allow"/"deny",Validaterejects anything else — done ininternal/harness/types.go. Mirrored at the tools layer (tools.NetworkPolicy,ContextKeyNetworkPolicy,WithNetworkPolicy,NetworkPolicyFromContext) ininternal/harness/tools/types.go, set alongside the existing sandbox-scope context value ininternal/harness/runner_step_engine.go, threaded frominternal/harness/runner.go'srunStepEnginecall (was line 3450 in the issue, now threadshtools.NetworkPolicy(effectivePermissions.Network)too).sandbox_darwin.go'sseatbeltProfileandsandbox_linux.go's bwrap args gate network confinement on the policy read from ctx;buildSandboxedCommandalready received ctx. Filesystem confinement andunrestrictedscope are unchanged.SandboxExecResult.NetworkPolicyreports the applied policy; the bash tool result map gainssandbox_network.network=allow|denyand, when denied, the required warning sentence. It is present on the first turn — previously this notice only existed for a continuation whose permissions changed. It's now a per-turn tail message (like working/observational memory and runtime context), not persisted into conversation history, so it doesn't inflate the stored transcript.harnesscli --sandbox/--networkpopulaterunCreateRequest.Permissionsonly when either flag is set; both omitted means no"permissions"key at all, so server defaults apply.Deviation from the literal task description, with reason:
CheckSandboxCommand'sSandboxScopeLocalheuristic (blocks barecurl/wget/nc/netcat/telnetregardless of the OS-level policy) is a second, string-based gate the issue's contract summary didn't call out explicitly. Left unchanged, it would have kept blocking those commands under the new default even after the OS-level fix, contradicting "workspace/local scopes now allow outbound network by default." Gated it on the sameNetworkPolicyparameter (only active whendeny) so the two enforcement layers agree. This is a minimal, same-file, same-mechanism fix required to make the stated default change actually true, not a new feature.Out of scope, confirmed untouched: per-host allowlists, the fetch tool SSRF guard,
checkWorkspaceScopeCommand's absolute-path heuristic,docs/runbooks/profile-authoring.md'sallow_net_accessprofile capability (a different mechanism gatingActionFetch/ActionDownload, not bash OS-level sandboxing).Impact analysis reconciliation
internal/harness/types.go,internal/harness/tools/types.go: additive fields/types only; no existing field renamed or removed.internal/harness/tools/sandbox.go:CheckSandboxCommandgained a parameter. Its only caller isinternal/harness/tools/bash_manager.go(updated) and tests (updated); confirmed no external callers via a repo-wide grep.internal/harness/tools/sandbox_darwin.go/sandbox_linux.go/sandbox_other.go:seatbeltProfilegained a parameter — its only caller isbuildSandboxedCommandin the same file.internal/harness/clone.go'sbuildTurnMessagesgained a parameter — its only two callers are inrunner_step_engine.go(both updated); no test calls it directly.internal/harness/runner.go/runner_step_engine.go:runStepEngine/newStepEngine/stepEnginegained a field/parameter — single call chain, no other callers.cmd/harnesscli/main.go:runCreateRequestgained an optional field;startRun/JSON marshaling unaffected for callers that don't set it (confirmed viaTestRunOmitsPermissionsWhenNoSandboxOrNetworkFlagSet).SandboxScopeLocal, or fixed tail-message positions the new per-turn permissions notice shifts by one, were updated in place (working memory / observational memory ordering tests, a trusted-origin system-prompt capture test, a red-team default-deny test) — see the red and green commit messages for the full list and reasoning per test.Architecture and duplication check
Reused the existing per-run context-injection pattern end to end rather than adding a new one:
tools.SandboxScope/ContextKeySandboxScope/WithSandboxScopealready existed as the mirror-at-tools-layer pattern for avoiding aharnesstotoolsimport cycle;NetworkPolicyfollows the identical shape (type, const, context key,With*,*FromContext) right next to it, per the issue's explicit instruction.JobManager.SetNetworkPolicy/networkPolicyForContextmirror the existingSetSandboxScope/sandboxScopeForContextmethods exactly. The permissions-notice text reuses the existingpermissionsNoticeLineshelper for both the (unchanged) continuation-changed notice and the new first-turn/per-turn notice, rather than duplicating the string formatting. No new tool catalog, no second sandbox-building code path, no parallel permissions struct.Test-first evidence
Red commit:
fdf22b52. Green commit:cdbc73dc. Regression commit:a4871c40.Red command:
go vet ./internal/harness/... ./cmd/harnesscli/...(plusgo test ./cmd/harnesscli -run 'TestRunParsesSandboxAndNetworkFlagsIntoPermissions|TestRunOmitsPermissionsWhenNoSandboxOrNetworkFlagSet' -v), captured by staging only the test files then stashing the implementation withgit stash push --keep-index -u.Observed failure:
Why the failure proved the missing/incorrect behavior: the compile failures name exactly the symbols the acceptance contract requires (
NetworkPolicy,WithNetworkPolicy,SandboxExecResult.NetworkPolicy,PermissionConfig.Network) — they don't exist yet, so no implementation could have accidentally made these tests pass. The CLI failure is a genuine runtime failure (unrecognized--sandbox/--networkflags), not an import/compile error.Green command:
go build ./internal/harness/... ./cmd/harnesscli/...thengo test ./internal/harness/... ./cmd/harnesscli -race. All green (see Verification evidence).Verification evidence
Targeted:
Live sandbox integration (real network, real
sandbox-exec), both as a Go test and manually against raw seatbelt profiles:Full regression, race-enabled:
Cross-compile check for the linux-only
sandbox_linux.go/sandbox_linux_test.go(this session is on macOS, so linux tests cannot execute here, only compile-verify):Not verified: actual bubblewrap enforcement on a real Linux host (no such host available in this session) — the linux implementation mirrors the darwin one's now-verified explicit-allow/explicit-deny pattern, and the linux test suite (updated in the red commit) asserts the correct
--unshare-netpresence/absence, but nobody has run it against a realbwrapbinary in this PR.Real-world manual browser/UI check: not applicable — this is a server/CLI-only change with no UI surface.
Rollout and rollback
No migration or data change. This is a pure default-behavior plus additive-field change, gated entirely by process restart: rebuild
harnessd/harnesscli(scripts/install.sh) and restart the daemon to pick it up — merging alone does not change a running process's behavior. Rollback is a plain revert of this PR (or the single commitcdbc73dcplus its docs); no persisted state needs repair sincePermissionConfig.Networkis per-run, not persisted across restarts. Observability: bash tool results now carrysandbox_network, so an operator can see the applied policy in tool output without extra instrumentation.Documentation
Updated:
website/docs/concepts/tools-and-permissions.md,website/docs/reference/tools-catalog.md,website/docs/reference/glossary.md,website/docs/reference/http-routes.md,website/docs/server/http-api-guide.md,website/docs/reference/cli-flags.md,website/docs/cli/harnesscli.md,docs/logs/engineering-log.md(new dated entry, security-relevant default change called out explicitly).Checked and left unchanged with reason:
website/docs/server/harnessd.mdanddocs/runbooks/benchmark-smoke.md(named in the task) contain no bash-sandbox-network statements; their "no network" mentions are about the fake provider needing no network calls, unrelated.docs/runbooks/profile-authoring.md'sallow_net_accessis a distinct profile-capability mechanism (ActionFetch/ActionDownloadgating), not this PR's OS-level bash sandbox axis.Contract checklist
CheckSandboxCommand's local-scope heuristic), explained abovesandbox-exec/curl against a live host, both via the Go test suite and manual raw-profile verification🤖 Generated with Claude Code
https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5