Skip to content

feat(cve-env): integrate cve-env as packages/cve_env (Phase 1 lift-and-shift) - #802

Open
gadievron wants to merge 27 commits into
mainfrom
feat/cve-env-integration
Open

feat(cve-env): integrate cve-env as packages/cve_env (Phase 1 lift-and-shift)#802
gadievron wants to merge 27 commits into
mainfrom
feat/cve-env-integration

Conversation

@gadievron

@gadievron gadievron commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Brings cve-env (an agentic CVE → Docker-environment builder) into raptor as packages/cve_env, wired like the sibling cve_diff tool. Phase 1 is a behavior-preserving lift-and-shift: cve-env keeps its own engine and adopts zero core/ modules, so the import is auditable as "same tool, new home."

Imported from gadievron/cve-env @ ba9f91c (see packages/cve_env/PROVENANCE.md).

Package import

What: vendored copy of cve-env at packages/cve_env/cve_env/ (flat layout, mirroring packages/cve_diff/cve_diff/) + its test suite + __init__/__main__ + LICENSE + PROVENANCE.md.
Counts (re-derived):

  • find packages/cve_env/cve_env -name '*.py' | wc -l → 41 source files
  • find packages/cve_env/tests -name '*.py' | wc -l → 89 test files
  • git diff --stat origin/main...HEAD | tail -1 → 142 files changed, 45379 insertions(+), 2 deletions(-)
    Scope note: all cve_env. imports are absolute, so the src/cve_env/packages/cve_env/cve_env/ move needs no source-import rewrites. grep -rE '^(from|import) (core|packages)\.' packages/cve_env/cve_env → 0 matches (zero raptor-core adoption this phase).

Pattern-B wiring (mirrors cve_diff)

Solution: bin/cve-env (launcher: env-strip + PYTHONPATH + trust marker), libexec/raptor-cve-env (non-interactive dispatcher with the inline trust-marker guard), .claude/commands/cve-env.md (dispatch: libexec/raptor-cve-env <subcommand> [args]). Subcommands are cve-env's own build / doctor (argparse), not Typer.
Verification: python3 .github/scripts/check_command_metadata.pyOK: 28 command .md files lint-clean (includes cve-env.md).

Layout-coupled path fixes

Problem: three sites resolved paths by file-depth and would mis-place artifacts under the new nesting (audit JSONL, outcome sidecar, refusals log).
Solution: added a CVE_ENV_OUTPUT_ROOT override (config._find_output_root) and re-homed agent/refusals.py:default_log_path; the launchers point it at raptor's out/ (consistent with cve_diff). Unset → byte-identical to standalone. 8 layout-coupled guard tests updated to derive the package dir from cve_env.__file__ (layout-independent).

Dependencies

  • requests 2.33.02.33.1 — the only cross-package pin reconciliation (cve-env floors at >=2.33.1; compatible with urllib3==2.7.0 + core/http).
  • claude-agent-sdk==0.1.71 added (pinned from cve-env's uv.lock; transitive deps anyio/mcp/sniffio, no clash).

CI + security-audit wiring

  • pytest.ini pythonpath += packages/cve_env.
  • .github/scripts/compute_filters.py: new cve_env path filter + cve-env's prompt files added to the prompt_audit filter (mirrors _PROMPT_CONSTRUCTION_FILES).
  • .github/workflows/tests.yml: changes output, fast-tier --ignore, a dedicated python-unit-tests-cve-env job, and the tests-passed aggregate.
  • core/security/prompt_envelope_audit.py: registered cve-env's prompt-construction files (agent/prompts.py, agent/loop.py) so the untrusted-interpolation audit covers them.

README

Added /cve-env (and the previously-missing /cve-diff) rows to the commands table.

Security context

cve-env is a research tool that builds and verifies intentionally-vulnerable environments for public CVEs — it does not fix or claim to fix any vulnerability, and this PR introduces no exploit. The "vulnerable"/"exploit"/"CVE" terms describe the tool's purpose (standing up a known-affected app at its pre-patch version for authorized analysis), not a security change to raptor. Container hardening it inherits: localhost-only port binding, --cap-drop ALL, no-new-privileges. The prompt-envelope audit registration above is the only security-surface change, and it tightens (not loosens) coverage by bringing cve-env's prompt builders under the existing untrusted-interpolation scan (pytest core/security/tests/test_prompt_envelope_audit.py → 18 passed, 0 new violations).

Tests

Verified locally this session:

  • pytest packages/cve_env/tests1529 passed, 4 skipped (identical to the standalone gadievron/cve-env baseline).
  • pytest core/http/tests against requests==2.33.1106 passed (dep bump causes no core/http regression).
  • pytest core/security/tests/test_prompt_envelope_audit.py18 passed (0 new violations from the registration).
  • pytest .github/tests/test_filter_coverage.py11 passed (the new cve_env/prompt_audit filters cover their imports).
  • Added test surface: git diff --numstat origin/main...HEAD -- 'packages/cve_env/tests/**' → +28145 lines across 90 files.
  • 5-CVE live smoke through bin/cve-env (Drupal/nginx/Elasticsearch/Tomcat/Laravel): 4 success + 1 turn_cap (with verify_passed=True), 0 tool-errors, 0 integration defects.

The full raptor suite is validated by CI on this PR (the dedicated python-unit-tests-cve-env job + the changed-tier jobs); locally only the core/http slice + the gate dry-runs above were run.

Compatibility

No breaking change to existing raptor behavior. cve_env imports no core/ module, so nothing in core/ or other packages/ changes meaning. The one shared-state edit is the requests pin bump (2.33.0 → 2.33.1, patch).

Open-PR file overlap (FYI, mechanical): #470 also edits compute_filters.py + tests.yml; #215 also edits requirements.txt. Each appends its own entries; whichever lands second rebases. No coordinated ordering required.

Adversarial review checklist

  • No argv / shell-injection vectors — launchers exec a fixed python3 -c; dispatcher uses the package's own argparse.
  • No path-resolution / traversal issues — output root is env-gated (CVE_ENV_OUTPUT_ROOT), defaults under raptor out/.
  • No signature break for public callers — additive package + additive wiring; no existing symbol changed.
  • No error-path / resource leaks — per-CVE container cleanup + image prune are opt-in flags, unchanged from upstream.
  • No hostile-input vectors newly exposed — CVE advisory text is routed through cve-env's existing exploit_text_sanitizer; prompt files now under the envelope audit.
  • No silent default that masks errors — CVE_ENV_OUTPUT_ROOT unset reproduces standalone behavior exactly.
  • No dead code introduced — whole-tool vendored import; zero raptor-core adoption deferred to a later phase.
  • No platform-specific assumption added — cve-env targets Docker-on-Linux/Colima as upstream; no new host assumption.

Author notes

Q1 — which lines implement the feature? The capability is the vendored packages/cve_env/ tree; the load-bearing integration glue is packages/cve_env/cve_env/config.py (_find_output_root), packages/cve_env/cve_env/agent/refusals.py (default_log_path), bin/cve-env, libexec/raptor-cve-env, .claude/commands/cve-env.md, and the 5 CI/audit edits.
Q2 — one concrete input handled right now. Standalone, refusals.py wrote refusals-log.md via Path(__file__).parents[3].parent (a fixed depth above the repo); under packages/cve_env/ that depth resolves to a different directory. The fix routes it through OUTPUT_ROOT, so a refusal during bin/cve-env build CVE-… lands in raptor's out/, not at an unexpected ancestor path.
Q3 — likely reviewer pushback + answer. "45k LOC / 142 files is too large to review — split it." It is a vendored whole-tool import where intermediate states fail CI (the package needs its wiring + deps in the same commit); it is behavior-faithful (zero core/ adoption), which the 1529/1529 standalone-parity run and the 5-CVE live smoke demonstrate, so the review surface is "is the wiring correct," not "is 45k lines of new logic correct."


Note

Medium Risk
Large vendored agent that runs Docker and interpolates CVE advisory text into LLM prompts; risk is mitigated by isolation from core/, hardened container defaults, tightened prompt-envelope audit, and no changes to existing package APIs beyond the shared requests patch bump.

Overview
Adds cve-env to RAPTOR as packages/cve_env — an agentic pipeline (research → resolve → acquire → launch → verify) that builds Docker environments running affected apps at pre-patch versions for a given CVE. Phase 1 is a behavior-preserving lift-and-shift from standalone gadievron/cve-env; the package keeps its own claude-agent-sdk loop and does not import any core/ modules yet.

User-facing wiring mirrors cve_diff: bin/cve-env (env-strip, PYTHONPATH, trust marker), libexec/raptor-cve-env (build / doctor), and .claude/commands/cve-env.md. Artifacts are routed through CVE_ENV_OUTPUT_ROOT (launchers default to raptor out/) so audit JSONL, outcome sidecars, and refusals logs land in the right tree under the deeper package layout.

CI / audit: new cve_env path filter, dedicated python-unit-tests-cve-env job, fast-tier ignore, and registration of agent/prompts.py / agent/loop.py in the prompt-envelope audit. README documents /cve-env (experimental) and /cve-diff.

Bundled upstream fixes (documented in PROVENANCE.md): interrupted-run cost floor keyed on status (not zero tokens — fixes dead floor under session auth) and ResultMessage cumulative usage merged via max() to stop ~2× token double-counting.

Deps: pins claude-agent-sdk==0.1.71; bumps requests to 2.33.1 for compatibility with cve-env’s floor.

Reviewed by Cursor Bugbot for commit f00d8a6. Bugbot is set up for automated code reviews on this repo. Configure here.

…d-shift)

Brings gadievron/cve-env (agentic CVE -> Docker environment builder) into
raptor as packages/cve_env, wired like the sibling cve_diff tool (Pattern B:
bin/ launcher + libexec/ dispatcher + .claude/commands slash command).

Provenance: imported from gadievron/cve-env @ ba9f91c (packages/cve_env/PROVENANCE.md).

Phase 1 is a behavior-preserving lift-and-shift: cve-env keeps its own agent
loop (claude-agent-sdk), Docker tooling, dockerfile generation, config, and
HTTP layer, and adopts ZERO core/ modules. Selective core adoption is deferred
to a later phase behind equivalence checks.

Contents (counts re-derived at commit time):
- packages/cve_env: 41 source .py + 89 test .py
    find packages/cve_env/cve_env -name '*.py' | wc -l  -> 41
    find packages/cve_env/tests   -name '*.py' | wc -l  -> 89
- git diff --cached --stat | tail -1 -> 142 files changed, 45379 insertions(+), 2 deletions(-)
- Wiring: bin/cve-env, libexec/raptor-cve-env (trust-guarded),
  .claude/commands/cve-env.md (dispatch: libexec/raptor-cve-env)
- Deps: requests 2.33.0 -> 2.33.1 (satisfies cve-env's >=2.33.1 floor;
  compatible with urllib3==2.7.0 + core/http) + claude-agent-sdk==0.1.71
  (pinned from cve-env's uv.lock)
- CI: pytest.ini pythonpath, compute_filters cve_env filter + prompt_audit
  mirror, tests.yml dedicated python-unit-tests-cve-env job, prompt-envelope
  audit registration of cve-env's prompt-construction files
- README: /cve-env (+ /cve-diff) commands-table rows

Local verification this session:
- pytest packages/cve_env/tests          -> 1529 passed, 4 skipped (identical
  to the standalone ship-11-june baseline)
- pytest core/http/tests (requests 2.33.1) -> 106 passed
- pytest core/security/tests/test_prompt_envelope_audit.py -> 18 passed
  (0 new violations from the cve-env registration)
- pytest .github/tests/test_filter_coverage.py -> 11 passed
- check_command_metadata.py -> 28 command files lint-clean
- 5-CVE live smoke through bin/cve-env: 4 success + 1 turn_cap
  (verify_passed=True), 0 tool-errors, 0 integration defects
Full raptor suite is validated by CI on this PR.

Path landmines fixed for the new layout (artifacts -> raptor out/ via
CVE_ENV_OUTPUT_ROOT): config._find_output_root, agent/refusals.py, and 8
layout-coupled guard tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread packages/cve_env/cve_env/agent/loop.py Outdated
Comment thread packages/cve_env/cve_env/tools/image_resolve.py Fixed
Comment thread packages/cve_env/cve_env/tools/source_build.py Fixed
Comment thread packages/cve_env/cve_env/tools/verify.py Fixed
Comment thread packages/cve_env/tests/unit/test_cascade_order_phase29.py Fixed
Comment thread packages/cve_env/tests/unit/test_experiment_env_vars.py Fixed
Comment thread packages/cve_env/tests/unit/test_experiment_env_vars.py Fixed
Comment thread packages/cve_env/tests/unit/test_experiment_env_vars.py Fixed
Comment thread packages/cve_env/tests/unit/test_source_build.py Fixed
Comment thread packages/cve_env/tests/unit/test_source_build.py Fixed
Comment thread packages/cve_env/tests/unit/test_source_build.py Fixed
@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

raptor-sca diff — /tmp/sca-base/findings.json/tmp/sca-pr/findings.json

  • New: 0
  • Resolved: 0
  • Persistent: 37 (37 medium)

No new or resolved findings; persistent backlog of 37 unchanged.

gadievron and others added 2 commits June 13, 2026 02:42
raptor's ruff-pr (F401/F811/F821/F841) and RAPTOR_MAX_TEST_SECONDS=10
fast-tier guard are stricter than cve-env's standalone CI; PR #802 CI
surfaced two failures:

- ruff (F401 unused-import / F841 unused-local):
    uvx ruff check --select F401,F811,F821,F841 <diff .py> -> Found 26 errors
  23 auto-fixed (unused imports); 3 F841 unused-locals renamed to `_`-dummy
  in test_bench200_bug_fixes / test_f9_b21_root_cause / test_loop — the
  asyncio.run(...) and path-construction calls are preserved.
- 2 docker_run tests ran the real host-port inspect poll to
  _INSPECT_POLL_TIMEOUT_S (10s) -> tripped the fast-tier 10s guard. Marked
  @pytest.mark.slow (nightly tier), per raptor's guard guidance.

Verification:
- uvx ruff check --select F401,F811,F821,F841 <diff> -> All checks passed!
- RAPTOR_MAX_TEST_SECONDS=10 pytest packages/cve_env/tests -n auto
    -> 1527 passed, 4 skipped (the 2 docker_run tests now slow-tier); slowest 1.89s

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ports upstream gadievron/cve-env 89917d8 (PR #2) into the vendored copy.

Both Outcome finalization sites in packages/cve_env/cve_env/agent/loop.py
floored total_cost_usd on max(last_cost, token_estimate) only. Under Claude
Code session auth usage is None on every message (tokens=0 -> token estimate=0);
on an interrupted exit the SDK cost is also implausibly low, so a 46-turn build
logged $0.013 while num_turns stayed accurate.

Add config.estimate_cost_from_turns + a shared _floor_cost() helper applied at
both Outcome sites, gated to every interrupted status {turn_cap,
budget_exhausted, error, interrupted, incomplete, rate_limited} with no token
usage and bounded by the budget cap. Clean exits (success, verified_partial,
verify_failed, launched_no_verify, unresolvable) report cost reliably and are
excluded.

Test functions added (grep -c 'def test_' on the new file): 5
Vendored-package run (pytest tests/unit/test_cost_floor_non_clean_exit.py):
14 passed. Regression (verify+map_status+loop+cost_floor): 244 passed.
raptor ruff-pr gate (F401,F811,F821,F841): clean.

Depends-on: #802

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gadievron

Copy link
Copy Markdown
Owner Author

Follow-up fix stacked on this branch: #803fix(cve-env): floor interrupted-exit cost by turns under session auth (ports upstream cve-env PR #2 / 89917d8). It depends on this PR; see #803 for merge ordering.

@gadievron

Copy link
Copy Markdown
Owner Author

Updated: this PR now also includes the cost-floor fix (commit 93de23e9, ports upstream cve-env PR #2 / 89917d8) — the previously-stacked #803 was folded in here so reviewers have a single PR with full CI. The vendored package ships already carrying the upstream fix; see packages/cve_env/PROVENANCE.md for the recorded divergence.

Follow-up to the cost-floor fix already in this PR. The floor was gated on
input_tokens==0 and output_tokens==0, but Claude Code session auth reports a
tiny NONZERO token stub (in=10, out=2) on interrupted runs, so the gate was
always False and the floor never fired in production -- a live 97-turn turn_cap
(CVE-2019-11043) logged $0.095. Drop the token gate; the floor is a max()
bounded by the budget cap, so it only raises and keeps a real token-bearing
run's cost unchanged.

Ports upstream cve-env 184d2c8. Vendored-package test run
(pytest tests/unit/test_cost_floor_non_clean_exit.py): 16 passed.
raptor ruff-pr gate (F401,F811,F821,F841): clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gadievron

Copy link
Copy Markdown
Owner Author

Updated again (commit 4ff38fd8): folded in a follow-up cost-floor fix. The floor shipped here was gated on input_tokens==0 and output_tokens==0, but Claude Code session auth emits a tiny nonzero token stub (in=10, out=2), so the gate never matched in production and the floor was dead code (a live CVE-2019-11043 turn_cap logged $0.095 for 97 turns). Gate now keys on interrupted-status membership only. Ports upstream cve-env #3. See packages/cve_env/PROVENANCE.md.

gadievron and others added 3 commits June 15, 2026 06:22
…(PR #4)

Ports upstream cve-env PR #4 (merged 43731d5) into the vendored copy.
state.total_*_tokens was accumulated from BOTH per-message
AssistantMessage.usage AND the session-cumulative ResultMessage.usage (SDK
types.py "Cumulative API usage for the session"), double-counting tokens ~2x
(worse across multi-ResultMessage retry storms). Merge the cumulative RM
usage via max() (new _merge_cumulative_tokens), keeping the per-message
_accum_tokens += path (which also covers give_up runs with no terminal
ResultMessage). Benign for cost under session auth (the token estimate never
wins _floor_cost's max()); fixes telemetry / API-key over-report.

Excludes an unrelated upstream Outcome hoist-refactor not part of PR #4.

Tests: 4 added (grep -c '^def test_' test_token_double_count.py => 4),
4 passed against the vendored module
(PYTHONPATH=packages/cve_env pytest test_token_double_count.py => 4 passed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The filter-coverage gate (.github/tests/test_filter_coverage.py) fails because
packages/codeql, packages/llm_analysis and core/orchestration import
core.threat_model (and llm_analysis imports core.dataflow.structural_validator),
added by the threat-model (#776) and structural-validator work on main, but the
codeql/llm_analysis/orchestration filter globs in compute_filters.py were never
updated to cover those core/ paths. Pre-existing on main — its CI skipped
ci-lint-tests (path-scoped); this PR touches .github/ so the gate runs and
surfaces it. Add the missing globs (pure additions, no glob narrowed):
  - codeql:        core/threat_model
  - llm_analysis:  core/dataflow, core/threat_model
  - orchestration: core/threat_model

Verified: .github/tests/test_filter_coverage.py 11 passed (0 uncovered);
test_compute_filters.py + libexec coverage 11 passed; prompt-envelope audit
18 passed; command-metadata 29 files clean; packages/cve_env suite 1547 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@grokjc

grokjc commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Some of those CodeQL issues are real and they can all be trivially fixed; dismissing them hides several bugs. Will pick up later but leaving note to self.

verify.py: pin minimum TLS to 1.2 when probing target containers.
Cert validation is disabled by design (throwaway certs); pinning
TLS 1.2 closes the remaining downgrade surface.

image_resolve.py: add _normalize_registry_token() so operators can
paste URL-ish deny values into CVE_ENV_DENY_REGISTRY and have
docker.io aliases (index.docker.io, registry-1.docker.io) collapse
consistently. Deny check now operates on normalized hosts rather
than substring of raw input.

source_build.py: rewrite normalize_github_url() to use urlparse
hostname comparison (was netloc, which trips on ports) and to match
SCP-style URLs (git@host:path) by regex rather than startswith.
Behavior preserved for all 8 existing edge-case tests including the
attacker-host spoofing suite.

tests: replace 14 incomplete URL-substring checks (.startswith /
"x" in y) with parsed-host equality assertions. The previous form
would have accepted hypothetical bad refs like
attacker.com/docker.io/drupal; the new form correctly rejects them.

Closes the 16 CodeQL alerts dismissed on PR #802.
Comment thread packages/cve_env/cve_env/tools/image_resolve.py Fixed
grokjc added 3 commits June 16, 2026 23:59
Update logic to check for denied Docker Hub access using set comparison.
…ns, formatting

- source_build.py: enforce https:// on _http_get_json/_http_get_bytes
  (S310 URL scheme audit)
- config.py: narrow except Exception to (OSError, ValueError)
- lifecycle.py: tempfile.gettempdir() instead of hardcoded /tmp (S108)
- image_resolve.py: strip trailing whitespace (W291)
- test_source_build.py: skip tests when claude_agent_sdk is absent
- ruff format across all source + test files (E501 593 → ~140)
@grokjc

grokjc commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

That's all the CodeQL issues fixed rather than suppressed...

grokjc added 9 commits June 20, 2026 22:50
25 test files import from cve_env.agent.loop or cve_env.agent.llm,
which transitively require claude_agent_sdk. Without the gate, these
fail at collection time on any machine without the SDK installed,
blocking the entire test suite. importorskip at module level cleanly
skips them.
…itization, compose safety

- Validate container ownership (cve-env.owner label) before docker exec/stop
- Expand safe_env dangerous vars: DOCKER_HOST, BASH_ENV, OPENSSL_CONF, etc.
- copytree symlinks=True prevents host file exfiltration via vulhub repos
- Compose security rewrite raises on parse failure instead of silently skipping
- Validate apt_packages against regex to prevent shell injection in Dockerfile RUN
- Add resource limits (4G mem, 2 CPU, 512 PIDs) to docker run containers
- Reduce tarball download cap 8G→512M, extraction cap 50G→2G
- Don't auto-load cve-env.toml from CWD (malicious repo config injection)
- Sanitize cve.references before embedding in agent prompt
- Guard module-scope int()/float() env parsing against ValueError crashes
- Fix _env_bool to handle explicit falsy values; align proprietary-verify getter
- Flag ADD-from-URL in Dockerfile validator
- Validate copy_ops dst for path traversal
- Empty docker stderr → unknown not transport
- Fix backslash collapsing dead code in dockerfile sanitizer
- Add Docker Hub/GitLab PAT patterns to audit secret redaction
_load_toml_config() no longer reads cve-env.toml from CWD without an
explicit CVE_ENV_CONFIG_FILE env var (malicious-repo config injection
defense). Split the old test_load_toml_reads_from_cwd_default into two:
one asserting CWD is ignored, one asserting explicit env var still works.
- Gate 10 additional test files with pytest.importorskip for
  function-level SDK imports
- Rewrite refusals wiring tests to verify production source via
  Path.read_text instead of importing SDK-dependent modules
- Fix test_verify: mock subprocess.run so regex validation is reached
- Fix test_reset_registry_complete: skip on ImportError
- Update stale docstrings referencing removed CWD auto-loading
Security:
- compose: staging tmpdir leaked on ComposeError
- compose: port-range extraction took only high end (80 never exposed)
- docker_run: env key containing '=' creates misnamed container var
- docker_build: GPG/patch recovery guards bypassed when image_tag=""
- source_build: tag API capped at 100; add pagination (10 pages)
- dockerfile_gen: empty workdir="" rendered invalid WORKDIR instruction

Correctness:
- loop.py: audit_path diverged from writer._path_for on sanitized CVE ID
- loop.py: Fix#8 continuation session_id asymmetry
- loop.py: _classify_api_overload used substring not startswith
- loop.py: GiveUpReceived from tool-cap left audit trail gap
- docker_run: duplicate_failing_attempt had reason_class="ok"
- config.py: stale comment documented removed CWD-autoload
- config.py: get_tool_attempt_cap silently swallowed malformed env var
- source_build: failed clone left partial target dir
- dockerfile_gen: P21 false-fired on standalone apt-get update
- run_in_container: docstring claimed -i flag not in argv

Hardening:
- audit.py: boundary-repair TOCTOU eliminated (single open)
- lifecycle.py: acquire_lock uses O_EXCL exclusive-create
- cli.py: double release_lock could delete another process's lock
- run.py: thread result passing via queue.Queue not bare dict
… prepend

fh.write("\n" + line) created an empty first line in new audit files,
causing json.loads to fail. Use fh.tell() within the same open to
conditionally prepend only when the file already has content.
fh.tell() in text append mode returns 0 on Linux (POSIX does not
require seeking to end on fopen "a"). Switch to a+b (binary
append+read): seek to end for file size, seek back one byte to check
if last char is newline, write with conditional prefix. Single file
handle eliminates the TOCTOU; binary mode gives reliable tell().
…, correctness, UX, performance

Security (H-class):
- agent/loop: isinstance checks for cap-reached exceptions (not string comparison)
- agent/tools: shutil cleanup on fuse build failure; thread-unsafety docstring
- agent/refusals: escape C1 control codes (0x80-0x9F) and bidi overrides
- tools/docker_run: reject flag-shaped env keys starting with '-'
- tools/docker_compose_up: YAML parse failure raises ComposeError (not silent None);
  reject parent-dir socket mounts; add SYS_RAWIO/NET_RAW/SYS_MODULE to dangerous caps
- tools/web_fetch: DNS failure returns blocking reason (fail-closed, not None)
- config: _env_parse helper with logger.warning on malformed env values
- utils/safe_env: expand _DANGEROUS_ENV_VARS from ~20 to 70+ (BASH_ENV, NODE_OPTIONS,
  DOCKER_HOST, GIT_CONFIG_*, KUBECONFIG, GCONV_PATH, TMPDIR, build-tool vars, etc.)

Correctness (C/M-class):
- config: negative budget clamped to 0; _env_bool explicit falsy set
- cli: prevent double lock release; PID in run_id
- agent/loop: session_id fallback; _map_status explicit allowlist; continuation
  cost/state tracking; suppress Exception not BaseException
- agent/audit: apiKey + ya29.* GCP OAuth patterns in secret detection
- tools/dockerfile_gen: CMD quote escaping; apt_packages regex validation;
  guard empty workdir; install_steps newline stripping
- tools/verify: IPv6 bracket wrapping; _normalize_kwargs canonical precedence;
  pop host_ip/host_port before **kwargs; ReDoS guard; broader DB hints
- tools/source_build: JSONC comment anchored to ^; word-boundary fuzzy match;
  tar filter='data' version compat; devcontainer continue-not-return
- tools/github_fetch: urllib.parse.quote for ref and clean_path
- tools/_image_resolve_state: try/except around int() env vars
- utils/lifecycle: atomic O_CREAT|O_EXCL lock
- utils/run: catch-all Exception in _target thread
- utils/dockerfile_hygiene: consume flag value after --flag; collapse 4+
  backslashes only; odd backslash count = continuation

Tests:
- Add pytest.importorskip("claude_agent_sdk") gate to all test files that
  transitively import SDK-dependent modules (35+ files)
- Fix wrong assertion messages (T6/T7), default values (T11/T25),
  exception capture (T12), monkeypatch patterns (T15)
- Update mocks for --pull missing, run_with_timeout, paginated tags
- Deduplicate importorskip calls introduced by merge of round 1-2 + round 3
… for isinstance refactor

cont_cost_acc += state.last_cost_usd double-counted because
state.last_cost_usd is cumulative across runs (+=), not a per-run
delta. Revert the 4 += sites to run.total_cost_usd (per-run, fresh
from each run_agent call). The initial assignment (line 2351) stays
on state.last_cost_usd — at that point cumulative == first-run total.

B2 structural test searched for quoted "TurnCapReached" / "BudgetCapExceeded"
but H7 changed the except handler from string comparison to isinstance,
removing the quoted literals. Update .find() to match unquoted class names.
@grokjc

grokjc commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator

Pausing here, I think this is good for the initial merge now; further changes involve actual integration work/consuming API from RAPTOR.

grokjc added 6 commits June 21, 2026 10:39
Previous SDK gating inserted `import pytest` + `importorskip` at the
top of each file; the pre-existing `import pytest` further down became
a dead duplicate. Harmless at runtime (Python caches modules) but noisy
in review.
The exploit_feasibility filter missed core/build/** after build_flags
was added as a dependency. The ELF planning test spawned nm on a
garbage binary, timing out at ~14s on CI runners.
DNS rebinding: post-connect peer-IP check catches short-TTL rebinding
that bypasses the pre-request getaddrinfo guard; fail-closed on DNS
resolution failure.

Container isolation: strip devices from compose specs; detect parent-dir
docker socket mounts; owner-label gate on docker_stop; per-container
memory/CPU/PID limits; reject env keys containing '=' or starting '-';
--cap-drop passed per-capability (not comma-joined).

Input validation: bound CVE_ENV_EXTRA_PROMPT_PREFIX (2000 chars, reject
control chars); bound CVE_ENV_DOCKER_RUN_TIMEOUT_S (10-3600); reject
registry-qualified image tags in docker_build; block cloud metadata IPs
in verify probes; ReDoS-safe regex in exploit_text_sanitizer and verify
log checker.

Tarball extraction: filter symlinks, device nodes, setuid bits on
Python <3.12 fallback path; reduce default tarball cap from 8 GiB to
512 MiB.

Audit log: broader secret redaction (Docker Hub PAT, GitLab PAT, Slack,
JWT, npm, PyPI tokens; git/ssh URL credentials); atomic single-fd
boundary repair eliminates TOCTOU in write path.

Assorted: paginated GitHub tag listing; IPv6 bracket-wrapping in verify
URLs; canonical alias precedence in _normalize_kwargs; compose port
range expansion; compose --pull missing (not always); compose YAML parse
errors surfaced as ComposeError.
Device mappings are now filtered by default: safe pseudo-devices
(/dev/null, /dev/zero, /dev/urandom, /dev/random, /dev/stdin,
/dev/stdout, /dev/stderr, /dev/fd/*) pass through; dangerous
mappings are stripped. Two override paths:

- allow_devices=True on docker_compose_up tool call (agent decides
  per-CVE based on NVD description / compose file context)
- CVE_ENV_ALLOW_DEVICES=1 env var (operator override for batch runs)

Also adds core/atomic_fs to fuzzing CI filter.
@grokjc
grokjc force-pushed the feat/cve-env-integration branch from a03f577 to a663e39 Compare July 21, 2026 15:21
…ation

# Conflicts:
#	.github/scripts/compute_filters.py
#	.github/workflows/tests.yml
@grokjc
grokjc force-pushed the feat/cve-env-integration branch from 241d7b2 to 1936b65 Compare July 21, 2026 16:46
@grokjc grokjc self-assigned this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants