Skip SWE sandbox scoring for errored rollouts

[rasdani]

Summary

• Skip SWE taskset sandbox scoring when a rollout state already has any error, not only vf.InfraError.
• Apply the guard consistently across R2E, SWE-bench, MultiSWE, OpenSWE, SWE-Lego, SWE-rebench, and SWESmith tasksets.

Why

SWE-style tasksets use keep_sandbox_for_scoring=True, so successful rollouts keep the sandbox around for deferred test-based scoring. When the rollout already failed, running the sandbox test suite is not useful and can keep sandboxes live longer than necessary.

The previous guard only skipped vf.InfraError. That meant non-infra rollout errors such as model/provider failures could still enter sandbox scoring. In particular, a vf.ModelError("No available workers...") should not run SWE tests after the rollout has already errored.

This PR is intentionally limited to the rubric error guards. Broader cleanup lifecycle hardening is left out of this change.

Tests

uv run ruff check verifiers/envs/experimental/composable/tasksets/swe/multi_swe.py verifiers/envs/experimental/composable/tasksets/swe/openswe.py verifiers/envs/experimental/composable/tasksets/swe/r2e_gym.py verifiers/envs/experimental/composable/tasksets/swe/swe_bench.py verifiers/envs/experimental/composable/tasksets/swe/swe_lego.py verifiers/envs/experimental/composable/tasksets/swe/swe_rebench_v2.py verifiers/envs/experimental/composable/tasksets/swe/swe_smith.py
uv run ruff format --check verifiers/envs/experimental/composable/tasksets/swe/multi_swe.py verifiers/envs/experimental/composable/tasksets/swe/openswe.py verifiers/envs/experimental/composable/tasksets/swe/r2e_gym.py verifiers/envs/experimental/composable/tasksets/swe/swe_bench.py verifiers/envs/experimental/composable/tasksets/swe/swe_lego.py verifiers/envs/experimental/composable/tasksets/swe/swe_rebench_v2.py verifiers/envs/experimental/composable/tasksets/swe/swe_smith.py

Push hooks also passed: ruff check, ruff format, Semgrep v1 policy, AGENTS sync, and ty ci parity.

Note

Skip SWE sandbox scoring for any errored rollout, not just vf.InfraError

All seven SWE rubric solved methods previously returned 0.0 early only when state['error'] was an instance of vf.InfraError. This change broadens the guard to return 0.0 for any non-None error, skipping test execution for all error types. Behavioral Change: rollouts with non-InfraError errors now score 0.0 instead of proceeding to test evaluation.

^{Macroscope summarized 472fb00.}

---

Note

Low Risk
Low risk guard change: expands the existing early-return condition so SWE taskset rubrics skip running sandbox tests whenever the rollout state contains any error, reducing wasted sandbox work without altering test execution logic for successful rollouts.

Overview
Skips deferred SWE sandbox scoring for errored rollouts. Across the SWE-style rubrics (MultiSWE, OpenSWE, R2E-Gym, SWE-bench, SWE-Lego, SWE-rebench-V2, and SWE-Smith), the solved guard now returns 0.0 whenever state["error"] is set (instead of only when it is a vf.InfraError), preventing unnecessary test runs and keeping sandboxes alive longer after already-failed rollouts.

^{Reviewed by Cursor Bugbot for commit 472fb00. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ed56ed2. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR modifies runtime behavior by broadening error conditions that skip SWE sandbox scoring. Additionally, there are unresolved review comments questioning design decisions on files not visible in the diff, suggesting outstanding concerns that need human attention.

No code changes detected at 472fb00. Prior analysis still applies.

^{You can customize Macroscope's approvability policy. Learn more.}

[rasdani]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Another round soon, please!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[rasdani]

Narrowed this PR down to only the SWE taskset error guards.

The earlier version also changed rollout/group cleanup and cleanup error-handling semantics. Those broader lifecycle changes are intentionally removed from this PR so the diff only prevents deferred SWE sandbox scoring when a rollout state already has error set.