feat(ai): per-seat difficulty, elimination/timing lines, action-cap flag for ai-commander#6195
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces structured diagnostic tracking for AI action loop breaks to help identify and debug driver stalls in ai_commander. It wraps the results of run_ai_actions in an AiActionsRun struct containing a detailed AiActionsBreakReason, and updates ai_commander to detect stalls, log diagnostic context, and exit with a distinct status code. Additionally, it adds per-seat difficulty overrides, custom action caps, post-resolution deck size logging, and stricter CLI argument validation. The review feedback highlights a maintenance concern regarding the duplication of accepted difficulty labels in ai_commander.rs and recommends centralizing this source of truth in config.rs.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Maintainer triage — held as a stacked dependency on #6194. This branch contains #6194's commits plus its own changes, and the PR description explicitly asks for #6194 to be reviewed first. #6194 currently has a blocking review finding in its stall-diagnostic path, so this PR is not independently reviewable or approvable at its current base. I will resume review after #6194 is corrected and lands/rebases. |
|
Addressed a review finding: the action-cap abort path ( For the record: as part of this same commit, 🤖 Generated with Claude Code |
|
Maintainer status — processed current head This head still contains the explicitly stacked #6194 work. #6194 has an unresolved formal changes-requested review for loss of the original |
|
Reprocessed current head This stacked PR remains held, not independently approved or enqueued: its base PR #6194 is still open at |
…lag for ai-commander
pod-lab's gauntlet harness runs mixed-skill 4-player tables and needs to
distinguish an expected early elimination from a stall, watch for slow
turns, and tune the action budget per run without recompiling.
- --difficulty-p0..--difficulty-p3 override a single seat's difficulty
(parsed through the existing single-authority parse_difficulty); the
table-wide --difficulty remains the default for any seat left unset.
Each seat's config is still built through the unchanged
create_config_for_players, called once per seat instead of once for the
whole table. Resolved per-seat difficulty is echoed at startup, mirroring
the anti-silent-downgrade guard --difficulty already has.
- The engine already flips player.is_eliminated per CR 800.4 when a player
leaves the game; the runner now watches that flag every action batch and
prints one ELIMINATED: P{n} turn=... actions=... elapsed=... line the
moment it flips.
- The existing per-turn snapshot line and the new elimination line both
carry wall-clock elapsed= timing, so slow turns are visible from stdout
without waiting for the final summary.
- The previously hardcoded MAX_TOTAL_ACTIONS constant is now --action-cap
<N> (default unchanged, 200,000), parsed through parse_action_cap, which
hard-fails a non-positive/non-numeric value instead of silently keeping
the default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An action-cap abort falls into the non-GameOver match arm and was printing the full softlock STALL block, including a factually wrong break_reason=unknown line (last_break_reason is never set on the abort door). Guard the STALL block with `if !aborted`; the ABORT line plus the pod-lab-matched "Game did NOT reach GameOver." line (printed in both cases, unchanged) are sufficient on the abort path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`ai_commander` restated the accepted `--difficulty` label list that `config.rs` already owns, so a new arm in `AiDifficulty::from_label` could drift out of the CLI's accepted set without any signal. Hoist the list to a module-level `pub const ACCEPTED_DIFFICULTY_LABELS` in `config.rs` next to `from_label`, and have the CLI validator consume it. `parse_difficulty`'s membership test and error message are unchanged. `from_label` itself is left alone: it is an exhaustive match producing enum values, not a membership test, and driving it off a `&[&str]` would trade a compiler-checked match for a positional lookup that can drift silently. The two are instead pinned together by a round-trip test — derived `Debug` renders the variant name, so a misspelled or unmapped label falls back to `Medium` and fails the assertion. Also corrects a comment on the per-seat difficulty echo that claimed a harness could catch a `--difficulty-pN` typo the way `parse_difficulty` hard-fails a bad `--difficulty`. It cannot: that arg branch silently no-ops on a non-numeric suffix, an index >= 4, or a missing value. The phase#6080 rationale for echoing the resolved tier is kept; the unsupported typo-catching claim is removed rather than papered over. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…st target Review follow-ups on the ai-commander CLI: - Drop `test = false` from the `ai-commander` `[[bin]]`. Its `#[cfg(test)]` region never compiled or ran, so the existing `parse_action_cap` test (and any bin-local test) was dead. The region now compiles under `-D warnings` for the first time. - Split `parse_action_cap` into a `Result`-returning `parse_action_cap_checked` core plus a thin exiting wrapper, so both the accept and reject paths are unit-testable (the wrapper's `exit(1)` branch was untestable as one function). - `--difficulty-pN` no longer silently drops a mistyped flag. A non-numeric or out-of-range (`>= 4`) seat index, or a missing label, now hard-fails at startup like its `parse_difficulty`/`parse_action_cap` siblings, via the pure, unit-tested `parse_seat_override`. The label value arg is consumed unconditionally so a bad index can no longer leak its label into the arg catch-all (`--difficulty-p9 Hard` previously applied nothing and let "Hard" cascade through, silently). - Extract the end-of-run aborted/stalled/completed decision (from `aborted` + `waiting_for`) into the pure, unit-tested `classify_run_outcome`; `main` matches on it for both the printed outcome and the exit code. Stdout is byte-identical on the three common paths (verified by diffing the `println!` literals). The one behavior change is intended: the narrowly-reachable `(aborted, GameOver)` corner — game ends on the last action of a batch, then the cap fires — now prints the abort line instead of the old inline code's contradictory "Game ended cleanly" (exit code 2 either way); the abort report is the honest label for a run the cap cut short. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The `accepted_difficulty_labels_round_trip_through_from_label` test only enforced list -> enum -> list: it could not catch a `from_label` arm (i.e. an enum variant) that never made it into `ACCEPTED_DIFFICULTY_LABELS`. Add `every_difficulty_variant_appears_in_accepted_labels` for the reverse direction, honest about its two layers: a wildcard-free `label_of` match forces a label arm for every variant (a compile error the moment a variant is added), and an `all.len()` vs label-count assertion catches the hand-maintained `all` array or the label list drifting out of step. The `ACCEPTED_DIFFICULTY_LABELS` doc comment now claims exactly what the two tests together enforce — no more. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
matthewevans
left a comment
There was a problem hiding this comment.
[MED] --action-cap does not bound the configured number of actions. Evidence: crates/phase-ai/src/bin/ai_commander.rs:313-324 calls run_ai_actions before testing total_actions >= action_cap, while crates/phase-ai/src/auto_play.rs:19,136 permits a batch of 200 actions. Why it matters: --action-cap 1 (and every non-multiple of 200) can execute up to 199 actions beyond the operator-specified budget, so a harness cannot use the new flag to cap expensive or looping runs as advertised. Suggested fix: thread the remaining action budget into the driver/batch boundary (or execute no more than the remaining number of actions) and add a regression that proves a small cap is never exceeded.
|
Rebased and ready for a fresh review.
Stdout is byte-identical on every reachable common path (pod-lab parses these lines); the one intended change is the Note on CI: the failing Card data check is repo-wide baseline staleness from tonight's Hobbit-set auto-unlock — Bard, King of Dale entered the pool at 00:00Z and added a row to the frozen draw-replacement corpus — so it fails identically on |
|
Review status — the current-head blocker remains unresolved.
|
… boundary Review finding on phase-rs#6195: --action-cap did not bound the configured number of actions. ai_commander called run_ai_actions — which always permits a batch of up to MAX_AI_ACTIONS_PER_SEQUENCE (200) actions — and only then compared total_actions with the cap, so --action-cap 1 (and every non-multiple of 200) could execute up to 199 actions past the operator's budget. Parameterize the batch loop as run_ai_actions_bounded(.., max_actions), clamped to min(max_actions, MAX_AI_ACTIONS_PER_SEQUENCE) so the module's safety cap stays the single authority (callers can shrink a batch, never enlarge it); run_ai_actions keeps its exact signature as a thin delegate passing the safety cap, so all existing callers are untouched. The commander driver now passes remaining = action_cap - total_actions into each batch, making total_actions == action_cap the exact worst case; the break-reason-before-cap check ordering is deliberately unchanged. Two exact-equality regressions in scenarios.rs (card-data-free, seeded 60-card libraries so the pass-priority stream provably outlasts any small budget): the bounded primitive returns exactly its budget with no break reason, and a mirror of the driver arithmetic proves a small cap of 5 is hit exactly, never exceeded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
matthewevans
left a comment
There was a problem hiding this comment.
Approved: the bounded AI action runner enforces the remaining --action-cap at the production batch boundary, and the long-stream regression proves a small cap cannot be exceeded.
matthewevans
left a comment
There was a problem hiding this comment.
Request changes — the cap implementation is at the right batch boundary, but its regression does not protect the commander driver that exposed the bug.
🔴 Blocker
[MED] The action-cap regressions do not exercise the ai_commander production loop. Evidence: crates/phase-ai/tests/scenarios.rs:283-345 invokes run_ai_actions_bounded and reproduces the budget loop locally, while the production call site is crates/phase-ai/src/bin/ai_commander.rs:251-333. Why it matters: changing that production call back to run_ai_actions would reintroduce the up-to-199-action overshoot from the prior review, yet both new tests would still pass. Suggested fix: extract the commander batch/remaining-budget driver boundary into a testable helper used by main, then assert a small-cap long stream through that helper never exceeds—and reaches—the cap.
✅ Clean
The implementation itself now passes action_cap - total_actions into run_ai_actions_bounded, and the bounded primitive preserves the existing 200-action safety ceiling (crates/phase-ai/src/bin/ai_commander.rs:251-267, crates/phase-ai/src/auto_play.rs:145-173). That resolves the prior production overrun by inspection.
Recommendation: add a discriminating commander-driver regression, then I can re-review the current implementation.
matthewevans
left a comment
There was a problem hiding this comment.
🔴 Blocking
The new action-cap regression still does not drive the production ai_commander loop. crates/phase-ai/tests/scenarios.rs:312-341 duplicates the remaining-budget loop, while crates/phase-ai/src/bin/ai_commander.rs:251-333 owns the actual driver and only has parser/outcome-helper tests later in the file. A regression in the CLI loop’s budget plumbing would therefore keep this test green.
Extract the production driver loop into a testable function used by main, then exercise a small action cap through that function. That gives the regression coverage its claimed protection without maintaining a mirrored implementation.
Recommendation: request changes — add a discriminating production-driver action-cap test before this is reconsidered.
…guards the production path Round-2 review finding on phase-rs#6195: the cap regressions reproduced the remaining-budget arithmetic in a hand-mirrored loop, so reverting the production call site back to unbounded batches would resurrect the overshoot while both tests stayed green. Extract the batch/remaining-budget boundary into run_driver_loop in auto_play.rs — the single authority main() now calls — returning a typed DriverOutcome { total_actions, exit: DriverExit::{CapReached, BatchBreak(reason)} }. The observer callback (per-batch, mirroring the duel suite's observer seam, mutable batch access for log draining plus the pre-batch running total) carries every printing/dump side effect in main unchanged: stdout is byte-identical, including the pre-batch totals in turn/ELIMINATED lines and both blank lines around the ABORT message. classify_run_outcome, driver_step, exit codes, and the STALL diagnostics are untouched. The helper's doc fences off drive_game_observed's deliberately different batch-boundary-overshoot contract (baseline- witnessed) so the two are never 'helpfully' unified. The replaced hand-mirror test now drives the production helper (cap 5: never exceeds, exactly reaches). A second regression uses cap 250 — beyond the 200-action per-batch safety clamp — to force two loop iterations and pin the across-batch accounting (batch sizes [200, 50]) where the original overshoot lived; a cap under one batch cannot discriminate that arm. The [200, 50] coupling to MAX_AI_ACTIONS_PER_SEQUENCE is deliberate and self-diagnosing (comment names the constant and file). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
matthewevans
left a comment
There was a problem hiding this comment.
Approved: the remaining-budget loop is now a production run_driver_loop boundary, and the two-batch regression proves a small --action-cap is exact rather than a batch-boundary approximation.
Previously STACKED on #6194 — that PR has now MERGED (squashed as
45c6b30b8), and this branch is rebased onto currentmain. This PR is now independently reviewable. Its diff is scoped to its own commits only; it does not re-list PR1's changes.Summary
Three additions to
ai-commanderfor pod-lab's mixed-skill gauntlet tables:--difficultystill sets the table-wide default, and new--difficulty-p0..--difficulty-p3flags override a single seat (parsed through the existing single-authorityparse_difficulty, so an override can never drift from the table-wide flag's validation). Each seat's config is still built throughcreate_config_for_players(unchanged, reused per seat rather than once for the whole table), and the resolved per-seat difficulty is echoed at startup before the game loop starts, mirroring the existing anti-silent-downgrade philosophyparse_difficultyalready established.player.is_eliminatedper CR 800.4 when a player leaves the game; the runner now watches that flag every batch and prints exactly oneELIMINATED: P{n} turn=… actions=… elapsed=…line the moment it flips, so a harness can tell an expected early elimination apart from a stall.elapsed=, and the elimination line above carries the same, so slow turns / perf regressions are visible from stdout alone instead of only the finalElapsed:summary.MAX_TOTAL_ACTIONSconstant is now--action-cap <N>(default unchanged, 200,000), parsed through a newparse_action_capthat hard-fails a non-positive/non-numeric value rather than silently keeping the default — the same silent-poison-failure class the difficulty parsing already guards against.Plus, from review on this round:
--difficultylabel list is hoisted out ofai_commander.rsinto a module-levelpub const ACCEPTED_DIFFICULTY_LABELSinconfig.rs, next to theAiDifficulty::from_labelit mirrors, and consumed by the CLI validator.from_labelitself is unchanged — it is an exhaustive match producing enum values, not a membership test, and driving it off a&[&str]would trade a compiler-checked match for a silently-driftable positional lookup. Instead, two tests inconfig.rspin the const and the enum together in both directions:accepted_difficulty_labels_round_trip_through_from_label(every listed label round-trips: list → enum → list) andevery_difficulty_variant_appears_in_accepted_labels(every enum variant, enumerated through a wildcard-free match that fails to compile when a variant is added, appears in the list and round-trips: enum → list → enum). One comment on the per-seat difficulty echo is also corrected; see the review comment for that and two disclosed follow-ups.ai-commandertest target — droppedtest = falsefrom its[[bin]]incrates/phase-ai/Cargo.toml, so the bin's#[cfg(test)]region actually compiles and runs. It never did before, which meant the existingparse_action_captest (and any bin-local test) was dead. The region now compiles for the first time under-D warnings.parse_action_capsplit into aResult-returningparse_action_cap_checkedcore + a thin exiting wrapper, so both the accept and the reject paths are unit-tested (the wrapper'sexit(1)branch was previously untestable).--difficulty-pNno longer silently drops a mistyped flag: a non-numeric or out-of-range (>= 4) seat index, or a missing label, now hard-fails at startup like itsparse_difficulty/parse_action_capsiblings, via a pure, unit-testedparse_seat_override. The label value arg is now consumed unconditionally, so a bad index can no longer leak its label into the arg catch-all.aborted+waiting_for) is extracted into a pure, unit-testedclassify_run_outcome;mainmatches on it for both the printed outcome and the exit code. Stdout is byte-identical on every reachable common path (verified by diffing theprintln!literals before/after) — pod-lab parses these lines. One narrowly-reachable corner does change:(aborted, GameOver)— the game ends on the last action of a batch and then the cap fires — now prints the abort line rather than the old inline code's contradictory "Game ended cleanly" (exit code 2 either way);Abortedis the honest label for a run the cap cut short.Test plan
Re-run after the rebase onto
main@45c6b30b8(the previous run's results were voided by the rebase):cargo fmt --allcleancargo clippy -p phase-ai --all-targets -- -D warningsclean (now includes the resurrected bincfg(test))cargo test -p phase-aiclean — new/resurrected tests:accepted_difficulty_labels_round_trip_through_from_label,every_difficulty_variant_appears_in_accepted_labels,parse_action_cap_accepts_positive_integer,parse_action_cap_rejects_zero_and_non_numeric,parse_seat_override_rejects_non_numeric_index,parse_seat_override_rejects_out_of_range_index,parse_seat_override_requires_a_label_value,parse_seat_override_accepts_valid_seat_and_label,classify_run_outcome_completed_only_on_clean_gameover,classify_run_outcome_aborted_when_cap_hit,classify_run_outcome_stalled_when_parked_off_gameover./scripts/check-parser-combinators.sh $(git merge-base upstream/main HEAD)) — Gate A + Gate G PASSprintln!literals byte-identical to the pre-change branch on every reachable common path (per-run output unchanged; the one intended change is the(aborted, GameOver)corner disclosed above)🤖 Generated with Claude Code