DFlash: let the gates-only dry run actually reach the DFlash gate

[anupsv]

The problem

#815 split the DFlash enablement interlock in two so the pipeline could be
validated without flipping the go-live flags: run_benchmark=false
publishes no ranked score, so a gates-only dispatch is admitted against an
inert track. The stated purpose of that split is to exercise
"DFlash correctness and parity gate (untimed)" — the one gate this track
owns and the serial pipeline does not.

It never could. Two steps sat above that gate and were not gated on
inputs.run_benchmark:

step	why it kills a dry run
Select hidden DFlash timed target from the pool	fails closed on an empty timed_prompt_pool
the timed-golden half of Prepare hidden DFlash goldens	the object key is that selection's output

The pool is deliberately empty on main — pinned by
DFlashEnablementInterlockTests.theTrackFixtureRemainsInertOnMain. So before
go-live every dispatch, ranked or dry, died red at pool selection: one
step short of the only DFlash-specific gate the dry run exists to run.

Two consequences, and the second is the worse one:

1. the interlock split bought a dry run that validated nothing the serial
   pipeline had not already validated; and
2. operators were being trained to read a red dry run as a successful one.

A dry run also downloaded a hidden timed golden it would never time
against.

The fix

Scope both to the ranked path. The DFlash correctness golden is pinned
independently of the pool (a fixed R2 key, not a pool entry), which is exactly
what lets the untimed leg — including the parity gate — run while the pool is
still empty. A clean dry run now ends green.

The ranked path is not relaxed

This is the mirror hazard, and it is the one worth reviewing hardest: a guard
that also let a ranked run past an empty pool would destroy contract layer
L6 (anti-lottery). Under L6 a frozen timed prompt makes a failed ranked run a
free retry, and every output-side gate degrades from "must be correct" into
"must be correct often enough", with a drafter-confidence threshold tunable by
submit-until-green.

• a ranked dispatch still runs the selection step and still fails
  closed on a pool that is empty, absent, null, or has an incomplete entry;
• the timed golden's pin check against the sampled entry's own
  sha256/bytes is unchanged;
• the row-count preflight is scoped, not relaxed — it stays unconditionally
  binding whenever the run times;
• new: Prepare hidden DFlash goldens independently refuses a ranked run
  whose selection outputs are empty. Honest scope: this is defence in depth
  plus a named diagnostic, not the only barrier —
  download-r2-object.sh already rejects an empty object key (exit 2), and an
  empty pin would mismatch a real digest. What it buys is that the job says
  "the selection step did not run" instead of dying on a generic
  argument-validation exit 2 inside a helper, ~25 minutes into a job, and that
  the refusal does not depend on either of those downstream behaviours staying
  as they are.

Tests

Tests/MLXFastTests/DFlashDryRunReachesItsGateTests.swift — 8 tests.

Six execute the real thing, not its text: the actual run: block scalars
of Select hidden DFlash timed target from the pool and
Prepare hidden DFlash goldens are extracted and run under /bin/bash against
sandboxed contracts and a stub R2 downloader that records every object key it
is asked for.

The two step-set tests evaluate the real if: expressions — parsing every
step of the job (including unnamed uses:-only steps; the parser was
cross-checked against a real YAML parse) and computing which steps run, with
GitHub's implicit-success() rule and steps.<id>.outcome resolved in order.
The evaluator hard-fails on any construct it does not recognise rather than
silently returning false: a test that quietly mis-parses a condition is worse
than no test.

test	asserts
aCleanDryRunReachesTheParityGateAndSkipsPoolSelection	dry-run step set includes the parity gate + the goldens step, excludes pool selection, and no dry-run step reads timed_prompt_pool
aRankedRunStillSelectsATimedTargetAndTimes	the mirror: ranked still samples and still times
theCorrectnessGoldenIsPinnedIndependentlyOfThePool	the fix's premise — the correctness key is a fixed pin, the timed key comes from the selection
aRankedSelectionStillFailsClosedOnAnUnusablePool	executed; 5 unusable-pool shapes, each must exit nonzero with ::error:: and emit no step outputs
aRankedSelectionSamplesFromAPopulatedPool	executed; samples from a real pool, never echoes the key to the job log, writes the private audit record, and returns more than one entry over 12 draws
aGatesOnlyGoldenPreparationSkipsTheTimedGolden	executed; exit 0, correctness golden downloaded and installed, timed golden never requested, no empty key
aRankedGoldenPreparationStillPinsTheTimedGolden	executed; ranked still downloads + pins; a golden that mismatches the sampled entry is still rejected
aRankedGoldenPreparationRefusesAnEmptySelection	executed; refuses by name

Attack evidence

Per the method requirement, each fix was attacked by reintroducing the defect.

Attack A — literal revert of .github/workflows/dflash-benchmark.yml to
origin/main (the exact pre-fix file). Fires 3 tests / 8 issues:

✘ aCleanDryRunReachesTheParityGateAndSkipsPoolSelection ... 2 issues
    !(<dry-run step set>.contains("Select hidden DFlash timed target from the pool") → true)
✘ aGatesOnlyGoldenPreparationSkipsTheTimedGolden ... 3 issues
    (result.exitStatus → 22) == 0
    !((result.requestedKeys → ["correctness_prompts/...json", ""]).contains("") → true)
    installedGolden → false
✘ aRankedGoldenPreparationRefusesAnEmptySelection ... 3 issues
✘ Test run with 8 tests in 1 suite failed with 8 issues.

Attack B — the mirror hazard: keep the if: but let a ranked run past an
empty pool (exit 1 → ::notice:: + exit 0):

✘ aRankedSelectionStillFailsClosedOnAnUnusablePool ... 6 issues
    (result.exitStatus → 0) != 0
✘ Test run with 8 tests in 1 suite failed with 6 issues.

Attack C — remove the independent empty-selection refusal:

✘ aRankedGoldenPreparationRefusesAnEmptySelection ... 3 issues
    !((result.requestedKeys → [..., ""]).contains("") → true)
✘ Test run with 8 tests in 1 suite failed with 3 issues.

Attack D — the tempting over-broad "fix": gate the whole
Prepare hidden DFlash goldens step ranked-only, so the dry run has no
correctness golden either:

✘ aCleanDryRunReachesTheParityGateAndSkipsPoolSelection ... 1 issue
✘ Test run with 8 tests in 1 suite failed with 1 issue.

Attack E — "fix" the red dry run by skipping the gate: gate
DFlash correctness and parity gate (untimed) on inputs.run_benchmark. This
makes a dry run green while deleting its entire purpose:

✘ aCleanDryRunReachesTheParityGateAndSkipsPoolSelection ... 1 issue
✘ Test run with 8 tests in 1 suite failed with 1 issue.

Attack F — drop the timed pin check while keeping the ranked download:

✘ aRankedGoldenPreparationStillPinsTheTimedGolden ... 2 issues
    (swapped.exitStatus → 0) != 0
✘ Test run with 8 tests in 1 suite failed with 2 issues.

Suite state

swift test: 543 tests / 25 suites pass (baseline on origin/main measured
in this workspace is 535/24; this adds 8 tests and 1 suite).

Pre-existing flakiness, unrelated to this change and reproduced with these
changes stashed on clean origin/main: the BenchmarkSupportTests
runtime-worker-protocol tests (... timed out waiting for protocol hello) fail
intermittently under machine load — measured 5 pass / 1 fail over 6 runs of
dflashReferenceRowsRequestCarriesTheVerifyBlockOnTheWire on unmodified
origin/main. This PR touches only workflow YAML and adds a test file that
spawns no runtime worker.

Notes

• Branched off origin/main (3e9f072), which already contains R2: sign the canonical request SigV4 actually specifies #818 and DFlash: stop a failed gates-only run uploading the raw sealed score #819 —
  both merged since the task was written. Touches no .github/scripts/*r2*.
• No CI workflow dispatched: dflash-benchmark.yml refuses to run from a
  branch ("Enforce trusted workflow context").
• One concern only; nothing bundled. Adjacent things noticed and deliberately
  not touched: the job-level MLXFAST_DFLASH_BENCH_GOLDEN_SHA256 /
  _BYTES pins are now dead (the timed golden is verified against the sampled
  pool entry instead) but are still required non-empty by
  Enforce DFlash track enablement.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.}