Skip to content

test(fixtures): replace out-of-focus annotated_frames with sharper images + re-expand set (post-#170) #230

Description

@kalwalt

Summary

Replace the current out-of-focus annotated_frames fixtures with sharper captures, and re-expand the fixture set now that the #170 determinism blocker is resolved.

Background

crates/core/tests/fixtures/annotated_frames/ feeds the absolute corner-error gate (tests/absolute_corner_error.rs, #166 Track A): each fixture is a JPEG of the printed pinball marker plus a .corners.json with the four corners in TL/TR/BR/BL order. The test reprojects each backend's homography corners and asserts max_err <= baseline + 0.5 px.

Two problems today:

  1. The images are poor. The fixtures' own README says it plainly: "The current pinball-seq{1..4}.jpg fixtures show what not to do (out of focus → matcher returns matched_id = -1)." An out-of-focus gate image is a weak signal — it exercises the matcher's failure mode more than its accuracy.
  2. The set is down to 2 frames (pinball-seq1, pinball-seq4). seq2/seq3 were dropped for run-to-run Rust non-determinism, and pinball-demo for cross-platform C++ divergence.

Why now: the blocker is lifted

The README says the dropped fixtures "will be re-added once #170 lands cross-platform determinism in both backends." That condition now appears met:

So we should be able to hold more frames stable across platforms than we could in May.

Scope

  • Capture new, sharp frames of the printed pinball marker per the README recipe: tap-to-focus, marker filling 40–80% of frame, no motion blur, good contrast.
  • Vet candidates quickly with simple_nft_dual (renders both backends' outlines) before annotating.
  • Annotate with tools/annotate_corners/index.html (TL → TR → BR → BL).
  • Replace pinball-seq1/seq4 and re-add the dropped frames (or fresh equivalents) — target ~4–5 stable fixtures.
  • Regenerate the baseline:
    WEBARKIT_REGEN_CORNER_BASELINE=1 cargo test --test absolute_corner_error --features dual-mode -- --nocapture
  • Verify the gate passes in normal mode on Linux CI (the gate is Ubuntu-only).
  • Consider tightening REGRESSION_EPSILON_PX (currently widened to ~2.0 px to absorb the old drift) — with determinism fixed and sharper images, a tighter epsilon means a stronger gate.
  • Fix the README's internal inconsistency (it calls seq1 "rock-solid (sub-pixel)" while the capture section lumps seq1..4 as out-of-focus), and refresh the "why only two frames" section.

Notes

  • No code change needed to add fixtures — the test globs *.corners.json.
  • Commit .jpg + .corners.json + baseline.json together so the baseline stays in sync.
  • Report matched_id + per-backend max-err in the PR body so reviewers can see what's being gated.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Backlog

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions