feat(bench): comparative S2 runset + H1-H21 re-evaluation + /bench page refresh

[blove]

Summary

Phase 4 of 4 for Tier 1 B2 comparative bench. First runset against real third-party grids — AG Grid Community v33, TanStack Table v8 + TanStack Virtual v3, and MUI X DataGrid Community v7 — replacing the gridalpha/gridbeta/gridgamma stubs.

• Comparative S2/hypothesis matrix (4 adapters × 12 scripts × 3 repeats, Chromium, ~4 min wall-clock).
• H1–H21 re-evaluated against real evidence. Threshold values in scripts/bench-matrix.mjs are unchanged — we ship the honest failing-H1 status rather than re-threshold.
• /bench page rewritten to cite real product names, measured deltas, and the new H1 status. LEGACY_ADAPTER_ID_MAP shim from Phase 1 removed; new milestone consumed directly.
• Two new milestone artifacts committed:
  • status/milestones/2026-05-08-b2-comparative-bench.hypotheses.json (full evaluator report)
  • status/milestones/2026-05-08-b2-scroll-summary.json (per-adapter scroll medians for the page)

Hypothesis status delta (vs 2026-05-07 bench-slab1 milestone)

H#	Before	After	Note
H1	satisfied	failing	Real MUI 11% faster than pretable on raw frame p95; comparators got real (was anchored on a 66.7ms/152px-clip stub baseline).
H5	satisfied	satisfied	Matrix still emits artifacts.
H6	satisfied	satisfied	Sort interaction (S2/pretable).
H7	satisfied	satisfied	Metadata filter (S2/pretable).
H8	satisfied	satisfied	Wrapped-text filter (S2/pretable).
H9	-	insufficient	S7 not in this matrix.
H10	-	insufficient	S7 not in this matrix.
H11	-	insufficient	S7 not in this matrix.
H12	-	insufficient	S7 not in this matrix.
H13	-	insufficient	S5/updates not in this matrix.
H14	-	insufficient	S5/updates not in this matrix.
H15	-	insufficient	S5/updates not in this matrix.
H16	satisfied	satisfied	Selection extend p95 = 10.2ms.
H17	satisfied	satisfied	Keyboard nav p95 = 10.1ms.
H18	satisfied	satisfied	Select-all latency = 8.7ms.
H19	satisfied	satisfied	Format overhead = 0.40ms.
H20	satisfied	satisfied	Cheap-render scroll p95 = 10.3ms.
H21	satisfied	satisfied	Heavy-render scroll p95 = 9.4ms.

H1 raw evidence (S2/hypothesis/Chromium, 3-repeat medians)

Adapter	Frame p95 (ms)	Row-height err (px)	Blank gaps	Anchor shift	Quality
MUI X DataGrid Community	8.7	1	0	0	passes
pretable	9.7	1	0	0	passes
AG Grid Community	16.7	2	1	0	fails (rhe>1, 1 blank)
TanStack Table v8	16.7	0	1	0	fails (1 blank)

Pretable is ~11% above MUI's frame p95 — outside the 10% parity threshold the H1 evaluator enforces. Pretable still passes every absolute quality sub-criterion (≤1px row-height error, 0 anchor shift, 0 blank gaps, 0 long tasks). The wedge moves from "raw frame speed" to "zero-artifact quality + headless-engine surface (sort/filter/selection/streaming primitives)."

Prose draft for /bench page

The page is rendered from the milestone JSONs; key prose blocks now read:

These numbers anchor every claim on the homepage. Generated by running the same dataset through apps/bench — pretable against three real third-party grids: AG Grid Community, TanStack Table v8 with TanStack Virtual, and MUI X DataGrid Community. We measure scroll frame timings, blank gaps under scroll, row-height fidelity for wrapped text, and anchor stability across rebuilds.

On this dataset, MUI X DataGrid Community renders frames a hair faster than pretable (8.7ms p95 vs 9.7ms) — both clear the single 60Hz frame budget with zero blank gaps and ≤ 1px row-height drift. AG Grid and TanStack land at roughly twice that frame p95 (~16.7ms) and both drop a blank gap during the scripted scroll; AG Grid additionally drifts 2px on row height, a sign that wrapped-cell layout doesn't round-trip through its line-height pipeline as cleanly as pretable's text-core does.

The honest read: pretable's wedge on this script isn't raw frame speed — it's the combination of zero blank gaps, zero anchor shift, and ≤ 1px row-height fidelity at full-grid feature weight. MUI matches pretable on quality but ships a fundamentally different feature surface (no headless engine, no streaming primitives, no theming-as-data). The H1 evaluator marks this run failing because the 10% parity threshold (pretable within 10% of the best full-grid comparator's frame p95) is not met on Chromium / hypothesis scale; pretable is roughly 11% above MUI. We're keeping the failing status visible rather than re-thresholding.

Verdict column on the table is computed from the data (no hardcoded literals).

Side fix: bench Playwright spec adapter-label assertion

Phase 1/2/3 added new aria-labels for each adapter ("AG Grid Community adapter", "TanStack Table adapter", "MUI X DataGrid adapter") but the comparator adapters render only the product name as visible text — getByText doesn't match aria-labels. The matrix run failed on the first AG Grid entry. Switched the spec to getByLabel(adapterLabel).first() (pretable mounts both an outer <section> and an inner <PretableSurface> with the same aria-label, so .first() is required for it). This is a test-side fix; no adapter implementations were touched.

Known gaps / What's NOT in this PR

• autosize script wiring. autosize is in BenchScriptName (packages/bench-runner/src/index.ts) and the AG Grid adapter has an onGridReady autosize branch, but the query-state parser (apps/bench/src/query-state.ts) does not accept it as a valid script query param, and validateSupportedP0aRequest's supportedScripts allowlist does not include it. No historical runset has autosize evidence; the AG Grid autosize branch is currently dead code. Phase 4's matrix run drops autosize from the script list as a result. End-to-end wiring (query-state + supportedScripts + pretable + mui handlers; tanstack returns unsupported per spec) is a separate follow-up PR.
• Comparative interaction evidence. sort / filter-text / filter-metadata are still gated to pretable-only. A future B-phase sub-project should expose comparative interaction latency for the new comparators.
• Comparative streaming evidence. All four bench adapters wire applyTransaction-style updates, but this matrix is S2-only; H13/H14/H15 stay insufficient until a S5/updates runset.
• Webkit/Firefox. Chromium-only.
• /bench page layout redesign. Prose was rewritten in place; the visual layout is unchanged.
• Threshold value changes. Only prose and rationale wording changed; numeric thresholds in scripts/bench-matrix.mjs are untouched.

Test plan

• pnpm -w typecheck passes
• pnpm -w test passes (190 tests, 62 files)
• pnpm -w lint 0 errors
• pnpm format clean
• All four real adapters run end-to-end at S2/hypothesis/Chromium for initial + scroll (Phase 4.1 pre-flight + the runset itself)
• Runset captured at status/milestones/2026-05-08-b2-comparative-bench.hypotheses.json
• apps/website builds with the rewritten /bench page

Do not auto-merge — block on user review of the prose draft above.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
pretable	Ready	Preview, Comment	May 9, 2026 1:09am

[github-actions]

Vercel preview ready

Preview: https://pretable-735nfheeg-cacheplane.vercel.app
Commit: 235f019235b2a64b4f8505176c2c15f0df39acfb

_{Updated automatically by the deploy-preview job.}