feat(strategy): rsvd captures 3 subspaces, not 4 (drop redundant col(B) sketch)#156
Conversation
|
Gate chain passed. This PR is queued for the next batched GPU evaluation window. |
|
This PR currently has merge conflicts with the base branch. Please resolve them within 12 hours or the bot will close the PR automatically. |
The subspace reconstruction is Ĉ = P A P B P with P = QQᵀ, which equals A@B
once range(Q) contains col(A), row(A), row(B):
P A = A needs col(A); A P = A needs row(A); B P = B needs row(B)
=> P A P B P = A B P = A B
col(B) is never needed. RandomizedSVDTransform split the M budget four ways
(col A, row A, col B, row B), so a quarter of every rank went to a subspace that
does not affect the result -- exact recovery of a rank-r product needed M ~ 4r.
Splitting three ways over {col(A), row(A), row(B)} recovers at M ~ 3r, a 25%
smaller subspace for the same accuracy (and better accuracy at any fixed
M < 4r). Total sketch width is still m, so basis_flops is unchanged.
Measured improvement over exact on compressible data at M = 3r (GTX 1650, fp32,
N=8192, lowrank rank-16, seed 0): accuracy 1.0000, 748ms vs 1774ms exact, 259
vs 1288 MiB, 42.4x fewer FLOPs. The old 4-way split at the same M=48 gates at
accuracy 0.35 (score 0); the 3-way split scores an improvement.
Fixes zeokin#91
7c7fe70 to
25bd375
Compare
|
Gate chain passed. This PR is queued for the next batched GPU evaluation window. |
zeokin
left a comment
There was a problem hiding this comment.
Approved — feat/strategy verdict after GPU re-measurement (RTX PRO 500 Blackwell, fp32).
Math verified independently (numpy) and confirmed by this run: 3-way rsvd {col(A), row(A), row(B)} recovers the low-rank product exactly (accuracy 0.99997), so col(B) is provably redundant.
GPU reproduction (lowrank rank-16, M=48, n=8192):
- seeded (seed 0): accuracy 0.99997, latency 0.295s, peak VRAM 259 MiB, 42.4x fewer FLOPs — dominates exact (0.44s / 1032 MiB) on all three cost axes.
- fresh unseen (2 runs, no --seed): accuracy 0.99996 / 0.99998 — stable; improvement:true both.
- full-rank safety: gated to ~0 (no regression on the reference regime).
Rebased onto current main during review: the 3-way change was combined with #211's vram_fraction threading (the sketch block was the only conflict); #211's frac test updated 4->3 to match; 262 tests green; Eval policy CI green on the rebased head.
Label: eval:BASELINE — first admitted strategy on the low-rank track. Scope: strategy/transforms.py (+ the merged test adjustment). Thanks for the clean derivation.
Change the pinned reference regime from N=12000 to N=8192 across CLI/config defaults, docs, and the GPU-batch workflow, so it matches the actual scoring hardware and the first admitted low-rank entry (#156). Reference device stays RTX 5090. Derived defaults updated to match: M = N//8 is now 1024 (was 1500), and the example VRAM figure ~0.8 GB (was ~1.7 GB). No semantics change beyond the default N; accuracy/dominance gates unchanged. 262 tests pass.
First real (non-mock) ledger entry. rsvd 3-way subspace on the low-rank track (rank-16, n=8192, M=48): accuracy 0.99997, dominates exact on latency/VRAM/FLOPs (0.30s vs 0.44s, 259 vs 1032 MiB, 42x fewer FLOPs). Measured on RTX PRO 500 Blackwell; reference device is RTX 5090. verdict BASELINE.
Second real frontier point and first tiered result: nystrom landmark-column transform beats the low-rank BASELINE (#156, 3-way rsvd) by ~30% (33% at equal M=64, 27% at natural M) — accuracy 0.99997, 0.23s vs 0.44s exact, 260 MiB, 42x fewer FLOPs. Regenerate dashboard/results.json from the ledger (build_dashboard_data): low-rank frontier now nystrom @ 17.11 (BASELINE #156 -> L #194).
The scorer gated every track at --min-accuracy 0.8, but the whitepaper (6.2) and the dashboard advertise per-track floors: full-rank 0.80, low-rank 0.95, decaying-spectrum 0.90. Low-rank is the easy case, so it must clear a higher bar to count. EvalConfig.accuracy_floor now defaults to None and resolves to the fill's track floor in __post_init__ (TRACK_FLOORS / default_floor); an explicit --min-accuracy still overrides. No verdict already recorded changes (#156/#194 clear 0.95). Regression test added.
…string The transform-authorship check wrongly flagged an UPDATE to an existing transform as unverified: modifying RandomizedSVDTransform.basis() (the #156 case) never touches the literal "rsvd", so the name-in-diff check missed it, while a NEW transform (#194) passed only because it adds `name = "nystrom"` + register_transform. Verify by the transform's CLASS instead: map the declared name -> its class via its `name = "..."` attribute (ast), then check whether the PR's diff hunks land inside that class's line range. Now a NEW transform (class added -> hunk overlaps the added range) and an UPDATE (hunk lands inside the existing class) validate the same way. Pure + unit-tested for both.
…ecaying-spectrum data
Replaces this branch's original content (nystrom 3-way split): repeat
GPU testing showed changing 4-way to 3-way landmark sampling does not
reliably improve latency -- the earlier single-run 19% difference did
not reproduce. The redundancy proof (col(B) is unnecessary) is still
mathematically correct, but with no measurable benefit there's no
reason to change the merged 4-way nystrom.
Per docs/developing-new-transforms.md's track guidance: low-rank is
accuracy-saturated (~1.0 for every method already), so the only lever
left there is latency -- exactly the fragile, noise-prone claim that
just failed to hold up. decaying-spectrum is flagged as the track with
real headroom and no admitted baseline yet.
New transform combining three ideas from this project's own history:
- the 3-way space split {col(A), row(A), row(B)} (zeokin#156)
- nystrom's landmark-column seed (zeokin#194): a memory gather, not a GEMM
against a random Gaussian projection
- power-iteration refinement (prior art: PR zeokin#185, unmerged): concentrates
a below-rank budget on the components that carry the product's energy
on a decaying-but-not-low-rank spectrum
zeokin#185 seeded its iterations with random Gaussian noise; its own GPU
measurement showed a real accuracy gain over rsvd on decaying-spectrum
data, but the extra streamed GEMMs were not amortized at n=8192, so it
never beat exact on latency. A landmark column is already correlated
with a matrix's dominant subspace on a decaying spectrum, unlike a
fully random projection, so seeding from landmarks instead should need
fewer refinement passes to reach a given accuracy.
At power_iters=0 this degenerates exactly to plain 3-way nystrom (same
basis_flops), so the CPU-safe suite verifies the iteration genuinely
earns its cost: test_nystrom_iter_beats_plain_nystrom_on_decaying_spectrum
confirms one pass measurably improves accuracy over the landmark seed
alone.
PR kind
Summary
The subspace reconstruction is
Ĉ = P·A·P·B·PwithP = QQᵀ, which equalsA@Bonce range(Q) contains col(A), row(A), row(B):col(B) is never needed.
RandomizedSVDTransformsplit the M-column budget four ways (col A, row A, col B, row B), so a quarter of every rank went to a subspace that does not affect the result — exact recovery of a rank-rproduct neededM ≈ 4r. Splitting three ways over {col(A), row(A), row(B)} recovers atM ≈ 3r: a 25% smaller subspace for the same accuracy (and strictly better accuracy at any fixedM < 4r). Total sketch width is stillm, sobasis_flops(2n²m + 2nm²) is unchanged.Fixes #91.
GPU Result (compressible regime — lowrank rank-16, M=3r=48, GTX 1650, fp32)
The reference regime is full-rank, where no subspace beats exact (accuracy ≈ 0) — that is unchanged and honestly still scores 0. The improvement is on compressible data, where the subspace method is designed to win; there this change turns a gated failure into a dominant improvement:
Raw scorecard (
python -m eval --n 8192 --pairs 3 --fill lowrank --data-rank 16 --rank-m 48 --transforms rsvd --seed 0):The before/after is fully reproducible at the same seed: old 4-way rsvd at M=48 gates at accuracy 0.3491 → score 0; 3-way scores an improvement. Measured on a GTX 1650 (named per CONTRIBUTING); reproduce on the RTX 5090 reference at N=12000.
CPU-safe validation
test_exact_when_m_equals_nand the rsvd low-rank recovery/monotonicity tests still pass. Device-independent proof (numpy): dropping col(B) keeps exact reconstruction (rel-error 1e-15); dropping col(A) or row(B) breaks it (~0.9).Scope
strategy/transforms.pyonly (Open — "the main event" zone); no API change,basis_flopsunchanged, no protected-path edits.