feat(strategy): add nystrom landmark-column transform for low-rank data#194
Merged
Merged
Conversation
Contributor
|
Gate chain passed. This PR is queued for the next batched GPU evaluation window. |
zeokin
approved these changes
Jul 13, 2026
zeokin
left a comment
Owner
There was a problem hiding this comment.
Approved — eval:L (first tiered frontier result on the low-rank track).
GPU re-measurement (RTX PRO 500 Blackwell, fp32, lowrank rank-16, n=8192, same seed 777):
- nystrom @ M=64: accuracy 0.99999, latency 0.230s, VRAM 260 MiB, 42x fewer FLOPs, score 17.11 (dominates exact on all axes).
- 3-way rsvd (current frontier, #156) @ M=64: score 12.85; @ M=48: score 13.51.
- nystrom beats the frontier by +33% at equal M=64, +27% at each method's natural M -> both clear the >=25% eval:L bar. Stable across 3 seeds (17.07-17.11).
Why it wins: nystrom's landmark-column basis (~NM^2, column gather + QR) is ~100x cheaper in FLOPs than rsvd's Gaussian sketch (~N^2M), so it's faster even using more columns. Merges cleanly (additive; keeps main's 3-way rsvd). Label: eval:L.
zeokin
added a commit
that referenced
this pull request
Jul 13, 2026
Second real frontier point and first tiered result: nystrom landmark-column transform beats the low-rank BASELINE (#156, 3-way rsvd) by ~30% (33% at equal M=64, 27% at natural M) — accuracy 0.99997, 0.23s vs 0.44s exact, 260 MiB, 42x fewer FLOPs. Regenerate dashboard/results.json from the ledger (build_dashboard_data): low-rank frontier now nystrom @ 17.11 (BASELINE #156 -> L #194).
zeokin
added a commit
that referenced
this pull request
Jul 13, 2026
The scorer gated every track at --min-accuracy 0.8, but the whitepaper (6.2) and the dashboard advertise per-track floors: full-rank 0.80, low-rank 0.95, decaying-spectrum 0.90. Low-rank is the easy case, so it must clear a higher bar to count. EvalConfig.accuracy_floor now defaults to None and resolves to the fill's track floor in __post_init__ (TRACK_FLOORS / default_floor); an explicit --min-accuracy still overrides. No verdict already recorded changes (#156/#194 clear 0.95). Regression test added.
zeokin
added a commit
that referenced
this pull request
Jul 13, 2026
Step 1 of the automated GPU-test workflow. TRACKS pins (fill, rank, M, floor) per track so a tier is reproducible and a submission can't cherry-pick the rank/M that flatters its method (the M choice flipped #194 from none to L). Low-rank pins M=64=4r for rank-16 -- the M where both 3-way and 4-way bases fully recover, so it's inclusive, not rigged to one basis. evaluator's per-track accuracy floor now resolves through tracks.track_floor, so the scorer and the registry can never disagree.
zeokin
added a commit
that referenced
this pull request
Jul 13, 2026
…string The transform-authorship check wrongly flagged an UPDATE to an existing transform as unverified: modifying RandomizedSVDTransform.basis() (the #156 case) never touches the literal "rsvd", so the name-in-diff check missed it, while a NEW transform (#194) passed only because it adds `name = "nystrom"` + register_transform. Verify by the transform's CLASS instead: map the declared name -> its class via its `name = "..."` attribute (ast), then check whether the PR's diff hunks land inside that class's line range. Now a NEW transform (class added -> hunk overlaps the added range) and an UPDATE (hunk lands inside the existing class) validate the same way. Pure + unit-tested for both.
RenzoMXD
added a commit
to RenzoMXD/Cuda-Compute-OSS
that referenced
this pull request
Jul 15, 2026
…ecaying-spectrum data
Replaces this branch's original content (nystrom 3-way split): repeat
GPU testing showed changing 4-way to 3-way landmark sampling does not
reliably improve latency -- the earlier single-run 19% difference did
not reproduce. The redundancy proof (col(B) is unnecessary) is still
mathematically correct, but with no measurable benefit there's no
reason to change the merged 4-way nystrom.
Per docs/developing-new-transforms.md's track guidance: low-rank is
accuracy-saturated (~1.0 for every method already), so the only lever
left there is latency -- exactly the fragile, noise-prone claim that
just failed to hold up. decaying-spectrum is flagged as the track with
real headroom and no admitted baseline yet.
New transform combining three ideas from this project's own history:
- the 3-way space split {col(A), row(A), row(B)} (zeokin#156)
- nystrom's landmark-column seed (zeokin#194): a memory gather, not a GEMM
against a random Gaussian projection
- power-iteration refinement (prior art: PR zeokin#185, unmerged): concentrates
a below-rank budget on the components that carry the product's energy
on a decaying-but-not-low-rank spectrum
zeokin#185 seeded its iterations with random Gaussian noise; its own GPU
measurement showed a real accuracy gain over rsvd on decaying-spectrum
data, but the extra streamed GEMMs were not amortized at n=8192, so it
never beat exact on latency. A landmark column is already correlated
with a matrix's dominant subspace on a decaying spectrum, unlike a
fully random projection, so seeding from landmarks instead should need
fewer refinement passes to reach a given accuracy.
At power_iters=0 this degenerates exactly to plain 3-way nystrom (same
basis_flops), so the CPU-safe suite verifies the iteration genuinely
earns its cost: test_nystrom_iter_beats_plain_nystrom_on_decaying_spectrum
confirms one pass measurably improves accuracy over the landmark seed
alone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR kind
Summary
New registered transform
nystrom. Splits theMbudget across col(A), row(A), col(B), and row(B) — the same four spaces rsvd sketches — but gathers random landmark columns/rows instead of random projections, then thin-QR intoQ.Basis FLOPs are
~2 N M²(QR only), not rsvd's~2 N² Msketches.Honest trade-off (GPU-measured): on
--fill lowrankaccuracy is 1.0000, VRAM and FLOPs beat exact, but latency is slightly above exact so the dominance gate scores 0 (not an improvement under the one rule). Versus rsvd at thesame accuracy/VRAM it is much faster and fewer FLOPs. Does not touch rsvd.
Fixes #193
Validation
python -m strategy.smoke— nystrom and rsvd both PASSpython -m pytest strategy/tests/test_nystrom.py tests/ strategy/tests/ eval/tests/ -q— 240 passed, 1 failed (
tests/test_correctness.py::test_fp16_incore_tiled_parity,pre-existing fp16 tolerance flake; unrelated to
strategy/)python -m eval …on NVIDIA GeForce RTX 5090 (see scorecard)GPU Result (required for feat PRs only)
Regime measured: N=12000, dtype=fp32, fill=lowrank, data-rank=16, rank M=1500, device=RTX 5090, seed=0, pairs=3
Comparison (same couples)
Raw scorecard (paste
python -m eval …output or--json)Checklist
strategy.smokeif relevant, pluspytest tests/ strategy/tests/ eval/tests/).Co-authored-byfooters for coding agents such as Cursor, Codex, Claude, Copilot, or similar tools.state honestly which axis it trades — see the one rule in CONTRIBUTING.md.
(Honest: accuracy held; VRAM + FLOPs beat exact; latency is slightly above exact → score 0.)
python eval/tests/test_eval.py,python strategy/tests/test_subspace.py,python tests/test_correctness.py.