Skip to content

fix(strategy): nystrom uses 3 landmark spaces, not 4 (drop redundant col(B))#270

Open
galuis116 wants to merge 1 commit into
zeokin:mainfrom
galuis116:fix/nystrom-drop-redundant-colb
Open

fix(strategy): nystrom uses 3 landmark spaces, not 4 (drop redundant col(B))#270
galuis116 wants to merge 1 commit into
zeokin:mainfrom
galuis116:fix/nystrom-drop-redundant-colb

Conversation

@galuis116

Copy link
Copy Markdown
Contributor

Summary

Fixes #269.

NystromTransform split its M-column landmark budget across four spaces —
col(A), row(A), col(B), row(B) — but col(B) is redundant. With the
projector P = Q Qᵀ, Ĉ = P A P B P = A B once range(Q) contains only
col(A), row(A), row(B):

P A = A  (col(A) ⊆ range(Q)),  A P = A  (row(A)),  B P = B  (row(B))
=>  P A P B P = A B P = A B

col(B) contributes nothing — the exact argument rsvd already uses (3 sketches,
#91 "rsvd wastes ~25% of the rank budget"). nystrom never got it, so a full
M/4 of the budget was wasted, and exact recovery of a rank-r product needed
M ≳ 4r instead of 3r — ~25% worse accuracy per M on the low-rank /
decaying-spectrum regimes it targets.

The fix splits M across the three necessary spaces (drops the col(B) block).
Same basis cost (gather M columns + thin QR — latency/VRAM unchanged),
strictly higher accuracy per M.

Test plan

  • python strategy/tests/test_nystrom_three_spaces.py3/3 (CPU): recovers a
    rank-4 A@B at M=15 (≈5 landmarks/space); basis orthonormal & correctly shaped.
    This test fails on the current 4-space nystrom (M/4 = 3 < r = 4 per space →
    cannot recover) and passes on the fix — a direct demonstration of the wasted budget.
  • ruff check strategy/transforms.py strategy/tests/test_nystrom_three_spaces.py — clean.

Notes

No scorecard: this is a correctness/efficiency fix: to a built-in transform (the
budget split), not a new strategy — fix: PRs are scorecard-exempt in the gate chain.
The change only reallocates existing landmark budget, so latency/VRAM are unchanged
while accuracy-per-M strictly improves.

…col(B))

NystromTransform split its M-column budget across col(A), row(A), col(B),
row(B) -- but col(B) is redundant. With P = Q Qᵀ, Ĉ = P A P B P = A B
once range(Q) contains col(A), row(A), row(B) (P A = A, A P = A, B P = B);
col(B) contributes nothing. This is the same argument rsvd already uses
(3 sketches, zeokin#91). So a full M/4 of the landmark budget was wasted and
exact recovery of a rank-r product needed M >= 4r instead of 3r -- ~25%
worse accuracy per M on the low-rank/decaying-spectrum regimes nystrom
targets.

Split M across the three necessary spaces (drop the col(B) block). Same
basis cost (gather M columns + thin QR), strictly higher accuracy per M.
Adds a CPU test that recovers a rank-4 product at M=15 (5 per space) --
which the old 4-space split (M/4=3 < 4 per space) could not. Distinct
from zeokin#226 (a new rrqr-nystrom transform; this fixes the existing one).

Fixes zeokin#269
@github-actions github-actions Bot added area:strategy Smart strategies / transforms (strategy/) area:tests Correctness gates and test suites (tests/) status:needs-review Awaiting maintainer review type:bug Something is incorrect or broken status:ready-non-gpu Fix/docs PR cleared non-GPU triage; maintainer review only labels Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:strategy Smart strategies / transforms (strategy/) area:tests Correctness gates and test suites (tests/) status:needs-review Awaiting maintainer review status:ready-non-gpu Fix/docs PR cleared non-GPU triage; maintainer review only type:bug Something is incorrect or broken

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] nystrom wastes M/4 of its budget on the redundant col(B) landmark block (3 spaces suffice, as rsvd/#91)

1 participant