Skip to content

[bug] nystrom wastes M/4 of its budget on the redundant col(B) landmark block (3 spaces suffice, as rsvd/#91) #269

Description

@galuis116

Summary

NystromTransform splits its M-column landmark budget across four spaces —
col(A), row(A), col(B), row(B) (strategy/transforms.py#L143-L165) — but col(B) is redundant. The reconstruction is Ĉ = P A P B P with P = Q Qᵀ, which equals A @ B once range(Q) contains only col(A), row(A), row(B):

P A = A   needs col(A) ⊆ range(Q)
A P = A   needs row(A) ⊆ range(Q)
B P = B   needs row(B) ⊆ range(Q)   =>  P A P B P = A B P = A B

This is the exact argument the rsvd transform already uses (it sketches 3 spaces, per its docstring and #91 "rsvd wastes ~25% of the rank budget"). nystrom never got the same fix and still spends a full landmark block on col(B), which contributes nothing to P A P B P.

Impact

With 4 spaces each block gets ~M/4 columns; with 3 it gets ~M/3. So a full M/4 of the landmark budget is wasted, and exact recovery of a rank-r product needs M ≳ 4r instead of 3r — i.e. ~25% lower accuracy-per-M on the low-rank / decaying-spectrum regimes nystrom targets.

Concrete failing scenario (CPU, verified)

Rank-4 couple, n=64, M=15: 3 spaces give ~5 landmarks each (spans rank 4) and recover A@B exactly (rel_err < 1e-6). The current 4-space split gives M/4 = 3 < r = 4 per space and cannot recover — rel_frobenius(A@B, P A P B P) stays large. (A regression test that recovers at M=15 fails on the current 4-space nystrom and passes on the 3-space fix.)

Fix

Split M across the three necessary spaces col(A), row(A), row(B) (drop the col(B) block), mirroring rsvd. Same basis cost (gather M columns + thin QR), strictly higher accuracy per M. Distinct from #226, which adds a new rrqr-nystrom transform and does not touch the existing NystromTransform.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:strategySmart strategies / transforms (strategy/)status:triageNewly opened, not yet triagedtype:bugSomething is incorrect or brokentype:strategyNew smart strategy / transform proposal

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions