Found during the /review of PR #60 (performance pass), measured on the live install.
verify._find_original (backend/backup/verify.py) checks for a bare originals/<hash> and then falls back to glob.glob(glob.escape(bare) + ".*"). Almost every original HAS an extension, so the glob runs for nearly every row — and glob with a magic pattern scandirs the whole directory.
At 314 rows against a 315-entry originals/ that is roughly 99k dirents per verification. It sits inside the measured 0.25s today so it is not a problem now, but it is the one O(n²) in the backup path and it grows with the square of the corpus.
Fix: scandir originals/ once into a {stem: name} map before the row loop and look each hash up in it, instead of globbing per row. The bare-hash fallback (from api/upload.py's splitext(filename or ".pdf"), which guards a None filename and not a missing extension) falls out of the same map for free.
Related: #45, #51.
Found during the /review of PR #60 (performance pass), measured on the live install.
verify._find_original(backend/backup/verify.py) checks for a bareoriginals/<hash>and then falls back toglob.glob(glob.escape(bare) + ".*"). Almost every original HAS an extension, so the glob runs for nearly every row — andglobwith a magic pattern scandirs the whole directory.At 314 rows against a 315-entry
originals/that is roughly 99k dirents per verification. It sits inside the measured 0.25s today so it is not a problem now, but it is the one O(n²) in the backup path and it grows with the square of the corpus.Fix: scandir
originals/once into a{stem: name}map before the row loop and look each hash up in it, instead of globbing per row. The bare-hash fallback (fromapi/upload.py'ssplitext(filename or ".pdf"), which guards a None filename and not a missing extension) falls out of the same map for free.Related: #45, #51.