Skip to content

perf: SyncJob transfer matching does full-table scan on every family sync #2447

Description

@JSONbored

Sentry: SURE-APP-WX (18,894 events, last seen today)

Family::Syncer#perform_post_sync calls auto_match_transfers! with no date filter, causing a full cartesian scan across all entries in a family's accounts after every sync.

For families with 10,000+ entries this is an O(N²) self-join:

SELECT ... FROM entries inflow_candidates
JOIN entries outflow_candidates ON (
  inflow_candidates.date BETWEEN outflow_candidates.date - 4 AND outflow_candidates.date + 4
  ...
)
JOIN accounts inflow_accounts ON ...
WHERE inflow_accounts.family_id = $1 AND outflow_accounts.family_id = $2
-- no date bound — scans every unmatched entry ever

Most entries added during an ongoing sync are recent (last few days). Scanning all historical entries is unnecessary.

Fix: add a since_date: parameter to transfer_match_candidates and auto_match_transfers!, and pass since_date: 90.days.ago.to_date from Family::Syncer#perform_post_sync. Account-level syncs keep since_date: nil for full-history matching on a specific account.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions