Skip to content

perf(worker): keyset-paginate quota-wall-nudge + custom-domain reconciler scans - #83

Merged
mastermanas805 merged 3 commits into
masterfrom
perf/keyset-nudge-domain-scans
Jun 3, 2026
Merged

perf(worker): keyset-paginate quota-wall-nudge + custom-domain reconciler scans#83
mastermanas805 merged 3 commits into
masterfrom
perf/keyset-nudge-domain-scans

Conversation

@mastermanas805

Copy link
Copy Markdown
Member

What

Two more periodic reconciler scans converted to keyset pagination, mirroring
the already-merged fetchLiveStackIDs loop in orphan_sweep_reconciler.go:

  • QuotaWallNudgeWorker.Work (quota_wall_nudge.go) — SELECT id, plan_tier FROM teams WHERE plan_tier NOT IN (team, anonymous, free)
  • CustomDomainReconciler.listActiveDomains (custom_domain_reconcile.go) —
    SELECT ... FROM custom_domains WHERE status NOT IN (live, failed)

Each scan still evaluates the WHOLE eligible set every tick — not a bare
LIMIT that drops rows.

Coverage block

Symptom:        unbounded full-table SELECT in 2 reconciler scans
Enumeration:    rg 'FROM teams|FROM custom_domains' internal/jobs/quota_wall_nudge.go internal/jobs/custom_domain_reconcile.go
Sites found:    2  (QuotaWallNudgeWorker.Work, listActiveDomains)
Sites touched:  2
Coverage test:  TestQuotaWallNudge_KeysetPagination/_SecondPageError/
                _KeysetRowsErr + TestCustomDomain_ListActiveDomains_*
Live verified:  worker auto-deploys on merge to master; verify via kubectl
                image SHA + /healthz commit_id (rule 14) post-merge

Notes

  • Batch sizes: quota_wall_nudge=1000 (cheap id+tier projection); custom_domain
    =100 (small — each row triggers an outbound DNS TXT lookup / HTTP HEAD probe,
    so a tight batch bounds each tick's network fan-out).
  • Keyset over OFFSET: rides the PK, restart-safe, no re-scan/drift under the
    concurrent status flips / tier changes each reconciler performs.
  • custom_domain ORDER BY moves created_atid::text; the caller explicitly
    does not depend on order ("purely for log readability").
  • All side effects preserved (wall-nudge: 24h dedupe + 3-axis evaluate +
    one-row-per-team insert; custom_domain: full reconcile loop).
  • make gate green locally.

🤖 Generated with Claude Code

…er scans

QuotaWallNudgeWorker.Work (`SELECT id, plan_tier FROM teams WHERE plan_tier
NOT IN (...)`) and CustomDomainReconciler.listActiveDomains (`SELECT ... FROM
custom_domains WHERE status NOT IN (live, failed)`) each issued ONE unbounded
SELECT per tick, materialising the entire eligible set into one allocation
before running the per-row work (the wall-nudge dedupe + aggregate
evaluations; the domain TXT lookup / HTTP probe).

Both now stream rows in keyset-paginated batches, advancing the cursor by the
last id::text and stopping on a short page — exactly mirroring
orphan_sweep_reconciler's fetchLiveStackIDs. Each scan still evaluates the
WHOLE eligible set every tick; only the per-fetch footprint is bounded so the
nested per-row queries / network calls don't run while one giant result set is
held open. Keyset (id::text > $cursor ORDER BY id::text ASC) rides the PK, is
restart-safe, and never re-scans or drifts under the concurrent status flips /
tier changes each reconciler performs.

Batch sizes: quota_wall_nudge=1000 (cheap id+tier projection); custom_domain
=100 (small — each row triggers an outbound DNS TXT lookup or HTTP HEAD probe,
so a tight batch keeps each tick's burst of network calls bounded). All side
effects preserved: wall-nudge keeps the 24h dedupe + three-axis evaluate +
single-row-per-team insert; custom_domain keeps the full reconcile loop (the
ORDER BY moves created_at → id::text, which the caller explicitly does not
depend on — it was "purely for log readability").

Tests: quota_wall_nudge_keyset_test.go + custom_domain_reconcile_keyset_test.go
add multi-page-advance, second-page-error, and mid-stream rows.Err() coverage,
mirroring the orphan_sweep keyset tests. Existing custom_domain coverage
expectations updated to the new WithArgs(statusLive, statusFailed, cursor,
limit) shape; wall-nudge expectations were arg-free and unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mastermanas805
mastermanas805 enabled auto-merge (squash) June 3, 2026 19:40
@mastermanas805
mastermanas805 merged commit 94b32c9 into master Jun 3, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant