perf(worker): keyset-paginate quota-wall-nudge + custom-domain reconciler scans - #83
Merged
Merged
Conversation
…er scans QuotaWallNudgeWorker.Work (`SELECT id, plan_tier FROM teams WHERE plan_tier NOT IN (...)`) and CustomDomainReconciler.listActiveDomains (`SELECT ... FROM custom_domains WHERE status NOT IN (live, failed)`) each issued ONE unbounded SELECT per tick, materialising the entire eligible set into one allocation before running the per-row work (the wall-nudge dedupe + aggregate evaluations; the domain TXT lookup / HTTP probe). Both now stream rows in keyset-paginated batches, advancing the cursor by the last id::text and stopping on a short page — exactly mirroring orphan_sweep_reconciler's fetchLiveStackIDs. Each scan still evaluates the WHOLE eligible set every tick; only the per-fetch footprint is bounded so the nested per-row queries / network calls don't run while one giant result set is held open. Keyset (id::text > $cursor ORDER BY id::text ASC) rides the PK, is restart-safe, and never re-scans or drifts under the concurrent status flips / tier changes each reconciler performs. Batch sizes: quota_wall_nudge=1000 (cheap id+tier projection); custom_domain =100 (small — each row triggers an outbound DNS TXT lookup or HTTP HEAD probe, so a tight batch keeps each tick's burst of network calls bounded). All side effects preserved: wall-nudge keeps the 24h dedupe + three-axis evaluate + single-row-per-team insert; custom_domain keeps the full reconcile loop (the ORDER BY moves created_at → id::text, which the caller explicitly does not depend on — it was "purely for log readability"). Tests: quota_wall_nudge_keyset_test.go + custom_domain_reconcile_keyset_test.go add multi-page-advance, second-page-error, and mid-stream rows.Err() coverage, mirroring the orphan_sweep keyset tests. Existing custom_domain coverage expectations updated to the new WithArgs(statusLive, statusFailed, cursor, limit) shape; wall-nudge expectations were arg-free and unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mastermanas805
enabled auto-merge (squash)
June 3, 2026 19:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two more periodic reconciler scans converted to keyset pagination, mirroring
the already-merged
fetchLiveStackIDsloop inorphan_sweep_reconciler.go:QuotaWallNudgeWorker.Work(quota_wall_nudge.go) —SELECT id, plan_tier FROM teams WHERE plan_tier NOT IN (team, anonymous, free)CustomDomainReconciler.listActiveDomains(custom_domain_reconcile.go) —SELECT ... FROM custom_domains WHERE status NOT IN (live, failed)Each scan still evaluates the WHOLE eligible set every tick — not a bare
LIMITthat drops rows.Coverage block
Notes
=100 (small — each row triggers an outbound DNS TXT lookup / HTTP HEAD probe,
so a tight batch bounds each tick's network fan-out).
concurrent status flips / tier changes each reconciler performs.
created_at→id::text; the caller explicitlydoes not depend on order ("purely for log readability").
one-row-per-team insert; custom_domain: full reconcile loop).
make gategreen locally.🤖 Generated with Claude Code