chore(infra): cut the Cloudflare Worker's D1 binding over to a fresh loopover database - #9502
Conversation
…loopover database The previous database (b2c79dd6-7771-4b1f-aa6b-085d5f3e9528, internally still named "gittensory" from before the 2026-07-14 rename) had grown to 8.96 GB of its 10 GB cap. D1 never runs VACUUM (cloudflare/workerd#1618), so the batch of now-pruned contributor-decision-pack rows (#9459 -- 18,336 rows / ~6.3 GB deleted) freed pages that get reused, not returned: the file was never going to shrink back down on its own. Cuts DB over to a freshly created, correctly-named "loopover" database (da7537cc-ab54-4dc1-8c38-2713f03f1130), populated from a verified export of the pruned source database: - Every table's row count matches the source exactly, except ordinary live-write drift on actively-written tables (webhook/audit/usage logs, which never stop changing between an export and a later comparison) -- confirmed via a full 103-table sweep. - d1_migrations matches exactly (202/202 rows, same max id). - signal_snapshots' contributor-decision-pack (the table #9459 fixed) is 213/213, exactly one row per contributor, confirming the dedup carried over correctly. - 206 rows whose payload_json exceeded D1's ~100KB single-statement text limit (SQLITE_TOOBIG) were re-imported via a chunked INSERT + payload_json ||= UPDATE sequence, verified byte-identical against the source before import. No other file in the repo references the old database_id. cf-typegen is unaffected (binding name/shape unchanged, only the id). Refs #9435
|
Warning ⏸️ LoopOver review result - manual review recommendedReview updated: 2026-07-27 22:15:07 UTC
Review summary Nits — 4 non-blocking
Concerns raised — review before merging
📋 Copy for AI agents — paste into your coding agentDecision drivers
Context & advisory signals — never blocks the verdict
Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. Decision record
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9502 +/- ##
==========================================
- Coverage 89.55% 88.65% -0.91%
==========================================
Files 843 843
Lines 110073 110073
Branches 26194 26194
==========================================
- Hits 98573 97582 -991
- Misses 10238 11520 +1282
+ Partials 1262 971 -291
Flags with carried forward coverage won't be shown. Click here to find out more. |
Why
The Worker's D1 database (
b2c79dd6-7771-4b1f-aa6b-085d5f3e9528, internally still namedgittensoryfrom before the 2026-07-14 repo/app rename) had grown to 8.96 GB of its 10 GB cap. #9459 found and fixed the dominant cause —contributor-decision-packsignal snapshots accumulating unbounded (6.3 GB / 71% of the file) — and I pruned the already-accumulated 18,336 stale rows directly against the live database.That prune shrank the live size (8.96 GB → 2.24 GB, confirmed via
wrangler d1 info), which was better than expected — but D1 does not runVACUUM(cloudflare/workerd#1618), so the file was never guaranteed to fully return to its logical size, and any future large one-off bloat has the same risk. Since a rename was overdue anyway (gittensory→loopover, matching the 2026-07-14 app rename), this cuts over to a fresh, correctly-named database rather than just living with the old one.What I verified before opening this
gittensory) vs new (loopover): every mismatch is explained by ordinary live-write drift on actively-written tables (webhook/audit/usage logs — the source database never stopped taking writes between the export and the comparison). One delta (github_rate_limit_observations, exactly −250,000) is the automatic hourlyretention.prunejob's per-table cap firing once on the source after the export — not data loss.d1_migrations: 202/202 rows, same max id, on both databases.contributor-decision-pack(the table fix(retention): dedupe contributor-decision-pack to latest-only, not a preserved series #9459 fixed): 213/213 rows on the new database, exactly one row per contributor.payload_jsonlarge enough to exceed D1's ~100KB single-statement text limit,SQLITE_TOOBIG) were re-imported via a chunkedINSERT+payload_json ||= UPDATEsequence instead of a raw literal insert. I verified this chunking byte-identical against the source data via a local SQLite reconstruction test before running it against the real database.cf-typegen:check: unaffected — the binding name/shape is unchanged, only thedatabase_id.wrangler.jsoncis the only file referencing the olddatabase_id.What this does NOT do
edge-nl-01) — that deployment uses its own Postgres, unrelated to this Cloudflare D1.CLOUDFLARE_D1_MONITOR_DATABASE_ID(the fix(selfhost): alert explicitly when the monitored D1 approaches its 10 GB cap #9458 size-alerting probe) — that's an env var on the self-host side, currently unset, and is a separate follow-up once this cutover is confirmed.Deploy mechanics — please read before merging
This repo's Worker (
loopover-api) deploys via Cloudflare Workers Builds, git-connected tomain— merging this PR triggers a live production deploy pointing the Worker at the new database. There is a brief window during the deploy where in-flight webhook-relay writes could be affected. I'm holding this for an explicit go rather than merging it myself, given it's the highest-blast-radius change in this batch.Refs #9435