|
| 1 | +# Event migration runbook (sub-calendar v1) |
| 2 | + |
| 3 | +How to run, verify, and — later — cut over the calendar-owned event migration |
| 4 | +from plan packet `02`. Read the whole page before touching production. |
| 5 | + |
| 6 | +Two forward migrations are involved: |
| 7 | + |
| 8 | +1. `calendar-record-migration` — reshapes the `calendar` collection to the |
| 9 | + strict record contract in place (preserving every `_id` and visibility |
| 10 | + preference) and creates each user's Compass-local calendar. |
| 11 | +2. `event-record-backfill` — rebuilds the inactive `event_new` collection from |
| 12 | + the legacy `event` collection using the deterministic transform, then |
| 13 | + verifies source/destination equivalence. |
| 14 | + |
| 15 | +Neither migration touches the legacy `event` collection, and nothing starts |
| 16 | +reading `event_new` until the separate runtime-cutover release. Running these |
| 17 | +early is safe; the app keeps serving from legacy data. |
| 18 | + |
| 19 | +## Preflight |
| 20 | + |
| 21 | +1. **Back up first.** Follow [Back up & restore](./backup-and-restore.md) in |
| 22 | + full. The calendar migration rewrites calendar rows in place — the backup is |
| 23 | + its only rollback. |
| 24 | +2. **Disk space.** The backfill writes a full copy of the event collection. |
| 25 | + Confirm free space of at least 2× `db.event.totalSize()`. |
| 26 | +3. **Quiet period.** The migrations tolerate concurrent writes (the app still |
| 27 | + writes legacy shapes), but run them at low traffic so the final |
| 28 | + verification scan isn't racing fresh edits. |
| 29 | +4. **Check what's pending:** |
| 30 | + |
| 31 | + ```bash |
| 32 | + bun run cli migrate pending |
| 33 | + ``` |
| 34 | + |
| 35 | + The migration runner reads the Mongo connection from your config file; set |
| 36 | + `COMPASS_CONFIG_FILE=~/compass/compass.yaml` when running outside the dev |
| 37 | + repo. On a Docker install the Mongo port is not published to the host, so |
| 38 | + run the CLI from a machine/container that can reach the compose network. |
| 39 | + |
| 40 | +## Run |
| 41 | + |
| 42 | +```bash |
| 43 | +bun run cli migrate up |
| 44 | +``` |
| 45 | + |
| 46 | +Expected behavior: |
| 47 | + |
| 48 | +- The calendar migration logs how many rows were reshaped and how many local |
| 49 | + calendars were created. Any unconvertible calendar row aborts the whole |
| 50 | + transaction with a summary of `{ id, reason }` pairs — nothing is half |
| 51 | + migrated. |
| 52 | +- The backfill logs per-run counts: attempted, inserted, failed, the time-zone |
| 53 | + derivation tally (`calendar` vs `utcFallback`), the number of someday sort |
| 54 | + orders assigned, and the legacy `allDayOrder` audit count. Event titles and |
| 55 | + descriptions never appear in logs. |
| 56 | +- The backfill is **fail-closed**: any transform failure, duplicate Google |
| 57 | + event id, or verification mismatch makes the migration throw with a compact |
| 58 | + summary. The legacy collection is untouched either way. Fix the reported |
| 59 | + rows (or file the fixture as a bug), then rerun. |
| 60 | +- Reruns are safe and convergent: the backfill clears and rebuilds the |
| 61 | + inactive destination from scratch, and the calendar migration passes over |
| 62 | + already-migrated rows. |
| 63 | + |
| 64 | +## Verify |
| 65 | + |
| 66 | +Verification runs automatically at the end of the backfill and rechecks: |
| 67 | + |
| 68 | +- total and per-user event counts, and per-category counts |
| 69 | + (timed / all-day / someday / series / occurrence); |
| 70 | +- no orphan `seriesId` or `calendarId` references; |
| 71 | +- no duplicate provider event ids per calendar; |
| 72 | +- a deterministic content hash of every behavior-bearing field, source vs |
| 73 | + destination. |
| 74 | + |
| 75 | +A successful run ends with the verification summary. If you need to re-check |
| 76 | +later without re-migrating, the same checks are exposed as |
| 77 | +`verifyEventMigration` in `packages/scripts/src/common/event-migration.verify.ts`. |
| 78 | + |
| 79 | +## Cutover (performed with the runtime-cutover release, not now) |
| 80 | + |
| 81 | +The cutover is a short, planned write pause. Do not improvise it outside a |
| 82 | +release window. |
| 83 | + |
| 84 | +1. Stop the backend so no writes land mid-rename: |
| 85 | + |
| 86 | + ```bash |
| 87 | + COMPOSE_PROFILES=selfhosted docker compose stop backend |
| 88 | + ``` |
| 89 | + |
| 90 | +2. Rerun `bun run cli migrate up` (idempotent) so the destination includes |
| 91 | + every write made since the last run, and confirm the verification summary. |
| 92 | +3. Rename collections so validators and indexes travel with the data |
| 93 | + (`prod_calendar` is the production database name): |
| 94 | + |
| 95 | + ```javascript |
| 96 | + // mongosh, e.g.: docker compose exec mongo mongosh -u ... -p ... |
| 97 | + use prod_calendar; |
| 98 | + db.event.renameCollection("event_legacy_v1"); |
| 99 | + db.event_new.renameCollection("event"); |
| 100 | + ``` |
| 101 | + |
| 102 | +4. Deploy the runtime-cutover app version and start the backend. |
| 103 | +5. Smoke-test: sign in, load a week with events, create and delete a test |
| 104 | + event. |
| 105 | + |
| 106 | +Keep `event_legacy_v1` — it is the rollback source. Do not drop it in v1. |
| 107 | + |
| 108 | +## Rollback after cutover |
| 109 | + |
| 110 | +Rolling back **abandons every write made since the cutover**. That loss window |
| 111 | +is the accepted trade for skipping dual-writes; keep it short by rolling back |
| 112 | +promptly or not at all. |
| 113 | + |
| 114 | +1. Stop the backend. |
| 115 | +2. Dump the new collection first so post-cutover writes stay recoverable by |
| 116 | + hand: |
| 117 | + |
| 118 | + ```bash |
| 119 | + mongodump --db prod_calendar --collection event --out /tmp/post-cutover-dump |
| 120 | + ``` |
| 121 | + |
| 122 | +3. Rename back and redeploy the previous app version: |
| 123 | + |
| 124 | + ```javascript |
| 125 | + use prod_calendar; |
| 126 | + db.event.renameCollection("event_new"); |
| 127 | + db.event_legacy_v1.renameCollection("event"); |
| 128 | + ``` |
| 129 | + |
| 130 | +4. Start the backend and verify legacy events are readable. |
| 131 | + |
| 132 | +## Notes |
| 133 | + |
| 134 | +- Executed 2025 migrations (`new-events-collection`, |
| 135 | + `migrate-events-to-new-events-collection`) stay recorded and untouched; the |
| 136 | + backfill supersedes their output by rebuilding the destination. |
| 137 | +- `allDayOrder` is audited (count logged) and intentionally not carried into |
| 138 | + the new schema — no production code reads it. |
| 139 | +- Dev databases use the `_dev.` collection prefix; the commands above show |
| 140 | + production names. |
0 commit comments