Check CBS codes and implausible dates; sweep 90 days weekly

[joepio]

Requested by Joep · project thread

Two responses to VNG's weekly status of 28 September, in separate commits.

1. CBS codes and implausible future dates

Before: Dordrecht had no cbsId in /api/status, so VNG's weekly check drops it from municipal coverage. Oirschot's newest content was 1 January 2078 (an ingekomen stuk whose iBabs "Datum" field reads 1-1-2078), and it heads every date-sorted search for Oirschot. VNG reports Schiermonnikoog with a 2099 date as well.

After: Dordrecht carries GM0505, and a catalog test fails when any gemeente lacks a well-formed, unshared GM code (the Amsterdam stadsdelen under GM0363 excepted). A date more than two years ahead is treated as a typo at the source.

How:

• src/util/plausible_date.ts: the two-year horizon.
• iBabs register entries whose date field is beyond it fall back to the entry's MutationDate.
• Projection leaves start_date and document_month empty for such a date (sorts last instead of first).
• /api/status computes latestContentDate in its own query bounded to the horizon, so rows already in the index with 2078 or 2099 stop reporting it as soon as this deploys. Kept separate from the last_indexed query because a range clause would drop rows without start_date from that one.

The Oirschot row in the index keeps its 2078 sort date until the entry is imported again. A manual import for Oirschot with a window covering that entry's MutationDate (or the next projection bump) fixes it.

2. Weekly 90-day sweep

Before: the nightly import only looks at meetings from 7 days back to 7 days ahead. Documents a council attaches to a meeting weeks later (besluitenlijsten, notulen, answers) are never imported, and the weekly coverage check counts them as missing. VNG saw the gaps grow by 677 documents across 179 municipalities in one week.

After: on Saturday night the scheduler also enqueues the 90..8 days before the nightly window for every source, so Sunday's coverage check measures what it found.

How: scheduledWindows() in src/scheduler.ts returns the nightly window, plus the adjacent sweep window on Saturdays (Europe/Amsterdam). Runs are ordinary scheduled / full runs, so blocklist checks, the S3 cache shortcut and iBabs pacing all apply unchanged. The existing "skip while scheduled runs are still active" guard covers both windows.

This is a hypothesis from the code, not yet measured: if the gaps in coverage do not shrink after the first Saturday, the cause lies elsewhere.

Tests: touched suites (plausible_date, registers, motions, source_catalog, status_api, scheduler, ingest_scheduler, scheduler_backfill_isolation) pass, deno check on all tests, oxlint, oxfmt on changed files, vite build.