ADFA-4739: Kotlin docs DB pipeline + Build Kotlin Docs GitHub Action - #24

- README: templates/assets are already included in the repo, not left
  for the user to place separately.
- sync_kdoc_json_to_db.py: default --db to documentation.db in cwd
  instead of a personal /home/alex path; add explicit BEGIN to match
  sibling scripts' transaction style.
- insert_optimized_media.py: rewrite_pages now substitutes renamed image
  references in a single regex pass over each row's original text,
  instead of sequential str.replace calls on a mutating buffer, which
  could double-rewrite a reference if one rename's new name chained into
  another rename's old name.
- run_e2e_pipeline_test.sh: add a verification step that reuses
  populate_db.py's own pruning logic to confirm blacklisted topic pages
  are actually absent from the database, replacing a weaker proxy check
  that only tested for k/html/wasm% paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Brings in the updated KDoc-to-JSON Dokka plugin (sourceSet whitelisting,
package-index/structure verification scripts) so this branch has the
current plugin used to generate the kotlin-stdlib docs consumed by
sync_kdoc_json_to_db.py.

- run_e2e_pipeline_test.sh: replace the manually-supplied STDLIB_ALL_LIBS
  path with STDLIB_DOCS_DIR (a kotlin repo's libraries/tools/kotlin-stdlib-docs
  checkout). New Step 4/5 derives the repo root and calls
  build-stdlib-json-docs.sh to freshly build/publish the kdoc-to-json plugin
  and generate kotlin-stdlib/-reflect/-test JSON docs (common+jvm source
  sets only, per the existing plugin config) before Step 5/5 syncs them into
  the database.
- Add build-stdlib-json-docs.sh: builds a fresh copy of the plugin, swaps in
  the JSON-plugin-enabled build.gradle.kts for the duration of the build
  (restored on exit via trap), and prints only the resulting all-libs path
  to stdout so it composes as STDLIB_ALL_LIBS="$(build-stdlib-json-docs.sh
  <kotlin-repo-root>)" - every gradlew invocation inside it is redirected to
  stderr so that capture isn't polluted by build console output.
- build.gradle.kts: re-add the Dokka dev-snapshot Maven repo (same
  dokka_repository property/default kotlin-stdlib-docs' own
  settings.gradle.kts uses) - kotlin-stdlib-docs' own
  dokka-samples-transformer-plugin subproject needs a Dokka dev build that
  isn't on Maven Central, and the allprojects{} repositories block here had
  removed access to it for every subproject.

Verified end-to-end against a real kotlin checkout: fresh plugin build,
JSON generation (common+jvm only), and sync into a scratch documentation.db
all succeed, with blacklist pruning verification passing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Documents what OfflineDocumentationTools is (the offline tooling that
builds/edits the documentation database consumed by Code on the Go),
how its pieces relate to each other, and — most importantly — the gap
between the schema this repo's tools expect and the schema the current
production documentation.db actually has (added templateId/Templates/
Bookshelf/BookCategories/PUCC_* tables, dropped the legacy
ide_tooltip_table).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CI counterpart to run_e2e_pipeline_test.sh: builds kotlin-stdlib/-reflect/-test
JSON docs and the Kotlin website docs, then loads both into a Drive-hosted
copy of documentation.db and uploads the result back to the same file.

Google Drive file IDs (database and Writerside's webHelpImages.zip) resolve
from secrets/inputs by default, with hard-codeable TEST_*_FILE_ID overrides
for one-off manual testing. Posts "Grabbing baton"/"...Dropping baton" to
Slack (SLACK_WEBHOOK_URL) around the run, since it mutates a single shared
Drive file. Defaults to dry_run so a first run never touches production.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…739)

Adds the code that loads converted Kotlin website content (build_nav.py,
populate_db.py, media insertion) and kotlin-stdlib/-reflect/-test JSON
content (sync_kdoc_json_to_db.py) into documentation.db, the local e2e test
script for that pipeline, and the CI workflow that runs it end-to-end
against a Drive-hosted copy of the database.

Split out of the larger Kotlin-docs pipeline PR (#21) so the DB-manipulation
side (this ticket) can be reviewed separately from producing the raw JSON
data for the Kotlin website (ADFA-5039, PR #23). This PR depends on ADFA-5039
merging first - populate_db.py, build_nav.py, and find_missing_assets.py all
import md_to_json.py, which isn't included here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Its own read_text(encoding="utf-8") sat outside any try/except, so a
malformed file raised uncaught and killed the whole process - bypassing
--allow-failures entirely, unlike every other file read in this script.
Now catches, counts, and reports it the same way the main conversion
loop already does; find_include_warnings returns (warnings, failed)
and main() folds that count into the existing failure total.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

populate_db.py trains a zstd fast-cover dictionary (256 KiB) from this run's
own pages/nav on first use and stores it in a new CompressionDictionary
table, then compresses every page/nav/image/asset row against it via the
brotli CLI's -D flag (the installed Python brotli package has no dictionary
API). Never retrains an existing dictionary: a dictionary-compressed row is
only decodable against the exact dictionary it was compressed with, verified
empirically to fail silently-wrong rather than loudly on a mismatch, so
retraining would orphan every already-migrated row.

insert_optimized_media.py rewrites the same rows populate_db.py writes (image
optimization, in-place URL rewrites), so it now loads and reuses the same
dictionary instead of the old plain-Brotli calls it would otherwise silently
corrupt those rows with.

ADFA-5153.

populate_db.py and insert_optimized_media.py only ever touch their own
subset of Content (k/html/%, assets/%). Every other Content row -- reference
docs, tooltip-linked pages, whatever else -- was still plain Brotli, no
dictionary. migrate_content_to_dictionary_brotli.py recompresses every
remaining 'brotli' row against the shared CompressionDictionary (training one
from a representative whole-corpus sample if none exists yet), so the "every
brotli row uses the dictionary" assumption WebServer.kt's reader depends on
actually holds.

Idempotent by construction: a plain decode reliably fails once a row is
already dictionary-compressed (verified over 200 trials), so re-running is
always a safe no-op. Backs up first (VACUUM INTO), runs in one transaction.

Run against the real documentation.db: 29,748/29,751 brotli rows migrated,
131.1MB -> 85.6MB compressed, 299.0MB -> 255.3MB overall.

ADFA-5153.

Every 'brotli' Content row in the real database is now compressed against
the shared CompressionDictionary (see the prior two commits), but
docdb_studio.py still read and wrote plain Brotli in three places:
get_html_anchors_for_path, fetch_content_for_path (both decode), and
compress_for_storage via import_content_files (encode). Against the
migrated database this wasn't a latent risk -- it was already broken: a
plain decode of dictionary-compressed content reliably fails, so anchor
validation and content preview were silently erroring on every real page,
and any new import would have written dictionary-incompatible plain Brotli
back into a database that assumes there is none left.

get_compression_dictionary(db_path) reads and caches a database's
CompressionDictionary (or None, for a database that predates ADFA-5153) --
docdb-studio never creates or retrains one itself, only ever reads whatever
another tool already produced. compress_for_storage/decompress_brotli shell
out to the brotli CLI's -D flag when a dictionary is present, matching
populate_db.py's approach, and fall back to the plain brotli package
otherwise. decompress_brotli deliberately raises brotli.error on failure so
the two existing call sites' `except brotli.error:` handling didn't need to
change.

Verified against the real (migrated) documentation.db: anchor lookup and
content fetch both now work on real pages that previously would have
errored.

ADFA-5153.

Each row's recompress spawns its own `brotli` subprocess, so the ~30,000-row
real migration was dominated by process-spawn overhead running strictly
sequentially. Retrospective feedback: this should have been parallelized
from the start rather than accepting a slow serial run.

migrate() now runs reassemble+plain-decompress+dictionary-recompress on a
ThreadPoolExecutor (defaults to ThreadPoolExecutor's own min(32,
cpu_count+4), tuned for exactly this I/O/subprocess-bound shape); each
worker opens its own read-only connection (a single sqlite3.Connection
isn't safe across threads) and reuses one DictionaryCompressor per thread
rather than one per row. The actual delete+insert writes stay serialized on
the caller's connection, which SQLite requires anyway. Measured 3-6x faster
than sequential on synthetic benchmarks.

DictionaryCompressor gets an atexit safety-net close(), since a per-thread
instance has no single call site that can cleanly scope a `with` block
around it the way populate_db.py's/insert_optimized_media.py's own
single-threaded usage already does.

Test fixture switched from :memory: to a real temp file, since worker
threads need an actual db_path to open their own connections against - an
in-memory database has none and can't be shared across connections at all.

ADFA-5153.

This is the pipeline that actually produces the live documentation.db
(scripts/DocumentationDatabase.py, fixed earlier on this ticket, turned
out to be dead code -- its tag-triggered workflow hasn't fired since
db-2025-07-16b). populate_db.py has always run its own bare VACUUM
with no page_size pin, so the real fix belongs here.

Extracted vacuum_and_pin_page_size(), mirroring docdb_studio.py's
vacuum_database(): pins page_size via PRAGMA before VACUUM, and works
around WAL journal mode silently preventing PRAGMA page_size from
taking effect (this file's own backup_database docstring already
anticipates a live/WAL-mode database).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

vacuum_and_pin_page_size (commit b203500) mirrored docdb-studio.py's
original vacuum_database(): in-place VACUUM + a journal_mode round-trip,
which requires exclusive access to db_path. SQLite refuses to switch a
WAL-mode database away from WAL while ANY other connection has it open
-- even one from a function that has already returned, since Python's
`with sqlite3.connect(...) as conn:` does not close conn on exit.
Empirically reproduced and fixed the identical bug in docdb-studio.py's
vacuum_database (PR #25); this mirrors that fix here since this
pipeline's own VACUUM is the one actually run against the live
documentation.db.

Rewritten on VACUUM INTO: rebuild into a temp file next to db_path
(read-only snapshot of the source, no exclusive access needed), then
atomically swap it into place with os.replace. journal_mode=WAL is
reapplied to the new file's final path (VACUUM INTO always produces a
plain rollback-journal file), and stale sidecars from the replaced file
are cleaned up.

Two new tests: the fix succeeds with both an unrelated open connection
and an unclosed caller-style connection present at once (the actual
scenario the old design was fragile against), and the original file is
left untouched if VACUUM INTO fails partway (temp file cleaned up, no
partial swap).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

tempfile.mkstemp() always creates its file mode 0600 regardless of the
original's mode or the process umask. The VACUUM INTO rewrite swaps
that temp file into db_path's place via os.replace, which never
restored the original permissions -- alexmmiller's QA of the mirrored
docdb-studio.py fix caught this silently dropping documentation.db
from 644 to 600 on every vacuum; same bug here since this pipeline's
vacuum_and_pin_page_size uses the identical mkstemp+replace pattern.

Capture db_path's mode before the rewrite and os.chmod it back after
the swap. New test confirms a 644 file stays 644 across
vacuum_and_pin_page_size (and fails against the pre-fix code, dropping
to 600).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…nal_mode-read connection

Same fix as the mirrored docdb-studio.py version: VACUUM INTO's target
accepts a bound parameter (already used by this file's own
backup_database for the same reason), sidestepping SQL string-literal
escaping for a path containing a single quote (e.g. "David's Docs")
rather than hand-rolling it. Also explicitly closes the journal_mode
-read connection instead of relying on it being reassigned by the next
`with` block.

New test: a quote in db_path's parent directory no longer breaks the
statement.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Same findings as the mirrored docdb-studio.py fix's third self-review:
- chmod the temp file to the original permissions before os.replace,
  not after -- fixing it up afterward left a real window where db_path
  was visible at mkstemp's 0600, and left permissions permanently
  wrong if the chmod itself failed.
- Explicitly close the VACUUM INTO and WAL-reapply connections, and
  give the WAL-reapply connection the same 30s timeout as its siblings
  in the same function.

19/19 local tests pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…to ADFA-5153

Benchmarking showed page_size=1024 vs 2048 has essentially the same
performance and a negligible size difference before compression (and
likely less after this PR's dictionary compression) -- adding
complexity without benefit. ADFA-5141 is declined; this PR is only
about the Brotli dictionary compression (ADFA-5153) and the page_size
work rode along on this branch by coincidence of timing, not by scope.

Restores populate_db.py's original plain VACUUM call and removes
vacuum_and_pin_page_size, SQLITE_PAGE_SIZE_BYTES, the now-unused os/stat
imports, and their dedicated test file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

WebServer.kt's reassembly loop always probes "<path>-1" first, but 14 of
19 chunked Content rows in the real documentation.db number their
continuations starting at "-2" instead, with no "-1" row at all. The
first lookup misses, the loop stops after the base 1 MB chunk, and the
row is served short: a corrupt image (compression='none', silent 200) or
a decode failure (compression='brotli', 500) - confirmed against a local
copy of the shipped database (md5 34c879595bd6fb87e5b68989369680a8).

No writer in this tool ever produced that numbering - populate_db.py,
insert_optimized_media.py, and migrate_content_to_dictionary_brotli.py
all go through insert_chunked_content, which has always started
fragments at -1. This is inherited data older than this pipeline, not
something it can regenerate correctly by re-running existing tools.

renumber_misnumbered_fragments.py finds base rows whose fragment chain
(via LIKE, sorted on the parsed numeric suffix rather than assumed
paths) doesn't start at 1, and renumbers it to a contiguous run starting
at -1, lowest-suffix first so each rename's target is the path just
vacated by the previous one. A chain with an actual gap (a genuinely
missing chunk, a different failure) is reported and left alone rather
than guessed at. Content bytes are never touched, only paths, so it's
safe regardless of a row's compression. Verified against a scratch copy
of the real database: renumbers exactly the 14 chains the ticket found,
and the two example rows (the devsite gif, the Javadoc index) reassemble
and decode correctly afterward.

…bering

ADFA-5171: Repair chunked Content rows misnumbered from -2

Every finding from the three reviews on PR #26 that lands in files this PR
touches, plus the one dictionary-consistency problem outside it that this PR
itself creates.

Data loss, both silent:

* The migration deleted and re-inserted each base row. Content carries
  AddBook/DeleteBook triggers on '%.pdf' paths, so that cycle replaced every
  curated Bookshelf entry with 'CURRENT_TIMESTAMP || id' under a fresh
  Content.id -- verified on the real database: (53507, category 5, "Android
  Notes for Professionals") became (53508, category NULL, "2026-08-21
  22:37:5553508"). 15 brotli-typed .pdf rows and all 7 Bookshelf rows are in
  scope. Writes are now UPDATE in place, with continuation rows reconciled by
  exact path.

* delete_content interpolated a path straight into LIKE, where `_` is a
  wildcard and the `-%` suffix was not restricted to digits, so unrelated rows
  could be deleted permanently (hal-eisen-adfa). No write path goes through
  LIKE any more.

Rows silently skipped while the run reported success:

* reassemble_content probed "<path>-1", so an ADFA-5171 chain numbered from -2
  reassembled truncated, failed to decode, and was counted as "already
  dictionary-compressed". The corpus has 29,751 base rows and exactly 3 with
  continuations; the run reported 29,748 migrated and 3 already-migrated in a
  first-ever migration, which is precisely those 3. Chain discovery is now
  shared with the repair script (populate_db.fragment_chain), so the two
  cannot drift apart again.

* Any decode failure counted as "already migrated" (hal-eisen-adfa). Rows are
  now classified by decoding both ways: identical either way means the encoder
  never referenced the dictionary and there is nothing to gain (~0.5% of the
  real corpus, and the reason a second run used to re-migrate them --
  alexmmiller); plain-only means migrate; dictionary-only means done; neither
  is an error, never a success.

* Recompressed bytes are verified to round-trip before being written.

Concurrency and memory:

* Each worker opened its own read connection while the caller held one write
  transaction over the whole run, which deadlocks under journal_mode=delete --
  documentation.db's actual mode (alexmmiller). All database access is now on
  the calling thread; workers receive bytes. Commits are batched, so an
  interrupted run keeps finished batches and resumes.

* Blobs are no longer selected for every row up front (~130 MB held at once).

Dictionary training, measured on the real corpus with only the sampling varied:

    first 300 rows by path (all under "a/")        36.2% smaller than plain
    300 rows stratified across doc sets            33.2%  <- worse
    stratified, 32 MiB plaintext budget            48.3%  <- best
    first-by-path, same 32 MiB budget              36.4%  <- volume alone: nil

The docstring promised "a random sample drawn across the WHOLE Content table"
and delivered the first 300 paths alphabetically -- 299 of them under "a/",
while j/ (10,326 rows) and k/ (3,757) trained nothing (hal-eisen-adfa). Fixing
it by stratifying alone makes things worse: quotas drawn from smaller doc sets
starve the trainer, which then cannot even fill a 256 KiB dictionary. Both
halves are needed, so sampling is now stratified by stored bytes and bounded
by a plaintext budget, seeded for reproducibility since a stored dictionary is
never retrained.

renumber_misnumbered_fragments:

* A chain numbered from -0 passed the "starts at 1?" guard and renamed onto an
  occupied slot, tripping UNIQUE(path) and rolling back every other repair in
  the pass (hal-eisen-adfa). Such a chain is repaired rather than skipped -- the
  app probes "-1", finds it, and serves the chain with "-0" dropped -- via a
  parking pass that is correct in either shift direction.

docdb-studio:

* sqlite3.OperationalError covers "database is locked", and caching that as
  "no dictionary" downgraded the whole session to plain Brotli
  (hal-eisen-adfa). Only definitive answers are cached now.

* The new `brotli` CLI dependency raised RuntimeError/OSError out of paths that
  guard only `brotli.error` (hal-eisen-adfa). Missing-binary now raises a
  BrotliCliMissing subclass of brotli.error, with an actionable message.

* decompress_brotli decoded dictionary-only, so it could not read plain rows --
  which a dictionary database always contains: anything a plugin contributes
  on-device, anything written outside populate_db.py, and everything mid-
  migration. It now falls back to a plain decode, as WebServer.kt does.

sync_kdoc_json_to_db (outside this PR's diff, but this PR is what makes
documentation.db a dictionary database):

* compress_for used plain brotli.compress, leaving every k/kotlin-stdlib row
  plain inside a dictionary database (hal-eisen-adfa). It now compresses
  against the database's dictionary when there is one.

* "Source file missing => delete the row" had no floor: a Dokka layout change
  makes every lookup miss, and the script would delete every stdlib row plus
  its parent Tooltips and exit 0 (hal-eisen-adfa). Sources are resolved up
  front and a wholesale miss aborts.

Corrected in populate_db's DictionaryCompressor docstring, because two reviews
reasoned from it: the two mismatch directions are not alike. Decoding a
dictionary row with NO dictionary is loud (398 of 400 real rows raised, 2
returned identical bytes, none wrong), which is what makes both this script's
idempotency check and WebServer.kt's fallback sound. Decoding with the WRONG
dictionary is the silent case (50% raised, 38% returned different bytes with no
error, 12% identical). The test asserting a wrong-dictionary decode does not
raise was asserting that coin flip; it now asserts the invariant that holds.

Tests: 25 in ProcessKotlinWebsiteJSON (up from 21) and 173 in docdb-studio (up
from 170) pass. New coverage for the -2 chain, an undecodable row, Bookshelf
survival through the triggers, a never-referenced-dictionary row across two
runs, stratified sample determinism and spread, zero-based renumbering, one bad
chain not blocking other repairs, a locked database not being cached, a plain
row in a dictionary database, and a missing brotli CLI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Two of these are misses in the previous commit's own fixes.

- The act check moved above the argument *validation* rather than below
  it, so five semantic errors (both --skip flags, --db-path missing or
  nonexistent, --images-zip-path missing or nonexistent) were still
  answered with "act is required" on a machine without act. It now sits
  immediately before the act invocation; all five report themselves, and
  act is still required once the arguments are good.
- possible_output_names' undetermined case claimed both names, which was
  widened from GIFs to every PNG and TIFF by the last commit. It now
  claims nothing: optimize_raster repeats the same Image.open and
  is_animated access, so whatever makes the probe fail makes process_file
  fail too, and the source writes no file at all. Holding two names for
  it de-conflicted a real image against an output that never appears.

And the deeper one behind both of the last two rounds' F26-shaped bugs:
optimize_directory now checks that the name process_file actually wrote
was one the planning pass claimed for that source, and fails loudly
otherwise. The prediction lives in a second place from the code it
predicts and has drifted twice, silently both times; this is the missing
connection between the halves.

Also:
- _is_animated_raster reports a probe failure through the logger instead
  of swallowing it - it runs for every PNG now, and a systematic failure
  would otherwise shift the whole de-confliction pass unexplained.
- outside_fences' `source` is required, not defaulted: the warning is the
  entire point of that change, and an optional argument is how a later
  call site quietly gets the old silence back.
- The unterminated-fence check runs once on spans[-1]. Only the last span
  can be unterminated (fenced_spans appends the run-to-EOF one after its
  loop), so checking every span re-split every code block for nothing.
- `animated: Optional[bool]`, and a non-bool argument is refused - the
  parameter took a Path one revision ago, and a Path is truthy, so a
  stale call reported "animated" for every file with no error.
- The case-handling test asserts the property rather than the
  "{stem}-{ext}" naming literal.

6 new tests, and every changed behaviour fails against the previous
commit. 192 + 33 + 189 pass; scripts/ unchanged from base. The real
299-image corpus is byte-for-byte identical again, and the local
workflow under act produces the same database md5 as the last two
rounds - with the new written-vs-claimed check silent across every path
the corpus exercises, including SVG rasterization.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The drift check added in the previous commit crashed on ordinary input.

possible_output_names' SVG branch returned the lowercase SVG_EXTENSION
literal instead of echoing the source's own suffix - every other branch
already echoes it - so "Diagram.SVG" was claimed as "Diagram.svg" while
optimize_svg writes `dst`, which keeps the source's spelling. Harmless
while it only fed the casefolded `claimed` map; once the check compared
what was written against what was claimed it aborted the entire run over
one ordinary file, after all 299 images had been optimized. The corpus
has no uppercase extensions, so neither the full-corpus run nor act
could have caught it.

Three things had to change for that to be safe, not just quiet:

- The check asks `claimed`, casefolded like every other lookup in the
  pass, instead of a second exact-case dict. De-confliction guarantees
  no two sources hold the same lowercased name, so "did this source
  claim this name" is exactly `claimed[name] is src` - and `claimed_for`
  is gone rather than kept in step by hand.
- A write from a source that claimed nothing (its animation could not be
  determined, so the planner expected optimize_raster to fail too) is one
  file's bad luck, not drift. It is now reported and the file kept - this
  module's rule is that a single file's problem does not end the run, and
  a dangling symlink doing exactly that was fixed as a bug in this same
  PR. Kept rather than dropped, deviating from the review's suggestion:
  discarding a successfully optimized image is the worse outcome, and
  insert_optimized_media's seen_names guard still covers the only
  residual risk.
- Genuine drift raises ValueError, not RuntimeError, so
  insert_optimized_media's `except ValueError` turns it into a clean
  "error: ..." line instead of letting a traceback escape main().

Also `logger: Optional[Logger]`, the annotation this commit's parent
fixed one function above and repeated here.

9 new tests, including the uppercase case across .SVG/.Svg/.PNG/.JPG/.TXT
with and without --webp; the six raster/passthrough parametrizations pass
against the parent commit too, which is what pins the crash to the SVG
branch. 204 + 33 + 189 pass; scripts/ unchanged from base. The real
299-image corpus is byte-for-byte identical, and act produces the same
database md5 as the last three rounds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The drift check's exemption keyed off the wrong thing, and let real
drift through.

`if src in unpredicted` read as "the planner claimed nothing for this
source", but it recorded "the probe failed" - and those come apart under
--webp false, where possible_output_names returns the source's own name
before it ever consults `animated`. So for any file whose probe failed,
a writer that produced a name nobody claimed got a warning and an exit 0
instead of the error the check exists to raise. The warning even said
"the de-confliction pass held no name for it" while `claimed` held one.

Rather than correct the condition, remove the case: optimize_raster now
takes the planner's `animated` answer instead of re-deriving it with its
own `getattr(img, "is_animated", False)` a few lines later. Two answers
to one question was the whole reason a source could be unpredictable
here and processable there. A source whose animation is undetermined is
now refused in optimize_raster, counted by process_file as one file's
error like any other, and never reaches the check - so the planner's
"claims nothing" is always accurate and the exemption has nothing left
to describe. Both answers already came from the same access on the same
file, so this only changes behaviour on a transient.

The per-source plan is one record (Plan: dst_rel, names, animated)
instead of a dst dict beside a parallel set - every parallel structure
this pass has grown ended up disagreeing with the one next to it, which
is what both of the last two rounds' bugs were.

Also:
- The drift message reports `plan.names`, the claim as the planner wrote
  it, not `claimed`'s casefolded keys - reporting "diagram.svg" for a
  source that claimed "Diagram.SVG" points the reader at a case mismatch
  that isn't the problem.
- The check compares with == rather than `is`: Path defines __eq__, and
  identity held only because two loops happened to share list objects.
- The uppercase test asserts the written name per case, not just that one
  file came out.

Dropped test_drift_is_refused_even_when_the_probe_failed as written: it
stubbed the probe to False - a probe that succeeded - so it demonstrated
nothing about the exemption and passed against the unfixed tree. The
regression for that hole is the --webp False half of
test_an_undetermined_probe_skips_one_file_and_no_more, which fails
against it.

206 + 33 + 189 pass; scripts/ unchanged from base. Real 299-image corpus
byte-for-byte identical, and act gives the same database md5 for the
fourth round running.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The previous commit flattened every animated WEBP to a single frame.

optimize_raster stopped deriving is_animated itself and took the
planner's answer instead - but the planner probed only
ANIMATABLE_EXTENSIONS, and .webp was deliberately excluded from that set
on the grounds that "an animated WEBP is copied through as a .webp,
which is what the conversion branch predicts anyway, so there is nothing
to resolve and no reason to pay for opening the file". True for
predicting a *name*; wrong once the same value decided whether to
re-encode. A 4-frame webp came out with 1 frame, exit 0, no warning -
and the written-vs-claimed guard could not see it, because the name was
exactly what was predicted. Only the frames were gone.

The set is gone with it. One question - "is this file animated" - now
gets one answer, taken from the file, for every raster; and the naming
rule follows from it uniformly: under --webp a raster keeps its own
extension exactly when it is animated, because that is when
optimize_raster copies it through instead of re-encoding. No curated
membership to get wrong, which it had been twice (first .gif alone, then
a four-entry set missing .webp and .jpg).

That also restores, and this time documents, the handling of an MPO
stored as .jpg - a phone stereo or burst capture, which Pillow reports
as animated. It is copied through rather than flattened to its primary
frame; the previous commit changed that silently as a side effect.

Also:
- optimize_raster's copy-through comment named webp as its example while
  that branch was unreachable for webp. It says what the branch really
  covers now, MPO included.
- process_file's docstring said `animated` was "irrelevant for SVG and
  passthrough files", implying it carried a real answer for every raster;
  it was a hard-coded False for four of the eight raster extensions.
- test_an_undetermined_probe_skips_one_file_and_no_more used other.jpg as
  its control, which survived only because .jpg was unprobed. Now a .txt
  passthrough, which no widening of the probe can touch.

New: test_every_animated_raster_keeps_its_frames_under_webp over
animated WEBP, APNG and MPO-as-.jpg, plus a static counterpart so the
exemption stays animation and not extension. The webp and MPO cases fail
against the previous commit; the APNG case passes, which is the control
showing what the old set did and did not cover.

214 + 33 + 189 pass; scripts/ unchanged from base. Real 299-image corpus
byte-for-byte identical, 39.5s -> 40.0s for probing every raster rather
than four extensions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

None of these is data loss; two are worth the change.

- The comment replacing ANIMATABLE_EXTENSIONS' rationale was appended
  rather than substituted, so the sentence it replaced survived directly
  above it: "opening every JPEG to learn that would cost an extra pass
  over the whole corpus", immediately followed by "Probed for every
  raster". The stale half is the exact argument that narrowed the probe
  set and flattened every animated WEBP, sitting there ready to persuade
  the next reader. The block is now one accurate paragraph, including
  what the widening really costs (39.5s -> 40.0s, which the previous
  comment called "measurable at neither end") and that .bmp is probed
  despite not being animatable - the price of having no list to get
  wrong.

- An animated raster is copied byte-for-byte: no resize, no re-encode.
  It was logged as "Optimized src -> dst" and counted in stats["raster"],
  so nothing anywhere said that the file ships at its original
  dimensions, past --max-width, into a database bundled into an offline
  app. optimize_raster now returns (path, copied_through) - the shape
  optimize_svg already uses for was_rasterized - and process_file counts
  it in stats["copied"] and logs "Copied unresized (animated)".

- possible_output_names checked `animated is None` only after the
  non-webp early return, so an unprobeable raster claimed a name there
  while optimize_raster refuses it and writes nothing. That held the name
  against other sources and broke the "claims nothing exactly when it
  writes nothing" invariant the written-vs-claimed check reads as a
  guarantee. The check is now above the --webp branch.

- test_every_animated_raster_keeps_its_frames now runs both --webp
  settings; every animation test ran only the webp path, which is not the
  one where the None case was mishandled.

New: the two reporting tests. The copy-through one fails against the
previous commit, printing the "Optimized" line it should not have.

219 + 33 + 189 pass; scripts/ unchanged from base. Real 299-image corpus
byte-for-byte identical, and the raster/copied tallies are unchanged at
223/1 - it holds no animated non-GIF, which is why none of this was
visible there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>