ADFA-5153: Migration script for the shared Brotli dictionary

[davidschachterADFA]

Stacked on #1677 — this PR's base is feature/ADFA-5153-content-brotli-dictionary, so its diff is just the three commits below. GitHub retargets it to stage once #1677 merges.

A maintenance script that migrates an existing documentation.db onto the shared Brotli dictionary #1677 teaches WebServer to read. Nothing here ships in the APK; the only production file touched is a Spotless exclusion.

Three phases, in this order

Each phase changes what the next one sees, so the order is load-bearing.

Phase	What it repairs	Effect on the 20-Aug database
retype	74 rows hold GIF/PNG/JPEG/QuickTime payloads but are typed text/plain (ADFA-5221), so they are Brotli-compressed for no gain and served as Content-Type: text/plain. Stores their plaintext and points them at the type their magic bytes prove.	61 → image/gif, 7 → image/png, 4 → video/quicktime, 2 → image/jpeg
renumber	14 of the 19 chunked items number continuations from -2 while the reassembly loop starts at -1 (ADFA-5171), so they serve as their first 1 MiB and nothing more. Shifts them down.	14 items renumbered from -1
migrate	Recompresses every ContentTypes.compression = 'brotli' row against the database's own CompressionDictionary.	29,515 of 29,677 items, 43.4 MiB saved (34.2%)

Phase 1 feeds phase 3 for free: a row retyped to image/gif inherits that type's compression = 'none', so phase 3's compression = 'brotli' selection stops seeing it. No exclusion list needed.

Why it is safe to run incrementally

WebServer tries a dictionary-attached decode first and falls back to a plain one, so a half-migrated database still serves every row. Classification deliberately tries the plain decode first: attaching no dictionary to a stream that needs one reliably fails, so a successful plain decode proves a row is not yet migrated. The reverse test is unsafe — a dictionary attached to a stream that never used one can decode to different bytes without erroring.

Rows over 1 MiB are raw slices of one stream, not independently compressed pieces, so the unit of work is a logical item (base row plus continuations) concatenated, decoded, rewritten and re-split. Migrating such rows one at a time would destroy the content.

Verified on a copy of the 20-Aug database

• 74/74 retyped rows byte-identical to the original decompressed plaintext.
• The chunked .mov reassembles from base + -1 to the same 1,357,576 bytes; all 19 chunked items reassemble to unchanged bytes.
• 250/250 sampled rows decode with the dictionary to identical content, including a 6 MB SVG and a 23 MB HTML index.
• Exactly 74 contentTypeID changes and 14 renames, nothing else. Content 30,649 → 30,649 rows, Bookshelf 7 → 7 (the .pdf AddBook/DeleteBook triggers never fire on continuation paths), PRAGMA integrity_check ok, no foreign-key violations.
• Idempotent: a second run reports 0 retype candidates and all 19 chunked items already at -1.
• 3.5 min at 20 workers (~73 min single-threaded); 313.8 → 268.1 MB after VACUUM.

Two judgement calls, both flags

• --mov-type quicktime (default) inserts an honest video/quicktime ContentTypes row. All four .mov files are genuine ftypqt QuickTime, which Chromium's demuxer generally will not play — so a correct type may still leave them blank. --mov-type mp4 labels them video/mp4 instead, which might coax playback. The real fix is transcoding in docdb-studio.
• --only-if-smaller stays off. Dictionary compression grows 9,098 rows by a median of 25 bytes — 257 KiB against 43.6 MiB saved — and turning it on would leave those rows plain and re-attempted on every future run.

Both data defects originate in docdb-studio's import path, so a freshly exported database carries them again until fixed there; ADFA-5221 and ADFA-5171 track that.

Notes for review

• build.gradle.kts excludes **/*.py from the Spotless shell block, which was reindenting Python to tabs.
• docs/documentation-database.md gains a paragraph on both data defects and which phase repairs each.
• No UI, so no font-scale check applies.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[davidschachterADFA]

Superseded by #1724. This PR was auto-closed when #1677 was squash-merged and its base branch feature/ADFA-5153-content-brotli-dictionary was deleted — nothing was wrong with the content. The same four commits are rebased onto the new stage in #1724, where the diff is just this branch's own work rather than the 19 commits #1677 carried.