[Sad Catapella B5][Scientific integrity] Reused TIFF-converted Zarr copies are not validated against their stored pixels #87

Baseline / classification

Adversarial audit Batch 5 at d7a538e56d41ac1093ba79adcad397c21b3dfd32. Confirmed missing verification in copy-reuse predicate; a missing Zarr chunk can be interpreted as fill value rather than a read error. End-to-end fixture not run.

Evidence

• run_experiment() unchanged-copy predicate checks only whether the copy folder exists, conversion settings match, axis units match and channel labels match. It never checks level-0 shape, dtype, chunks, chunk presence/content or a copy integrity digest.
• _load_previous() validates the CSV hash and source TIFF signatures, not the converted Zarr chunks.
• OmeZarrWriter._open() creates Zarr arrays with a fill value of zero (see _create_array()). A deleted, previously nonzero chunk can therefore read as zeros from a sparse Zarr array rather than failing loudly.
• Tests check that deleting the entire copy directory triggers re-conversion, not an interior 0/ chunk.

Reproducer to implement

Convert a TIFF with a nonzero level-0 chunk. Save table/manifest. Delete or overwrite one level-0 Zarr chunk while leaving .zattrs, .zarray and the source TIFF unchanged. Rerun with convert_tiffs=True. The current predicate can treat the source as unchanged and reuse its prior histogram; the reused copy's pixel values no longer match the histogram. Validate also truncation, dtype/shape edits and changes to non-level-0 pyramids.

Impact

A manifest and histogram table can claim to describe the exported data while that persisted exported data has been changed or lost. Downstream consumers of converted copies can silently analyze zero-filled/mismatching pixels.

Fix

Record and validate copy integrity (versioned per-chunk hashes or a trustworthy immutable manifest/index, including metadata), or rebuild/verify copies that cannot be trusted; the chosen trade-off should be documented and benchmarked for large stores. Ensure successful reuse implies level-0 data exactly matches recorded source histograms. Do not silently interpret missing expected chunk payload as a real zero image. Test corrupt/missing chunks, changed metadata, intact unchanged copies and rollback. No Numba benefit for I/O verification; hashing codecs may already be accelerated.