You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Sad Catapella B3][Performance] TIFF-to-Zarr conversion rereads the converted pixels to compute histograms #79
Baseline:d7a538e56d41ac1093ba79adcad397c21b3dfd32. Classification: confirmed redundant I/O by design; benefit of a fused pass is a performance hypothesis.
Evidence:_CopyStaging.convert() lines 885–897 reads TIFF planes and writes Zarr. run_experiment() lines 600–610 then invokes _read_file() on the new Zarr copy for histogram calculation. Existing test asserts each TIFF is read once for conversion and the Zarr copy is read again for histograms: intended behavior. It should not be altered without preserving the invariant that the table describes the persisted converted pixels.
Optimization plan: investigate a fused, chunk-aware stream that feeds level-0 writer and histogram accumulator from the same canonical pixel blocks, with a post-write integrity check to ensure copied data and histograms match. Consider direct concurrent accumulation and conversion only if retry/rollback, channel-ordering, clipping, endianness and range-check multi-pass behavior can remain exactly equivalent. For 32/64-bit and float robust range, extra quantile passes are still necessary unless spooling/compact intermediate state is used. Benchmark source TIFF reads, Zarr reads/decode calls, disk traffic, wall time, peak RSS and end-to-end CSV equivalence. Do not add Numba to the I/O orchestrator.
Obsolete in v0.4.0: convert_tiffs, zarr_export and OME-Zarr copy reuse were removed in PP-2 (#66, commit 3c2802d; plan question 46). Named-axis copies of images with axis errors are now the OME-TIFF convert step (PP-11, #124, merged as 8db1f4a). Closing as not planned (owner-approved triage, 2026-11-02).
After conversion the run reads the copies themselves and recomputes every row (design §5).
Activity
guijacquemet commentedon Nov 2, 2026
Obsolete in v0.4.0:
convert_tiffs,zarr_exportand OME-Zarr copy reuse were removed in PP-2 (#66, commit 3c2802d; plan question 46). Named-axis copies of images with axis errors are now the OME-TIFFconvertstep (PP-11, #124, merged as 8db1f4a). Closing as not planned (owner-approved triage, 2026-11-02).After conversion the run reads the copies themselves and recomputes every row (design §5).
Generated by Claude Code