You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Sad Catapella B3][Performance] Misaligned Zarr block reads may decode the same chunk repeatedly #78
Baseline:d7a538e56d41ac1093ba79adcad397c21b3dfd32. Classification: confirmed sub-chunk repeated-slice access pattern; decompression/caching cost is an unmeasured performance hypothesis.
Evidence:_choose_block() lines 1021–1058 permits partial rows; _ranges() makes multiple sub-chunk ranges; _zarr_layout() lines 1338–1385 always enables partial_rows=True; _zarr_plane() lines 1392–1404 executes a separate Zarr array[selection] for each range. On compressed Zarr stores, each sub-chunk selection may trigger another whole-chunk decode absent effective caching. The same blocks are revisited across histogram passes via the pinned normalization brick (compute_block_histogram). Test asserts pixel correctness of sub-chunk reads, not decode counts or throughput.
Impact: Very poor I/O/decompression efficiency when chunk Y is large relative to affordable block rows, especially remote/object stores and robust histogram passes. No measured slowdown is asserted.
Fix: instrument actual store get/decompress calls; compare chunk-aligned reads, cache-by-chunk with explicit memory bound, and rechunking/preprocessing options. Avoid assuming smaller NumPy selection means smaller decompression. Include 1/2/4-D chunk shapes spanning channels and planes; compression=none/lz4/zstd; local SSD and network object store where available; RSS, decompressed bytes, storage requests and wall time. No Numba role for codec/I/O itself.
Activity