Skip to content

[Sad Catapella B3][Scientific correctness] OME-Zarr pyramid mean corrupts uint64/int64 precision #75

Closed as not planned

Description

@guijacquemet

Baseline: d7a538e56d41ac1093ba79adcad397c21b3dfd32. Classification: confirmed arithmetic precision loss by source inspection; no test/benchmark executed.

Evidence: downsample() lines 268–280 first converts every pixel to float64, then averages and casts back. IEEE float64 cannot exactly represent every integer beyond 2^53. OmeZarrWriter.write() and _write_blocks() use this function for all pyramid levels; the histogram/source level 0 is unaffected. Tests only use small uint8 values, and block/whole parity uses uint16/int32/float32.

Analytic counterexample: 2×2 uint64 array with all entries 2**53+1: their exact mean is 2**53+1. The float64 conversion rounds every entry to 2**53, and the overview pixel therefore differs by 1. Values close to uint64 maximum also risk inaccurate clipping/casting, because that limit itself is not exactly representable as float64. This is a scientific copy/overview error, not a claim that raw level-0 pixels or histograms are wrong.

Fix: implement an integer-safe 2×2 rounded mean for 64-bit integer types without float64 loss or sum overflow, with explicit tie/rounding policy matching existing np.rint (ties-to-even). Test 0, signed extreme, uint64 maximum, values >2^53, odd image dimensions and both whole-plane and blocked export. Preserve cross-format agreement. Consider a Numba fused kernel only after exact arithmetic is defined; benchmark against NumPy on uint8/16/32/64 and float images, reporting cold JIT and warmed throughput.

Activity

  1. guijacquemet commented on Nov 2, 2026

    @guijacquemet
    ContributorAuthor

    Obsolete in v0.4.0: convert_tiffs, zarr_export and OME-Zarr copy reuse were removed in PP-2 (#66, commit 3c2802d; plan question 46). Named-axis copies of images with axis errors are now the OME-TIFF convert step (PP-11, #124, merged as 8db1f4a). Closing as not planned (owner-approved triage, 2026-11-02).

    PP-11 does no pixel arithmetic on copies (planes are copied bit-for-bit) and refuses int64/uint64 images, which OME-TIFF cannot hold.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions