You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Normalization Batch 2, commit f784c95e3049483f87466150ffc5764375fa631d. Confirmed precision defect from source inspection and an isolated NumPy probe. No repository pytest run.
Evidence
PooledHistogram.__init__, lines 107–138 combines per-plane int64 counts using np.bincount(inverse, weights=...), which returns float64, then casts rounded values back to int64. Pixel totals are consequently derived from the rounded merged counts, rather than the original exact integer totals.
_cumulative and percentile then operate on these rounded values; WeightedHistogram also relies on float64 weighted accumulation, which is appropriate for genuinely fractional weights but requires a stated precision policy.
One valid logical singleton-bin histogram with counts = np.array([2**53 + 1], dtype=np.int64), and total_pixel_count = valid_pixel_count = 2**53 + 1. np.bincount(np.array([0]), weights=np.array([2**53 + 1], dtype=np.int64)).astype(np.int64)[0] is 2**53 on float64 NumPy: one pixel disappears. PooledHistogram.pixel_count and percentile rank inputs are then wrong. This does not require the histogram itself to be inexact; it is separate from #28 (high-intensity value rounding).
Impact and fix
Silent loss of exact count/provenance in very large or pre-aggregated datasets; count error might affect extreme percentile ranks and record validation. Rare for one actual FOV; important for a library advertising exact integer pooling without a documented limit.
For unweighted PooledHistogram, aggregate int64 counts with integer-safe operations (including explicit overflow detection near 2^63), e.g. ordered unique-key reduction with integer summation rather than float64 bincount weights. Distinguish the weighted API's floating tolerance and range. Assert pixel_count == sum(hist.valid_pixel_count) and counts/tail totals exactly, or explicitly refuse out-of-range counts. Add synthetic >2^53, near-int64-max, multi-histogram overlapping-bin and q=100 boundary tests, plus ordinary fixtures. Any schema-level change needs an explicit decision; no silent cap or float coercion.
Activity