[guijacquemet]

Numerical-correctness batch against main c952263 (v0.4.0). There is one commit per issue. Each issue was reproduced on main with a failing test before it was fixed.

Fixes #28, fixes #33, fixes #41.

#27: not reproduced, no change

Float histogram binning of near-zero and subnormal ranges is already correct on main, through #84/#85 (float_bin_position uses a bounded fraction where halving would underflow or n / half_range would overflow) and #86. Evidence on c952263:

• [0, 1e-308] with 100 bins puts the values in bins 0 and 99.
• [0, 5e-324] with 2 and 100 bins puts them in the first and last bin.
• [-1e-300, 1e-300] with 64 bins puts them in bins 0 and 63.
• A constant 1e-300 plane puts everything in bin 0.
• Robust plus range-check histograms of 20,000 values at scales 1e-308, 1e-300 and 1e-320, read in two blocks, match an exact Fraction oracle (floor((v - lo) / (hi - lo) * n)) bin for bin, with 0 differences.
• The existing tests test_float_range_of_the_smallest_subnormal_keeps_distinct_bins, test_float_narrow_range_does_not_overflow_the_bin_scale and test_float_range_between_subnormals_keeps_every_pixel_in_its_bin cover the issue's cases, including _bin_index.

#27 can be closed as fixed by #85/#87.

#28: 64-bit integers that float64 cannot hold (5551e2e)

Problem. [2**53 + 1] (uint64) was reported is_exact=True with max_representative_error == 0, although its representative is 2**53. A second problem: representatives were rounded twice (binning_min, then the sum). For example, the bin holding only 2**53 + 2, which float64 can represent, got the representative 2**53.

Fix. The fix is in the new pure module integer_representatives.py. It lives there because histogram.py is at its shrink-only line baseline, which is now 861.

• Representatives: each one is (first + last) / 2, divided once as Python integers, so it is the float64 nearest the midpoint. Below 2**51 this gives exactly the old value.
• Error bound: for a plane with a value of magnitude 2**51 or more, max_representative_error is the exact largest distance between each bin's (and tail's) integers and the float64 that stands in for them, rounded up. A histogram is therefore exact only when every representative equals its value.
• What does not change: counts and bin limits never change, and the schema stays the same. This applies the "counts/partition exact, float64 summaries with honest bounds" option from the issue. normalization_exact and Histogram_Exact_C<i> become false for such planes.

Outputs that move (intended). Only int64 and uint64 lines with values of magnitude 2**51 or more. Every moved histogram line was checked against an independent oracle. It recomputes the bin limits with its own ceil formula, takes the nearest float to each exact Fraction midpoint, and computes the exact bound rounded up. All lines match it, and counts, extrema and tails are unchanged on every line.

tools/dump_outputs.py (8c2d3895… → 1723cafb…), 186 of 1788 lines:

Lines	Change
int64-wide, uint64-wide histograms (48 lines)	Representatives move by at most 1536 (int64) and 2048 (uint64), out of values around 9e18, i.e. 1-2 ulp. The bound changes in its last bits and is now a true upper bound.
int64-constant, uint64-constant (48 lines)	Error goes from 0 to 170 and 341, exact goes from True to False. The values 3074457345618258602 and 6148914691236517205 are not float64 values; their float64 forms differ from them by exactly 170 and 341.
Pooled, summary and fit lines of those types	They follow from the moved representatives, and the summaries' Histogram_Exact_C0 goes from True to False for the constant images. Fitted int64 offsets and scales move by about 1e-13 relative. Example: the zscore image offset goes from 9618469625711740 to 9618469625711942.

Is the new mean more accurate? The exact model mean (every pixel at its exact bin midpoint, computed with Fraction) is 9618469625711938. The pooled mean is now 4.7 away from it; before it was 197.3 away. uint64 means are unchanged.

--corpus (a4c37fdf… → 8bb7c3c1…), 90 of 4764 lines:

Corpus planes	Change
int64 [-2**63, -2**63+1]	Error goes from 0 or 0.5 to 1, exact from True to False.
int64 [2**63-2, 2**63-1] and uint64 [2**64-2, 2**64-1]	Error goes from 0 or 0.5 to 2: 2**63 - 2 and 2**64 - 2 are stored as 2**63 and 2**64.
Full-range int64	Representatives move by at most 1024, and the bound changes in its last bits.
Full-range uint64	The bound changes in its last bits only.

The 400 golden fits are bit-identical: they contain no 64-bit values of magnitude 2**51 or more.

#33: standard deviations of large finite values (47e5339)

Problem.

• PooledHistogram.std and WeightedHistogram.std returned inf for [-1e200, 1e200], 1e200 ± 1e185 and ±1e308, so zscore was refused even though the true spread is finite.
• A tail of values around 1e200 raised a bare OverflowError from float ** 2.

Fix.

• Both now go through _std_with_tails. The plain sum is tried first, so every finite result is bit-identical to before.
• When the plain sum is not finite but the mean and every value are, the sum is redone with all values divided by a power of two near the largest (exact), and the result is multiplied back.
• A tail whose stored Histogram_*SumOfSquares is infinite is refused by name, e.g. "tail 2 of 2 has sum of squares inf … use percentile or robust z-score, or the min-max range", because its spread cannot be known. Mean, percentiles, median and MAD stay available.

Tests. They compare against an exact Fraction/Decimal oracle for ±1e200, 1e200 ± 1e185, ±1e308, constant 1e200, weighted parts and robust tails near 1e153. They also cover a CSV round trip of an infinite tail through to the fit refusal.

Outputs that move: none. Dump and corpus digests are identical to the #28 commit, and the goldens are bit-identical.

Not fixed here (needs an owner decision and a new schema version). Schema v4 stores raw tail sums of squares, so:

• A tail of large values with a small spread still loses precision through cancellation in s2 - s * mean.
• Squares that overflowed when the file was written cannot be recovered.

A centred or scaled moment representation is a file-format change (AGENTS §2.6). It may belong with the v5 schema work in #90.

Also seen, not in scope (follow-up). The pooled mean can still overflow when count * value exceeds the float64 maximum, e.g. two pixels of 1e308 with weights. A non-finite mean leaves the std non-finite and zscore refused with its existing message.

#41: missing conditions per time point (c084a35)

Problem. With biological_repeat and per_time_point=True, the rows (control, r1, t=0) and (treated, r1, t=1) produced the fitted groups r1::t=0 and r1::t=1, each with one condition, and no repeat_missing_conditions warning.

Fix.

• The check now runs on the groups that scope_groups returns, i.e. the same keys as the fit, and names them with group_name (e.g. r1::t=0).
• The expected set is unchanged: every condition of the statistics (contract §4.2).
• Without per_time_point the items and the message are unchanged.

Tests. The controls are: complete same-time groups, a condition missing a whole time point in every repeat, renamed pairs, pooled time, and the image scope.

Outputs that move: none.

Gate

• python tools/check.py (Python 3.13, Linux) runs ruff format, pytest and the code rules. Pytest: 1579 passed, 2 skipped. Both skips are tests/test_rgb.py:80 (CSBDeep not installed locally; the CSBDeep CI job runs them).
• The code-rules baseline only shrank:
  • histogram.py lines: 862 → 861
  • the _incomplete_repeat_groups docstring entry was removed

Overlap with #90 (claude/stage3-v5-histograms)

#90 does not touch histogram.py or scopes.py. Both PRs edit these files:

• pooling.py: here only std and _std_with_tails
• pyproject.toml: one added layer line
• tools/code_rules_baseline.json: two lines
• CHANGELOG.md: a new "Unreleased" section
• docs/contract.md: §1.2.2, §1.2.3, §1.4 and §4.2
• AGENTS.md: the module map
• tests/test_scopes.py and tests/test_pooling.py: tests appended at the end

Small textual conflicts are possible in the docs, CHANGELOG and baseline.

---

Generated by Claude Code