Repository navigation
Bin narrow float64 ranges with a bounded fraction - #85
Merged
Merged
Conversation
Halving rounded a subnormal endpoint (half range zero for distinct endpoints) and n / half_range overflowed for very narrow ranges, mis-binning finite pixels. Use (value - lo) / (hi - lo) * n there; ordinary ranges keep their exact arithmetic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
This was referenced Nov 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #84
What changed and why
For a very narrow float64 range,
hi*0.5 - lo*0.5rounds a subnormal endpoint (a half range of zero for distinct endpoints) andn / half_rangeoverflows to inf before the integer cast. Finite pixels were mis-binned (orZeroDivisionErrorin_bin_index).float_bin_position(inbin_centres.py) is used by_float_binsand_bin_index, so both share one arithmetic. Where halving is exact andn / half_rangeis finite (neither endpoint subnormal, half range at leastn * 2**-1000) it keeps the existing halved arithmetic. Elsewhere the position is the bounded fraction(value - lo) / (hi - lo)timesn; the halves are used only whenhi - loitself overflows.float_bin_centresuseslo + (hi - lo) * (fraction / 2)whenhi - lois finite, the halved/midpoint form otherwise (the Compute float bin centres without overflowing near the float64 limits #76 large-range case still passes)._bin_indexclips the value into[lo, hi]first, as its bin is clipped._float_binstestslo == hiinstead of a zero half range for coinciding robust limits.--update-baseline(two entries shrank).Deviation from the proposed fix
The review proposed
(value - lo) / (hi - lo)wheneverhi - lois finite. I tried that first: in a sweep of 1603 histograms it changed 18 (a pixel exactly on a bin edge moved one bin) because the rounding differs from the existing arithmetic. The requirement is that existing histograms stay bit-identical, so the existing arithmetic is kept where it is exact and the bounded fraction is used only where it is not.Regression tests (tests/test_histogram.py)
test_float_range_of_the_smallest_subnormal_keeps_distinct_binstest_float_narrow_range_does_not_overflow_the_bin_scaletest_float_range_between_subnormals_keeps_every_pixel_in_its_binOn the old code all three FAIL (
3 failed, 10 passedfor the selected tests); on the new code they pass.Bit-identity of existing histograms
A sweep of 1603 histograms (float32 and float64; normal, uniform, on-bin-edge, exponential, linspace, NaN-containing planes; 8 to 4096 bins; with and without the robust-range check, bin counts, limits and representatives; plus three ±float64-max ranges) gives byte-identical JSON on
origin/mainand on this branch (sha2568f119879...f1f8for both).Checks
python tools/check.py: 724 passed, 2 skipped (tests/test_rgb.py:80,csbdeepnot installed; not claimed as passed), Ruff and code rules clean, exit 0.Generated by Claude Code