Skip to content

Bin narrow float64 ranges with a bounded fraction - #85

Merged
guijacquemet merged 1 commit into
mainfrom
claude/narrow-float-range
Nov 7, 2026
Merged

guijacquemet merged 1 commit into
mainfrom
claude/narrow-float-range

Conversation

@guijacquemet

Copy link
Copy Markdown
Contributor

Fixes #84

What changed and why

For a very narrow float64 range, hi*0.5 - lo*0.5 rounds a subnormal endpoint (a half range of zero for distinct endpoints) and n / half_range overflows to inf before the integer cast. Finite pixels were mis-binned (or ZeroDivisionError in _bin_index).

  • New float_bin_position (in bin_centres.py) is used by _float_bins and _bin_index, so both share one arithmetic. Where halving is exact and n / half_range is finite (neither endpoint subnormal, half range at least n * 2**-1000) it keeps the existing halved arithmetic. Elsewhere the position is the bounded fraction (value - lo) / (hi - lo) times n; the halves are used only when hi - lo itself overflows.
  • float_bin_centres uses lo + (hi - lo) * (fraction / 2) when hi - lo is finite, the halved/midpoint form otherwise (the Compute float bin centres without overflowing near the float64 limits #76 large-range case still passes).
  • _bin_index clips the value into [lo, hi] first, as its bin is clipped.
  • _float_bins tests lo == hi instead of a zero half range for coinciding robust limits.
  • Code-rule baseline lowered by --update-baseline (two entries shrank).

Deviation from the proposed fix

The review proposed (value - lo) / (hi - lo) whenever hi - lo is finite. I tried that first: in a sweep of 1603 histograms it changed 18 (a pixel exactly on a bin edge moved one bin) because the rounding differs from the existing arithmetic. The requirement is that existing histograms stay bit-identical, so the existing arithmetic is kept where it is exact and the bounded fraction is used only where it is not.

Regression tests (tests/test_histogram.py)

  • test_float_range_of_the_smallest_subnormal_keeps_distinct_bins
  • test_float_narrow_range_does_not_overflow_the_bin_scale
  • test_float_range_between_subnormals_keeps_every_pixel_in_its_bin

On the old code all three FAIL (3 failed, 10 passed for the selected tests); on the new code they pass.

Bit-identity of existing histograms

A sweep of 1603 histograms (float32 and float64; normal, uniform, on-bin-edge, exponential, linspace, NaN-containing planes; 8 to 4096 bins; with and without the robust-range check, bin counts, limits and representatives; plus three ±float64-max ranges) gives byte-identical JSON on origin/main and on this branch (sha256 8f119879...f1f8 for both).

Checks

python tools/check.py: 724 passed, 2 skipped (tests/test_rgb.py:80, csbdeep not installed; not claimed as passed), Ruff and code rules clean, exit 0.


Generated by Claude Code

Halving rounded a subnormal endpoint (half range zero for distinct endpoints) and
n / half_range overflowed for very narrow ranges, mis-binning finite pixels. Use
(value - lo) / (hi - lo) * n there; ordinary ranges keep their exact arithmetic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Narrow float64 histogram ranges mis-bin pixels (halved range rounds to zero, n / half_range overflows)

2 participants