[Sad Catapella N4][Scientific integrity] Duplicate plane_id rows are accepted and silently double-counted in normalization #43

Audit basis

Normalization Batch 4, commit f784c95e3049483f87466150ffc5764375fa631d. Confirmed missing uniqueness enforcement, with deterministic counterexample; no package test execution.

Evidence

• read_plane_histograms() lines 190–200 appends each syntactically valid PlaneRecord without tracking seen plane_id values. _record() lines 308–330 verifies a row's derived plane_id equals its own metadata, not uniqueness across rows. This is distinct from the previously closed read_plane_histograms ignores the plane_id, image_file and bit_depth columns #18 about mismatched derived columns.
• scope_groups() lines 115–126 appends all records, including duplicates. fit() and _records() accept arbitrary lists of PlaneRecord objects without checking duplicated plane identity.
• tests/test_csv_io.py lines 156–162 verify plane_id formatting; no duplicate-row regression is present.

Counterexample

Construct one source plane A and another plane B with different histograms but the same group; write [A, B, B] as the CSV or pass them to scope_groups/fit. Every row is individually valid. Repeating B weights its pixels twice, so pooled mean, percentiles and normalization scales can change although the physical source had only two planes. In the case of an identical duplicate row, the table is still scientifically invalid even if some normalized parameters are unchanged.

Fix / acceptance

Enforce unique plane_id within a statistics table/read and within each public grouping/fit entry path at a clearly specified trust boundary; report source path, z/t and first/duplicate row indexes. For a streaming reader, keep bounded identity tracking (O(number of planes) keys) and reject prior to parameter publication; do not silently deduplicate since that changes the user's data. Include exact duplicates, conflicting duplicate histograms, same image at different t/z (valid), same basename in different folders (valid), and multiple channels in one XYC row (valid). Document ownership with preprocessing's run manifest and do not imply an input tamper can always be detected by this library alone. Benchmark N=1k–100k records; no Numba needed.