Skip to content

[Sad Catapella N7][Scientific QC] Intensity outlier comparison accepts incompatible image percentile definitions #55

Closed

Description

@guijacquemet

Baseline and classification

Normalization Batch 7, commit f784c95e3049483f87466150ffc5764375fa631d. Confirmed missing QC-comparability validation, affecting the standalone public intensity_checks API. No package tests executed.

Evidence

  • ImageSummary.percentile and percentile values explicitly determine the meaning of each p_high; image_summaries() produces one consistent percentile pair if called once, but the public QC function accepts arbitrary Iterable[ImageSummary].
  • intensity_checks() lines 153–180 sorts and compares summaries without verifying their percentile values match.
  • _image_outliers() lines 188–211 labels the comparison as the first member's high percentile, then computes an outlier statistic using all members' p_high values. A 95th percentile and a 99.8th percentile are different measurements and must not silently enter the same reference. Tests use manually constructed summaries but all with (1.0,99.8).

Minimal case

Generate 6 summary objects in one condition and repeat from identically distributed source pixels; store five image summaries with percentile=(1,95) and one with percentile=(1,99.8). intensity_checks treats these values as the same high-percentile statistic; it may flag the 99.8th image as an outlier even if no image-level technical change exists. If the 99.8th image sorts first, its percentile name also mislabels the 95th comparison for the rest. The exact flag depends on distribution and zero-MAD policy (#22).

Fix/acceptance

Validate the same percentile pair for all summaries in each comparison group (preferably globally, unless explicitly supporting separate cohorts); fail fast with paths, expected and observed thresholds. Do not reinterpret values or silently group by rounded threshold names. Test 95 vs 99.8, same thresholds, different low threshold, mixed groups, order independence, and direct summary inputs. If separate percentile cohorts are supported, prevent cross-cohort comparisons and report insufficient reference sizes. Benchmark negligible; Numba is inappropriate.

Activity

  1. added a commit that references this issue on Nov 7, 2026
    a822872
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions