Bucket A (2/2): refuse invalid parameters and channel indices; report unchecked channels

AGENTS.md

@@ -49,7 +49,10 @@ src/morphobricks_normalization/
                     Normalization_Offset_C<i> and Normalization_Scale_C<i>)
   summaries.py      ImageSummary, image_summaries, summary_columns: one row per image,
                     Z and T pooled, from plane_histograms.csv rows (Milestone 4)
-  checks.py         IntensityCheckSettings, intensity_checks: the four intensity warnings
+  checks.py         IntensityCheckSettings, intensity_checks: the four intensity warnings;
+                    unchecked_channels: channels without finite pixels
+  parameter_checks.py  finite offsets, positive scales and whole channel indices that
+                    NormalizationParameters and Parameters check when made
   issues.py         Issue: warnings returned to the calling brick (channel for the checks)
 tests/              pytest with simulated planes (tests/simulation.py); tests/data holds
                     a schema-version-2 fixture written by the Milestone 2 code

README.md

@@ -117,6 +117,8 @@ Milestone 4 added the intensity checks, computed from `plane_histograms.csv` row
   (`Issue.channel`).
 - The limits are an `IntensityCheckSettings` (default 3 robust deviations, 1 % saturated
   pixels, at least 5 images per comparison, provisional).
+- `unchecked_channels(summaries)` names each image channel without finite pixels
+  (`channel_without_finite_pixels`): its summaries are blank, so the checks cannot see it.
 
 ```python
 from morphobricks_normalization import IntensityCheckSettings, image_summaries, intensity_checks

docs/contract.md

@@ -313,7 +313,8 @@ issues = range_check_issues([(plane_id, channel, check), ...])  # §1.4
 | `fit` | One entry point for every method; dispatches on `settings.image_type` and `settings.method`. Statistics are pooled histograms per channel, or pixels; with `pooling="conditions"` the histograms carry their condition (§4.3). Never reads files and never walks folders. |
 | `apply` | Returns a new float32 array; the input is never modified. `channel_axis` gives the channel axis of `pixels`; it may be left out for one channel. No clipping (N19). Non-finite pixels stay non-finite. Computed in float64 for fluorescence (§3.2) and with CSBDeep's float32 arithmetic for `rgb` (§3.3). |
 | `select` | The parameters of some of the image's channels, e.g. of a single-channel patch; `channels` keeps their indices (N46). |
-| `to_columns` / `from_columns` | The columns of §8.1 as text in the format of §8.2; `from_columns(to_columns(p)) == p` exactly. `to_columns` writes the pairs of `channels` only; `from_columns` reads every channel whose pair is filled and skips channels whose pair is blank (N46). |
+| `to_columns` / `from_columns` | The columns of §8.1 as text in the format of §8.2; `from_columns(to_columns(p)) == p` exactly. `to_columns` writes the pairs of `channels` only; `from_columns` reads every channel whose pair is filled and skips channels whose pair is blank (N46). `from_columns` refuses a `normalization_scope` that is not one of §4.1's scopes; it does not compare the row's scope with the settings' (a calling brick may read rows of any scope with the run's method settings). |
+| Validity | `Parameters` and `NormalizationParameters` refuse, when made, an offset that is not finite and a scale that is not finite and above 0, naming the channel or method; `Parameters.channels` and `select` take whole numbers 0 or above only (Python or NumPy integers; booleans, fractions, whole floats and text are refused, never truncated). Non-finite pixels stay non-finite in `apply`. |
 | `to_dict` / `from_dict` | JSON-compatible dictionaries (§8.3) that round-trip exactly. |
 | `image_summaries`, `intensity_checks` | §7. |
 
@@ -559,11 +560,14 @@ library writes no file (N50). The columns are the shared columns `source_relativ
 | `Intensity_MedianIntensity_C<i>` | Median |
 | `Intensity_LowPercentile_C<i>`, `Intensity_HighPercentile_C<i>` | Default the 1st and 99.8th percentiles |
 | `Intensity_SaturatedFraction_C<i>` | Pixels equal to the type's maximum over valid pixels, for integer types whose histogram holds the maximum value in a bin of its own; blank otherwise (float images, or inexact top bins) |
-| `Histogram_Exact_C<i>` | Whether every pooled histogram of the channel was exact |
+| `Histogram_Exact_C<i>` | Whether every pooled histogram of the channel was exact; blank (`None`) for a channel without finite pixels, which has no statistics to be exact |
 
 The checks below call these values `median`, `p_low`, `p_high` and `saturated_fraction`, the
 attributes of `ImageSummary` that hold one value per channel. A channel without finite pixels
-has blank values and is not checked.
+has blank values and is not checked; `unchecked_channels(summaries)` returns one
+`channel_without_finite_pixels` warning per such image and channel (`Issue.channel` set), so
+that a calling brick can show that the channel was not checked rather than healthy. It is
+separate from `intensity_checks`, whose four codes are unchanged.
 
 ### 7.2 Checks
 
@@ -665,7 +669,8 @@ Raised as exceptions before any pixel is normalized; the calling brick blocks th
 | Statistics file of schema version 3 (N56) | `CsvFormatError` | "This plane_histograms.csv was written with schema version 3, without the underflow and overflow columns. Run the preprocessing brick again on this experiment: it rewrites the table with these columns (schema version 4)." |
 | `HistogramSettings` with an unknown range, invalid quantiles or `min_central_bins` below 1 | `ValueError` | Names the value and the allowed ones |
 | Rows or images with different numbers of channels (N43) | `CsvFormatError` (table), `ValueError` (pixels) | Names the images and their channel counts, and says that every image of an experiment must have the same channels |
-| `from_columns` with a missing or unreadable column, or a channel without both its offset and scale | `ValueError` | Names the column |
+| `from_columns` with a missing or unreadable column, an unknown `normalization_scope`, or a channel without both its offset and scale | `ValueError` | Names the column |
+| `Parameters` or `NormalizationParameters` with a non-finite offset, or a scale that is not finite and above 0; a channel index that is not a whole number 0 or above | `ValueError` | Names the channel (or method) and the value |
 
 ### 9.2 Warnings
 
@@ -679,6 +684,7 @@ Returned as `Issue` objects; processing continues and the calling brick reports
 | `planes_without_finite_pixels` | Planes without finite pixels excluded from a group that still has others |
 | `patches_discarded` | Patches without spread or finite pixels, listed in one warning with a suggested scope |
 | The four codes of §7.2 | Intensity checks |
+| `channel_without_finite_pixels` | An image channel without finite pixels, which the intensity checks cannot check (§7.1, `unchecked_channels`) |
 | `narrow_histogram_range` | A channel's central range covers fewer than `min_central_bins` bins in some planes (§1.4); reported by the preprocessing brick only |
 
 ---

src/morphobricks_normalization/__init__.py

@@ -8,7 +8,7 @@
 """
 
 from ._version import __version__
-from .checks import IntensityCheckSettings, intensity_checks
+from .checks import IntensityCheckSettings, intensity_checks, unchecked_channels
 from .csv_io import OUTPUT_FILENAME, read_plane_histograms, write_plane_histograms
 from .dtypes import UnsupportedDtypeError, default_bin_count
 from .fitting import Parameters, fit
@@ -84,6 +84,7 @@
     "robust_z_score",
     "scope_groups",
     "summary_columns",
+    "unchecked_channels",
     "write_plane_histograms",
     "z_score",
 ]

src/morphobricks_normalization/checks.py

@@ -34,6 +34,7 @@
 
 from ._version import __version__
 from .issues import (
+    CHANNEL_WITHOUT_FINITE_PIXELS,
     IMAGE_INTENSITY_OUTLIER,
     NO_INTENSITY_SPREAD,
     REPEAT_INTENSITY_SHIFT,
@@ -63,6 +64,11 @@
     "underestimated. Check the exposure, gain or laser power, and consider excluding the image "
     "or its saturated regions from intensity measurements."
 )
+_UNASSESSED_ACTION = (
+    "Check the image: this channel is empty or holds only NaN or infinite values, so it was not "
+    "checked and cannot be normalized from its own pixels. Exclude the image, or fix the channel "
+    "in the source data and run the preprocessing brick again."
+)
 _SPREAD_ACTION = (
     "Check whether the channel is empty or the image is blank. Exclude the image, or normalize "
     "it at the biological-repeat scope, which pools it with the other images of its repeat."
@@ -180,6 +186,43 @@ def intensity_checks(
     ]
 
 
+def unchecked_channels(summaries: Iterable[ImageSummary]) -> list[Issue]:
+    """Return a ``channel_without_finite_pixels`` warning per image channel that was not checked.
+
+    A channel without finite pixels (all NaN or infinite) has blank summaries,
+    so :func:`intensity_checks` cannot check it and stays silent about it; this
+    names it instead, so that silence is not read as a healthy channel (issue
+    #57). It is separate from :func:`intensity_checks`, whose four codes the
+    calling bricks list.
+
+    Parameters
+    ----------
+    summaries
+        One summary per image, from
+        :func:`~morphobricks_normalization.summaries.image_summaries`.
+
+    Returns
+    -------
+    list of Issue
+        One per image and channel without finite pixels, sorted by condition,
+        biological repeat, image and channel; ``Issue.channel`` is set.
+    """
+    return [
+        Issue(
+            WARNING,
+            CHANNEL_WITHOUT_FINITE_PIXELS,
+            f"Image {summary.source_relative_path}, channel C{channel}: it has no finite pixels "
+            "(every value is NaN or infinite), so its intensities were not checked.",
+            items=(summary.source_relative_path,),
+            action=_UNASSESSED_ACTION,
+            channel=channel,
+        )
+        for summary in sorted(summaries, key=_order)
+        for channel, median in enumerate(summary.median)
+        if median is None
+    ]
+
+
 def _image_outliers(summaries: list[ImageSummary], settings: IntensityCheckSettings) -> list[Issue]:
     groups: dict[tuple[str, str], list[ImageSummary]] = {}
     for summary in summaries:

src/morphobricks_normalization/fitting.py

@@ -37,10 +37,11 @@
     robust_z_score,
     z_score,
 )
+from .parameter_checks import channel_indices, check_offset_scale
 from .pooling import NoValidPixelsError, PooledHistogram, WeightedHistogram
 from .records import PlaneRecord, different_channel_counts
 from .rgb import RGB_CHANNEL_COUNT, apply_rgb, check_rgb, rgb_percentile
-from .settings import NormalizationSettings, check_settings
+from .settings import SCOPES, NormalizationSettings, check_settings
 
 # Columns written once per row; the per-channel pairs follow (offset_column,
 # scale_column).
@@ -81,11 +82,13 @@ class Parameters:
     exact
         Whether every pooled histogram was exact.
     offset, scale
-        One value per channel, in the order of ``channels``.
+        One value per channel, in the order of ``channels``: finite offsets,
+        finite scales above zero.
     channels
         The image's channel indices the parameters belong to, in increasing
         order; by default ``0, 1, ...``. A single-channel patch of channel 2 of
-        an image has ``channels == (2,)`` (contract §8.1).
+        an image has ``channels == (2,)`` (contract §8.1). Whole numbers only:
+        booleans, fractions and text are refused, never truncated.
     issues
         Warnings about these parameters (``approximate_parameters``,
         ``planes_without_finite_pixels``); not stored in the columns and not
@@ -110,14 +113,16 @@ def __post_init__(self) -> None:
                 f"offset(s) and {len(self.scale)} scale(s)."
             )
         channels = range(len(self.offset)) if self.channels is None else self.channels
-        channels = tuple(int(c) for c in channels)
+        channels = channel_indices(channels)
         increasing = all(b > a for a, b in zip((-1, *channels), channels, strict=False))
         if len(channels) != len(self.offset) or not increasing:
             raise ValueError(
                 "Expected one channel index per offset, from 0 upwards in increasing order; "
                 f"got channels {channels} for {len(self.offset)} offset(s)."
             )
         object.__setattr__(self, "channels", channels)
+        for channel, offset, scale in zip(channels, self.offset, self.scale, strict=True):
+            check_offset_scale(offset, scale, f"Channel {channel}")
 
     @property
     def channel_count(self) -> int:
@@ -139,9 +144,9 @@ def select(self, channels: Iterable[int]) -> Parameters:
         Raises
         ------
         ValueError
-            If a channel is not among :attr:`channels`.
+            If a channel is not among :attr:`channels`, or is not a whole number.
         """
-        wanted = sorted({int(c) for c in channels})
+        wanted = sorted(set(channel_indices(channels)))
         missing = [c for c in wanted if c not in self.channels]
         if not wanted:
             raise ValueError("Select at least one channel of these parameters.")
@@ -242,14 +247,15 @@ def from_columns(
             its own channel).
         settings
             The settings of the run that wrote the row; its method must be the
-            row's ``normalization_method``.
+            row's ``normalization_method``. The row's ``normalization_scope``
+            must be one of the scopes of contract §4.1.
 
         Raises
         ------
         ValueError
-            Naming the column that is missing or cannot be read, or whose
-            method differs from the settings; also for a channel with only one
-            of its two columns.
+            Naming the column that is missing or cannot be read, whose method
+            differs from the settings, or whose scope is unknown; also for a
+            channel with only one of its two columns.
         """
         missing = [name for name in COLUMNS if name not in columns]
         if missing:
@@ -302,7 +308,7 @@ def from_columns(
             _check_rgb_columns(channels, offset, scale)
         return cls(
             method=method,
-            scope=_text(columns["normalization_scope"]),
+            scope=_scope(columns),
             group=_text(columns["normalization_group"]) or None,
             exact=_flag(columns, "normalization_exact"),
             offset=tuple(offset),
@@ -561,6 +567,16 @@ def _text(value: Any) -> str:
     return str(value)
 
 
+def _scope(columns: Mapping[str, Any]) -> str:
+    scope = _text(columns["normalization_scope"])
+    if scope not in SCOPES:
+        raise ValueError(
+            f"Column normalization_scope is {scope!r}, which is not a scope; the scopes are "
+            f"{', '.join(SCOPES)}. Read the parameters from a row written with normalization."
+        )
+    return scope
+
+
 def _number(columns: Mapping[str, Any], name: str) -> float:
     text = _text(columns[name])
     try:

src/morphobricks_normalization/issues.py

@@ -22,6 +22,8 @@
 REPEAT_INTENSITY_SHIFT = "repeat_intensity_shift"
 SATURATED_IMAGE = "saturated_image"
 NO_INTENSITY_SPREAD = "no_intensity_spread"
+# Channels whose intensities could not be checked (issue #57); not among the four of §7.2.
+CHANNEL_WITHOUT_FINITE_PIXELS = "channel_without_finite_pixels"
 
 # Histogram range check of contract §1.4, Milestone 5; reported by the preprocessing brick.
 NARROW_HISTOGRAM_RANGE = "narrow_histogram_range"

src/morphobricks_normalization/normalization.py

@@ -24,6 +24,7 @@
 
 import numpy as np
 
+from .parameter_checks import check_offset_scale
 from .pooling import PooledHistogram
 
 # Makes the MAD a consistent estimator of the standard deviation for normal data.
@@ -32,12 +33,19 @@
 
 @dataclass(frozen=True)
 class NormalizationParameters:
-    """How to normalize pixel values: ``(pixels - offset) / scale``."""
+    """How to normalize pixel values: ``(pixels - offset) / scale``.
+
+    The offset must be finite and the scale finite and above zero; other values
+    are refused with ``ValueError``.
+    """
 
     method: str
     offset: float
     scale: float
 
+    def __post_init__(self) -> None:
+        check_offset_scale(float(self.offset), float(self.scale), f"The {self.method} method")
+
     def apply(
         self,
         pixels: np.ndarray,

src/morphobricks_normalization/parameter_checks.py

@@ -0,0 +1,70 @@
+"""Checks of normalization parameters: offsets, scales and channel indices (contract §8).
+
+:class:`~morphobricks_normalization.normalization.NormalizationParameters` and
+:class:`~morphobricks_normalization.fitting.Parameters` check their values when
+they are made, so parameters fitted here, read from columns or built by a
+calling brick follow the same rules. A value that breaks one is refused with a
+``ValueError``; nothing is rounded, truncated or replaced.
+"""
+
+from __future__ import annotations
+
+import math
+import numbers
+from collections.abc import Iterable
+
+import numpy as np
+
+
+def check_offset_scale(offset: float, scale: float, where: str) -> None:
+    """Refuse an offset that is not finite, or a scale that is not finite and above zero.
+
+    Parameters
+    ----------
+    offset, scale
+        The values of one channel.
+    where
+        Names the channel or method in the message, e.g. ``"Channel 2"``.
+
+    Raises
+    ------
+    ValueError
+        Naming ``where``, the value and what to do.
+    """
+    if not math.isfinite(offset):
+        raise ValueError(
+            f"{where}: the normalization offset must be a finite number; got {offset!r}. Fit the "
+            "parameters again, or correct the stored offset."
+        )
+    if not (math.isfinite(scale) and scale > 0):
+        raise ValueError(
+            f"{where}: the normalization scale must be a finite number above 0; got {scale!r}. "
+            "Fit the parameters again, or correct the stored scale."
+        )
+
+
+def channel_indices(channels: Iterable[object]) -> tuple[int, ...]:
+    """Return channel indices as ints, refusing anything that is not a whole number 0 or above.
+
+    Python and NumPy integers are accepted; booleans, fractions (even whole
+    floats such as ``1.0``), text and negative numbers are refused rather than
+    rounded, truncated or parsed.
+
+    Raises
+    ------
+    ValueError
+        Naming the value that is not a channel index.
+    """
+    indices = []
+    for channel in channels:
+        if (
+            isinstance(channel, bool | np.bool_)
+            or not isinstance(channel, numbers.Integral)
+            or channel < 0
+        ):
+            raise ValueError(
+                f"A channel index is a whole number counted from 0; got {channel!r}. Give the "
+                "channels as integers (0 for the first channel)."
+            )
+        indices.append(int(channel))
+    return tuple(indices)