PP-8b: refuse X-before-Y masks in the run; RunInputs names unrecorded files

AGENTS.md

@@ -99,6 +99,9 @@ src/morphobricks_preprocessing/
   pixel_size_checks.py the §11 pixel-size checks on resolved sizes (moved from metadata.py)
   narrow_ranges.py  NarrowRange (moved from manifest.py)
   sources.py        check_sources (moved from manifest.py in PP-4)
+  unrecorded.py     unrecorded_files: images and masks the run neither recorded nor excluded, by
+                    preprocessing's discovery and pairing; RunInputs.open raises
+                    UnrecordedFilesError for them (PP-8b, contract §9.1)
   mask_record.py    the manifest's masks entry with fingerprints and roles (manifest v2)
   axis_check.py     the axis check (contract §2): header_axis_issue, axis_disagreements,
                     likely_axis_cause, check_header_axes, ConfirmedAxes (a confirmation covers

docs/plan.md

@@ -287,6 +287,9 @@ API, CLI). The notebook and CLI only call the API.
   *v0.4.0 (PP-8, question 49): stated calibration in the output folder's `calibration.csv`
   (experiment, folder or image; a size or `pixels`), recorded per axis with its source;
   the pixel-size checks run on the resolved sizes; CLI `calibrate` (contract §9.3).*
+  *v0.4.0 (PP-8b, questions 50 and 51): a mask whose header names X before Y is refused by
+  the run (`mask_layout`); `RunInputs.open` refuses an experiment holding images or masks
+  the run did not record (`UnrecordedFilesError`, contract §9.1).*
 - **Milestone 5 — Integration and docs:** public API, notebook, CLI, user documentation.
   *Implemented: summaries (`report`), notebook widgets (`notebook_ui`,
   `notebooks/MorphoBricks_Preprocessing.ipynb`), `check`/`run` command line (`cli`) and the
@@ -1122,3 +1125,36 @@ Raised during review; to be decided before the milestones that need them:
     (owner, 1 November 2026). The manifest records `calibration_sha256`, checked by
     `RunInputs`; a changed file is an input change that recomputes no row. Y and X are
     stated per role and never exchanged. Contract §9.3.*
+50. What happens to a mask whose own header names X before Y (PP-8b, from CellPatcher PR #69
+    question 2)?
+    *Decided (2 November 2026, owner): the run refuses it. It is the existing `mask_layout`
+    error (no new code), naming the mask with its header axes and the `numpy.swapaxes` fix,
+    from `check` and `run` as from `preflight`. Before, `run` dropped it with the header size
+    comparison it supersedes (`mask_issues`) and recorded the mask with `axes: ""`, which
+    `RunInputs.read_mask` then called "changed or missing". A manifest never records a mask
+    without roles (`mask_manifest` raises), and `RunInputs.open` refuses such an older record
+    with its own message. Images differ: an image whose header names X before Y is read in
+    canonical order (design change log, 1 November 2026); masks are never reordered.
+    Contract §9.1, §15.*
+51. How do bricks learn of images added after the run (PP-8b, from CellPatcher PR #69
+    question 1)?
+    *Decided (2 November 2026, owner): bricks never discover files and never leave an image
+    out silently; preprocessing gives the check. `unrecorded_files` lists, with
+    preprocessing's own rules, the images discovery would analyse and the masks it would pair
+    with a recorded image (masks folder, OME-Zarr label images) that the run neither recorded
+    nor excluded; files discovery ignores or refuses do not count. `RunInputs.open` always
+    runs it and raises `UnrecordedFilesError` (a `RunInputsError` with the files) telling the
+    user to run preprocessing again or exclude them. Chosen over an explicit call because it
+    reads no pixel and no TIFF header (a folder walk, plus OME-Zarr label metadata), and a
+    check bricks must remember to call could be forgotten; there is no opt-out. Contract
+    §9.1.*
+    *Also decided (2 November 2026, owner, on the PP-8b review questions): (1) a run that
+    v0.4.0rc1 recorded with a mask of empty axes is refused as a whole by `RunInputs.open`,
+    not only at `read_mask`; (2) a `masks_root` given for a run that recorded no masks
+    folder is refused: every mask there paired with a recorded image is unrecorded, and the
+    message says to run preprocessing again with the masks folder (`--masks`); (3) OME-Zarr
+    label images are never skipped: without the `zarr` extra, `RunInputs.open` refuses a run
+    holding OME-Zarr images, naming `morphobricks-preprocessing[zarr]`, and a recorded
+    OME-Zarr image whose `labels` group cannot be read is its own refusal naming the image,
+    not a silent pass and not an "unrecorded" file; (4) the version stays 0.3.4 until PP-13,
+    and v0.4.0rc2 is tagged by the owner after this PR.*

docs/user_guide.md

@@ -591,6 +591,10 @@ Run the notebook or `run` again after adding, changing or removing images:
 - changing only the image type reads nothing again: the histograms are the same for both
   types, and only the manifest records the new type.
 
+Run the brick again after adding images or masks before using a later brick: later bricks
+read the images and masks this brick's run recorded, and refuse an experiment holding images
+or masks the run did not record (they name them).
+
 To leave out images, for example a damaged file, list them under *Exclude images* (one per
 line) or pass `--exclude`. Excluded images are not read, are listed in the messages and the
 manifest, and their earlier rows are removed. Remove them from the list to analyse them
@@ -677,7 +681,7 @@ continues.
 | `unreadable_mask` | Error | A mask cannot be opened, or holds several images, or the `labels` group of an OME-Zarr image cannot be read (it must list its label images). Save the mask again as a TIFF label image, or repair the OME-Zarr labels. |
 | `mask_not_2d` | Error | A mask has dimensions other than Y and X that its metadata does not name as depth (Z) or time (T), for example a plain stack. (A mask one pixel high or wide is a 2D mask, not this error.) State the axes of its mask type once (`--mask-axes Cells=TYX`), or save it as a 2D label image, or with its axes named (an ImageJ hyperstack or OME-TIFF with axes TYX or ZYX). |
 | `invalid_mask_axes`, `mask_axes_contradict_header` | Error | Axes stated for a mask type do not fit its masks (one role, Y, X, Z or T, per axis; Y and X once each), name a type that is not found, or contradict the axes a mask's metadata names. Correct the statement. |
-| `mask_layout` | Error | A mask is stored with its Y and X the other way round from its image as read (X before Y, or the image's height and width swapped). Masks are never transposed: save it again with `numpy.swapaxes(mask, -1, -2)`. |
+| `mask_layout` | Error | A mask is stored with its Y and X the other way round from its image as read (its header names X before Y, for example `XY`, named in the message with the mask; or the image's height and width swapped). Masks are never transposed: save it again with `numpy.swapaxes(mask, -1, -2)`. |
 | `mask_planes_differ` | Error | The masks of one image have different numbers of depth slices or time points. Make every mask of an image with the image's planes. |
 | `mask_planes_mismatch` | Error (`run`) | A mask does not have the depth slices and time points of its image under the confirmed axis roles: a 2D mask fits only an image with one plane (not a Z-stack or time-lapse) unless its type is stated `YX` (`--mask-axes Cells=YX`: one mask shared by every plane), and a mask with Z or T needs as many slices or time points as its image. Save one mask plane per image plane, with the axes named, or check the axis roles. |
 | `mask_size_mismatch` | Error | A mask does not have the width and height of its image. Check that it was made from that image at full resolution. |
@@ -686,6 +690,7 @@ continues.
 | `ambiguous_mask` | Error | A mask's name could belong to two images (one image name starts another's) or fit both layouts. Rename the masks or the images so each names one image and one kind. |
 | `mask_without_image`, `mask_outside_type_folder`, `mask_of_excluded_image` | Warning | Files in the masks folder that match no image of the experiment (check the folder and file names), other files directly in the masks folder, or masks of excluded images. They are not used. |
 | `manifest_before_v040`, `images_not_in_manifest`, `images_missing`, `images_changed`, `position_id_not_recorded`, `identity_differs`, `axis_mapping_mismatch` | Error (other bricks) | A later brick checked its images against this brick's run (`check_sources`): a run written before v0.4.0 (redo it), an image the run did not analyse, no longer there, changed since, recorded with other labels or position (or by a version that did not record positions), or read with axes the run gave other roles. Run this brick again on the experiment as it is, or use the run's labels and axis roles. |
+| `UnrecordedFilesError` | Error (other bricks) | The experiment holds images or masks that the preprocessing run neither recorded nor excluded, for example added after the run. Later bricks never look for images themselves, so they refuse rather than leave them out: run this brick again on the experiment as it is (with its masks folder, `--masks`, for masks), or exclude the images. Files the run ignores (non-image files, system files, masks matching no image) do not count. A run holding OME-Zarr images also needs the `zarr` extra here (`pip install "morphobricks-preprocessing[zarr]"`), so that their label images can be checked; an OME-Zarr image whose `labels` group cannot be read is refused too. |
 | `zarr_not_installed` | Error | OME-Zarr images need the `zarr` extra. Install it, or exclude the OME-Zarr images. |
 
 ## 9. Python interface

pyproject.toml

@@ -146,6 +146,7 @@ baseline = "tools/code_rules_baseline.json"
 "morphobricks_preprocessing.narrow_ranges" = "pure"
 "morphobricks_preprocessing.recorded_sizes" = "pure"
 "morphobricks_preprocessing.run_inputs" = "io"
+"morphobricks_preprocessing.unrecorded" = "io"
 "morphobricks_preprocessing.testing" = "orchestration"
 "morphobricks_preprocessing.mask_scan" = "io"
 "morphobricks_preprocessing.masks" = "io"

src/morphobricks_preprocessing/__init__.py

@@ -192,8 +192,9 @@
     sample_planes,
     write_calibration,
 )
-from .run_inputs import Fov, LazyImage, RunInputs, RunInputsError
+from .run_inputs import Fov, LazyImage, RunInputs, RunInputsError, UnrecordedFilesError
 from .sources import check_sources
+from .unrecorded import UnrecordedFiles, unrecorded_files
 from .validation import Severity, ValidationIssue, has_errors
 from .zarr_input import (
     NgffError,
@@ -283,6 +284,9 @@
     "RunResult",
     "RunInputs",
     "RunInputsError",
+    "UnrecordedFiles",
+    "UnrecordedFilesError",
+    "unrecorded_files",
     "Fov",
     "LazyImage",
     "RunSettings",

src/morphobricks_preprocessing/mask_checks.py

@@ -10,6 +10,7 @@
 from __future__ import annotations
 
 from collections.abc import Iterable, Mapping
+from dataclasses import replace
 
 from .axes import AxisMapping, AxisReview
 from .masks import MaskInventory, mask_plane_issues
@@ -87,7 +88,8 @@ def mask_issues(
     :func:`mask_size_issues`, and the planes are checked by
     :func:`.masks.mask_plane_issues`. Images with an axis error are never read, so their
     masks are left out of both: a size or plane message would only repeat the axis error
-    in another form.
+    in another form. A mask whose own header puts X before Y has no roles and stays
+    refused (``mask_layout``): only the header's size comparison is replaced.
 
     Parameters
     ----------
@@ -106,6 +108,24 @@ def mask_issues(
     skip = review.axis_error_files
     return [
         *(i for i in masks.issues if i.code not in SIZE_CODES),
+        *_header_refusals(masks),
         *mask_size_issues(masks, metadata.files, review.file_mappings, skip=skip),
         *mask_plane_issues(masks, metadata.files, review.file_mappings, skip=skip),
     ]
+
+
+def _header_refusals(masks: MaskInventory) -> list[ValidationIssue]:
+    """Return the size-code issues of masks whose header was refused, not size-compared.
+
+    :func:`.masks.find_masks` gives ``mask_layout`` both to a mask whose own header puts X
+    before Y (it gets no roles) and to a mask whose Y and X are its image's header's
+    swapped. Only the second is superseded by :func:`mask_size_issues`; the first is kept,
+    or the run would record the mask without axes.
+    """
+    refused = {mask.path for mask in masks.files if not mask.axes}
+    kept = []
+    for issue in masks.issues:
+        paths = tuple(path for path in issue.paths if path in refused)
+        if issue.code in SIZE_CODES and paths:
+            kept.append(replace(issue, paths=paths))
+    return kept

src/morphobricks_preprocessing/mask_record.py

@@ -35,7 +35,16 @@ def mask_manifest(inventory: MaskInventory, experiment_root: str | os.PathLike)
         ``roles``, ``axes_from`` and ``shared``. ``axes`` are the mask's axes as stored
         (its header's letters, ``"IYX"`` for a plain stack), ``roles`` what they are read as
         (``"TYX"`` when stated), so a brick reads the mask as stored, never reordered.
+
+    Raises
+    ------
+    ValueError
+        For a mask without roles: its header was refused, so the run has an error and
+        records no mask without axes.
     """
+    unread = [mask.path for mask in inventory.files if not mask.axes]
+    if unread:
+        raise ValueError(f"Masks without roles cannot be recorded: {', '.join(unread)}")
     data = inventory.to_manifest()
     by_path = {mask.path: mask for mask in inventory.files}
     for entry in data["files"]:

src/morphobricks_preprocessing/masks/files.py

@@ -7,7 +7,7 @@
 
 import logging
 import os
-from collections.abc import Iterable, Mapping
+from collections.abc import Collection, Iterable, Mapping
 from pathlib import Path
 
 from ..discovery import DiscoveryResult, is_system_name, is_tiff_name
@@ -127,7 +127,7 @@ def _folder_issues(
 
 
 def _folder_masks(
-    folder: Path, discovery: DiscoveryResult, images: Mapping[str, TiffMetadata]
+    folder: Path, discovery: DiscoveryResult, images: Collection[str]
 ) -> tuple[list[MaskFile], list[ValidationIssue]]:
     """Pair the TIFF files of the masks folder with the images, in either layout.
 
@@ -137,7 +137,8 @@ def _folder_masks(
     an analysed image; a file matching two images or both layouts is refused, never guessed.
     A mask matching no image is the warning ``mask_without_image``, and a mask of an excluded
     image the warning ``mask_of_excluded_image``: neither is used (design v0.4 §3 keeps the
-    severity of every existing check).
+    severity of every existing check). ``images`` are the analysed images' paths (excluded
+    images left out).
     """
     by_key, by_leaf = {}, {}
     for leaf in discovery.leaves:

src/morphobricks_preprocessing/masks/stated.py

@@ -84,7 +84,8 @@ def mask_roles(header: str, named: str, stated: str | None) -> tuple[str, str |
             detail = f"header axes {header}, stated {stated}: {', '.join(contradicted)}"
             return "", "mask_axes_contradict_header", detail
     if roles.index("X") < roles.index("Y"):
-        return "", "mask_layout", f"axes {roles}: X is stored before Y"
+        given = f"header axes {header}" + ("" if stated is None else f", stated {stated}")
+        return "", "mask_layout", f"{given}: X is stored before Y"
     return roles, None, ""