CP-2a: file runs read through the preprocessing run's RunInputs

AGENTS.md

@@ -176,7 +176,22 @@ environment.yaml
 
 CellPatcher-specific conventions:
 
-* `image_axes`/`mask_axes` are explicit and axis-aware (see `axes.py`).
+* **File runs read through a clean preprocessing run** (v0.2.0, CP-2a; design
+  v0.4 §6): `run_patching_pipeline`, `run_patching_condition_repeat_tree` and
+  `run_patching_from_folders` need `common.preprocessing_run` (else
+  `no_preprocessing_run` with the preprocessing command). `cellpatcher.file_runs`
+  opens it with `RunInputs.open` with the image folder/root and masks folder
+  given (a moved or copied experiment is checked and read where it is now),
+  patches the run's FOVs with their recorded keys, and reads images and masks
+  with `RunInputs.read_plane`/`read_mask` (`cellpatcher.run_reading`) in one
+  canonical layout per run (T, Z, C, then Y, X). No CellPatcher-side axis,
+  pairing or calibration logic is used for them; `common.image_axes`/`mask_axes`
+  other than `"YX"`, `mask_source="ngff_labels"`, `mask_label_name` and labels
+  other than `"single"` are refused with a migration message (CP-2b removes
+  them for file runs). The run is opened once per call and kept for that call
+  only (`cellpatcher.run_session`).
+* `image_axes`/`mask_axes` are explicit and axis-aware (see `axes.py`) for
+  in-memory arrays and lazy delivery from a path.
   Image/mask pairing is the preprocessing brick's `find_masks`, called by
   `io.pair_masks` (issue #45, owner decision 2026-10-30): TIFF masks named
   `<image>_<type>.tif` or `<type>/<image>.tif`; `matching.mask_type` picks
@@ -229,7 +244,15 @@ in notebooks where practical.
 
 ### Inputs this repo consumes
 
-CellPatcher reads image files, and (for `cell` mode) matching label/mask
+Since v0.2.0 (CP-2a) folder and tree runs read files only through a clean
+preprocessing run (`common.preprocessing_run`, opened with `RunInputs`; see §4 and
+`docs/configuration_reference.md`, "File runs read through the preprocessing
+run"): the images, their labels and `position_id`, their roles and their masks
+are the run's. A folder mask and a label image of one type for one image are
+refused by the preprocessing run (`duplicate_mask`, rc1). The rest of this
+paragraph describes discovery and pairing as `validate_image_and_mask_folders`
+and lazy delivery from paths still do them (until CP-2b), and as file runs did
+before v0.2.0. CellPatcher reads image files, and (for `cell` mode) matching label/mask
 files, directly from user-provided folders. Images are found with the
 preprocessing brick's `discover_experiment` (`io.discover_images`; owner
 decision of 2026-10-30, issue #45): TIFF files and OME-Zarr images. PNG, JPEG,
@@ -263,11 +286,10 @@ guessed, and `common.mask_axes` with a `C` is refused for mask-folder masks
 
 Two ways to point CellPatcher at input data:
 
-- `run_patching_pipeline(image_folder_path, ...)` — a single flat folder of
-  images (optionally with a matching mask folder). `condition`/
-  `biological_repeat` must be set explicitly via
-  `CellPatcherConfig.common.condition`/`.biological_repeat` when using this
-  entry point; they are not parsed from the folder path.
+- `run_patching_pipeline(image_folder_path, ...)` — the images of a
+  preprocessing run of one condition and biological repeat (optionally with
+  its masks folder). Since v0.2.0 their labels are the run's; before, they
+  were set with `CellPatcherConfig.common.condition`/`.biological_repeat`.
 - `run_patching_condition_repeat_tree(image_root_path, ...)` — the
   `<condition>/<biological_repeat_YYYY_MM_DD>[/<technical_repeat>]/` layout
   from §6, walked

docs/normalization.md

@@ -169,10 +169,10 @@ planes and what to do:
 
 | Message | Meaning and fix |
 |---|---|
-| *The preprocessing run in ... cannot be used* (`preprocessing_run_refused`) | The preprocessing brick's `RunInputs` refuses the run (since v0.2.0): no `run_manifest.json` next to the table, a run written before preprocessing v0.4.0, a table or `experiment_design.csv` changed since the run, an image or mask changed or missing since the run, or an experiment moved since the run. Run preprocessing v0.4 on the experiment where it is now. |
+| *The preprocessing run in ... cannot be used* (`preprocessing_run_refused`) | The preprocessing brick's `RunInputs` refuses the run (since v0.2.0): no `run_manifest.json` next to the table, a run written before preprocessing v0.4.0, a table or `experiment_design.csv` changed since the run, or an image or mask changed or missing since the run where CellPatcher reads it. Run preprocessing v0.4 on the experiment as it is now; an experiment only moved or copied (times kept) since its run is read where it is now when CellPatcher is given that folder and its masks folder (since CP-2a). |
 | *normalization.plane_histograms/manifest/experiment_design names ...* (`not_the_run_output`) | Only the run's own files, unrenamed in its output folder, are read. Give that folder's `plane_histograms.csv` (and leave `manifest` and `experiment_design` empty). |
 | *... the preprocessing run recorded no design* (`missing_experiment_design`) | The `biological_repeat` scope needs a run whose experiment design was confirmed (step 1 of the preprocessing brick). |
-| *... image(s) found by CellPatcher have no rows in plane_histograms.csv* | Images were excluded or skipped by the preprocessing brick, or added after it ran. Run the preprocessing again on the same folder, remove those images, or use the patch scope. |
+| *... image(s) found by CellPatcher have no rows in plane_histograms.csv* | In-memory runs: images whose `source_relative_path` the run has no rows for. Folder and tree runs patch only the run's images (since CP-2a; an image added after the run is not seen). Run the preprocessing again, or use the patch scope. |
 | *... changed after the preprocessing brick read them* / *... the images changed after the preprocessing run* | An image's size, time or pixel values differ from the statistics. Run the preprocessing again. |
 | *... other condition, biological repeat or technical repeat labels* | CellPatcher and the preprocessing read different folder levels: give both the same experiment folder. |
 | *... another position_id than their rows* | A `position_id` given in `image_metadata` is not the one the preprocessing recorded (the image name). Leave it to the image name, or give the recorded one. |

docs/normalization_contract.md

@@ -60,8 +60,8 @@ brick's
 
 | Rule | Detail |
 |---|---|
-| Discovery | CellPatcher always discovers images with its own folder rules (`discover_condition_repeat_folders`), whether or not preprocessing was run. The manifest never decides which images are patched. |
-| Experiment root | The folder given to CellPatcher as image root must be the folder given to the preprocessing brick, so relative paths match. |
+| Discovery | Since v0.2.0 (CP-2a) folder and tree runs patch the preprocessing run's images (`RunInputs.fovs()`), with their FOV keys as the run recorded them; CellPatcher discovers nothing for them (design v0.4 §6). An image added after the run is not patched, and nothing says so yet: a known limitation, closed when CellPatcher calls the check for added images that preprocessing will give bricks (not in v0.4.0rc1). Before, CellPatcher discovered images with its own folder rules and the manifest never decided which were patched. |
+| Experiment root | The folder given to CellPatcher as image root is where the preprocessed experiment is now: the run is opened with it (`RunInputs.open(experiment_root=...)`, and the masks folder with `masks_root`), so an experiment moved or copied since the run is checked and read there. Relative paths are the run's. |
 | Image types | TIFF files (`.tif`, `.tiff`, any case) and, with the extra, OME-Zarr images: folders whose names end in `.zarr`, read from level 0 of their first image. A store with several images is an error, as in preprocessing. |
 | OME-Zarr copies | Copies written by the preprocessing brick (`<output>/ome_zarr/...`) are not read; CellPatcher reads the source images. |
 | `source_relative_path` | The image's path relative to the experiment root, with `/` separators, from its real location in the tree (`control/img.tif` for an image directly in `control/`; a linked image or folder keeps the link's path under the root, as the brick records it when it walks the tree). Never built from `ConditionRepeatFolder.relative_folder`, which inserts `single` for missing levels. |
@@ -132,19 +132,23 @@ Checks before any patch is normalized; every failure blocks the run:
 - the plane's exact finite minimum and maximum equal `pixel_min` and `pixel_max`, and its
   finite pixel count equals `valid_pixel_count`.
 
-An image CellPatcher found without any row in the CSV (excluded or skipped in preprocessing,
-or added later) is reported with all such images in one message.
+An image without any row in the CSV is reported with all such images in one message (in a
+file run every image is the run's, so this means a row the run did not write).
 
 **The run (since v0.2.0):** the table's folder is opened with the preprocessing brick's
 `RunInputs.open`, and any refusal stops the run (`preprocessing_run_refused`): no manifest,
 a manifest before v0.4, a table or design whose SHA-256 is not the manifest's, an image or
-mask of the run changed or missing (which includes an experiment moved since the run). The
+mask of the run changed or missing, where CellPatcher was given the experiment (since CP-2a
+a file run opens the run with its image root and masks folder; in-memory runs and
+`load_plane_statistics` without `experiment_root` open it where the run recorded it). The
 CSV and manifest must be the run's own files, unrenamed (`not_the_run_output`). Then:
 
-- each patched image's `manifest.source_fingerprint()` (size and modification time) equals its
-  manifest entry;
-- CellPatcher's image axes give the same role for every axis as the manifest's
-  `axis_mapping`;
+- a file run reads every image and mask through `RunInputs` (`read_plane`, `read_mask`), which
+  checks each file's fingerprint again on every read, in the roles the run recorded (before
+  CP-2a: the patched images' size, time and declared axes were checked with `check_sources`);
+- the run is opened once per CellPatcher call and kept for that call only
+  (`cellpatcher.run_session`): the next call checks every image and mask again, patched or
+  not (before CP-2a an opened run was cached for the session, keyed on the table and manifest);
 - the manifest's `image_type`, when it records one, equals `normalization.image_type`
   (otherwise `image_type_differs`, checked before any image is read). A run that recorded no
   image type leaves the choice to the configuration (normalization brick decision N34).

notebooks/CellPatcher/CHANGELOG.md

@@ -4,6 +4,34 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## [Unreleased] - File runs read through the preprocessing run (CP-2a)
+
+### Changed
+- **Breaking.** `run_patching_pipeline`, `run_patching_condition_repeat_tree` and
+  `run_patching_from_folders` read files only through a clean preprocessing run (v0.4):
+  `common.preprocessing_run`, its output folder, is required in every mode, normalization on or
+  off (`no_preprocessing_run`, with the preprocessing command). The images patched are the
+  run's, with their labels and `position_id` as it recorded them; images and masks are read
+  with `RunInputs.read_plane`/`read_mask` in the roles it recorded, in one canonical layout per
+  run (T, Z, C, then Y, X), recorded in `run_config.json` as `common.image_axes`/`mask_axes`.
+  An image added to the experiment after the run is not patched, and nothing says so yet (known
+  limitation, closed when CellPatcher calls the preprocessing check for added images, which is
+  not in v0.4.0rc1).
+- The run is opened with the image folder/root and the masks folder given, so an experiment
+  moved or copied since its run (times kept) is checked and read where it is now. A mask
+  changed in a copy is refused (before, only the original was checked).
+- For file runs, `common.image_axes`/`mask_axes` other than `"YX"`, `mask_source="ngff_labels"`,
+  `mask_label_name`, and `condition`/`biological_repeat`/`technical_repeat` other than
+  `"single"` are refused with a migration message. `matching.mask_type` chooses among the run's
+  mask types (a label image's type is its name); `random` mode reads masks when a masks folder
+  or `matching.mask_type` is given. `run_patching_pipeline` patches a run of one condition and
+  biological repeat.
+- The statistics cache is gone: a run is opened once per CellPatcher call and kept for that
+  call only, so a mask or an unpatched image changed between two calls is refused.
+- `load_plane_statistics`, `recorded_image_type` and `cached_statistics` take
+  `experiment_root=`/`masks_root=`; they and `open_run` moved to `cellpatcher.run_open`
+  (still importable from `cellpatcher.statistics`).
+
 ## [Unreleased] - The preprocessing run is read through RunInputs (CP-1, preprocessing v0.4.0rc1)
 
 ### Changed

src/cellpatcher/config.py

@@ -19,22 +19,22 @@
 @accept_deprecated_repeat
 @dataclass
 class CommonConfig:
+    # The axes of in-memory arrays given to run_patching (canonical, Y before X). File runs
+    # read each image and mask in the roles the preprocessing run recorded and refuse other
+    # values than "YX" (CP-2a; removed for them in CP-2b).
     image_axes: str = "YX"
     mask_axes: str = "YX"
     output_dir: str | None = None
     extract_arrays: bool = False
-    # Stamped onto every PatchRegion/metadata-CSV row for this run. These are
-    # the MorphoBricks shared-CSV-contract `condition`/`biological_repeat`/
-    # `technical_repeat` columns (see AGENTS.md §5). A label that is not
-    # given is "single", as for a missing folder level; None is stamped as
-    # "single" too. run_patching()/run_patching_pipeline() never infer them
-    # from paths; set them explicitly, or use
-    # orchestration.run_patching_condition_repeat_tree to derive them per
-    # leaf from a <condition>/<biological_repeat>[/<technical_repeat>]/
-    # folder tree. technical_repeat is the well/dish/coverslip within the
-    # biological repeat; per-image image_metadata["technical_repeat"]
-    # overrides it. `repeat` stays accepted as a deprecated alias of
-    # biological_repeat (DeprecationWarning).
+    # The output folder of the clean preprocessing run (v0.4) a folder or tree run reads its
+    # images, masks and labels through (RunInputs, design v0.4 §6); required for file runs.
+    preprocessing_run: str | None = None
+    # The MorphoBricks shared-CSV-contract `condition`/`biological_repeat`/`technical_repeat`
+    # columns (AGENTS.md §5) of in-memory run_patching; a label not given (or None) is
+    # "single", as for a missing folder level. Folder and tree runs take each image's labels
+    # from the preprocessing run and refuse other values. technical_repeat is the well/dish/
+    # coverslip; per-image image_metadata["technical_repeat"] overrides it. `repeat` stays
+    # accepted as a deprecated alias of biological_repeat (DeprecationWarning).
     condition: str | None = SINGLE_CONDITION_LABEL
     biological_repeat: str | None = SINGLE_BIOLOGICAL_REPEAT_LABEL
     technical_repeat: str | None = SINGLE_TECHNICAL_REPEAT_LABEL
@@ -48,18 +48,16 @@ class CommonConfig:
     z_index: int | None = None
     t_policy: str = "keep"
     t_index: int | None = None
-    # Where folder and tree runs read masks (docs/normalization_contract.md
-    # §4): "tiff_tree", a mask folder (tree) mirroring the images and matched
-    # by file name with `matching`; or "ngff_labels", the label image named
-    # mask_label_name in each OME-Zarr image's own labels group (the name may
-    # be left None when every image has exactly one).
+    # Where folder and tree runs read masks before CP-2a. They now read the masks the
+    # preprocessing run paired (choose the type with matching.mask_type) and refuse other
+    # values than these defaults.
     mask_source: str = "tiff_tree"
     mask_label_name: str | None = None
     # The object set of cell-mode patches (the `object_set` column and a
     # level of patch_id, cellpatcher.ids). None: folder and tree runs name it
-    # after the mask (its OME-NGFF label name, or its file name's suffix after
-    # the image's stem), and "mask" when the mask gives no name (in-memory
-    # run_patching, or a mask named like its image). Set it to name it
+    # after the mask type the preprocessing run recorded (a folder name, or an
+    # OME-NGFF label name), and "mask" when the mask gives no name (in-memory
+    # run_patching). Set it to name it
     # explicitly; image_metadata["object_set"] overrides it per image.
     object_set: str | None = None