Bind masks to their fingerprints for the whole run and the relationship scan

AGENTS.md

@@ -127,7 +127,12 @@ src/morphobricks_preprocessing/
                     preprocessing's discovery and pairing; RunInputs.open raises
                     UnrecordedFilesError for them (PP-8b, contract §9.1)
   run_inputs_text.py the texts RunInputs refuses a run with (moved from run_inputs.py in PP-12)
-  mask_record.py    the manifest's masks entry with fingerprints and roles (manifest v2)
+  mask_record.py    the manifest's masks entry with fingerprints and roles (manifest v2); the
+                    fingerprint is the one bound to the mask's header (issue #160)
+  mask_binding.py   masks bound to their fingerprints (issues #160, #165): changed_masks,
+                    mask_contents/content_changed_masks (verify_content),
+                    mask_changed_issue (mask_changed_during_run, mask_changed_during_scan),
+                    scan_fingerprints/check_scanned and MaskChangedError (scan_masks)
   axis_check.py     the axis check (contract §2): header_axis_issue, axis_disagreements,
                     likely_axis_cause, check_header_axes, ConfirmedAxes (a confirmation covers
                     only its files as fingerprinted, plain_stack_not_confirmed); AXIS_ERROR_CODES, AXIS_CHECK_CODES
@@ -181,7 +186,8 @@ src/morphobricks_preprocessing/
                     mask_axes_questions (the types whose axes must be stated, with the
                     statements mask_roles accepts)
   mask_scan.py      the optional mask step (Q32): piece_counts (scipy.ndimage), pair_counts,
-                    scan_masks -> MaskReport (kinds, relations, direct-parent hierarchy, evidence),
+                    scan_masks -> MaskReport (kinds, relations, direct-parent hierarchy, evidence;
+                    each image's masks fingerprinted before and after they are read, #165),
                     write_mask_report (mask_relationships.json); DEFAULT_THRESHOLDS,
                     mask_thresholds (checked changes)
   mask_pairs.py     pair_counts' pure helpers: best_parents (a tie between parents gives no
@@ -484,7 +490,10 @@ functions the tests use.
   into place (the output folder is fsynced right after the commit); the next holder of the
   lock finishes or discards an interrupted publication.
   Every image is fingerprinted when its header is read, before and after its pixels are
-  read, and before the commit (`source_changed_during_run`, issue #90). Nothing is
+  read, and before the commit (`source_changed_during_run`, issue #90). Every mask's header
+  is read between two fingerprints (`mask_changed_during_check`), that fingerprint is the one
+  recorded, and the mask must still have it before the commit (with `verify_content` also its
+  content SHA-256; `mask_changed_during_run`, issue #160). Nothing is
   published if any error is found.
 - **Exclusions:** the user may exclude images explicitly (`exclude=[...]`, e.g. damaged
   files). Excluded images are not read, are reported (`excluded_images` warning), are

CHANGELOG.md

@@ -37,6 +37,30 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the
   New `MaskReport.hierarchy_cycles` and `mask_scan.hierarchy_cycles`. Acyclic hierarchies
   print as before; `mask_relationships.json` is unchanged (report version 1).
 
+- **Masks are bound to their fingerprints for the whole run** (#160). Each mask's header is
+  read between two fingerprints that must agree (`find_masks(bind=True)`,
+  `MaskFile.bound`; else the new error `mask_changed_during_check`), the manifest records
+  that fingerprint (before, it was taken when the manifest was written, so a mask rewritten
+  during the run was recorded under headers checked on another version), and every mask
+  must still have it right before the commit; otherwise the new error
+  `mask_changed_during_run` names the mask and what changed, and nothing is published. A
+  `verify_content` run also hashes every mask after the header checks and again as the last
+  step before the commit. Content hashing stays opt-in (#159).
+  - *Records and migration:* no schema change. The manifest (versions 2 and 3) and its mask
+    entries keep their keys and values: for masks unchanged during the run the fingerprint
+    recorded is the same, so an unchanged rerun keeps the manifest byte for byte. The masks'
+    content SHA-256 is compared in memory and not recorded. Earlier manifests read as
+    before.
+- **No mask relationship report from masks that change while they are scanned** (#165). The
+  mask step fingerprints each mask with its header (`prepare_mask_step`) and each image's
+  masks before and after their planes are read; a mask that changed or went missing raises
+  `MaskChangedError` (a `ValueError`; the new error `mask_changed_during_scan`, which the
+  `masks` command and the notebook's **Describe the masks** show with what to do), no report is written and an earlier one is left as it was. With
+  `verify_content` (`prepare_mask_step(verify_content=...)`, new `masks --verify-content`;
+  by default the recorded run's setting) the masks' content SHA-256 is compared too.
+  *Records:* `mask_relationships.json` is unchanged (report version 3) and unchanged masks
+  give the same report.
+
 ## v0.4.0 (5 November 2026)
 
 The final v0.4.0, with the small changes since v0.4.0rc5 (#147) and the release step: the

docs/user_guide.md

@@ -655,7 +655,7 @@ SHA-256 still accept it; the report records the rerun. Other options:
 | `--image-type fluorescence\|rgb` | The image type (default: the earlier run's, else the suggested one, printed) |
 | `--design FILE` | The confirmed experiment design (default: `experiment_design.csv` in the output folder); it is copied to the output folder |
 | `--mode restart\|full` | `restart` (default) reuses the rows of unchanged images; `full` recomputes every row and check. Both keep your confirmations and replace the results only when the run succeeds |
-| `--verify-content`, `--no-verify-content` | Reuse rows (earlier or staged) only when each image's content SHA-256 is the one recorded with them, not only its size and modification time; every image is read once more. Default: the earlier run's choice, else off. Turning it on or off recomputes every row |
+| `--verify-content`, `--no-verify-content` | Reuse rows (earlier or staged) only when each image's content SHA-256 is the one recorded with them, not only its size and modification time; every image is read once more. Default: the earlier run's choice, else off. Turning it on or off recomputes every row. With `masks`: also compare each mask's content SHA-256 before and after it is read (same default) |
 | `--no-masks` | Drop the masks folder an earlier run recorded, with the mask axes stated for its masks (those of OME-Zarr label images are kept) |
 | `--no-exclude` | Clear the exclusions an earlier run recorded, so every image is analysed again (not together with `--exclude`) |
 | `--remove-stale-lock` | Remove the lock left by a run that was stopped or whose computer restarted (`check`, `run`, `masks`, `design` and `calibrate`); give it only when no other run uses the output folder |
@@ -727,7 +727,12 @@ objects have positive labels: a mask with a negative value stops the command wit
 naming the mask, and no report is written.
 The output folder's lock is held from before the first mask is read until the report is
 written: when another run holds it, the command stops with `output_locked` before reading
-any mask.
+any mask. A mask that changes after the command read its header or while its pixels are read
+(a segmentation still writing it) stops the command with `mask_changed_during_scan`, naming
+the mask: no report is written, and an earlier `mask_relationships.json` is left as it was.
+`--verify-content` also compares each mask's content (SHA-256) before and after it is read
+(each mask is read whole twice more); given neither it nor `--no-verify-content`, the run's
+own setting is used.
 
 The rules behind each suggestion use thresholds you can change, for example
 `--inside-object-share 0.8` to call a kind of mask "inside" another when 80% of its objects
@@ -959,7 +964,11 @@ that program and run again. This needs temporary disk space for the new outputs.
 **Images that change during the run.** If an image changes or disappears while the run
 checks or reads it (an acquisition or a copy still writing into the experiment), the run
 stops with `source_changed_during_run` and publishes nothing. Wait until nothing writes to
-the experiment, then run again: the images already read resume.
+the experiment, then run again: the images already read resume. Masks are checked the same
+way: a mask that changes while the run checks its header, or at any time before the run
+publishes its results (a segmentation still writing it), stops the run with
+`mask_changed_during_check` or `mask_changed_during_run`, naming the mask and what changed,
+and nothing is published. With `--verify-content` the masks' content is compared too.
 
 Every output file gets the permissions a file you save yourself would get (your umask;
 usually readable by everyone, writable by you), so colleagues can open the results in a
@@ -1070,6 +1079,9 @@ continues.
 | `conversion_source_changed` | Warning | An original changed or went missing after its named-axis copy was made; the copies hold the images as converted. If the originals were corrected, convert again into a new folder. |
 | `copy_changed_after_conversion` | Error | An image of the folder of named-axis copies changed (its size, time or SHA-256), went missing or was added after the conversion, so the conversion record does not describe it, and the bricks (`RunInputs`) would refuse a run on it. The run stops before it writes anything but the report: convert the originals again into a new folder, and run preprocessing on that one (same output folder). |
 | `source_changed_during_run` | Error | An image changed or went missing while the run was checking or reading it, so its rows would not describe it. No scientific outputs are published and sources are untouched; the discovery report is written. Wait until nothing writes to the experiment (an acquisition or a copy still running), then run again: the images already read resume from their staged rows. |
+| `mask_changed_during_check` | Error | A mask changed while its header was read, so the check cannot tell which version it describes. Wait until nothing writes to the masks (a segmentation or a copy still running), then check or run again. |
+| `mask_changed_during_run` | Error | A mask changed or went missing after the run checked its header and before it published its results (its size or time, or for an OME-Zarr label image its files; with `--verify-content` also its content). Nothing is published; the discovery report is written. Wait until nothing writes to the masks, then run again: the images already read resume from their staged rows. |
+| `mask_changed_during_scan` | Error | The `masks` command (or **Describe the masks**): a mask changed or went missing after its header was read or while its pixels were read. No report is written; an earlier `mask_relationships.json` is left as it was. Wait until nothing writes to the masks, then describe them again. |
 | `design_unreadable` | Error | The `experiment_design.csv` an earlier run wrote cannot be read. Correct it, or delete it to confirm the design again. |
 | `no_usable_planes` | Error | No plane has finite pixel values. Check the images. |
 | `tiff_outside_leaf` | Error | Images sit in a folder that also holds subfolders, so they have no condition or repeat of their own. A folder holds either images or subfolders, never both. Move the images into a folder of their own, or out of the experiment folder. |

pyproject.toml

@@ -140,6 +140,7 @@ baseline = "tools/code_rules_baseline.json"
 "morphobricks_preprocessing.axis_review" = "pure"
 "morphobricks_preprocessing.mask_checks" = "pure"
 "morphobricks_preprocessing.mask_review" = "pure"
+"morphobricks_preprocessing.mask_binding" = "io"
 "morphobricks_preprocessing.mask_record" = "io"
 "morphobricks_preprocessing.notebook_axes" = "ui"
 "morphobricks_preprocessing.notebook_axis_preview" = "ui"

src/morphobricks_preprocessing/cli_masks.py

@@ -8,13 +8,16 @@
 axes and the exclusions; a value given that differs is used and named. The pairing, the
 scan and the write are :mod:`.run.mask_step`'s, shared with the notebook: the output
 folder's lock is held from before the first mask is read to the write, so a locked folder
-is refused before any mask is read.
+is refused before any mask is read. A mask that changes after its header is read or while it
+is read refuses the step (``mask_changed_during_scan``, issue #165); ``--verify-content``
+also compares each mask's content SHA-256.
 """
 
 from __future__ import annotations
 
 import argparse
 
+from .mask_binding import MaskChangedError
 from .mask_scan import DEFAULT_THRESHOLDS, mask_thresholds
 from .mask_text import mask_report_summary, mask_summary
 from .report import format_issues
@@ -46,6 +49,12 @@ def add_masks_arguments(masks: argparse.ArgumentParser) -> None:
         help="clear the exclusions of the earlier run for this scan (not with --exclude)",
     )
     masks.add_argument("--log", metavar="FILE", help="write technical details to this file")
+    masks.add_argument(
+        "--verify-content",
+        action=argparse.BooleanOptionalAction,
+        help="also compare each mask's content SHA-256 before and after it is read (reads "
+        "each mask whole twice more; default: as in the earlier run, else no)",
+    )
     limits = masks.add_argument_group(
         "relationship thresholds", "change the defaults (contract §15.1); shares are 0 to 1"
     )
@@ -73,6 +82,7 @@ def masks_command(args: argparse.Namespace) -> int:
         exclude=() if args.no_exclude else (tuple(args.exclude) or None),
         masks_folder=args.masks,
         mask_axes=args.mask_axes or None,
+        verify_content=args.verify_content,
     )
     if step.refusal is not None:
         print(format_issues([step.refusal]))
@@ -96,6 +106,9 @@ def progress(done: int, total: int, path: str) -> None:
     print()
     try:
         result = describe_masks(step, progress=progress, thresholds=thresholds)
+    except MaskChangedError as exc:  # changed since its header was read, or while read
+        print(format_issues([exc.issue]))
+        return 1
     except ValueError as exc:  # masks whose pixels cannot be used (planes, labels)
         print(f"The masks cannot be used: {exc}")
         return 1