v0.4.0rc5: tie-break gives no parent, ImageJ unit without resolution, plan Q66/Q73/Q74, changelog

AGENTS.md

@@ -180,6 +180,8 @@ src/morphobricks_preprocessing/
                     scan_masks -> MaskReport (kinds, relations, direct-parent hierarchy, evidence),
                     write_mask_report (mask_relationships.json); DEFAULT_THRESHOLDS,
                     mask_thresholds (checked changes)
+  mask_pairs.py     pair_counts' pure helpers: best_parents (a tie between parents gives no
+                    parent, ambiguous_parent; Q74), one_to_one_matches and match_counts
   copy_axes.py      convert (PP-11, contract §17): the axes, shape, pixel sizes (by role; a
                     relabelled axis gets an unknown scale; equal X and Y kept through a Y/X
                     swap, question 59) and OME metadata of a named-axis copy

CHANGELOG.md

@@ -6,15 +6,67 @@ The design is `docs/restart/preprocessing-design-v0.4.md` in MorphoFeature. The
 are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the
 [migration guide](docs/migration-0.3-to-0.4.md).
 
-## 0.4.0 (unreleased; release candidates rc1 to rc4)
+## 0.4.0rc5 (since v0.4.0rc4)
+
+What changed between v0.4.0rc4 and v0.4.0rc5: PRs #139 (cross-brick test of 3 November
+2026), #140 (review of rc4), #142 (round-2 audit) and #143 (this release candidate). Each
+item is described in full under 0.4.0 below. Decisions: [plan](docs/plan.md) §12 questions
+66, 72, 73 and 74.
+
+### Migration notes (from rc4)
+
+- **Mount points are refused** (#140): a mount point below the experiment or masks folder
+  is the error `mounted_folder` in discovery, `check`, `run`, `convert` and `RunInputs`.
+  Copy the files into the experiment, or select the mounted folder itself. A file with
+  several hard links is the warning `hard_linked_file`.
+- **`position_id` is required in a version 2 manifest** (#142, A-m5), equal to the image's
+  name without its extension; anything else is a `ManifestFormatError`. Manifests written
+  by rc1 to rc4 open unchanged (rc4's is tested); a hand-edited one may not.
+- **Sizes in micrometres may change by up to 1 ULP** (#142, B-m7) when computed from a unit
+  other than µm (`9 nm` is now `0.009`, not `0.009000000000000001`). Manifests are
+  unchanged; exact comparisons of the same length in two units now agree.
+- **`--verify-content`** (#140, I3), opt-in: rows are reused only for the same content
+  SHA-256 (`RunMode(verify_content=True)`, the notebook's **Verify each image's content**).
+  Turning it on or off recomputes every row; a default run's manifest has the rc4 keys.
+- **A child tied between parents has no parent** (#143, plan Q74): a child lying exactly as
+  much in two or more parents is no longer given the one with the lower label. When that
+  share reaches `inside_pixel_share` (at the default, exactly half in each of two parents)
+  it is counted as `ambiguous_parent`, not `inside`; a tie below it stays
+  `split_between_parents`, as before. `mask_relationships.json` is report version
+  3; `inside` and `parents_with_N_children` may be lower than with rc4 when masks have such
+  ties. CellPatcher and MorphoFeature read neither count.
+
+### Changed and fixed
+
+- `pair_counts`: an exact tie between parents gives no parent; the new evidence
+  `ambiguous_parent` is shown by `masks`, the notebook's **Describe the masks** card and its
+  relationship table; report version 3 (#143, plan Q74).
+- An ImageJ TIFF with a unit but neither resolution tag is `invalid_physical_calibration`
+  with the reason; before, uncalibrated with no reason. `read_physical_sizes` gives a TIFF
+  without pages no sizes with the reason in `invalid`; in `check` and `run` such a file is
+  the error `unreadable_tiff`, as before (#143).
+- A TIFF calibration that cannot be read is invalid, and defects are raised (#142, A-m6);
+  `position_id` required in v2 (#142, A-m5); one length in any unit is one double (#142,
+  B-m7).
+- Mount points refused, hard links named (#140, B1/I1); the output folder flushed after the
+  commit (#140, I2); the identity rule stated and `--verify-content` added (#140, I3).
+- `masks` accepts per-plane masks with a stated shared 2-D mask (F1); `run` needs no
+  `--axes` when every image names its axes (F9); a rerun that changes nothing keeps the
+  manifest byte-identical (F10); stale locks and resumed runs are worded (F11); `convert
+  --output` and the order of copies are explained (F8) (#139, plan Q73).
+- Plan Q66 confirmed by the owner; Q73's follow-up (a stated page reshape or reordered
+  copies) is not in v0.4 (issue #144).
+- Version 0.4.0rc5; the notebook installs `v0.4.0rc5`.
+
+## 0.4.0 (unreleased; release candidates rc1 to rc5)
 
 Preprocessing becomes the one step that discovers, validates, records and serves an
 experiment. Every later brick reads the run through `RunInputs`. Nothing is guessed: axes,
 calibration, masks and design come from the headers or from the user's recorded answers.
 
 Release candidates: **v0.4.0rc1** (after PP-6, #112), **v0.4.0rc2** (after PP-8d, #118),
-**v0.4.0rc3** (#125, after PP-10d) and **v0.4.0rc4** (PP-13, for the round-2 data-flow audit
-and the bricks' migration).
+**v0.4.0rc3** (#125, after PP-10d), **v0.4.0rc4** (PP-13, for the round-2 data-flow audit
+and the bricks' migration) and **v0.4.0rc5** (#143, after the round-2 audit fixes).
 
 ### Breaking
 
@@ -111,7 +163,18 @@ and the bricks' migration).
   no calibration; any other exception is raised, and a value that is not a path is a
   `TypeError`. An ImageJ resolution tag that tifffile dropped because it could not read it
   (it only logs), next to the other one, is named instead of leaving that axis silently
-  uncalibrated.
+  uncalibrated. An ImageJ unit with neither resolution tag ("a unit without a resolution")
+  is `invalid_physical_calibration` too, no longer uncalibrated with no reason (owner, 3
+  November 2026; #143). `read_physical_sizes` gives a TIFF file without pages ("no pages")
+  no sizes with the reason in `invalid`; the folder check reports such a file as the error
+  `unreadable_tiff`, as before.
+
+- **A child lying equally in two or more parents has no parent** (owner, 3 November 2026;
+  plan §12 question 74, #143): `pair_counts` gave it the parent with the lower label index,
+  so `inside` and `parents_with_N_children` depended on the label numbering. An exact equal
+  largest overlap now gives no parent; when the share would make the child inside it is
+  counted as `ambiguous_parent` (new evidence), shown by `masks`, the notebook and the
+  relationship table. `mask_relationships.json` is report version 3.
 
 - **A version 2 manifest names every image's position by its name** (round-2 audit, A-m5):
   the reader accepted a file entry without `position_id`, or with null, and `RunInputs`

README.md

@@ -110,7 +110,7 @@ LabConstrictor's
    needs ipywidgets 8). Locally Step 0 prints the Python version, platform and working folder.
    No shell or line magics are used; pip runs with the kernel's Python.
 3. When the package is missing (always on a fresh Colab runtime), Step 0 installs the release
-   named by `RELEASE_TAG` (currently `v0.4.0rc4`), never the moving `main` branch, and prints
+   named by `RELEASE_TAG` (currently `v0.4.0rc5`), never the moving `main` branch, and prints
    which source and version it uses. A checkout around the notebook, or an installed package,
    is used first.
 4. `current_version` and `notebook_name` in Step 0 are the names LabConstrictor's version check

docs/contract.md

@@ -342,7 +342,7 @@ row (follow-up P3b, normalization plan N43).
 | `different_pixel_sizes` | Warning | The physical pixel size (X by Y) differs between images, or the z-step differs between images with more than one depth slice. Sizes are compared in micrometres by the shared rule of decision D-a (§16): two sizes are the same when they differ by at most 0.1 % of the larger, on every axis. The warning is given exactly when, on some axis, the smallest and the largest size differ, that is when some two images differ. The images are grouped by single linkage (`pixel_size_groups`) and those outside the largest group are listed. When every image is linked to the others by near-equal sizes (1.0, 1.0009, 1.0018 µm), the sizes drift, no image stands out, and the message says that none is listed for that size. Until D86 sizes were rounded to four significant digits, so 1.0004 and 1.0006 µm differed (1 and 1.001); they are now the same. |
 | `missing_pixel_size` | Warning | The pixel size (or the z-step of images with more than one depth slice) is known for some images and not for others. Nothing is reported when no image has it. |
 | `unsupported_pixel_size_unit` | Warning | An image's metadata give a pixel size or z-step (OME `PhysicalSize{X,Y,Z}`, ImageJ resolution or `spacing`, OME-Zarr level-0 scale) in a length unit that is not one of decision D-a2 (§16), for example OME `in` or ImageJ `inch`. The size is not converted and the axis is treated as uncalibrated (as before); the warning names each image, its axes and the unit as written, so the missing calibration is never silent. An OME-Zarr scale of X, Y or Z without a unit is named the same way ("X and Y without a unit"; review of #136): it is recorded as unknown (audit I3), never taken as micrometres, except on the axes listed as `unknown_scale` (§10). A unit without a size, and a plain TIFF's resolution tags (decision D-c), are not reported. The run is not refused. |
-| `invalid_physical_calibration` | Warning | An image's metadata give a calibration that cannot be used (issue #94): OME-Zarr `coordinateTransformations` that are not a list or of an unsupported type, more than one scale, a scale stored in a separate file or not one number per axis, or a scale value of X, Y, Z or T that is not a positive finite number; or, from any TIFF or OME-Zarr metadata, a size that is not a positive finite length in micrometres once converted (issue #93: `1e308 m`, `1e-320 pm`); or OME-XML in a TIFF that cannot be parsed (audit m6: before, any error reading the sizes read as merely uncalibrated); or TIFF metadata whose bytes cannot be read (`OSError`, `struct.error`, `TiffFileError` while a tag is read), or an ImageJ `XResolution` or `YResolution` tag that is missing or unreadable while the other is given (tifffile drops a tag it cannot read with only a log message), with a unit (round-2 audit A-m6); `read_physical_sizes` also gives a file it cannot open as a TIFF no sizes with the reason in `invalid` (in the folder check such a file is `unreadable_tiff`), and raises any other exception as the defect it is; or a size of OME-TIFF (`PhysicalSizeX/Y/Z`, `TimeIncrement`) or ImageJ metadata (`spacing`, `finterval` and the resolution tags, with a unit) that is itself not a positive finite number (`inf`, `NaN`, zero, negative or not a number; review of #133, before silently unknown); or an ImageJ `XResolution` or `YResolution` tag, with a unit, that is not two whole numbers (a tag of another type; review of #136, before silently unknown). The message names each image and what is wrong, with the axis and value. Its sizes are unknown on those axes (never a substituted value) and recorded as uncalibrated; the warning is in the `check` and `run` issues and the report (`discovery_report.json`). An image is not named once every size the malformed calibration leaves unknown (X and Y, and Z or T with more than one plane) is stated in `calibration.csv`; a time step cannot be stated there, so a lost time step of a time series is always named. A calibration that is not given (no transformations, or no scale) is not reported. The run is not refused. |
+| `invalid_physical_calibration` | Warning | An image's metadata give a calibration that cannot be used (issue #94): OME-Zarr `coordinateTransformations` that are not a list or of an unsupported type, more than one scale, a scale stored in a separate file or not one number per axis, or a scale value of X, Y, Z or T that is not a positive finite number; or, from any TIFF or OME-Zarr metadata, a size that is not a positive finite length in micrometres once converted (issue #93: `1e308 m`, `1e-320 pm`); or OME-XML in a TIFF that cannot be parsed (audit m6: before, any error reading the sizes read as merely uncalibrated); or TIFF metadata whose bytes cannot be read (`OSError`, `struct.error`, `TiffFileError` while a tag is read), or an ImageJ `XResolution` or `YResolution` tag that is missing or unreadable while the other is given (tifffile drops a tag it cannot read with only a log message), with a unit (round-2 audit A-m6); or an ImageJ unit with neither resolution tag ("a unit without a resolution"; owner, 3 November 2026, v0.4.0rc5: before, uncalibrated with no reason); or, from `read_physical_sizes`, a TIFF file without pages ("no pages", v0.4.0rc5: tifffile opens it with only a log message; in the folder check such a file is `unreadable_tiff`, as before); `read_physical_sizes` also gives a file it cannot open as a TIFF no sizes with the reason in `invalid` (in the folder check such a file is `unreadable_tiff`), and raises any other exception as the defect it is; or a size of OME-TIFF (`PhysicalSizeX/Y/Z`, `TimeIncrement`) or ImageJ metadata (`spacing`, `finterval` and the resolution tags, with a unit) that is itself not a positive finite number (`inf`, `NaN`, zero, negative or not a number; review of #133, before silently unknown); or an ImageJ `XResolution` or `YResolution` tag, with a unit, that is not two whole numbers (a tag of another type; review of #136, before silently unknown). The message names each image and what is wrong, with the axis and value. Its sizes are unknown on those axes (never a substituted value) and recorded as uncalibrated; the warning is in the `check` and `run` issues and the report (`discovery_report.json`). An image is not named once every size the malformed calibration leaves unknown (X and Y, and Z or T with more than one plane) is stated in `calibration.csv`; a time step cannot be stated there, so a lost time step of a time series is always named. A calibration that is not given (no transformations, or no scale) is not reported. The run is not refused. |
 | `partial_pixel_size` | Warning | An image's recorded pixel size (header or `calibration.csv`) is known in micrometres along only one of X and Y; the other is unknown or stated as `pixels` (audit m5, plan §12 question 72). The message names each image with both sizes. Bricks that measure lengths refuse such an image (calibration decision 8, 2 November 2026): a partial calibration is never completed or approximated. The run is not refused. |
 | `non_square_pixels` | Warning | An image's recorded X and Y sizes are both known but not exactly equal in micrometres (calibration decision 1: no tolerance, so 0.5 and 0.5004 µm differ). The message names each image with both sizes. Bricks that measure lengths (perimeter, Feret diameter, axis lengths) refuse such an image; areas and intensities are exact. The run is not refused. |
 | `different_channel_counts` | Error | The number of channels differs between images: the `C` dimension, or the RGB samples `S` when there is no `C`; once the axis roles are confirmed, the length of the C axis (1 without one), which replaces the header's count. The images not having the most common number are listed, with each number's image count. Every image of an experiment must have the same channels: split the others into another experiment or exclude them. |
@@ -448,7 +448,7 @@ masks/                         masks folder, outside the experiment folder
 | Warnings | `mask_missing` (images without a mask of a type that other images have: their objects of that type cannot be measured), `mask_without_image` (files matching no image of the experiment), `mask_of_excluded_image` (masks of excluded images, not used), `mask_outside_type_folder` (files directly in the masks folder that are not TIFF masks). |
 | Run | `run_experiment(..., masks_folder=..., mask_axes=...)` stops on a mask error and records the masks in the manifest's `masks` key (§9). Without masks, the run and its outputs are unchanged. |
 
-### 15.1 The mask step (`mask_relationships.json`, report version 2)
+### 15.1 The mask step (`mask_relationships.json`, report version 3)
 
 The optional mask step reads every image's masks, one image at a time, and describes each
 mask type and every ordered pair of types over the whole experiment. Pieces are counted with
@@ -473,7 +473,12 @@ plane it is in, and `images` still counts images.
 
 For a child type and a parent type, the counts are summed over the images (and planes) with
 both masks.
-Each child object's parent is the parent object sharing most of its pixels.
+Each child object's parent is the parent object sharing most of its pixels. When two or more
+parents share exactly that many (a child lying half in each of two cells), the child has
+**no parent**, so that no count depends on how the objects are numbered (owner, 3 November
+2026; before, the parent with the lower label was taken): it is never `inside`, and when
+the tied share would have made it inside it is counted as `ambiguous_parent`. Each child is
+counted once in `inside`, `ambiguous_parent`, `split_between_parents` or `outside_parents`.
 
 | Relation of child to parent | Rule (default thresholds; the values used are written in the report) |
 |---|---|
@@ -485,9 +490,10 @@ Each child object's parent is the parent object sharing most of its pixels.
 | Evidence (per pair) | Meaning |
 |---|---|
 | `children`, `parents` | Objects of each type. |
-| `inside`, `inside_parent_larger` | Children with at least half of their pixels in one parent; of those, children whose parent is larger. |
-| `split_between_parents`, `outside_parents` | Children touching parents but with less than half of their pixels in any one; children touching no parent. |
-| `parents_with_0_children`, `parents_with_1_child`, `parents_with_2_children`, `parents_with_3_or_more_children` | How many children each parent holds (two nuclei in one cell counts as 2). |
+| `inside`, `inside_parent_larger` | Children with at least half of their pixels in one parent, which is their parent (no tie); of those, children whose parent is larger. |
+| `ambiguous_parent` | Children with at least half of their pixels in a parent, for which two or more parents share that many pixels exactly: they have no parent and are not `inside` (report version 3). `masks`, the notebook's **Describe the masks** and the relationship table name them. |
+| `split_between_parents`, `outside_parents` | Children touching parents but with less than half of their pixels in any one (tied or not); children touching no parent. |
+| `parents_with_0_children`, `parents_with_1_child`, `parents_with_2_children`, `parents_with_3_or_more_children` | How many `inside` children each parent holds (two nuclei in one cell counts as 2); a child with no parent counts for none. |
 | `child_pixels`, `child_pixels_with_parent_label` | Child pixels; those whose label equals the parent label under them. |
 | `children_matched_one_to_one`, `parents_matched_one_to_one` | Objects matched one to one (the maximum matching above, at an intersection over union of at least 0.5); the two counts are equal. The number of matches does not depend on which of several equally large matchings is found, so it is deterministic. |
 | `children_not_matched`, `parents_not_matched` | Objects of each type left without a match. |
@@ -512,7 +518,7 @@ The report is UTF-8 JSON (two-space indent, LF), written atomically to the outpu
 
 | Key | Meaning |
 |---|---|
-| `report_version` | `2` (since issue #67: the one-to-one matches are a matching, and the four counts of unmatched and several-partner objects were added). |
+| `report_version` | `3` (since v0.4.0rc5: a tie between parents gives no parent, and `ambiguous_parent` was added; plan §12 question 74). `2` since issue #67: the one-to-one matches are a matching, and the four counts of unmatched and several-partner objects were added. |
 | `thresholds` | The thresholds used, all seven. |
 | `images` | Images scanned (every image with at least one mask). |
 | `masks` | Per mask type: `kind`, `images`, `objects` (labels over all images) and `median_pieces_per_label`. |