Input support for MorphoRelate M3: single folders, physical sizes per axis (TIFF and OME-Zarr), embeddable notebook steps

AGENTS.md

@@ -62,10 +62,11 @@ src/morphobricks_preprocessing/
   discovery.py      folder walk, leaf folders, labels (condition / biological_repeat / technical_repeat),
                     position_id per image and the duplicate check
   metadata.py       TIFF header inspection (tifffile, no pixel reads), dtype/dimension/series checks,
-                    OME channel names + significant bits, physical pixel sizes; header consistency
-                    warnings (width/height, plane counts, pixel sizes)
-  pixel_sizes.py    PhysicalSizes from OME/ImageJ TIFF metadata and the OME-Zarr level-0 scale,
-                    sizes in micrometres for comparing (read_physical_sizes re-exported by zarr_export)
+                    OME channel names + significant bits, physical sizes (TiffMetadata.physical_sizes);
+                    header consistency warnings (width/height, plane counts, pixel sizes)
+  physical_sizes.py PhysicalSizes (pixel sizes, z-step, time step, a unit per axis, micrometers()
+                    for comparing), read_physical_sizes (OME / ImageJ TIFF metadata; also importable
+                    from zarr_export) and ngff_physical_sizes (OME-NGFF scale and axis units)
   dataset.py        dataset consistency from the folders: biological repeats per condition, balance
                     table, unbalanced_images, repeat_missing_conditions, CONSISTENCY_CODES (P2)
   design.py         experiment design (P3): ExperimentDesign/DesignRow, default pairing, reconcile with
@@ -85,14 +86,17 @@ src/morphobricks_preprocessing/
   run.py            preflight(), run_experiment() (incremental), sample_planes() for previews
   report.py         plain-language summaries shared by CLI and notebook, group statistics tables
   cli.py, __main__  `morphobricks-preprocessing check|run` (also `python -m morphobricks_preprocessing`)
-  zarr_input.py     OME-Zarr input: open_ngff (OME-NGFF 0.4/0.5 metadata -> level-0 array and axis
-                    letters), ngff_label_names/open_ngff_labels (label images, for later bricks),
+  zarr_input.py     OME-Zarr input: open_ngff (OME-NGFF 0.4/0.5 metadata -> level-0 array, axis
+                    letters and physical sizes from the scale and axis units),
+                    ngff_label_names/open_ngff_labels (label images, for later bricks),
                     zarr 2/3 helpers, canonical_dtype; needs the `zarr` extra
   zarr_export.py    TIFF -> OME-Zarr conversion (OmeZarrWriter, OME-NGFF 0.4, plane by plane or
                     block by block),
                     ConversionSettings (chunk size, compression, pyramid levels), estimate_conversion
   notebook_style.py notebook look and feel: cards, status cards, tiles, tables, ConfirmToggle, Browse dialogs
-  notebook_ui.py    PreprocessingNotebook: the four notebook steps (needs the `notebook` extra)
+  notebook_ui.py    PreprocessingNotebook: the four notebook steps (needs the `notebook` extra);
+                    folder_step/axis_step take title, purpose, intro and show_next (and
+                    show_output for step 1) so other bricks can embed them (plan §12 Q26)
 notebooks/MorphoBricks_Preprocessing.ipynb   thin notebook: every code cell calls notebook_ui only
 tests/              pytest with simulated TIFFs (tests/tiff_tree.py, tests/simulation.py)
 experiments/binning_study/      the study behind the binning decisions (not part of the package)
@@ -157,8 +161,10 @@ functions the tests use.
 - **Hierarchy:** `<experiment>/<condition>/<biological repeat>/<technical repeat>/<image>.tif`.
 - **Leaf folders:** a leaf folder has images and no subfolders. Only TIFFs directly in
   leaf folders are analysed.
-- **Missing levels:** a branch may stop early; missing labels are the reserved value
-  `single`. A folder *named* `single` (any case) is an error.
+- **Missing levels:** a branch may stop early; missing labels are the value `single`. A
+  folder *named* `single` (lower case) that is the only folder in its place is that level's
+  value `single` (a level with one member, plan §12 question 23); spelled in another case,
+  or beside other folders, it is the error `reserved_folder_name`.
 - **Uniqueness:** labels are unique only among siblings, compared case-insensitively;
   siblings differing only in case are an error. Groups are identified by the full label
   path.
@@ -356,6 +362,7 @@ Technical details go to `logging` (shown in the notebook's "Technical details" p
 | OME-Zarr (Q12) | Input may be TIFF and/or OME-Zarr (folders ending in `.zarr`), mixed. OME-Zarr axes come from their metadata and are matched by role; level 0 of a single image only, several images per store is `multiple_series`. TIFF choice once per experiment (`convert_tiffs`): read directly (default) or convert to `<output>/ome_zarr` (OME-NGFF 0.4) and compute rows from the copies; switching recomputes TIFF rows and removes unused copies. Reads OME-NGFF 0.4 and 0.5 (0.5 needs zarr 3). In the notebook: its own step-3 card "(Optional) Image conversion" after Settings (Q13): radio buttons (read directly = recommended), a disk-space warning (amber, red when the estimate exceeds free space, never blocking), and conversion settings (chunk size, compression + level, pyramid levels) shown only when converting. CLI `--convert-tiff` with `--chunk-size`, `--compression`, `--compression-level`, `--pyramid-levels`. No overlap setting: chunks never overlap |
 | System files (Q5) | Ignored silently; `single` check and sibling uniqueness are case-insensitive |
 | Too-deep folders (Q7) | Error |
+| Folders named `single` (Q23) | Accepted as the level's value `single` when lower case and the only folder in its place (MorphoBricks one-member level, MorphoRelate M3); another case or other folders beside it stay `reserved_folder_name` |
 | Channel names (Q1) | Manifest only (from OME), warning `channel_names_differ`; CSV stays at schema 1 |
 | Plane coordinates (Q17) | `z`, `c`, `t` always written (0 for absent axes), `plane_id` always `::z=…,c=…,t=…`; CSV schema 2, version-1 tables recomputed on the next run |
 | OME-NGFF labels (Q18) | Read-only helpers `ngff_label_names`/`open_ngff_labels` for later bricks (CellPatcher masks); label images are never analysed and the outputs do not change |
@@ -486,6 +493,11 @@ plainer version "not interactive enough". Descriptive, guided and forgiving is t
 
 ### Page layout (`notebook_style`)
 
+- **Embedding steps 1 and 2 (Q26):** `folder_step` and `axis_step` keep this notebook's
+  words by default; another brick (MorphoRelate) passes its own `title`, `purpose` and
+  `intro` (HTML; `""` leaves the box out), `show_output=False` to hide the output folder
+  card (shown again when the check reports `output_inside_experiment`), and `show_next=False` to hide the "What next?" box naming this notebook's
+  steps. The widgets and their behaviour are the same either way.
 - **Page and cards:** `ui.page(title, subtitle, children)` wraps a step in a grey page
   (`mb-root`) with a title and a one-sentence purpose. `ui.card(header, children)` gives a
   white card with an upper-case header. `ui.intro(html)` adds a blue "what happens here"
@@ -642,6 +654,11 @@ jupyter lab notebooks/MorphoBricks_Preprocessing.ipynb
   and contract §3–§7 moved to the normalization contract (PRs #16 and #17, question 19),
   and `position_id` filled per cross-brick decision D02 (question 20), and follow-ups
   P2 (dataset consistency warnings, question 21) and P3 (experiment design, question 22).
+- **Unreleased (for MorphoRelate milestone M3):** folders named `single` read as a
+  one-member level (question 23), a unit per axis in `PhysicalSizes` and physical sizes in
+  `inspect_image` (question 24), OME-Zarr physical sizes from the OME-NGFF scale (question
+  25), and notebook steps 1 and 2 embeddable with their own words (question 26). No
+  version bump or tag.
 - **Next (not started):** follow-ups P4–P5 of the normalization plan (image type,
   intensity checks and plots).
 - **Not yet verified:**

README.md

@@ -22,7 +22,7 @@ Pre-alpha. Implemented and tested:
 - TIFF header inspection without reading pixels (`metadata`)
 - Dataset consistency warnings in the folder check: images per condition and biological
   repeat, repeats missing from some conditions, differing plane counts and pixel sizes
-  (`dataset`, `pixel_sizes`)
+  (`dataset`, `physical_sizes`)
 - Axis mapping: prefill from metadata, validation, applying one mapping to every
   file (`axes`)
 - Reading TIFFs one 2D plane at a time, with a memory limit and a preview plane;

docs/contract.md

@@ -23,7 +23,7 @@ whole experiment; any change to them goes through the normalization brick.
 | Leaf folder | A folder that contains images and no subfolders. Only images directly in leaf folders are analysed. |
 | Missing levels | A branch may stop early; the missing labels are `single`. If the root itself is the leaf, all three labels are `single`. |
 | Skipped levels | Not possible: a level only exists below its parent level. |
-| Reserved name | A folder named `single` (in any upper/lower case) is an error. |
+| Folders named `single` | `single` is the MorphoBricks value of a level with one member, the value missing levels get. A folder named `single` in lower case is accepted as that value when it is the only folder in its place (system folders aside): `<condition>/single/<image>.tif` gives the biological repeat and the technical repeat `single`, the labels of `<condition>/<image>.tif`. It stays an error (`reserved_folder_name`) when it is spelled in another case (`Single`, a different label) or has other folders beside it (its level would then have several members, one called `single`). |
 | Uniqueness | Folder names need only be unique among siblings, ignoring case: siblings whose names differ only in case are an error. |
 | Depth | At most three folder levels below the root. A folder below the technical-repeat level is an error. |
 | System files | Dot-files and dot-folders (`.DS_Store`, `._*`, `.ipynb_checkpoints`), `Thumbs.db`, `desktop.ini` and `__MACOSX` are ignored without a warning. |
@@ -51,6 +51,7 @@ length 1 and no role, has coordinate 0.
 | OME-Zarr labels | The label images of an image's OME-NGFF `labels` group (0.4 and 0.5) are listed with `ngff_label_names` and opened, level 0, with `open_ngff_labels` (by name; the name may be left out when there is one). They are never analysed; later bricks such as patch extraction use them as masks. |
 | Experiment mapping | Defined on the TIFF arrangement when the experiment has TIFF files; in an experiment of OME-Zarr images only, on their arrangement with the most axes. |
 | Channel names, significant bits | Recorded in the run manifest when metadata provides them; not part of the CSV. Channel names that differ at the same channel index are a warning. |
+| Physical sizes | Read with the header, not part of the CSV or the manifest, and compared between images in the folder check (§11, in micrometres with each axis's own unit; a size without a known unit cannot be compared): `PhysicalSizes` with the size of one step along X, Y, Z and T and each axis's unit (OME-NGFF unit names). TIFF: OME `PhysicalSize{X,Y,Z}` with their own units (micrometres when left out) and `TimeIncrement`, or ImageJ resolution, `spacing` and `finterval` with `unit`, `yunit`, `zunit`. OME-Zarr (0.4 and 0.5): the `scale` of level 0's `coordinateTransformations`, times the multiscales entry's own `scale` when given, with each axis's `unit` (common spellings such as `µm`, `um`, `micron` and `s` given as UDUNITS-2 names, others as written). Ångström (`Å`) and ImageJ's escaped micro sign (`\u00B5m`) are recognised. Nothing is guessed: what the metadata does not give is None; a scale in a separate file (`path`) or not one number per axis gives no sizes; an OME-Zarr scale without a unit is kept with the unit None. |
 | Plane counts | Images may have different numbers of channels, depth slices, time points or planes along an unassigned dimension; this is a warning (§11). |
 
 ## 3. Pixel types
@@ -93,7 +94,7 @@ method is an offset and a scale, `(pixels − offset) / scale`.
 
 | Severity | Situations | Effect |
 |---|---|---|
-| Error | Experiment design not confirmed (`design_not_confirmed`) or a repeat group holding two folders of one condition (`repeat_group_conflict`), when a design is used (§12); mixed dtypes; unreadable TIFF (header or pixel data) or OME-Zarr folder in a leaf folder that is not excluded; OME-Zarr images, or converting TIFF images to OME-Zarr, without the `zarr` extra; OME-Zarr images whose declared axes do not fit the experiment's roles; every image excluded; more than one image series in a file; unsupported sample format or axis; missing/duplicate X or Y, duplicate roles, unassigned non-singleton axis, inconsistent mapping; plane that cannot be read within the memory limit, not even in blocks (for example a plane stored compressed in one piece); folder named `single`; sibling folders differing only in case; images in one leaf folder with the same position identifier (§1); folder below the technical-repeat level; output folder inside the experiment; no usable planes | Processing is blocked |
+| Error | Experiment design not confirmed (`design_not_confirmed`) or a repeat group holding two folders of one condition (`repeat_group_conflict`), when a design is used (§12); mixed dtypes; unreadable TIFF (header or pixel data) or OME-Zarr folder in a leaf folder that is not excluded; OME-Zarr images, or converting TIFF images to OME-Zarr, without the `zarr` extra; OME-Zarr images whose declared axes do not fit the experiment's roles; every image excluded; more than one image series in a file; unsupported sample format or axis; missing/duplicate X or Y, duplicate roles, unassigned non-singleton axis, inconsistent mapping; plane that cannot be read within the memory limit, not even in blocks (for example a plane stored compressed in one piece); folder named `single` in another case, or beside other folders (§1); sibling folders differing only in case; images in one leaf folder with the same position identifier (§1); folder below the technical-repeat level; output folder inside the experiment; no usable planes | Processing is blocked |
 | Warning | Files that are not images; empty folders; images outside leaf folders; images excluded by the user, or excluded paths that match no image; channel names differing between files; non-finite pixels; different image widths or heights; different numbers of channels, depth slices, time points or other planes; physical pixel sizes that differ or are known for only some images; unbalanced numbers of images; biological repeats missing from some conditions (§11); planes without finite pixels while others remain; planes over the memory limit that are read in blocks | Processing continues; the item is reported |
 
 Every message states what happened in plain language, lists the affected paths, and
@@ -162,11 +163,12 @@ presentation does not change the conversion rules below.
 | Level 0 | The source values, unchanged, in the source pixel type; chunks of one plane along Z, C and T and at most the chunk size (16 to 8,192 px, default 1,024) in X and Y |
 | Further levels | 2 x 2 block means rounded to the pixel type; automatic (default: until a plane fits in 512 x 512, at most 8 levels in all), none, or 1 to 7 levels; for viewing only |
 | Large planes | A plane read in blocks is written block by block; its further levels are then built from the stored level above, in regions of even size, and equal those of a plane written whole |
-| Scale | Physical pixel sizes, z-step and time step from OME or ImageJ metadata, with OME-NGFF units; 1 when unknown |
+| Scale | Physical pixel sizes, z-step and time step from OME or ImageJ metadata, with OME-NGFF units, each axis with its own unit (OME `PhysicalSizeXUnit`, `PhysicalSizeYUnit`, `PhysicalSizeZUnit`, micrometres when left out; ImageJ `unit`, `yunit`, `zunit`); 1 when unknown |
 | Channels | `omero` block with channel names from metadata (or `Channel <i>`) and each channel's observed minimum and maximum as display window |
 | Updates | Copies of unchanged images are kept, of new or changed images rewritten, and of images no longer analysed removed; a failed run leaves the earlier copies as they were (see §9 for publication) |
 | Switching | Turning conversion on or off recomputes every TIFF row; turning it off removes the copies the manifest recorded |
 | Missing copy | A converted TIFF whose copy was deleted counts as changed and is converted again |
+| Outdated units | A copy whose axis units differ from those its TIFF's metadata gives now (copies written before each axis had its own unit, plan §12 question 24) counts as changed and is converted again; its rows do not change. Only the copy's `.zattrs` is read to tell |
 | Settings change | Copies written with other conversion settings (recorded in the manifest) are written again; their rows do not change |
 
 ## 11. Dataset consistency (folder check)