Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ src/morphobricks_preprocessing/
metadata.py TIFF header inspection (tifffile, no pixel reads), dtype/dimension/series checks,
OME channel names + significant bits, physical sizes (TiffMetadata.physical_sizes);
header consistency warnings (width/height, plane counts, pixel sizes);
header_axes (the axes the metadata names, contract §16)
header_axes and named_axes (the axes the metadata names, contract §16)
physical_sizes.py PhysicalSizes (pixel sizes, z-step, time step, a unit per axis, micrometers()
for comparing), space_unit/to_micrometers (the shared unit rule, contract §16),
read_physical_sizes (OME / ImageJ TIFF metadata; also importable
Expand Down
4 changes: 2 additions & 2 deletions docs/contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -406,8 +406,8 @@ instead of keeping copies.
| Length units (D-a2) | `space_unit(text)`, `to_micrometers(value, unit)`, `SPACE_UNITS`, `UnsupportedUnitError` (`physical_sizes`) | Microscopy units only: `pm`; `Å`/`angstrom`; `nm`; `µm`/`um`/`micron(s)`/`micrometer(s)`/`micrometre(s)`; `mm`; `cm`; `m`; with each name in US and British spelling and in the plural (`nanometre`, `millimeters`, ...). Symbols of two letters or fewer are matched exactly (`Mm` is not `mm`), longer names in any case; ImageJ's `\u00B5` is decoded. `space_unit` returns the OME-NGFF name (`"micrometer"`), None when no unit is given, and refuses any other unit by name (`UnsupportedUnitError`); `to_micrometers` converts a positive finite length, None when the length or its unit is not known. This brick's readers (§2) use the same table: since D86, ImageJ and OME-NGFF sizes in British spellings (`micrometre`) are calibrated, where they were unknown before. |
| Plain-TIFF resolution (D-c) | `read_physical_sizes` | A TIFF's resolution tags give a pixel size only with OME or ImageJ metadata and a unit; a plain TIFF (scanner or image-editor dpi) is uncalibrated, never converted from inches or centimetres. |
| Labels (D-b) | `check_labels(pixels, name)`, `LARGEST_LABEL` (`masks`) | A mask holds whole-number labels from 0 (background) to `LARGEST_LABEL` = 2,147,483,647: MorphoBricks labels are int32 in every brick. A negative label, a label above the limit, or a pixel type that is not integer or boolean is refused with a `ValueError` naming the mask and the value. `read_mask`, `piece_counts`, `pair_counts` and `scan_masks` call it. Since D86 the limit is int32; before, this brick refused only labels above 2^63 − 1 (int64), so a `uint32` or wider mask with a label of 3,000,000,000 is now refused. |
| Header axes (D-d) | `header_axes(meta)`, `PLACEHOLDER_AXES` (`metadata`) | The axes an image's own metadata names, as `tifffile` reads them with length-1 axes left out: OME, ImageJ and `tifffile` "shaped" metadata (`{"axes": "CYX"}`, decision D-d), a plain 2D `YX` (or `YXS` RGB) TIFF, and an OME-Zarr image's OME-NGFF axes. None when an axis is a placeholder (`I`: a plain stack of pages; `Q`: shaped without axes, or an OME-NGFF axis of another type), or the image does not open: the file does not say which dimension is depth, time or channel. |
| Declared axes | `check_declared_axes(meta, declared)` (`axes`) | Axes a brick or user declares for an image or mask (`image_axes="CYX"`) may name what the header leaves open, never contradict it: one letter per header axis (any case), roles X, Y, Z, C, T each at most once; an axis the header names keeps its letter; a placeholder axis may be declared Z, C or T (never X or Y); the RGB samples `S` may be declared C when the header has no C; any other header axis (tile `R`, ...) cannot be declared. Otherwise the error `declared_axes_mismatch` naming the image and both axes (`unreadable_tiff` or `unreadable_ome_zarr` when the header does not open). |
| Header axes (D-d) | `header_axes(meta)`, `named_axes(meta)`, `series_axes_from_metadata(series)`, `PLACEHOLDER_AXES`, `OPEN_AXIS` (`metadata`) | The axes an image's own metadata names, as `tifffile` reads them with length-1 axes left out: OME, ImageJ and `tifffile` "shaped" metadata (`{"axes": "CYX"}`, decision D-d), a plain 2D `YX` (or `YXS` RGB) TIFF, and an OME-Zarr image's OME-NGFF axes. None when an axis is a placeholder (`I`: a plain stack of pages; `Q`: shaped without axes, or an OME-NGFF axis of another type), or the image does not open: the file does not say which dimension is depth, time or channel. `named_axes` gives the rule position by position, with `?` (`OPEN_AXIS`) for a dimension the metadata does not name: a placeholder, and, in a plain stack of pages without axis metadata (`IYX`, `QYX`, `QQYX`; `TiffMetadata.axes_from_metadata` False, from `series_axes_from_metadata`; True, the strict reading, for a header built by hand), the `Y` and `X` too, which only say where each page's rows and columns are stored (a (Y, X, C) array written without metadata reads as `QYX`). `header_axes` gives axes exactly when nothing is open, and `check_declared_axes` then accepts those axes, them with `Y` and `X` swapped, and (without a C axis) `S` declared C; nothing else. |
| Declared axes | `check_declared_axes(meta, declared)`, `declared_axes_transposed(meta, declared)` (`axes`) | Axes a brick or user declares for an image or mask (`image_axes="CYX"`) may name what the header leaves open, never contradict it: one letter per header axis (any case), roles X, Y, Z, C, T each at most once, with `Y` and `X` declared when the header has them (a plain stack too); an axis the metadata names (`named_axes`) keeps its letter, except that the named `Y` and `X` may be declared swapped (header `CYX` declared `CXY`, `ZYX` declared `ZXY`): the image is read transposed, and `declared_axes_transposed` returns True so the caller can record it (owner decision of 31 October 2026; it raises `ValueError` for a declaration that does not fit); any other disagreement is refused; a dimension it leaves open may be declared any role, so a plain `QYX` stack may be declared `YXC` as well as `CYX` (since v0.3.4; before, its positional `Y` and `X` were binding and a placeholder could not be declared X or Y), while an ImageJ `ZYX` declared `CYX`, an OME `CYX` declared `YXC` or a shaped `CYX` declared `YXC` are refused; the RGB samples `S` may be declared C when the header has no C; any other header axis (tile `R`, ...) cannot be declared. Otherwise the error `declared_axes_mismatch` naming the image and both axes (`unreadable_tiff` or `unreadable_ome_zarr` when the header does not open). |
| Mask files | `read_mask_file(path, z=, t=)` (`masks`) | Reads one plane of a TIFF mask or of an OME-Zarr label image (`<image>.zarr/labels/<name>`) given by its path, with `read_mask`'s rules (§15): its axes are its own metadata's, Y and X plus at most a Z and a T axis named so; any other or unnamed dimension (a plain stack, `IYX`; a `C` axis) is refused, never taken for depth or time; `z` and `t` are required for a mask with that axis; integer or boolean pixels; labels checked with `check_labels`. A `ValueError` names the file and the problem. |
| Identity (D-e) | `FOV_KEY`, `check_identity_text(value, field)`, `check_plane_index(value, axis)`, `check_identity_case(keys)`, `MISSING_TEXT`, `IdentityError` (`discovery`) | `FOV_KEY` = (`condition`, `biological_repeat`, `technical_repeat`, `position_id`), the position key of §1. `check_identity_text` accepts a label or `position_id` that is text without leading or trailing whitespace and not `""`, `NA`, `nan`, `none`, `null`, `n/a` or `<NA>` in any case (`MISSING_TEXT`); `check_plane_index` accepts None or an integer of at least 0, a Python `int` or a NumPy integer (returned as a plain `int`), and refuses everything else (bool, float, `"1"`) without converting it. Both raise `IdentityError` (a `ValueError`) naming the field and the value. Discovery applies the text rule to every folder and image name (`invalid_name`) and refuses names of one level that differ only in case anywhere in the experiment (`case_conflict`, §1). `position_id` keeps its own scope: unique within its technical repeat, ignoring case (`duplicate_position_id`). `check_identity_case` applies the same case rule to identities a brick reads from a table, one key per row in `FOV_KEY` order (or the three labels): a label of one level spelled two ways anywhere in the run, or two `position_id` values of one technical repeat equal ignoring case, raise `IdentityError` naming both spellings; the same key may repeat (several planes of one position). |
| Sources of a run | `check_sources(manifest, experiment_root, relative_paths, image_axes=, identities=)`, `source_fingerprint`, `file_sha256` (`manifest`) | The checks a brick makes before using a run's table for some images (formerly CellPatcher `check_images` and MorphoFeature `histogram_fit`). Each image must be in the manifest (`images_not_in_manifest`), still exist (`images_missing`), and be unchanged by `source_fingerprint`, its size and modification time, for an OME-Zarr folder its files' total size and newest time, so a rewritten chunk counts (`images_changed`). Its recorded `position_id` must be the one its name gives; with `identities` (`FOV_KEY` order), its labels and `position_id` must be the caller's (`identity_differs`; `position_id_not_recorded` for a manifest written before positions were recorded). With `image_axes`, the caller's axes must give each axis the role recorded for the file (`FileEntry.roles`, or the run's mapping resolved for the header in older manifests); an axis without a role (length 1) fits any letter (`axis_mapping_mismatch`). All errors, naming their images; the table's own hash is checked with `file_sha256` against `csv_sha256`. |
Expand Down
12 changes: 12 additions & 0 deletions docs/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -938,6 +938,18 @@ Raised during review; to be decided before the milestones that need them:
only drift and no image stands out; it replaces the four-significant-digit rounding.
Owner's follow-up decisions: `null`, `n/a` and `<NA>` are refused as names too, and
`check_plane_index` returns NumPy integers as plain ints.*
*Amended (31 October 2026, for v0.3.4): only the axes the metadata names bind a
declaration. The `Y` and `X` `tifffile` reports for a plain stack of pages (`IYX`,
`QYX`, `QQYX`, no axis metadata) only say where each page's rows and columns are
stored, so like the placeholders they are left open: a plain `QYX` stack may be
declared `YXC` or `CYX`. `named_axes` is the one rule behind `header_axes` and
`check_declared_axes`; a declaration the metadata contradicts (ImageJ `ZYX` as `CYX`,
OME or shaped `CYX` as `YXC`) is still refused. MorphoRelate's workaround (checking only
when `header_axes` names every axis) is no longer needed. A declaration that differs
from the named axes only by swapping Y and X (`CYX` declared `CXY`) is accepted: the
image is read transposed, and `declared_axes_transposed` tells the caller so it can
record it (as CellPatcher and MorphoFeature allowed). Any other disagreement stays
refused.*
39. What does v0.3.3 hold?
*Decided (31 October 2026): v0.3.2 and the shared rules of question 38 (PRs #49–#55),
with the warning `unsupported_pixel_size_unit` naming an image, axis and unit outside
Expand Down
8 changes: 8 additions & 0 deletions src/morphobricks_preprocessing/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
axis_length_ranges,
check_declared_axes,
check_mapping,
declared_axes_transposed,
declared_mapping,
file_mapping_for,
prefill_experiment_mapping,
Expand Down Expand Up @@ -121,6 +122,7 @@
read_mask_file,
)
from .metadata import (
OPEN_AXIS,
PLACEHOLDER_AXES,
ImageMetadata,
MetadataReport,
Expand All @@ -132,6 +134,8 @@
inspect_images,
inspect_ome_zarr,
inspect_tiff,
named_axes,
series_axes_from_metadata,
)
from .physical_sizes import (
PIXEL_SIZE_TOLERANCE,
Expand Down Expand Up @@ -217,6 +221,7 @@ def __dir__() -> list[str]:
"MASK_NAME_SUFFIXES",
"MASK_REPORT_FILENAME",
"MISSING_TEXT",
"OPEN_AXIS",
"OUTPUT_FILENAME",
"PIXEL_SIZE_TOLERANCE",
"PLACEHOLDER_AXES",
Expand Down Expand Up @@ -298,6 +303,7 @@ def __dir__() -> list[str]:
"check_table",
"compute_block_histogram",
"compute_plane_histogram",
"declared_axes_transposed",
"declared_mapping",
"default_bin_count",
"default_design",
Expand Down Expand Up @@ -329,6 +335,7 @@ def __dir__() -> list[str]:
"mask_plane_issues",
"mask_thresholds",
"min_max",
"named_axes",
"ngff_label_names",
"ome_zarr_relative_path",
"open_ngff",
Expand Down Expand Up @@ -362,6 +369,7 @@ def __dir__() -> list[str]:
"same_pixel_size",
"sample_planes",
"scan_masks",
"series_axes_from_metadata",
"source_fingerprint",
"space_unit",
"to_micrometers",
Expand Down
80 changes: 67 additions & 13 deletions src/morphobricks_preprocessing/axes.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,13 @@
from dataclasses import dataclass, field
from types import MappingProxyType

from .metadata import PLACEHOLDER_AXES, MetadataReport, TiffMetadata, channel_count_issues
from .metadata import (
OPEN_AXIS,
MetadataReport,
TiffMetadata,
channel_count_issues,
named_axes,
)
from .validation import ValidationIssue, error, has_errors, plural

ROLES = ("X", "Y", "Z", "C", "T")
Expand Down Expand Up @@ -240,18 +246,25 @@ def check_declared_axes(meta: TiffMetadata, declared: str) -> ValidationIssue |

One rule for every brick that is told an image's (or mask's) axes, for example
``image_axes="CYX"`` (MorphoBricks D86): a declaration can name what the header leaves
open, never contradict it. Position by position:

- an axis the metadata names (:func:`.metadata.header_axes`: X, Y, Z, C, T) must be
declared with that letter;
- a placeholder axis (``I``, ``Q``: the file does not say) may be declared Z, C or T,
never X or Y;
open, never contradict it. Only what the metadata names binds
(:func:`.metadata.named_axes`, the same rule as :func:`.metadata.header_axes`).
Position by position:

- an axis the metadata names (X, Y, Z, C, T) must be declared with that letter, except
that the named Y and X may be declared swapped (header ``CYX`` declared ``CXY``): the
image is then read transposed, which :func:`declared_axes_transposed` reports so the
caller can record it (owner decision, 31 October 2026);
- a dimension it leaves open may be declared any role: a placeholder axis (``I``,
``Q``), and in a plain stack of pages without axis metadata (``IYX``, ``QYX``,
``QQYX``) the ``Y`` and ``X`` too, which only say where the pages' rows and columns
are stored (a (Y, X, C) array written without metadata reads as ``QYX``);
- the RGB samples axis ``S`` may be declared C when the header has no C (as the axis
review's prefill does);
- any other header axis (tile ``R``, angle ``A``, ...) has no role and cannot be
declared.

The declaration must have one letter per header axis, each role at most once.
The declaration must have one letter per header axis, each role at most once, and
declare Y and X when the header has them (also where a plain stack leaves them open).

Parameters
----------
Expand All @@ -276,7 +289,7 @@ def check_declared_axes(meta: TiffMetadata, declared: str) -> ValidationIssue |
"Export the image again, or leave it out.",
)
text = declared.upper() if isinstance(declared, str) else ""
problem = _declared_axes_problem(meta.axes, text)
problem = _declared_axes_problem(meta.axes, named_axes(meta), text)
if problem is None:
return None
return error(
Expand All @@ -289,7 +302,41 @@ def check_declared_axes(meta: TiffMetadata, declared: str) -> ValidationIssue |
)


def _declared_axes_problem(header: str, declared: str) -> str | None:
def declared_axes_transposed(meta: TiffMetadata, declared: str) -> bool:
"""Return True when a declaration fits the header only with its named Y and X swapped.

:func:`check_declared_axes` accepts a declaration that differs from the axes the
metadata names only by swapping Y and X (header ``CYX`` declared ``CXY``): the image is
read transposed. A brick that accepts such a declaration records it with this function.
A plain stack's Y and X are not named (:func:`.metadata.named_axes`), so declaring them
the other way round is never a swap.

Parameters
----------
meta
The image's header (:func:`.metadata.inspect_image`).
declared
The declared axes, one letter per dimension (any case).

Returns
-------
bool
True when the named Y is declared X (and the named X declared Y).

Raises
------
ValueError
When the declaration does not fit the header at all (:func:`check_declared_axes`
gives an error), with that error's message.
"""
issue = check_declared_axes(meta, declared)
if issue is not None:
raise ValueError(issue.message)
named = named_axes(meta)
return "Y" in named and declared.upper()[named.index("Y")] == "X"


def _declared_axes_problem(header: str, named: str, declared: str) -> str | None:
if not declared or len(declared) != len(header):
return f"{len(declared)} declared for {len(header)} dimensions"
unknown = sorted(set(declared) - set(ROLES))
Expand All @@ -298,10 +345,17 @@ def _declared_axes_problem(header: str, declared: str) -> str | None:
repeated = sorted({axis for axis in declared if declared.count(axis) > 1})
if repeated:
return f"{', '.join(repeated)} declared more than once"
for index, (have, said) in enumerate(zip(header, declared, strict=True)):
if have == said:
# Even where a plain stack leaves its Y and X open, the image has rows and columns:
# a declaration without them would read no plane at all.
undeclared = [axis for axis in "YX" if axis in header and axis not in declared]
if undeclared:
return f"{' and '.join(undeclared)} {'is' if len(undeclared) == 1 else 'are'} not declared"
for index, (have, said) in enumerate(zip(named, declared, strict=True)):
if have in (said, OPEN_AXIS):
continue
if have in PLACEHOLDER_AXES and said in PLANE_ROLES:
# The named Y and X swapped: with one letter per role, the other position must then
# be swapped too, so this accepts a swap of the two and nothing else.
if have in "YX" and said in "YX":
continue
if have == "S" and said == "C" and "C" not in header:
continue
Expand Down
Loading
Loading