Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,44 @@ published `morphobricks.toml` first), `run_experiment(design=...)` and the `desc
of `PreprocessingNotebook` are gone, and a run written by v0.5.0 or earlier is refused until it
is run again.

### PP-0: `RunInputs.resolve_request`, the shared request resolver (#256)

Resolver contract version 1 (`RESOLVER_VERSION = 1`; MorphoBricks-Spec ADR 0002 AR1-AR11,
contract §9.1 and §16). **API additions only.**

- **One interpreter of a described run for every analysis route** (a brick's direct run and a
run through MorphoFeature alike). A `Request` (canonical channel IDs; masks by ID with a role,
`objects` or `region`, and, for a `binary` mask used as objects, the explicit
`binary_mask_policy`; `plane` or `volume`; the consumer's `Capabilities`, `rgb_support` and
`volume_support`, both False by default; the explicit `device`, `cpu` or `cuda`) resolves to
an immutable, picklable `ResolvedRequest`: one `FovSelection` per position with the C index of
every requested channel **in that image**, the masks as `MaskSelection`s, the planes to read
and the canonical IDs beside the unchanged key; every applicable refusal at once
(`A_SCOPE_INCOMPLETE` "present in k of n acquisitions", `A_MASK_UNCHECKED` "k of n files",
`A_MASK_NOT_INSTANCES`, `A_RGB_NOT_SUPPORTED`, and the contract's own
`A_VOLUME_NOT_SUPPORTED`); the coverage of every biological repeat per channel and mask
(`full`, `partial`, `unavailable`), refused or not; and the `Provenance` (the resolver
version, the description's SHA-256 and section digests as the manifest recorded them, and the
request with its device). A refused request selects nothing; `require()` raises the first
refusal's error class with every refusal in its text.
- **It only classifies.** It reads the manifest, the published description (checked against
the run's SHA-256) and `mask_representation.json`; it reads no pixel, runs no kernel, writes
nothing, converts no mask (a recorded policy comes back as `MaskSelection.conversion` for the
route's shared labeller), checks only the device's spelling (the consumer refuses `cuda`
without a GPU) and imports no brick, CellPatcher or provider module, directly or through the
package (tested in a fresh interpreter). A malformed request is a `ValueError`: an ID twice;
an unknown role, policy, plane kind or device; a policy on a mask that is not `binary` or in a
`region` role; channels or masks not a tuple (of `str` IDs, of `MaskRequest`), or a capability
that is not a `bool` (the request is recorded as given, so it must be immutable).
- **Every run it sees is described** (AR10): a run without `morphobricks.toml` is refused by
`RunInputs.open` with `A_DESCRIPTION_REQUIRED` (PP-C), so the resolver has no undescribed
branch.
- `FovIdentity` gains the unit's `image_type`; `Fov` is unchanged. New exports:
`RESOLVER_VERSION`, `Request`, `MaskRequest`, `Capabilities`, `ResolvedRequest`,
`FovSelection`, `MaskSelection`, `PlaneSelection`, `Refusal`, `RepeatCoverage`, `Provenance`,
`VolumeNotSupportedError`. Suite: `tests/test_spec_resolve_request.py`, a Spec module
(`spec_gate.EXPECTED_MODULES`).

## 0.5.0 - 2026-11-09

Stage 1 of the experiment description (MorphoBricks-Spec experiment-v1, pinned at `1bed024`):
Expand Down
40 changes: 39 additions & 1 deletion docs/contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,44 @@ It gives:
of a mask file the mask step has not checked (§15; `mask_representation.json`) is refused with
`MaskUncheckedError` (`A_MASK_UNCHECKED`); the error classes carry the Spec code as `code`. `ChannelNotFoundError` is `A_SCOPE_INCOMPLETE` (Stage 2, owner answer 2): one rule and one code for a channel that no image has and for one that some images lack, and its message names the image. `section_digests()` returns a copy of the manifest's `experiment_description.section_digests` (SHA-256 of the `conditions`, `channels`, `repeats` and `masks` sections; the published file is not read again).

- **The request resolver (resolver contract version 1, `RESOLVER_VERSION`; MorphoBricks
architecture decision of 10 November 2026).** `resolve_request(request)` is the one
interpreter of a described run for an analysis, on every route (a brick's direct run and a run
through MorphoFeature alike; `run_inputs_request`, `run_inputs_resolve`). A `Request` names
canonical channel IDs, masks by ID with a role (`objects` or `region`) and, for a `binary`
mask used as objects, the explicit `binary_mask_policy` (`connected_components`,
experiment-v1 §4.5c), the planes (`plane`, one selection per (t, z), or `volume`, every Z of
one t), the consumer's `Capabilities` (`rgb_support`, `volume_support`, both False by
default) and the explicit `device` (`cpu` or `cuda`; recorded, never chosen here). It returns
an immutable, picklable `ResolvedRequest`: one `FovSelection` per position (the unchanged key,
`repeat_group`, `condition_id`, `technical_repeat_id`, the unit's declared `image_type`, the C
index of every requested channel **in that image** (`channel_indices`, `index_of`), the masks
as `MaskSelection(mask_id, source_type, representation, role, check_status, conversion)` and
the `PlaneSelection`s to read), the `refusals`, the `coverage` of every biological repeat per
channel and mask (`full`, `partial`, `unavailable`; experiment-v1 §8) and the `Provenance`
(the resolver version, the description's SHA-256 and section digests as the manifest recorded
them, and the request). A refused request has no selections: no subset is taken. Every
applicable refusal is reported, each a `Refusal(code, subject, message, files)`:
`A_SCOPE_INCOMPLETE` for a channel or mask some or all images lack ("present in k of n
acquisitions", k may be 0; an undeclared mask ID is 0 of n), `A_MASK_UNCHECKED` ("k of n
files"), `A_MASK_NOT_INSTANCES` (a non-`instances` mask as objects without the policy),
`A_RGB_NOT_SUPPORTED` (`rgb` units without `rgb_support`) and `A_VOLUME_NOT_SUPPORTED`
(`VolumeNotSupportedError`, the resolver contract's own code: a volume without
`volume_support`). `ResolvedRequest.require()` returns the selections or raises the first
refusal's error class with every refusal in its text. The resolver reads the manifest, the
published description and `mask_representation.json` only: no pixel, no kernel, no brick,
CellPatcher or provider import (a test scans its modules), and it writes nothing. It converts
nothing: a recorded policy comes back as `MaskSelection.conversion` and each route applies it
with the one shared labeller (CellPatcher's), so both routes give the same objects. The
`device` is checked for its spelling only and recorded in the provenance; the consumer refuses
`cuda` without a GPU and never switches. A malformed request (an ID twice, an unknown role,
policy, plane kind or device, a policy on a mask that is not `binary` or in a `region` role,
channels or masks not a tuple of `str` IDs or of `MaskRequest`, a capability that is not a
`bool`) is a `ValueError`, not a refusal. Every run the resolver sees is described: a run
without a published description is refused by `open` with `A_DESCRIPTION_REQUIRED` (§0;
`DescriptionRequiredError`, the resolver contract's other own code, MorphoBricks-Spec ADR 0002
AR10, AR11 (1)), so there is no undescribed path.

A `RunInputs` is picklable, so worker processes can use it.
`morphobricks_preprocessing.testing.make_clean_run(folder, ...)` writes a tiny experiment of
named-axis OME-TIFFs (optionally with masks), publishes its description
Expand Down Expand Up @@ -672,7 +710,7 @@ instead of keeping copies.
| Masks named as images (D-i) | `mask_name_issues(leaves, mask_names=())`, `images_named_as_masks(leaves, suffixes=MASK_NAME_SUFFIXES)`, `mask_folder_part(path)`, `MASK_NAME_SUFFIXES`, `MASK_FOLDER_WORDS` (`discovery`) | The screen MorphoRelate's `init` ran (`project_image_names`), now discovery's warnings `image_named_as_mask` and `mask_folder_in_experiment` (§1) for every brick. Names are compared in lower case across the whole experiment; the longest `position_id` owns a file, an image of the same folder first. `mask_names` adds the caller's expected mask names (`cells` for `f1_cells.tif`) to `MASK_NAME_SUFFIXES`; `discover_experiment` gives none. A folder counts as a masks folder when a word of its name (`CellLabels` is `Cell` and `Labels`) is in `MASK_FOLDER_WORDS`. `images_named_as_masks` returns the `(image, owner)` pairs of that detection for the caller's own name endings (`["_cells"]`), for a brick that acts on the files itself (MorphoRelate leaves out the images named with a declared mask name). |
| Pixel sizes (D-a) | `same_pixel_size(a, b, roles=("X", "Y"))`, `pixel_size_groups(sizes, roles=)`, `PIXEL_SIZE_TOLERANCE` (`physical_sizes`) | Two pixel sizes are the same when `|a - b| <= 1e-3 * max(a, b)` on every compared axis, in micrometres; one rule for a declared size against a header and for two headers. `same_pixel_size` takes `PhysicalSizes` (converted with `micrometers`) or sizes already in micrometres, one per role, and returns None when either is unknown (no size, or a unit that cannot be converted), never True. The rule is not transitive (1.0 ≈ 1.0009 ≈ 1.0018, but 1.0 ≠ 1.0018), so there is no per-image grouping key: `pixel_size_groups` links images by single linkage, so images of two groups always differ while the extremes of one group may drift; unknown sizes are left out; the largest group first. A size that cannot be converted to a positive finite length in micrometres (`1e308 m`) is unknown, never infinite, and an infinite or NaN value is never the same as any other (issue #93; before, `inf` matched every size). The folder check's `different_pixel_sizes` uses both (§11). |
| Experiment design | `read_confirmed_design(path, expected_sha256=)`, `ConfirmedDesign`, `DesignCheckError` (`design`) | The one reader of `experiment_design.csv` (§12) for a brick that pools biological repeats (formerly CellPatcher `load_experiment_design` and MorphoFeature `ExperimentDesign.read`). It takes the file as this brick writes it and fills nothing in: UTF-8 without a byte-order mark; the header exactly the §12 columns in order (a column named twice or without a name is refused naming it, as by the lenient reader, §12); one value per column in every row (an unclosed quote, a missing or extra value, or a blank line is refused); no value with outer whitespace; `control` and `confirmed` exactly `true` or `false`; counts written as whole numbers of at least 0; `schema_version` 1; `confirmed_on` empty or `YYYY-MM-DD`; `condition`, `biological_repeat` and `repeat_group` pass `check_identity_text` and do not differ only in case (`check_identity_case`); no folder twice, one `control` per condition, at least one row; every group confirmed with one folder per condition (`design_issues`); with `expected_sha256` (the manifest's `experiment_design_sha256`) the file must be that one. The SHA-256 is of the bytes parsed. `ConfirmedDesign` gives `path`, `sha256`, the `design`, read-only `repeat_groups` (`{(condition, biological_repeat): repeat_group}`, the normalization brick's input), `group_members()` (`{repeat_group: {condition: biological_repeat}}`), `conditions`, `controls` and `groups`. Refusals raise `DesignCheckError` (a `ValueError`) with its `issues`: `missing_experiment_design`, `unreadable_experiment_design`, `invalid_experiment_design`, `repeat_group_conflict`, `design_not_confirmed` or `changed_experiment_design`, each an error naming the file (or the group's folders). This brick's step 1 and `check` read the file being edited with the lenient `load_design(output_folder, repeats)` (§12), which returns `(design, changes)`: the saved design brought up to date with the folders, or the default design when there is no file. A saved file that cannot be read raises `DesignCheckError` with the one error `unreadable_experiment_design`, whose message is the same "The experiment design cannot be read: …" as here followed by the lenient reader's refusal, naming the line and column where it can (the `DesignFormatError` is its `__cause__`); no default design is returned and the file is not touched (owner decision of 31 October 2026, plan §12 question 43). **Breaking in v0.4.0:** up to v0.3.4 `load_design` returned a third value, `problem`, the reason as text, together with the default design. |
| Analysis refusals (Stage 2) | `ChannelNotFoundError` (`A_SCOPE_INCOMPLETE`), `MaskNotInstancesError` (`A_MASK_NOT_INSTANCES`), `RgbNotSupportedError` (`A_RGB_NOT_SUPPORTED`), `DraftRefusedError` (`A_DRAFT_REFUSED`), `MaskUncheckedError` (`A_MASK_UNCHECKED`), `RunInputsError.code` (`run_inputs_error`, exported at the package top level) | The Spec's analysis-layer codes (experiment-v1 §8) as one class each, all `RunInputsError` subclasses (a `ValueError`) with the code as the class attribute `code` (`None` on `RunInputsError` itself), so a brick matches the class or the code and keeps no copy. Preprocessing raises `A_DRAFT_REFUSED`, `A_MASK_UNCHECKED` and `A_SCOPE_INCOMPLETE` itself (§9.1); `A_MASK_NOT_INSTANCES` (a mask that is not `instances` used as an object set) and `A_RGB_NOT_SUPPORTED` (`rgb` units given to a brick that does not declare RGB support) are for the bricks to raise, and no check for them lives here. A brick writes the code in its message text, as preprocessing does. The classes pickle, so a worker process can raise them. `RunInputs.section_digests()` (§9.1) gives the description's section digests for cache keys. |
| Analysis refusals (Stage 2) | `ChannelNotFoundError` (`A_SCOPE_INCOMPLETE`), `MaskNotInstancesError` (`A_MASK_NOT_INSTANCES`), `RgbNotSupportedError` (`A_RGB_NOT_SUPPORTED`), `DraftRefusedError` (`A_DRAFT_REFUSED`), `MaskUncheckedError` (`A_MASK_UNCHECKED`), `RunInputsError.code` (`run_inputs_error`, exported at the package top level) | The Spec's analysis-layer codes (experiment-v1 §8) as one class each, all `RunInputsError` subclasses (a `ValueError`) with the code as the class attribute `code` (`None` on `RunInputsError` itself), so a brick matches the class or the code and keeps no copy. Preprocessing raises `A_DRAFT_REFUSED`, `A_MASK_UNCHECKED` and `A_SCOPE_INCOMPLETE` itself (§9.1); `A_MASK_NOT_INSTANCES` (a mask that is not `instances` used as an object set) and `A_RGB_NOT_SUPPORTED` (`rgb` units given to a brick that does not declare RGB support) are found by `resolve_request` (§9.1) against the consumer's declared capability, and a brick raises them through `ResolvedRequest.require()` (or reports its `refusals`). A brick writes the code in its message text, as preprocessing does. The classes pickle, so a worker process can raise them. `RunInputs.section_digests()` (§9.1) gives the description's section digests for cache keys. `RunInputs.resolve_request()` (§9.1) is the one interpreter of a described run for every route: a brick builds a `Request` and reads through the `FovSelection`s it returns, so channel-by-ID, scope, mask-representation and RGB rules have one implementation; `VolumeNotSupportedError` (`A_VOLUME_NOT_SUPPORTED`) and `DescriptionRequiredError` (`A_DESCRIPTION_REQUIRED`, raised by `RunInputs.open`) are the resolver contract's own codes (MorphoBricks-Spec ADR 0002), not experiment-v1 §8's. |
| Experiment description (Stage 1) | `read_description(path)`, `description_consume.read_published_description(path)` (the published `morphobricks.toml` only), `run.description_file.publish_description` (the only writer), `draft.draft_from_discovery`, `draft.migrate_legacy_design`, `description_reconcile.reconcile` | A brick reads the **published** `morphobricks.toml` through `RunInputs` (§9.1) and never a draft: a draft is internal to preprocessing, is never named `morphobricks.toml` and is refused with `A_DRAFT_REFUSED` (experiment-v1 §12). The command line shows the draft (`describe`, §12) and confirms nothing: the user's answers are the only way an ID, a repeat or a removal becomes confirmed. The owner's answers Q1 to Q8 of the Stage 1 design are in plan §12, question 78. |

## 17. Named-axis copies (`convert`, v0.4.0, design v0.4 §5)
Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,8 @@ baseline = "tools/code_rules_baseline.json"
"morphobricks_preprocessing.run_inputs_error" = "pure"
"morphobricks_preprocessing.required_description" = "io"
"morphobricks_preprocessing.run_inputs_semantic" = "io"
"morphobricks_preprocessing.run_inputs_request" = "pure"
"morphobricks_preprocessing.run_inputs_resolve" = "io"
"morphobricks_preprocessing.run_inputs_image" = "pure"
"morphobricks_preprocessing.description_layouts" = "pure"
"morphobricks_preprocessing.description_record" = "pure"
Expand Down
26 changes: 26 additions & 0 deletions src/morphobricks_preprocessing/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,20 @@
MaskUncheckedError,
RgbNotSupportedError,
)
from .run_inputs_request import (
RESOLVER_VERSION,
Capabilities,
FovSelection,
MaskRequest,
MaskSelection,
PlaneSelection,
Provenance,
Refusal,
RepeatCoverage,
Request,
ResolvedRequest,
VolumeNotSupportedError,
)
from .sources import check_sources
from .unrecorded import UnrecordedFiles, unrecorded_files
from .validation import Severity, ValidationIssue, has_errors
Expand Down Expand Up @@ -260,6 +274,7 @@
"OPEN_AXIS",
"PIXEL_SIZE_TOLERANCE",
"PLACEHOLDER_AXES",
"RESOLVER_VERSION",
"RGB",
"SPACE_UNITS",
"AxisMapping",
Expand Down Expand Up @@ -324,6 +339,17 @@
"MaskNotInstancesError",
"MaskUncheckedError",
"RgbNotSupportedError",
"VolumeNotSupportedError",
"Capabilities",
"FovSelection",
"MaskRequest",
"MaskSelection",
"PlaneSelection",
"Provenance",
"Refusal",
"RepeatCoverage",
"Request",
"ResolvedRequest",
"UnrecordedFiles",
"UnrecordedFilesError",
"unrecorded_files",
Expand Down
7 changes: 5 additions & 2 deletions src/morphobricks_preprocessing/fov_identity.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
function of the key: the repeat ID (``repeat_group``) of the member folder, the condition ID of
the condition folder, the technical repeat ID of the unit folder, and the unit's channel IDs
by C index. Nothing is looked up by the label alone: a label that names no declared member, or
two, is refused.
two, is refused. The unit's declared image type rides along (``image_type``) for the
request resolver (:mod:`.run_inputs_resolve`).
"""

from __future__ import annotations
Expand All @@ -22,6 +23,8 @@ class FovIdentity:
condition_id: str
technical_repeat_id: str
channel_ids: tuple[str, ...]
image_type: str | None = None
"""The unit's declared image type (``rgb``), or None for an ordinary channel image."""


def fov_identity(description: Description, labels: tuple[str, str, str]) -> FovIdentity:
Expand All @@ -41,7 +44,7 @@ def fov_identity(description: Description, labels: tuple[str, str, str]) -> FovI
"""
condition, repeat, technical = labels
found = [
FovIdentity(repeat_id, condition_id, unit_id, unit.channels)
FovIdentity(repeat_id, condition_id, unit_id, unit.channels, unit.image_type)
for repeat_id, entry in description.repeats.items()
for condition_id, member in entry.members.items()
if description.conditions[condition_id].source_folder == condition
Expand Down
3 changes: 2 additions & 1 deletion src/morphobricks_preprocessing/run_inputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
from .run_inputs_error import RunInputsError as RunInputsError # re-exported (moved)
from .run_inputs_image import LazyImage as LazyImage # re-exported (moved)
from .run_inputs_projection import ProjectionInputs
from .run_inputs_resolve import ResolvingInputs
from .run_inputs_semantic import SemanticInputs, check_description, with_identity
from .run_inputs_text import (
_ZARR_MISSING,
Expand Down Expand Up @@ -153,7 +154,7 @@ class Fov:


@dataclass(frozen=True)
class RunInputs(SemanticInputs, ProjectionInputs):
class RunInputs(SemanticInputs, ProjectionInputs, ResolvingInputs):
"""The inputs of a clean preprocessing run, read as the run recorded them.

Use :meth:`open`; the attributes are the snapshot taken then.
Expand Down
Loading
Loading