From ca30beefe5ace57502a5a5cbcfd79114d895cb73 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 08:21:38 +0000 Subject: [PATCH] Add the A_* error classes and RunInputs.section_digests() Stage 2 design PR 2. ChannelNotFoundError carries A_SCOPE_INCOMPLETE (owner answer 2: one rule for a channel no image has and one some images lack; the message names the image). MaskNotInstancesError (A_MASK_NOT_INSTANCES) and RgbNotSupportedError (A_RGB_NOT_SUPPORTED) are shared classes for the bricks to raise, and all five analysis error classes are exported. section_digests() returns the manifest's description digests, or None without a description. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm --- CHANGELOG.md | 9 ++ docs/contract.md | 3 +- src/morphobricks_preprocessing/__init__.py | 12 ++ .../run_inputs_error.py | 35 +++++- .../run_inputs_semantic.py | 24 +++- tests/test_a_error_classes.py | 108 ++++++++++++++++++ 6 files changed, 184 insertions(+), 7 deletions(-) create mode 100644 tests/test_a_error_classes.py diff --git a/CHANGELOG.md b/CHANGELOG.md index e35fd32a..a9e71f44 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,15 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the ## Unreleased +- **`A_*` error classes and `RunInputs.section_digests()` (Stage 2 design PR 2, owner answer 2).** + `ChannelNotFoundError` now has `code = "A_SCOPE_INCOMPLETE"`: one rule and one code for a channel + no image has and for one some images lack; the message names the image and ends with the code. + New `MaskNotInstancesError` (`A_MASK_NOT_INSTANCES`) and `RgbNotSupportedError` + (`A_RGB_NOT_SUPPORTED`), both `RunInputsError` subclasses for the bricks to raise (preprocessing + checks neither). The five analysis codes' classes (also `DraftRefusedError` and + `MaskUncheckedError`) are exported at the package top level. `RunInputs.section_digests()` returns + a copy of the manifest's `section_digests` of the description (None without one), read from the + manifest only. Contract §9.1 and §16. - **Images without header channel names are listed in a described run (follow-up of #218, owner decision 2026-11-08).** An image whose header names no channel, or not every one, still passes the per-unit header checks (§4.3, fixture v12: no evidence needed, the confirmed layout is diff --git a/docs/contract.md b/docs/contract.md index bdc98b18..10b8cd99 100644 --- a/docs/contract.md +++ b/docs/contract.md @@ -320,7 +320,7 @@ It gives: `morphobricks.toml` (`description_consume.read_published_description`) or a published file with an unconfirmed repeat is refused with `DraftRefusedError` (`A_DRAFT_REFUSED`). `read_mask` of a mask file the mask step has not checked (§15; `mask_representation.json`) is refused with - `MaskUncheckedError` (`A_MASK_UNCHECKED`); the error classes carry the Spec code as `code`. + `MaskUncheckedError` (`A_MASK_UNCHECKED`); the error classes carry the Spec code as `code`. `ChannelNotFoundError` is `A_SCOPE_INCOMPLETE` (Stage 2, owner answer 2): one rule and one code for a channel that no image has and for one that some images lack, and its message names the image. `section_digests()` returns a copy of the manifest's `experiment_description.section_digests` (SHA-256 of the `conditions`, `channels`, `repeats` and `masks` sections; the published file is not read again), or `None` for a run without a description. A `RunInputs` is picklable, so worker processes can use it. `morphobricks_preprocessing.testing.make_clean_run(folder, ...)` writes a tiny experiment of @@ -624,6 +624,7 @@ instead of keeping copies. | Masks named as images (D-i) | `mask_name_issues(leaves, mask_names=())`, `images_named_as_masks(leaves, suffixes=MASK_NAME_SUFFIXES)`, `mask_folder_part(path)`, `MASK_NAME_SUFFIXES`, `MASK_FOLDER_WORDS` (`discovery`) | The screen MorphoRelate's `init` ran (`project_image_names`), now discovery's warnings `image_named_as_mask` and `mask_folder_in_experiment` (§1) for every brick. Names are compared in lower case across the whole experiment; the longest `position_id` owns a file, an image of the same folder first. `mask_names` adds the caller's expected mask names (`cells` for `f1_cells.tif`) to `MASK_NAME_SUFFIXES`; `discover_experiment` gives none. A folder counts as a masks folder when a word of its name (`CellLabels` is `Cell` and `Labels`) is in `MASK_FOLDER_WORDS`. `images_named_as_masks` returns the `(image, owner)` pairs of that detection for the caller's own name endings (`["_cells"]`), for a brick that acts on the files itself (MorphoRelate leaves out the images named with a declared mask name). | | Pixel sizes (D-a) | `same_pixel_size(a, b, roles=("X", "Y"))`, `pixel_size_groups(sizes, roles=)`, `PIXEL_SIZE_TOLERANCE` (`physical_sizes`) | Two pixel sizes are the same when `|a - b| <= 1e-3 * max(a, b)` on every compared axis, in micrometres; one rule for a declared size against a header and for two headers. `same_pixel_size` takes `PhysicalSizes` (converted with `micrometers`) or sizes already in micrometres, one per role, and returns None when either is unknown (no size, or a unit that cannot be converted), never True. The rule is not transitive (1.0 ≈ 1.0009 ≈ 1.0018, but 1.0 ≠ 1.0018), so there is no per-image grouping key: `pixel_size_groups` links images by single linkage, so images of two groups always differ while the extremes of one group may drift; unknown sizes are left out; the largest group first. A size that cannot be converted to a positive finite length in micrometres (`1e308 m`) is unknown, never infinite, and an infinite or NaN value is never the same as any other (issue #93; before, `inf` matched every size). The folder check's `different_pixel_sizes` uses both (§11). | | Experiment design | `read_confirmed_design(path, expected_sha256=)`, `ConfirmedDesign`, `DesignCheckError` (`design`) | The one reader of `experiment_design.csv` (§12) for a brick that pools biological repeats (formerly CellPatcher `load_experiment_design` and MorphoFeature `ExperimentDesign.read`). It takes the file as this brick writes it and fills nothing in: UTF-8 without a byte-order mark; the header exactly the §12 columns in order (a column named twice or without a name is refused naming it, as by the lenient reader, §12); one value per column in every row (an unclosed quote, a missing or extra value, or a blank line is refused); no value with outer whitespace; `control` and `confirmed` exactly `true` or `false`; counts written as whole numbers of at least 0; `schema_version` 1; `confirmed_on` empty or `YYYY-MM-DD`; `condition`, `biological_repeat` and `repeat_group` pass `check_identity_text` and do not differ only in case (`check_identity_case`); no folder twice, one `control` per condition, at least one row; every group confirmed with one folder per condition (`design_issues`); with `expected_sha256` (the manifest's `experiment_design_sha256`) the file must be that one. The SHA-256 is of the bytes parsed. `ConfirmedDesign` gives `path`, `sha256`, the `design`, read-only `repeat_groups` (`{(condition, biological_repeat): repeat_group}`, the normalization brick's input), `group_members()` (`{repeat_group: {condition: biological_repeat}}`), `conditions`, `controls` and `groups`. Refusals raise `DesignCheckError` (a `ValueError`) with its `issues`: `missing_experiment_design`, `unreadable_experiment_design`, `invalid_experiment_design`, `repeat_group_conflict`, `design_not_confirmed` or `changed_experiment_design`, each an error naming the file (or the group's folders). This brick's step 1 and `check` read the file being edited with the lenient `load_design(output_folder, repeats)` (§12), which returns `(design, changes)`: the saved design brought up to date with the folders, or the default design when there is no file. A saved file that cannot be read raises `DesignCheckError` with the one error `unreadable_experiment_design`, whose message is the same "The experiment design cannot be read: …" as here followed by the lenient reader's refusal, naming the line and column where it can (the `DesignFormatError` is its `__cause__`); no default design is returned and the file is not touched (owner decision of 31 October 2026, plan §12 question 43). **Breaking in v0.4.0:** up to v0.3.4 `load_design` returned a third value, `problem`, the reason as text, together with the default design. | +| Analysis refusals (Stage 2) | `ChannelNotFoundError` (`A_SCOPE_INCOMPLETE`), `MaskNotInstancesError` (`A_MASK_NOT_INSTANCES`), `RgbNotSupportedError` (`A_RGB_NOT_SUPPORTED`), `DraftRefusedError` (`A_DRAFT_REFUSED`), `MaskUncheckedError` (`A_MASK_UNCHECKED`), `RunInputsError.code` (`run_inputs_error`, exported at the package top level) | The Spec's analysis-layer codes (experiment-v1 §8) as one class each, all `RunInputsError` subclasses (a `ValueError`) with the code as the class attribute `code` (`None` on `RunInputsError` itself), so a brick matches the class or the code and keeps no copy. Preprocessing raises `A_DRAFT_REFUSED`, `A_MASK_UNCHECKED` and `A_SCOPE_INCOMPLETE` itself (§9.1); `A_MASK_NOT_INSTANCES` (a mask that is not `instances` used as an object set) and `A_RGB_NOT_SUPPORTED` (`rgb` units given to a brick that does not declare RGB support) are for the bricks to raise, and no check for them lives here. A brick writes the code in its message text, as preprocessing does. The classes pickle, so a worker process can raise them. `RunInputs.section_digests()` (§9.1) gives the description's section digests for cache keys. | | Experiment description (Stage 1) | `read_description(path)`, `description_consume.read_published_description(path)` (the published `morphobricks.toml` only), `run.description_file.publish_description` (the only writer), `draft.draft_from_discovery`, `draft.migrate_legacy_design`, `description_reconcile.reconcile` | A brick reads the **published** `morphobricks.toml` through `RunInputs` (§9.1) and never a draft: a draft is internal to preprocessing, is never named `morphobricks.toml` and is refused with `A_DRAFT_REFUSED` (experiment-v1 §12). The command line shows the draft (`describe`, §12) and confirms nothing: the user's answers are the only way an ID, a repeat or a removal becomes confirmed. The owner's answers Q1 to Q8 of the Stage 1 design are in plan §12, question 78. | ## 17. Named-axis copies (`convert`, v0.4.0, design v0.4 §5) diff --git a/src/morphobricks_preprocessing/__init__.py b/src/morphobricks_preprocessing/__init__.py index d6812c82..c43e0609 100644 --- a/src/morphobricks_preprocessing/__init__.py +++ b/src/morphobricks_preprocessing/__init__.py @@ -209,6 +209,13 @@ write_calibration, ) from .run_inputs import Fov, LazyImage, RunInputs, RunInputsError, UnrecordedFilesError +from .run_inputs_error import ( + ChannelNotFoundError, + DraftRefusedError, + MaskNotInstancesError, + MaskUncheckedError, + RgbNotSupportedError, +) from .sources import check_sources from .unrecorded import UnrecordedFiles, unrecorded_files from .validation import Severity, ValidationIssue, has_errors @@ -311,6 +318,11 @@ "RunResult", "RunInputs", "RunInputsError", + "ChannelNotFoundError", + "DraftRefusedError", + "MaskNotInstancesError", + "MaskUncheckedError", + "RgbNotSupportedError", "UnrecordedFiles", "UnrecordedFilesError", "unrecorded_files", diff --git a/src/morphobricks_preprocessing/run_inputs_error.py b/src/morphobricks_preprocessing/run_inputs_error.py index 3c26174d..9be7027b 100644 --- a/src/morphobricks_preprocessing/run_inputs_error.py +++ b/src/morphobricks_preprocessing/run_inputs_error.py @@ -2,8 +2,12 @@ ``RunInputsError`` is the general refusal: run preprocessing again. The Spec's analysis-layer refusals (experiment-v1 section 8) are subclasses with their code, so a caller can match them: -:class:`DraftRefusedError` (``A_DRAFT_REFUSED``) and :class:`MaskUncheckedError` -(``A_MASK_UNCHECKED``). +:class:`DraftRefusedError` (``A_DRAFT_REFUSED``), :class:`MaskUncheckedError` +(``A_MASK_UNCHECKED``), :class:`ChannelNotFoundError` (``A_SCOPE_INCOMPLETE``), +:class:`MaskNotInstancesError` (``A_MASK_NOT_INSTANCES``) and :class:`RgbNotSupportedError` +(``A_RGB_NOT_SUPPORTED``). Preprocessing raises the first three of these itself; the last two +are for the bricks to raise (contract section 16, Stage 2 design PR 2), so that no brick keeps +its own copy of the code. """ from __future__ import annotations @@ -29,4 +33,29 @@ class MaskUncheckedError(RunInputsError): class ChannelNotFoundError(RunInputsError): - """A canonical channel ID is not one of the image's channels (never a default index).""" + """A canonical channel ID is not one of the image's channels (never a default index). + + One rule for "no image has the channel" and "some images lack it" (owner answer 2): both + are ``A_SCOPE_INCOMPLETE``, and the message names the images. + """ + + code = "A_SCOPE_INCOMPLETE" + + +class MaskNotInstancesError(RunInputsError): + """A mask whose representation is not ``instances`` was used as an object set. + + For the bricks to raise: a mask that is not ``instances`` (for example ``binary``) has no + object identity, so it is refused unless the request asks for a recorded conversion. + """ + + code = "A_MASK_NOT_INSTANCES" + + +class RgbNotSupportedError(RunInputsError): + """The run's units are ``rgb`` and the receiving brick does not declare RGB support. + + For the bricks to raise: the request is refused, never split or converted to grey. + """ + + code = "A_RGB_NOT_SUPPORTED" diff --git a/src/morphobricks_preprocessing/run_inputs_semantic.py b/src/morphobricks_preprocessing/run_inputs_semantic.py index 986f259d..5bb1c9ec 100644 --- a/src/morphobricks_preprocessing/run_inputs_semantic.py +++ b/src/morphobricks_preprocessing/run_inputs_semantic.py @@ -123,6 +123,22 @@ def description(self) -> Description | None: """ return check_description(self.output_folder, self.manifest) + def section_digests(self) -> dict[str, str] | None: + """Return the description's section digests as the run's manifest recorded them. + + The SHA-256 (hex) of the ``conditions``, ``channels``, ``repeats`` and ``masks`` + sections (:func:`.description_write.section_digests`), so a brick can key a cache or a + memo on the section it depends on. Read from the manifest only: the published file is + not read again (``description()`` does that). + + Returns + ------- + dict[str, str] or None + A new dict, or None when the run had no description. + """ + recorded = self.manifest.experiment_description + return None if recorded is None else dict(recorded["section_digests"]) + def images_without_header_channel_names(self) -> dict[str, tuple[int, ...]] | None: """Return the images the run passed without header channel names, as it recorded them. @@ -152,7 +168,8 @@ def channel_index(self, fov: object, channel_id: str) -> int: Raises ------ ChannelNotFoundError - When the image does not have that channel (nothing is defaulted). + ``A_SCOPE_INCOMPLETE``: the image does not have that channel (nothing is + defaulted); the message names the image. RunInputsError When the run had no description, so its channels have no IDs. """ @@ -161,8 +178,9 @@ def channel_index(self, fov: object, channel_id: str) -> int: ids = self._channel_ids(fov) if channel_id not in ids: raise ChannelNotFoundError( - f"The channel {channel_id!r} is not in this image; its channels are " - f"{', '.join(ids)}." + f"The channel {channel_id!r} is not in the image " + f"{getattr(fov, 'relative_path', fov)}; its channels are {', '.join(ids)} " + "(A_SCOPE_INCOMPLETE)." ) return ids.index(channel_id) diff --git a/tests/test_a_error_classes.py b/tests/test_a_error_classes.py new file mode 100644 index 00000000..e6c76874 --- /dev/null +++ b/tests/test_a_error_classes.py @@ -0,0 +1,108 @@ +"""The ``A_*`` error classes and ``RunInputs.section_digests()`` (Stage 2 design PR 2). + +Spec codes (experiment-v1 section 8) as classes the bricks share (contract section 16): one +class per code, each a ``RunInputsError`` that carries its ``code``, exported at the package +top level, picklable (a worker process raises them), and with the code in the message text. +""" + +from __future__ import annotations + +import pickle +import warnings + +import pytest + +import morphobricks_preprocessing as package +from description_tree import MAPPING, build_experiment +from morphobricks_preprocessing import run_experiment +from morphobricks_preprocessing.description import description_from_bytes +from morphobricks_preprocessing.description_write import section_digests +from morphobricks_preprocessing.manifest import read_manifest +from morphobricks_preprocessing.run_inputs import RunInputs, RunInputsError +from morphobricks_preprocessing.run_inputs_error import ( + ChannelNotFoundError, + DraftRefusedError, + MaskNotInstancesError, + MaskUncheckedError, + RgbNotSupportedError, +) +from spec_pin import fixtures + +VALID = {item.name.split("_")[0]: item for item in fixtures("valid")} +CODES = [ + (DraftRefusedError, "A_DRAFT_REFUSED"), + (MaskUncheckedError, "A_MASK_UNCHECKED"), + (ChannelNotFoundError, "A_SCOPE_INCOMPLETE"), + (MaskNotInstancesError, "A_MASK_NOT_INSTANCES"), + (RgbNotSupportedError, "A_RGB_NOT_SUPPORTED"), +] + + +def _open(tmp_path, name="v01", *, publish=True): + data = VALID[name].data + root, out = tmp_path / "exp", tmp_path / "out" + out.mkdir() + build_experiment(root, description_from_bytes(data)) + if publish: + (out / "morphobricks.toml").write_bytes(data) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + assert run_experiment(root, MAPPING, output_folder=out).written + return RunInputs.open(out), out + + +@pytest.mark.parametrize(("error", "code"), CODES) +def test_each_class_is_a_run_inputs_error_with_its_code_and_is_exported(error, code): + assert issubclass(error, RunInputsError) and error.code == code + assert getattr(package, error.__name__) is error and error.__name__ in package.__all__ + raised = error(f"refused ({code})") + assert raised.code == code and isinstance(raised, ValueError) + with pytest.raises(RunInputsError, match=code): + raise raised + + +@pytest.mark.parametrize(("error", "code"), CODES) +def test_each_class_survives_pickling_with_its_code(error, code): + again = pickle.loads(pickle.dumps(error("a message"))) + assert type(again) is error and again.code == code and str(again) == "a message" + + +def test_the_general_refusal_has_no_code(): + assert RunInputsError.code is None and RunInputsError("x").code is None + + +def test_an_absent_channel_is_a_scope_incomplete_refusal_naming_the_image(tmp_path): + inputs, _ = _open(tmp_path) + fov = inputs.fovs()[0] + with pytest.raises(ChannelNotFoundError) as raised: + inputs.channel_index(fov, "PHASE") + assert raised.value.code == "A_SCOPE_INCOMPLETE" + message = str(raised.value) + assert "PHASE" in message and fov.relative_path in message and "DNA, ACTIN" in message + assert "A_SCOPE_INCOMPLETE" in message + with pytest.raises(ChannelNotFoundError, match=fov.relative_path) as by_path: + inputs.channel_index(fov.relative_path, "dna") # by path; case is part of the ID + assert by_path.value.code == "A_SCOPE_INCOMPLETE" + + +def test_section_digests_equal_the_manifests(tmp_path): + inputs, out = _open(tmp_path) + recorded = read_manifest(out / "run_manifest.json").experiment_description["section_digests"] + digests = inputs.section_digests() + assert digests == recorded == section_digests(description_from_bytes(VALID["v01"].data)) + assert set(digests) == {"conditions", "channels", "repeats", "masks"} + digests["masks"] = "changed" + assert inputs.section_digests() == recorded # a copy: the manifest is not changed + + +def test_section_digests_are_none_without_a_description(tmp_path): + inputs, _ = _open(tmp_path, publish=False) + assert inputs.description() is None and inputs.section_digests() is None + + +def test_section_digests_read_the_manifest_only(tmp_path): + inputs, out = _open(tmp_path) + (out / "morphobricks.toml").unlink() # description() would now refuse the run + with pytest.raises(RunInputsError): + inputs.description() + assert inputs.section_digests() == section_digests(description_from_bytes(VALID["v01"].data))