From 7418da1b5aa9d6012364e1a2b857bc1be887a184 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 04:00:51 +0000 Subject: [PATCH 1/2] CLI describe, docs and a recorded conformance run (Stage 1, PR 17) describe prints the draft experiment description in plain language, writes nothing and confirms nothing; --publish lists what is missing and exits 1 unless nothing is open. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm --- CHANGELOG.md | 15 + README.md | 2 +- docs/conformance.md | 30 ++ docs/contract.md | 2 + docs/development.md | 3 + docs/plan.md | 25 ++ docs/user_guide.md | 45 ++- pyproject.toml | 1 + src/morphobricks_preprocessing/cli.py | 8 +- .../cli_describe.py | 283 ++++++++++++++++++ tests/test_cli_describe.py | 223 ++++++++++++++ 11 files changed, 634 insertions(+), 3 deletions(-) create mode 100644 docs/conformance.md create mode 100644 src/morphobricks_preprocessing/cli_describe.py create mode 100644 tests/test_cli_describe.py diff --git a/CHANGELOG.md b/CHANGELOG.md index b7d4921e..48e2a99a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,21 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the ## Unreleased +- **`describe` command, docs and a recorded conformance run (Stage 1, PR 17).** + `morphobricks-preprocessing describe EXPERIMENT [--output FOLDER] [--exclude PATH] + [--publish]` prints the draft experiment description in plain language: the proposed IDs + (none, never repaired, for a folder that cannot be an ID), the repeats with their folders and + image counts, the references and every open question with what the user has to do. The draft + is the legacy `experiment_design.csv` of the output folder migrated, or the folders alone; + against a published `morphobricks.toml` it compares the folders (new, gone, image counts, + repeats to confirm again; no header is read). It reads no image and writes nothing, and + **confirms nothing**: confirmations are the user's (owner rule). `--publish` publishes only + a draft with no open question and every decision confirmed, else it lists what is missing and + exits with status 1; since no answers live outside the notebook yet, it publishes nothing + from a draft of folders (open question to the owner), and says `Nothing to publish` for a + published description that matches the folders. Docs: contract §12 and §16, user guide + "Describe your experiment", plan §12 question 77 (the owner's answers Q1 to Q8), + `docs/conformance.md` (the recorded run of the suite with no Spec skip). - **Z-projection arithmetic (part of #176).** New pure module `projection.py`: `Projection` (method `max`, `mean` or `sum`, no default; optional zero-based half-open slice range), `projection_dtype` (the output type from the input type, method and slice count only: max diff --git a/README.md b/README.md index e0cc0212..a8987cb6 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ Pre-alpha; the current release is v0.4.0 (what is planned next is in - Reading a run from another brick: `RunInputs.open(output_folder)` - A step-by-step notebook ([notebooks/MorphoBricks_Preprocessing.ipynb](notebooks/MorphoBricks_Preprocessing.ipynb)) and a command line (`morphobricks-preprocessing check` / `run` / `masks` / `calibrate` / - `design` / `convert`) + `design` / `describe` / `convert`) Every feature, message code and output file is described in the [user guide](docs/user_guide.md). diff --git a/docs/conformance.md b/docs/conformance.md new file mode 100644 index 00000000..16301a9c --- /dev/null +++ b/docs/conformance.md @@ -0,0 +1,30 @@ +# Conformance run record (Stage 1, PR 17) + +The design (Stage 1, PR 17) names no place for this record, so it is kept here. Every row is a +real run; a skip is never counted as a pass. + +## Run of 8 November 2026 + +| | | +|---|---| +| Base commit | `68b2a4b` (`origin/main`, merge of PR 211); the run included the changes of PR 17 | +| Spec pin | the full SHA in `tests/spec_fixtures/PIN.toml` (vendored `experiment-v1.md` at `b197ac1`) | +| Command | `python tools/check.py` (Ruff, Ruff format, then `pytest -q` over the whole `tests/` folder), then `python tools/check.py --rules` | +| Whole suite | 4324 passed, 4 skipped, 0 failed (301 s) | +| Spec modules (`tests/test_spec_*.py`) | 1535 passed, 0 skipped, 0 failed (the Spec gate fails a skip or xfail there) | +| Code rules | passed, no baseline entry grew | + +The 4 skips are outside the Spec modules, each an environment limit of the machine that ran: + +- `tests/test_cli_log.py:165`, `tests/test_folder_rules.py:201`, `tests/test_folder_rules.py:229`: + the file system tells names apart by case, so the case-only collision cannot be built; +- `tests/test_discovery.py:665`: the run had permissions that ignore file modes (root). + +These four behaviours are therefore **not verified by this run**. + +## Not covered + +- The Spec's `fixtures/expected/` golden CSVs and the conformance items that need PRs 13 to 16 + (notebook cards) are outside this run; this is not a release-candidate sign-off for those. +- `describe --publish` publishes nothing from a draft of folders (see the pull request's open + questions), so no end-to-end publication through the command line is part of this record. diff --git a/docs/contract.md b/docs/contract.md index cd740d80..c003ceda 100644 --- a/docs/contract.md +++ b/docs/contract.md @@ -434,6 +434,7 @@ folder. | Command line | `run` reads the file from the output folder, or `--design FILE`. Without a fully confirmed design it writes the design brought up to date with the folders (a template with `confirmed=false` for the groups still to confirm, keeping the answers already given) to the output folder, explains the columns, and stops with status 1; the run stops on `design_not_confirmed` first and writes its report (§9.2). Confirmed rows without a date get the date of the run. | | Template from `check` (v0.4.0, plan §12 question 57) | `check` (CLI; `run.write_check_outputs` in Python) also writes the design prefilled from the folders, under the output folder's lock: one row per condition and biological-repeat folder, `repeat_group` the folder name, the current counts, `confirmed=false`, `control=false`. Only when the output folder holds no `experiment_design.csv` (a design there, edited or unreadable, is never replaced) and the folders have no error. | | `design` command (v0.4.0, plan §12 question 57) | `design --output FOLDER` prints the design as a table and the repeat groups not confirmed yet, and any `repeat_group_conflict` or `invalid_experiment_design`. Options change it without editing the file, in this order: `--repeat-group CONDITION/REPEAT=GROUP` (repeatable; moves a folder, and the groups it leaves and joins are no longer confirmed, as `ExperimentDesign.with_group`), `--control CONDITION` and `--no-control CONDITION` (repeatable), `--confirm GROUP` (repeatable) and `--confirm-all` (every group not confirmed yet). It reads, changes and writes the file under the lock (`output_locked` while a run holds it), with the run's reader (`read_experiment_design`) and validator (`design_issues`); a confirmation fills `confirmed_on` with today's date and the counts are those in the file; a group already confirmed keeps its date. An unknown folder, condition or group, a condition both marked and unmarked, a group to confirm whose rows break the rules (`repeat_group_conflict`, `invalid_experiment_design`) or a design that cannot be written is refused, and nothing is changed (exit status 1). Nothing is confirmed by default. Python: `edit_design(output_folder, DesignEdit(...), on=None)` returns the design and the file written (None when nothing changed), raising `DesignEditError`; `apply_design_edit(design, edit, on)` is the same change without files. `DesignEdit.unconfirm` (v0.4.0, PP-10; not a CLI option) takes confirmations back, after the moves and before the confirmations (an unknown group, or one both confirmed and unconfirmed, also through `confirm_all`, is refused; it never confirms anything). The notebook applies every design answer with `apply_design_edit` (the "Undo" with `unconfirm`) and saves it with `run.design_file.write_design(design, output_folder)`, under the lock. The run reconciles the design with the folders as before, so a group that gained a folder since is asked again. | +| `describe` command (Stage 1, PR 17) | `describe EXPERIMENT [--output FOLDER] [--exclude PATH] [--publish]` prints the draft experiment description (experiment-v1 §12) in plain language: the proposed ID of each condition, technical-repeat folder and biological repeat (no proposal, with the rule's code, for a folder that cannot be an ID: never repaired), each repeat with its folders and image counts and whether it is confirmed (and the date), the references, the notes of a migration with several dates, and every open question with what the user has to do. The draft is the legacy `experiment_design.csv` of the output folder migrated (`draft.migrate_legacy_design`; one that cannot be read or breaks a rule is exit status 1 and left as it is) when there is one and no published `morphobricks.toml`, else the draft of the folders (`draft.draft_from_discovery`); when `morphobricks.toml` is published (`run.description_run.load_description`), the folders are compared with it (`description_reconcile.reconcile`: unchanged, new, gone, image counts, repeats to confirm again), reading the folder structure only, so the channels are not compared. It reads no image and writes nothing: no output folder, draft, description, design or records file. **It confirms nothing**: an ID, the comparability of a repeat (N35), a reference, a removal and every date are the user's answers (notebook), never given by the command or read from the clock. `--publish` publishes a draft only when no question is open and every decision is confirmed; otherwise it prints `Not published. What is missing:` with every item and exits with status 1. This version holds no answers outside the notebook (the draft file is not in the package), so `--publish` refuses every draft built from folders and publishes nothing; for a published description that matches the folders it says `Nothing to publish` (exit 0); a new, gone or changed folder is listed and the file is left as it is. An error (experiment folder not found, discovery errors, unreadable description or records) is exit status 1; the command takes no lock. | | Run | `run_experiment(..., design=...)` brings the design up to date with the folders and stops with `design_not_confirmed` or `repeat_group_conflict` if it is not fully confirmed. Otherwise the file is written with the table and published before the manifest, which records its SHA-256 (§9). Without a design the run is unchanged: no file is written, an existing one is left as it is, and `experiment_design_sha256` is null. `plane_histograms.csv` does not depend on the design. | ## 13. Image type @@ -623,6 +624,7 @@ instead of keeping copies. | Masks named as images (D-i) | `mask_name_issues(leaves, mask_names=())`, `images_named_as_masks(leaves, suffixes=MASK_NAME_SUFFIXES)`, `mask_folder_part(path)`, `MASK_NAME_SUFFIXES`, `MASK_FOLDER_WORDS` (`discovery`) | The screen MorphoRelate's `init` ran (`project_image_names`), now discovery's warnings `image_named_as_mask` and `mask_folder_in_experiment` (§1) for every brick. Names are compared in lower case across the whole experiment; the longest `position_id` owns a file, an image of the same folder first. `mask_names` adds the caller's expected mask names (`cells` for `f1_cells.tif`) to `MASK_NAME_SUFFIXES`; `discover_experiment` gives none. A folder counts as a masks folder when a word of its name (`CellLabels` is `Cell` and `Labels`) is in `MASK_FOLDER_WORDS`. `images_named_as_masks` returns the `(image, owner)` pairs of that detection for the caller's own name endings (`["_cells"]`), for a brick that acts on the files itself (MorphoRelate leaves out the images named with a declared mask name). | | Pixel sizes (D-a) | `same_pixel_size(a, b, roles=("X", "Y"))`, `pixel_size_groups(sizes, roles=)`, `PIXEL_SIZE_TOLERANCE` (`physical_sizes`) | Two pixel sizes are the same when `|a - b| <= 1e-3 * max(a, b)` on every compared axis, in micrometres; one rule for a declared size against a header and for two headers. `same_pixel_size` takes `PhysicalSizes` (converted with `micrometers`) or sizes already in micrometres, one per role, and returns None when either is unknown (no size, or a unit that cannot be converted), never True. The rule is not transitive (1.0 ≈ 1.0009 ≈ 1.0018, but 1.0 ≠ 1.0018), so there is no per-image grouping key: `pixel_size_groups` links images by single linkage, so images of two groups always differ while the extremes of one group may drift; unknown sizes are left out; the largest group first. A size that cannot be converted to a positive finite length in micrometres (`1e308 m`) is unknown, never infinite, and an infinite or NaN value is never the same as any other (issue #93; before, `inf` matched every size). The folder check's `different_pixel_sizes` uses both (§11). | | Experiment design | `read_confirmed_design(path, expected_sha256=)`, `ConfirmedDesign`, `DesignCheckError` (`design`) | The one reader of `experiment_design.csv` (§12) for a brick that pools biological repeats (formerly CellPatcher `load_experiment_design` and MorphoFeature `ExperimentDesign.read`). It takes the file as this brick writes it and fills nothing in: UTF-8 without a byte-order mark; the header exactly the §12 columns in order (a column named twice or without a name is refused naming it, as by the lenient reader, §12); one value per column in every row (an unclosed quote, a missing or extra value, or a blank line is refused); no value with outer whitespace; `control` and `confirmed` exactly `true` or `false`; counts written as whole numbers of at least 0; `schema_version` 1; `confirmed_on` empty or `YYYY-MM-DD`; `condition`, `biological_repeat` and `repeat_group` pass `check_identity_text` and do not differ only in case (`check_identity_case`); no folder twice, one `control` per condition, at least one row; every group confirmed with one folder per condition (`design_issues`); with `expected_sha256` (the manifest's `experiment_design_sha256`) the file must be that one. The SHA-256 is of the bytes parsed. `ConfirmedDesign` gives `path`, `sha256`, the `design`, read-only `repeat_groups` (`{(condition, biological_repeat): repeat_group}`, the normalization brick's input), `group_members()` (`{repeat_group: {condition: biological_repeat}}`), `conditions`, `controls` and `groups`. Refusals raise `DesignCheckError` (a `ValueError`) with its `issues`: `missing_experiment_design`, `unreadable_experiment_design`, `invalid_experiment_design`, `repeat_group_conflict`, `design_not_confirmed` or `changed_experiment_design`, each an error naming the file (or the group's folders). This brick's step 1 and `check` read the file being edited with the lenient `load_design(output_folder, repeats)` (§12), which returns `(design, changes)`: the saved design brought up to date with the folders, or the default design when there is no file. A saved file that cannot be read raises `DesignCheckError` with the one error `unreadable_experiment_design`, whose message is the same "The experiment design cannot be read: …" as here followed by the lenient reader's refusal, naming the line and column where it can (the `DesignFormatError` is its `__cause__`); no default design is returned and the file is not touched (owner decision of 31 October 2026, plan §12 question 43). **Breaking in v0.4.0:** up to v0.3.4 `load_design` returned a third value, `problem`, the reason as text, together with the default design. | +| Experiment description (Stage 1) | `read_description(path)`, `description_consume.read_published_description(path)` (the published `morphobricks.toml` only), `run.description_file.publish_description` (the only writer), `draft.draft_from_discovery`, `draft.migrate_legacy_design`, `description_reconcile.reconcile` | A brick reads the **published** `morphobricks.toml` through `RunInputs` (§9.1) and never a draft: a draft is internal to preprocessing, is never named `morphobricks.toml` and is refused with `A_DRAFT_REFUSED` (experiment-v1 §12). The command line shows the draft (`describe`, §12) and confirms nothing: the user's answers are the only way an ID, a repeat or a removal becomes confirmed. The owner's answers Q1 to Q8 of the Stage 1 design are in plan §12, question 77. | ## 17. Named-axis copies (`convert`, v0.4.0, design v0.4 §5) diff --git a/docs/development.md b/docs/development.md index 6f318265..0e3e375c 100644 --- a/docs/development.md +++ b/docs/development.md @@ -121,6 +121,9 @@ commit SHA in its `PIN.toml`, never a branch. modules (`description*.py`) for a second table of codes or names and for lenient, older or default-version paths. +The result of the last recorded whole-suite run, with its skips, is in +[conformance.md](conformance.md). + ## Versioning - Semantic versioning for the package. diff --git a/docs/plan.md b/docs/plan.md index 9de84359..ebf3a474 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -1554,3 +1554,28 @@ Raised during review; to be decided before the milestones that need them: `RunInputs.open` refuses a manifest recording `shared: true`. Plain-stack masks with stated axes (`Cells=TYX`) remain and are checked against the image. (3) The other bricks are not changed: their own Z-plane policies stay as a second defence. +77. Stage 1: the experiment description (`morphobricks.toml`, Spec experiment-v1) and the + notebook that writes it. *Decided (owner, 7 November 2026; all eight recommendations of + the Stage 1 design, section 6, accepted):* (Q1) the notebook always writes the + description; the command line and `run_experiment` without one keep the v0.4 behaviour + with a warning, so the NFC and mask-type-name refusals and the raw header-name checks + apply only with a description; a description is required from Stage 3. (Q2) The + publication records (re-confirmations, removals, image-count changes) live in the + append-only `morphobricks.records.jsonl` written under the lock at publication and copied + unchanged into the next run's manifest; Spec §4.6 says so. (Q3) Layouts whose units are + identical and header-named are one confirmation; unnamed layouts are pre-filled but need + one tick per acquisition beside its thumbnails. (Q4) A raw header name is proposed as a + channel ID only when it is valid as it stands (never repaired), and nothing is adopted + without the user's confirmation. (Q5) A mask has no default representation: `instances` + or `binary` is the user's answer, and the dtype and the values seen are evidence only. + (Q6) A Spec change before PR 7: a condition gets the `single` rule and the ID proposal + rule of §4.6a covers repeat IDs. (Q7) Time-lapses are allowed as before; homogeneity + compares axis semantics, not the number of time points; previews show t = 0 with a T + slider. (Q8) The added fields are `condition_id` and `technical_repeat_id` beside + `repeat_group`, the source folder labels staying in `Fov.key` (contract §9.1, §16). + *Implemented in PR 17:* the command `describe` (contract §12) prints the draft and + confirms nothing (the owner's rule: confirmations are given only by the user); its + `--publish` publishes only a draft with no open question and every decision confirmed. + The draft file that would hold the user's answers outside the notebook is not part of the + package yet, so `--publish` cannot complete a draft built from folders (open question to + the owner, PR 17). diff --git a/docs/user_guide.md b/docs/user_guide.md index 498be07a..1b706c3a 100644 --- a/docs/user_guide.md +++ b/docs/user_guide.md @@ -673,7 +673,7 @@ After writing, `run` prints the intensity checks of the table, grouped by kind, the channel, the image or biological repeat and the values compared, and what to do once per kind; it draws no plots (the notebook does). -**Exit status.** Every command (`check`, `run`, `masks`, `calibrate`, `design` and +**Exit status.** Every command (`check`, `run`, `masks`, `calibrate`, `design`, `describe` and `convert`) exits with status 1 when it prints an error, and with 0 otherwise: warnings, intensity warnings included, never change it. So `check` exits with 1 on every error it reports, also an experiment design it cannot read (it never replaces that file) or one @@ -746,6 +746,49 @@ kinds of mask count as the same objects. `morphobricks-preprocessing masks --hel all seven with their defaults; the values used are printed and written in the report. `python -m morphobricks_preprocessing` works the same way. +### Describe your experiment (`describe`) + +```bash +morphobricks-preprocessing describe /data/exp1 +``` + +prints, in plain language, the draft description of the experiment that the folders (and, +when there is one, the output folder) allow: the ID proposed for each condition, technical +repeat and biological repeat (a folder such as `control` or `z`, whose name cannot be an ID, +gets no proposal and is never repaired), each repeat with its folders and image counts, and +every question still open, each with what you have to do: choose or accept an ID, confirm +that the folders of a repeat were prepared and acquired under the same circumstances, say +what each of two controls is compared against. + +- It reads the folder structure only (no image is opened) and **writes nothing**: no output + folder, no draft, no description. `--output FOLDER` names the output folder to read (default: + next to the experiment), `--exclude PATH` leaves an image out, as in `check`. +- If the output folder holds an `experiment_design.csv` from an earlier version and no + published `morphobricks.toml`, the draft is that design migrated: its confirmed repeats + and their dates are shown as you gave them, and the IDs are still only proposals. A design + that cannot be read, or breaks the rules of the description, is refused with the reason and + left as it is. +- If a `morphobricks.toml` is published, `describe` compares it with the folders instead: how + many acquisitions are as published, which are new or gone, which image counts changed and + which repeats must be confirmed again. Only folders and counts are compared; the channels + are not (no header is read). +- **Nothing is confirmed by the command.** An ID, the comparability of a repeat, a reference + and the removal of a folder are your answers, given in the notebook; the command never + answers for you and never fills in a date. + +```bash +morphobricks-preprocessing describe /data/exp1 --publish +``` + +publishes the description only when no question is open and every decision is confirmed; +otherwise it prints `Not published. What is missing:` with every item and exits with status 1. +In this version the command line holds none of your answers (they are given in the notebook), +so `--publish` publishes nothing from a draft made of folders: it lists what is missing. For +a published description that matches the folders it says `Nothing to publish` and exits with +status 0; for one that no longer matches, it lists the new, gone or changed folders and leaves +the file as it is. Like the other commands, `describe` exits with status 1 when it reports an +error (an experiment folder that is not found, an unreadable description) and 0 otherwise. + ### Stating pixel sizes (`calibrate`) Each image's pixel size (X and Y) and z-step (Z) come from its metadata. When the diff --git a/pyproject.toml b/pyproject.toml index d4dea34c..175ac6aa 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -162,6 +162,7 @@ baseline = "tools/code_rules_baseline.json" "morphobricks_preprocessing.cli_calibrate" = "ui" "morphobricks_preprocessing.cli_masks" = "ui" "morphobricks_preprocessing.cli_design" = "ui" +"morphobricks_preprocessing.cli_describe" = "ui" "morphobricks_preprocessing.cli_convert" = "ui" "morphobricks_preprocessing.conversion" = "orchestration" "morphobricks_preprocessing.conversion_check" = "io" diff --git a/src/morphobricks_preprocessing/cli.py b/src/morphobricks_preprocessing/cli.py index afb37006..5d043a45 100644 --- a/src/morphobricks_preprocessing/cli.py +++ b/src/morphobricks_preprocessing/cli.py @@ -1,10 +1,12 @@ -"""Command-line interface: ``check``, ``run``, ``masks``, ``calibrate``, ``design``, ``convert``. +"""Command-line interface: ``check``, ``run``, ``masks``, ``calibrate``, ``design``, ``describe``. ``check`` runs the preflight (folders and image headers, no pixels) and prints what will be analysed, the problems found and the suggested axis roles; it writes the report (``discovery_report.json`` and ``.txt``) to the output folder, and the experiment design template, prefilled from the folders, when the folder holds no design. ``design`` (:mod:`.cli_design`) shows, checks and changes that design without editing it. +``describe`` (:mod:`.cli_describe`) prints the draft experiment description in plain language; +it writes nothing and confirms nothing. ``convert`` is described below. ``run`` (:mod:`.cli_run`) processes the experiment with the axis roles given in ``--axes``, or those of the earlier run in the same output folder; roles are never taken from metadata without being stated. It also needs the @@ -34,6 +36,7 @@ from .cli_calibrate import add_calibrate_parser, calibrate_command, calibration_summary from .cli_convert import add_convert_parser, convert_command +from .cli_describe import add_describe_parser, describe_command from .cli_design import add_design_parser, design_command, design_errors, design_instructions from .cli_masks import add_masks_arguments, masks_command from .cli_run import ( @@ -104,6 +107,7 @@ def build_parser() -> argparse.ArgumentParser: add_masks_arguments(masks) for command in (add_calibrate_parser(commands), add_design_parser(commands)): _remove_stale_lock_option(command) + add_describe_parser(commands) add_convert_parser(commands) return parser @@ -113,6 +117,8 @@ def main(argv: Sequence[str] | None = None) -> int: _safe_output() parser = build_parser() args = parser.parse_args(argv) + if args.command == "describe": + return describe_command(args) if args.command in ("calibrate", "design"): if args.remove_stale_lock and not _remove_stale_lock(args): return 1 diff --git a/src/morphobricks_preprocessing/cli_describe.py b/src/morphobricks_preprocessing/cli_describe.py new file mode 100644 index 00000000..155ff9f7 --- /dev/null +++ b/src/morphobricks_preprocessing/cli_describe.py @@ -0,0 +1,283 @@ +"""The command line's ``describe``: show the experiment description draft (Stage 1, PR 17). + +``describe EXPERIMENT [--output FOLDER]`` prints what the folders and the output folder say +about the experiment description (experiment-v1) in plain language: the IDs proposed for +the conditions, technical repeats and biological repeats, the repeats and their folders, the +references, and every question still open. It reads the folder structure and nothing else (no +image is opened, no pixel read) and writes nothing. + +The draft comes from the existing functions only: the legacy ``experiment_design.csv`` of the +output folder, when there is one and no published description, is migrated +(:func:`.draft.migrate_legacy_design`); otherwise the draft is built from the folders +(:func:`.draft.draft_from_discovery`); when ``morphobricks.toml`` is published, the folders are +compared with it (:func:`.description_reconcile.reconcile`). This module adds no rule. + +**Nothing is confirmed here.** An ID, a repeat's comparability (N35), a reference and a removal +are the user's answers, given in the notebook; the command never answers for the user and never +reads the clock. ``--publish`` publishes only a draft that has no open question and holds every +decision confirmed; otherwise it exits with status 1 and lists what is missing. In this version +the command line cannot hold answers (the draft file the notebook writes is not part of the +package yet), so ``--publish`` refuses every draft built from the folders and publishes nothing; +for a published description that matches the folders it reports that there is nothing to +publish. + +Exit status: 1 for an error (a folder that cannot be described, a description that cannot be +read, a legacy design that breaks the rules, a publication refused), else 0, as every command. +""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +from .description import Description +from .description_reconcile import ( + NEW_UNIT, + REMOVED_UNIT, + ObservedUnit, + Reconciliation, + Status, + UnitFolders, + reconcile, +) +from .design import DESIGN_FILENAME, DesignFormatError +from .discovery import DiscoveryResult, default_output_folder, discover_experiment +from .draft import Draft, DraftError, draft_from_discovery, migrate_legacy_design +from .draft_ids import IdKind, id_problems +from .report import format_issues +from .run.description_run import load_description +from .run.recorded import normalise_relative_path +from .validation import has_errors + +QUESTION_TEXT = { + "id_pending": "choose the ID of {subject} (accept the proposal or type another)", + "repeat_not_confirmed": ( + "confirm that the folders of {subject} were prepared and acquired under the same " + "circumstances, so they are comparable" + ), + "confirmed_on_missing": "{subject} is confirmed but has no date: confirm it again", + "control_unresolved": ( + "condition {subject} is marked as a control, as is another one: say what each " + "condition is compared against" + ), + "legacy_folder_missing": ( + "the legacy design lists {subject}, which is no longer in the folder: restore it or " + "confirm that it is removed" + ), +} +"""What the user must do for each open question of a draft, in plain words.""" + +NOT_IN_DRAFT = ( + "the channels of each acquisition, the image type and the meaning of the masks are not " + "part of this draft and are answered in the notebook; the command line cannot publish " + "without them" +) +REASON_TEXT = { + NEW_UNIT: "a new folder joined it", + REMOVED_UNIT: "a folder of it is gone", +} + + +def add_describe_parser(commands: argparse._SubParsersAction) -> argparse.ArgumentParser: + """Add the ``describe`` command and its options; return its parser.""" + describe = commands.add_parser( + "describe", + help="show the draft experiment description (proposals, open questions) in plain " + "language; writes nothing, and confirms nothing", + ) + describe.add_argument("experiment", help="the experiment folder") + describe.add_argument( + "--output", + metavar="FOLDER", + help="output folder holding experiment_design.csv or morphobricks.toml " + "(default: next to the experiment, _MorphoBricks_Preprocessing)", + ) + describe.add_argument( + "--exclude", + action="append", + default=[], + metavar="PATH", + help="image to leave out, relative to the experiment folder (repeatable)", + ) + describe.add_argument( + "--publish", + action="store_true", + help="publish the description only when no question is open and every decision is " + "confirmed; otherwise exit with status 1 and list what is missing. Nothing is " + "confirmed for you", + ) + return describe + + +def describe_command(args: argparse.Namespace) -> int: + """Print the draft (or the comparison with the published description); return the status.""" + root = Path(args.experiment).resolve() + output = Path(args.output).resolve() if args.output else default_output_folder(root) + excluded = sorted({normalise_relative_path(path) for path in args.exclude}) + discovery = discover_experiment(root, output) + if has_errors(discovery.issues): + print(format_issues(discovery.issues)) + return 1 + if discovery.issues: + print(format_issues(discovery.issues), end="\n\n") + found, problems = load_description(output) + if problems: + print(format_issues(problems)) + return 1 + if found is not None: + lines, blockers = _published(found.description, discovery, excluded, output) + else: + try: + draft = _draft(discovery, excluded, output) + except (DesignFormatError, DraftError) as exc: + print(f"The experiment description cannot be drafted from {output / DESIGN_FILENAME}") + print(f" {exc}") + return 1 + lines, blockers = draft_lines(draft, root), draft_blockers(draft) + print("\n".join(lines)) + if not args.publish: + return 0 + print() + if not blockers: + print("Nothing to publish: the published description matches the folders.") + return 0 + print("Not published. What is missing:") + print("\n".join(f" - {item}" for item in blockers)) + return 1 + + +def _draft(discovery: DiscoveryResult, excluded: list[str], output: Path) -> Draft: + """Build the draft: the legacy design migrated when the output folder holds one.""" + legacy = output / DESIGN_FILENAME + if legacy.is_file(): + return migrate_legacy_design(legacy.read_bytes(), discovery, excluded, name=legacy.name) + return draft_from_discovery(discovery, excluded) + + +def _proposal(kind: IdKind, folder: str, slot_proposed: str | None) -> str: + """Say what is proposed for a folder, or why nothing is.""" + if slot_proposed is not None: + return f"proposed ID {slot_proposed!r}" + why = ", ".join(id_problems(kind, folder)) + return f"no ID can be proposed ({why or 'in use'}): you name it" + + +def draft_lines(draft: Draft, root: Path) -> list[str]: + """Return the plain-language report of a draft.""" + source = ( + "the legacy experiment_design.csv of the output folder" + if draft.origin == "legacy" + else "the folders alone" + ) + out = [ + f"Draft experiment description of {root}", + f"Drawn from {source}. Nothing here is confirmed: IDs below are proposals.", + "", + "Conditions (folder: proposal)", + ] + out += [ + f" {folder}: {_proposal('conditions', folder, slot.proposed)}" + for folder, slot in sorted(draft.conditions.items()) + ] + out += ["", "Technical repeats (folder: proposal)"] + out += [ + f" {folder}: {_proposal('technical_repeats', folder, slot.proposed)}" + for folder, slot in sorted(draft.technical_repeats.items()) + ] + out += ["", "Biological repeats"] + for group, repeat in sorted(draft.repeats.items()): + state = f"confirmed on {repeat.confirmed_on}" if repeat.confirmed else "not confirmed" + out.append(f" {group}: {_proposal('repeats', group, repeat.id.proposed)}; {state}") + out += [ + f" {m.condition} / {m.source_repeat}: {m.image_count} images" for m in repeat.members + ] + if draft.references: + out += ["", "References (condition is compared against)"] + out += [f" {c} -> {r}" for c, r in sorted(draft.references.items())] + for note in draft.notes: + listed = "; ".join(f"{c}/{r}: {d or 'no date'}" for c, r, d in note.dates) + out += ["", f"Several dates in the legacy design for {note.repeat_group} ({listed})."] + out.append(f" The draft keeps the latest, {note.kept or 'none'}; you confirm it again.") + out += _question_lines(draft) + return out + + +def _question_lines(draft: Draft) -> list[str]: + """Return what the user still has to answer, one line each.""" + questions = draft.open_questions() + if not questions: + return ["", "No question is open."] + out = ["", f"{len(questions)} open questions (answered by you, in the notebook):"] + out += [f" - {QUESTION_TEXT[q.code].format(subject=q.subject)}" for q in questions] + return out + + +def draft_blockers(draft: Draft) -> list[str]: + """Return everything that stops this draft from being published, in plain words.""" + blockers = [QUESTION_TEXT[q.code].format(subject=q.subject) for q in draft.open_questions()] + return blockers or [NOT_IN_DRAFT] + + +def _observed(discovery: DiscoveryResult, excluded: list[str]) -> dict[UnitFolders, ObservedUnit]: + """Return the units the folders hold now, with their included image counts.""" + found = {} + for leaf in discovery.leaves: + count = sum(image not in excluded for image in leaf.images) + if count: + found[UnitFolders(*leaf.labels)] = ObservedUnit(count) + return found + + +def _published( + published: Description, discovery: DiscoveryResult, excluded: list[str], output: Path +) -> tuple[list[str], list[str]]: + """Compare the published description with the folders; return the report and blockers.""" + rec = reconcile(published, _observed(discovery, excluded)) + return reconciliation_lines(rec, output), reconciliation_blockers(rec) + + +def _folders(unit: UnitFolders) -> str: + return f"{unit.condition} / {unit.source_repeat} / {unit.technical}" + + +def reconciliation_lines(rec: Reconciliation, output: Path) -> list[str]: + """Return the plain-language comparison of the published description with the folders.""" + out = [ + f"The published description ({output / 'morphobricks.toml'}) compared with the folders.", + "Only the folder structure and image counts are compared: no image header is read, so " + "the channels are not.", + "", + f"{len(rec.unchanged)} acquisitions as published.", + ] + out += [f" new, no ID yet: {_folders(u.folders)}" for u in rec.new] + out += [f" gone, kept until you confirm: {_folders(u.folders)}" for u in rec.removed] + for change in rec.counts: + out.append( + f" {change.repeat_id} / {change.condition_id}: {change.before} images published, " + f"{change.after} now" + ) + for repeat, reasons in rec.reopened.items(): + why = ", ".join(REASON_TEXT.get(reason, reason) for reason in reasons) + out.append(f" repeat {repeat} must be confirmed again: {why}") + if not (rec.new or rec.removed or rec.counts or rec.reopened): + out.append(" No change.") + return out + + +def reconciliation_blockers(rec: Reconciliation) -> list[str]: + """Return what stops a publication from the comparison, in plain words.""" + blockers = [ + f"{_folders(u.folders)} is new: say which repeat, condition and technical repeat it is" + for u in rec.new + ] + blockers += [ + f"{_folders(u.folders)} is gone: confirm that it is removed, or restore it" + for u in rec.with_status(Status.REMOVED) + ] + blockers += [f"repeat {repeat}: confirm it again" for repeat in rec.reopened] + blockers += [ + f"{c.repeat_id} / {c.condition_id}: the image count changed ({c.before} to {c.after}); " + "this command cannot publish it" + for c in rec.counts + ] + return blockers diff --git a/tests/test_cli_describe.py b/tests/test_cli_describe.py new file mode 100644 index 00000000..d1719dc8 --- /dev/null +++ b/tests/test_cli_describe.py @@ -0,0 +1,223 @@ +"""The command line's ``describe`` (Stage 1, PR 17): print the draft, publish nothing unasked. + +- ``describe`` prints the proposals, the open questions and what needs the user's confirmation, + and writes nothing (no output folder, no draft, no ``morphobricks.toml``). +- ``--publish`` refuses a draft with an open question (exit status 1, every missing item + listed); it never confirms an ID, a repeat or a date for the user. +- For a published description that matches the folders there is nothing to publish; for one + the folders no longer match, what needs the user's answer is listed and the file is untouched. +""" + +from __future__ import annotations + +from dataclasses import replace + +import pytest + +from description_tree import build_experiment +from morphobricks_preprocessing.cli import main +from morphobricks_preprocessing.description import description_from_bytes +from morphobricks_preprocessing.description_file import sha256_hex +from morphobricks_preprocessing.design import DesignRow, ExperimentDesign, write_experiment_design +from morphobricks_preprocessing.discovery import discover_experiment +from morphobricks_preprocessing.draft import draft_from_discovery +from morphobricks_preprocessing.run.description_file import publish_description +from spec_pin import fixtures +from tiff_tree import make_tree + +TOML = "morphobricks.toml" +DESIGN = "experiment_design.csv" + + +def row(condition, repeat, *, control=False, on="2026-10-30", confirmed=True): + return DesignRow( + condition, + repeat, + repeat, + control=control, + image_count=2, + confirmed=confirmed, + confirmed_on=on if confirmed else "", + ) + + +def tree(root, *paths): + make_tree(root, [f"{p}/{name}.tif" for p in paths for name in ("a", "b")]) + return root + + +def listing(folder): + """Every path under ``folder`` with its bytes, to prove that nothing was written.""" + return {p: p.read_bytes() for p in sorted(folder.rglob("*")) if p.is_file()} + + +@pytest.fixture +def experiment(tmp_path): + return tree(tmp_path / "exp", "CTRL/rep1", "CTRL/rep2", "DrugA/rep1", "DrugA/rep2") + + +def test_describe_prints_the_proposals_and_the_open_questions_and_writes_nothing( + experiment, tmp_path, capsys +): + output = tmp_path / "out" + before = listing(tmp_path) + assert main(["describe", str(experiment), "--output", str(output)]) == 0 + out = capsys.readouterr().out + assert "Drawn from the folders alone. Nothing here is confirmed" in out + assert "CTRL: proposed ID 'CTRL'" in out + assert "rep1: proposed ID 'rep1'; not confirmed" in out + assert "CTRL / rep1: 2 images" in out + assert "open questions (answered by you, in the notebook)" in out + assert "confirm that the folders of repeat rep1 were prepared and acquired" in out + assert "choose the ID of condition CTRL" in out + assert listing(tmp_path) == before and not output.exists() + + +def test_a_folder_that_cannot_be_an_id_has_no_proposal_and_is_not_repaired(tmp_path, capsys): + root = tree(tmp_path / "exp", "control/rep1", "DrugA/rep1") + assert main(["describe", str(root)]) == 0 + out = capsys.readouterr().out + assert "control: no ID can be proposed" in out and "you name it" in out + assert "DrugA: proposed ID 'DrugA'" in out + + +def test_describe_reads_the_legacy_design_without_confirming_anything(experiment, tmp_path, capsys): + output = tmp_path / "out" + rows = [ + row("CTRL", "rep1", control=True), + row("CTRL", "rep2", control=True), + row("DrugA", "rep1", control=True, confirmed=False, on=""), + row("DrugA", "rep2", control=True, confirmed=False, on=""), + ] + output.mkdir() + write_experiment_design(ExperimentDesign(tuple(rows)), output / DESIGN) + before = listing(tmp_path) + assert main(["describe", str(experiment), "--output", str(output)]) == 0 + out = capsys.readouterr().out + assert "legacy experiment_design.csv" in out + assert "DrugA -> CTRL" not in out # two controls: nothing is guessed + assert "marked as a control, as is another one" in out + assert "rep1: proposed ID 'rep1'; not confirmed" in out + assert listing(tmp_path) == before + + +def test_publish_refuses_a_draft_with_open_questions_and_lists_them(experiment, tmp_path, capsys): + output = tmp_path / "out" + before = listing(tmp_path) + assert main(["describe", str(experiment), "--output", str(output), "--publish"]) == 1 + out = capsys.readouterr().out + assert "Not published. What is missing:" in out + missing = out.split("Not published. What is missing:")[1] + assert missing.count(" - choose the ID of") == 5 # 2 conditions, single, 2 repeats + assert " - confirm that the folders of repeat rep2" in out + assert not (output / TOML).exists() and not output.exists() + assert listing(tmp_path) == before + + +def test_publish_never_confirms_for_the_user(experiment, tmp_path, capsys): + """A fully confirmed legacy design still leaves the IDs to the user: nothing is adopted.""" + output = tmp_path / "out" + output.mkdir() + rows = [row(c, r) for c in ("CTRL", "DrugA") for r in ("rep1", "rep2")] + write_experiment_design(ExperimentDesign(tuple(rows)), output / DESIGN) + before = listing(tmp_path) + assert main(["describe", str(experiment), "--output", str(output), "--publish"]) == 1 + out = capsys.readouterr().out + assert "choose the ID of condition CTRL" in out + assert "confirmed on 2026-10-30" in out # the user's own earlier confirmation, as given + assert listing(tmp_path) == before # no design rewritten, no description, no draft + + +def test_a_complete_draft_is_still_not_publishable_from_the_command_line(tmp_path): + """Every ID and repeat confirmed leaves only what the draft does not hold.""" + from morphobricks_preprocessing.cli_describe import NOT_IN_DRAFT, draft_blockers + + discovery = discover_experiment(tree(tmp_path / "exp", "CTRL/rep1", "DrugA/rep1")) + draft = draft_from_discovery(discovery) + for kind, key in [ + ("conditions", "CTRL"), + ("conditions", "DrugA"), + ("technical_repeats", "single"), + ("repeats", "rep1"), + ]: + draft = draft.confirm_id(kind, key, key) + assert [q.code for q in draft.open_questions()] == ["repeat_not_confirmed"] + assert any("same circumstances" in item for item in draft_blockers(draft)) + done = replace( + draft, + repeats={"rep1": replace(draft.repeats["rep1"], confirmed=True, confirmed_on="2026-11-01")}, + ) + assert done.publishable + assert draft_blockers(done) == [NOT_IN_DRAFT] + + +def test_a_legacy_design_that_cannot_be_read_is_refused_and_left_alone( + experiment, tmp_path, capsys +): + output = tmp_path / "out" + output.mkdir() + (output / DESIGN).write_text("not,a,design\n1,2,3\n", encoding="utf-8") + before = listing(tmp_path) + assert main(["describe", str(experiment), "--output", str(output)]) == 1 + assert "cannot be drafted" in capsys.readouterr().out + assert listing(tmp_path) == before + + +def test_a_missing_experiment_folder_is_an_error(tmp_path, capsys): + assert main(["describe", str(tmp_path / "nowhere")]) == 1 + assert "does not exist or is not a folder" in capsys.readouterr().out + + +# --- a published description ------------------------------------------------------------- + + +V01 = next(f for f in fixtures("valid") if f.name.startswith("v01_")) + + +def published(tmp_path): + """Publish a description whose counts match the folders written for it (2 images a unit).""" + description = description_from_bytes(V01.data) + repeats = {} + for key, repeat in description.repeats.items(): + members = { + cid: replace(m, image_count=2 * len(m.technical_repeats)) + for cid, m in repeat.members.items() + } + repeats[key] = replace(repeat, members=members) + description = replace(description, repeats=repeats) + root, output = tmp_path / "exp", tmp_path / "out" + output.mkdir() + build_experiment(root, description) + publish_description(output, description, base_sha256="absent") + return root, output + + +def test_a_published_description_that_matches_the_folders_has_nothing_to_publish(tmp_path, capsys): + root, output = published(tmp_path) + before = listing(tmp_path) + assert main(["describe", str(root), "--output", str(output), "--publish"]) == 0 + out = capsys.readouterr().out + assert "No change." in out and "no image header is read" in out + assert "Nothing to publish" in out + assert listing(tmp_path) == before + + +def test_a_new_folder_beside_a_published_description_needs_the_users_answer(tmp_path, capsys): + root, output = published(tmp_path) + make_tree(root, ["Extra/day9/a.tif"]) + sha = sha256_hex((output / TOML).read_bytes()) + before = listing(tmp_path) + assert main(["describe", str(root), "--output", str(output)]) == 0 + assert "new, no ID yet: Extra / day9 / single" in capsys.readouterr().out + assert main(["describe", str(root), "--output", str(output), "--publish"]) == 1 + out = capsys.readouterr().out + assert "Extra / day9 / single is new: say which repeat" in out + assert sha256_hex((output / TOML).read_bytes()) == sha + assert listing(tmp_path) == before + + +def test_a_description_that_cannot_be_read_is_refused(tmp_path, capsys): + root, output = published(tmp_path) + (output / TOML).write_text("schema = 'nope'\n", encoding="utf-8") + assert main(["describe", str(root), "--output", str(output)]) == 1 + assert "cannot be used" in capsys.readouterr().out From 6b07278cb327bd4e2ee2e8b769270fbce462a69f Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 04:20:41 +0000 Subject: [PATCH 2/2] Renumber the Stage 1 plan question to 78 Question 77 is taken on main by the Z-projection decision (#212). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm --- CHANGELOG.md | 2 +- docs/contract.md | 2 +- docs/plan.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 48e2a99a..a23e6ac2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,7 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the exits with status 1; since no answers live outside the notebook yet, it publishes nothing from a draft of folders (open question to the owner), and says `Nothing to publish` for a published description that matches the folders. Docs: contract §12 and §16, user guide - "Describe your experiment", plan §12 question 77 (the owner's answers Q1 to Q8), + "Describe your experiment", plan §12 question 78 (the owner's answers Q1 to Q8), `docs/conformance.md` (the recorded run of the suite with no Spec skip). - **Z-projection arithmetic (part of #176).** New pure module `projection.py`: `Projection` (method `max`, `mean` or `sum`, no default; optional zero-based half-open slice range), diff --git a/docs/contract.md b/docs/contract.md index c003ceda..00f86196 100644 --- a/docs/contract.md +++ b/docs/contract.md @@ -624,7 +624,7 @@ instead of keeping copies. | Masks named as images (D-i) | `mask_name_issues(leaves, mask_names=())`, `images_named_as_masks(leaves, suffixes=MASK_NAME_SUFFIXES)`, `mask_folder_part(path)`, `MASK_NAME_SUFFIXES`, `MASK_FOLDER_WORDS` (`discovery`) | The screen MorphoRelate's `init` ran (`project_image_names`), now discovery's warnings `image_named_as_mask` and `mask_folder_in_experiment` (§1) for every brick. Names are compared in lower case across the whole experiment; the longest `position_id` owns a file, an image of the same folder first. `mask_names` adds the caller's expected mask names (`cells` for `f1_cells.tif`) to `MASK_NAME_SUFFIXES`; `discover_experiment` gives none. A folder counts as a masks folder when a word of its name (`CellLabels` is `Cell` and `Labels`) is in `MASK_FOLDER_WORDS`. `images_named_as_masks` returns the `(image, owner)` pairs of that detection for the caller's own name endings (`["_cells"]`), for a brick that acts on the files itself (MorphoRelate leaves out the images named with a declared mask name). | | Pixel sizes (D-a) | `same_pixel_size(a, b, roles=("X", "Y"))`, `pixel_size_groups(sizes, roles=)`, `PIXEL_SIZE_TOLERANCE` (`physical_sizes`) | Two pixel sizes are the same when `|a - b| <= 1e-3 * max(a, b)` on every compared axis, in micrometres; one rule for a declared size against a header and for two headers. `same_pixel_size` takes `PhysicalSizes` (converted with `micrometers`) or sizes already in micrometres, one per role, and returns None when either is unknown (no size, or a unit that cannot be converted), never True. The rule is not transitive (1.0 ≈ 1.0009 ≈ 1.0018, but 1.0 ≠ 1.0018), so there is no per-image grouping key: `pixel_size_groups` links images by single linkage, so images of two groups always differ while the extremes of one group may drift; unknown sizes are left out; the largest group first. A size that cannot be converted to a positive finite length in micrometres (`1e308 m`) is unknown, never infinite, and an infinite or NaN value is never the same as any other (issue #93; before, `inf` matched every size). The folder check's `different_pixel_sizes` uses both (§11). | | Experiment design | `read_confirmed_design(path, expected_sha256=)`, `ConfirmedDesign`, `DesignCheckError` (`design`) | The one reader of `experiment_design.csv` (§12) for a brick that pools biological repeats (formerly CellPatcher `load_experiment_design` and MorphoFeature `ExperimentDesign.read`). It takes the file as this brick writes it and fills nothing in: UTF-8 without a byte-order mark; the header exactly the §12 columns in order (a column named twice or without a name is refused naming it, as by the lenient reader, §12); one value per column in every row (an unclosed quote, a missing or extra value, or a blank line is refused); no value with outer whitespace; `control` and `confirmed` exactly `true` or `false`; counts written as whole numbers of at least 0; `schema_version` 1; `confirmed_on` empty or `YYYY-MM-DD`; `condition`, `biological_repeat` and `repeat_group` pass `check_identity_text` and do not differ only in case (`check_identity_case`); no folder twice, one `control` per condition, at least one row; every group confirmed with one folder per condition (`design_issues`); with `expected_sha256` (the manifest's `experiment_design_sha256`) the file must be that one. The SHA-256 is of the bytes parsed. `ConfirmedDesign` gives `path`, `sha256`, the `design`, read-only `repeat_groups` (`{(condition, biological_repeat): repeat_group}`, the normalization brick's input), `group_members()` (`{repeat_group: {condition: biological_repeat}}`), `conditions`, `controls` and `groups`. Refusals raise `DesignCheckError` (a `ValueError`) with its `issues`: `missing_experiment_design`, `unreadable_experiment_design`, `invalid_experiment_design`, `repeat_group_conflict`, `design_not_confirmed` or `changed_experiment_design`, each an error naming the file (or the group's folders). This brick's step 1 and `check` read the file being edited with the lenient `load_design(output_folder, repeats)` (§12), which returns `(design, changes)`: the saved design brought up to date with the folders, or the default design when there is no file. A saved file that cannot be read raises `DesignCheckError` with the one error `unreadable_experiment_design`, whose message is the same "The experiment design cannot be read: …" as here followed by the lenient reader's refusal, naming the line and column where it can (the `DesignFormatError` is its `__cause__`); no default design is returned and the file is not touched (owner decision of 31 October 2026, plan §12 question 43). **Breaking in v0.4.0:** up to v0.3.4 `load_design` returned a third value, `problem`, the reason as text, together with the default design. | -| Experiment description (Stage 1) | `read_description(path)`, `description_consume.read_published_description(path)` (the published `morphobricks.toml` only), `run.description_file.publish_description` (the only writer), `draft.draft_from_discovery`, `draft.migrate_legacy_design`, `description_reconcile.reconcile` | A brick reads the **published** `morphobricks.toml` through `RunInputs` (§9.1) and never a draft: a draft is internal to preprocessing, is never named `morphobricks.toml` and is refused with `A_DRAFT_REFUSED` (experiment-v1 §12). The command line shows the draft (`describe`, §12) and confirms nothing: the user's answers are the only way an ID, a repeat or a removal becomes confirmed. The owner's answers Q1 to Q8 of the Stage 1 design are in plan §12, question 77. | +| Experiment description (Stage 1) | `read_description(path)`, `description_consume.read_published_description(path)` (the published `morphobricks.toml` only), `run.description_file.publish_description` (the only writer), `draft.draft_from_discovery`, `draft.migrate_legacy_design`, `description_reconcile.reconcile` | A brick reads the **published** `morphobricks.toml` through `RunInputs` (§9.1) and never a draft: a draft is internal to preprocessing, is never named `morphobricks.toml` and is refused with `A_DRAFT_REFUSED` (experiment-v1 §12). The command line shows the draft (`describe`, §12) and confirms nothing: the user's answers are the only way an ID, a repeat or a removal becomes confirmed. The owner's answers Q1 to Q8 of the Stage 1 design are in plan §12, question 78. | ## 17. Named-axis copies (`convert`, v0.4.0, design v0.4 §5) diff --git a/docs/plan.md b/docs/plan.md index ebf3a474..aeb185d1 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -1554,7 +1554,7 @@ Raised during review; to be decided before the milestones that need them: `RunInputs.open` refuses a manifest recording `shared: true`. Plain-stack masks with stated axes (`Cells=TYX`) remain and are checked against the image. (3) The other bricks are not changed: their own Z-plane policies stay as a second defence. -77. Stage 1: the experiment description (`morphobricks.toml`, Spec experiment-v1) and the +78. Stage 1: the experiment description (`morphobricks.toml`, Spec experiment-v1) and the notebook that writes it. *Decided (owner, 7 November 2026; all eight recommendations of the Stage 1 design, section 6, accepted):* (Q1) the notebook always writes the description; the command line and `run_experiment` without one keep the v0.4 behaviour