CLI describe, docs and conformance run (Stage 1, PR 17)

CHANGELOG.md

@@ -8,6 +8,21 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the
 
 ## Unreleased
 
+- **`describe` command, docs and a recorded conformance run (Stage 1, PR 17).**
+  `morphobricks-preprocessing describe EXPERIMENT [--output FOLDER] [--exclude PATH]
+  [--publish]` prints the draft experiment description in plain language: the proposed IDs
+  (none, never repaired, for a folder that cannot be an ID), the repeats with their folders and
+  image counts, the references and every open question with what the user has to do. The draft
+  is the legacy `experiment_design.csv` of the output folder migrated, or the folders alone;
+  against a published `morphobricks.toml` it compares the folders (new, gone, image counts,
+  repeats to confirm again; no header is read). It reads no image and writes nothing, and
+  **confirms nothing**: confirmations are the user's (owner rule). `--publish` publishes only
+  a draft with no open question and every decision confirmed, else it lists what is missing and
+  exits with status 1; since no answers live outside the notebook yet, it publishes nothing
+  from a draft of folders (open question to the owner), and says `Nothing to publish` for a
+  published description that matches the folders. Docs: contract §12 and §16, user guide
+  "Describe your experiment", plan §12 question 78 (the owner's answers Q1 to Q8),
+  `docs/conformance.md` (the recorded run of the suite with no Spec skip).
 - **Reader owns its rule vocabulary; five refusals pinned by path (#204).**
   `description_identity` no longer imports `check_identity_text` from `discovery` or `SINGLE`
   from normalization (Spec §2.7): `description_rules` holds `MISSING_TEXT`, `SINGLE` and

README.md

@@ -33,7 +33,7 @@ Pre-alpha; the current release is v0.4.0 (what is planned next is in
 - Reading a run from another brick: `RunInputs.open(output_folder)`
 - A step-by-step notebook ([notebooks/MorphoBricks_Preprocessing.ipynb](notebooks/MorphoBricks_Preprocessing.ipynb))
   and a command line (`morphobricks-preprocessing check` / `run` / `masks` / `calibrate` /
-  `design` / `convert`)
+  `design` / `describe` / `convert`)
 
 Every feature, message code and output file is described in the [user guide](docs/user_guide.md).
 

docs/conformance.md

@@ -0,0 +1,30 @@
+# Conformance run record (Stage 1, PR 17)
+
+The design (Stage 1, PR 17) names no place for this record, so it is kept here. Every row is a
+real run; a skip is never counted as a pass.
+
+## Run of 8 November 2026
+
+| | |
+|---|---|
+| Base commit | `68b2a4b` (`origin/main`, merge of PR 211); the run included the changes of PR 17 |
+| Spec pin | the full SHA in `tests/spec_fixtures/PIN.toml` (vendored `experiment-v1.md` at `b197ac1`) |
+| Command | `python tools/check.py` (Ruff, Ruff format, then `pytest -q` over the whole `tests/` folder), then `python tools/check.py --rules` |
+| Whole suite | 4324 passed, 4 skipped, 0 failed (301 s) |
+| Spec modules (`tests/test_spec_*.py`) | 1535 passed, 0 skipped, 0 failed (the Spec gate fails a skip or xfail there) |
+| Code rules | passed, no baseline entry grew |
+
+The 4 skips are outside the Spec modules, each an environment limit of the machine that ran:
+
+- `tests/test_cli_log.py:165`, `tests/test_folder_rules.py:201`, `tests/test_folder_rules.py:229`:
+  the file system tells names apart by case, so the case-only collision cannot be built;
+- `tests/test_discovery.py:665`: the run had permissions that ignore file modes (root).
+
+These four behaviours are therefore **not verified by this run**.
+
+## Not covered
+
+- The Spec's `fixtures/expected/` golden CSVs and the conformance items that need PRs 13 to 16
+  (notebook cards) are outside this run; this is not a release-candidate sign-off for those.
+- `describe --publish` publishes nothing from a draft of folders (see the pull request's open
+  questions), so no end-to-end publication through the command line is part of this record.

docs/contract.md

@@ -434,6 +434,7 @@ folder.
 | Command line | `run` reads the file from the output folder, or `--design FILE`. Without a fully confirmed design it writes the design brought up to date with the folders (a template with `confirmed=false` for the groups still to confirm, keeping the answers already given) to the output folder, explains the columns, and stops with status 1; the run stops on `design_not_confirmed` first and writes its report (§9.2). Confirmed rows without a date get the date of the run. |
 | Template from `check` (v0.4.0, plan §12 question 57) | `check` (CLI; `run.write_check_outputs` in Python) also writes the design prefilled from the folders, under the output folder's lock: one row per condition and biological-repeat folder, `repeat_group` the folder name, the current counts, `confirmed=false`, `control=false`. Only when the output folder holds no `experiment_design.csv` (a design there, edited or unreadable, is never replaced) and the folders have no error. |
 | `design` command (v0.4.0, plan §12 question 57) | `design --output FOLDER` prints the design as a table and the repeat groups not confirmed yet, and any `repeat_group_conflict` or `invalid_experiment_design`. Options change it without editing the file, in this order: `--repeat-group CONDITION/REPEAT=GROUP` (repeatable; moves a folder, and the groups it leaves and joins are no longer confirmed, as `ExperimentDesign.with_group`), `--control CONDITION` and `--no-control CONDITION` (repeatable), `--confirm GROUP` (repeatable) and `--confirm-all` (every group not confirmed yet). It reads, changes and writes the file under the lock (`output_locked` while a run holds it), with the run's reader (`read_experiment_design`) and validator (`design_issues`); a confirmation fills `confirmed_on` with today's date and the counts are those in the file; a group already confirmed keeps its date. An unknown folder, condition or group, a condition both marked and unmarked, a group to confirm whose rows break the rules (`repeat_group_conflict`, `invalid_experiment_design`) or a design that cannot be written is refused, and nothing is changed (exit status 1). Nothing is confirmed by default. Python: `edit_design(output_folder, DesignEdit(...), on=None)` returns the design and the file written (None when nothing changed), raising `DesignEditError`; `apply_design_edit(design, edit, on)` is the same change without files. `DesignEdit.unconfirm` (v0.4.0, PP-10; not a CLI option) takes confirmations back, after the moves and before the confirmations (an unknown group, or one both confirmed and unconfirmed, also through `confirm_all`, is refused; it never confirms anything). The notebook applies every design answer with `apply_design_edit` (the "Undo" with `unconfirm`) and saves it with `run.design_file.write_design(design, output_folder)`, under the lock. The run reconciles the design with the folders as before, so a group that gained a folder since is asked again. |
+| `describe` command (Stage 1, PR 17) | `describe EXPERIMENT [--output FOLDER] [--exclude PATH] [--publish]` prints the draft experiment description (experiment-v1 §12) in plain language: the proposed ID of each condition, technical-repeat folder and biological repeat (no proposal, with the rule's code, for a folder that cannot be an ID: never repaired), each repeat with its folders and image counts and whether it is confirmed (and the date), the references, the notes of a migration with several dates, and every open question with what the user has to do. The draft is the legacy `experiment_design.csv` of the output folder migrated (`draft.migrate_legacy_design`; one that cannot be read or breaks a rule is exit status 1 and left as it is) when there is one and no published `morphobricks.toml`, else the draft of the folders (`draft.draft_from_discovery`); when `morphobricks.toml` is published (`run.description_run.load_description`), the folders are compared with it (`description_reconcile.reconcile`: unchanged, new, gone, image counts, repeats to confirm again), reading the folder structure only, so the channels are not compared. It reads no image and writes nothing: no output folder, draft, description, design or records file. **It confirms nothing**: an ID, the comparability of a repeat (N35), a reference, a removal and every date are the user's answers (notebook), never given by the command or read from the clock. `--publish` publishes a draft only when no question is open and every decision is confirmed; otherwise it prints `Not published. What is missing:` with every item and exits with status 1. This version holds no answers outside the notebook (the draft file is not in the package), so `--publish` refuses every draft built from folders and publishes nothing; for a published description that matches the folders it says `Nothing to publish` (exit 0); a new, gone or changed folder is listed and the file is left as it is. An error (experiment folder not found, discovery errors, unreadable description or records) is exit status 1; the command takes no lock. |
 | Run | `run_experiment(..., design=...)` brings the design up to date with the folders and stops with `design_not_confirmed` or `repeat_group_conflict` if it is not fully confirmed. Otherwise the file is written with the table and published before the manifest, which records its SHA-256 (§9). Without a design the run is unchanged: no file is written, an existing one is left as it is, and `experiment_design_sha256` is null. `plane_histograms.csv` does not depend on the design. |
 
 ## 13. Image type
@@ -623,6 +624,7 @@ instead of keeping copies.
 | Masks named as images (D-i) | `mask_name_issues(leaves, mask_names=())`, `images_named_as_masks(leaves, suffixes=MASK_NAME_SUFFIXES)`, `mask_folder_part(path)`, `MASK_NAME_SUFFIXES`, `MASK_FOLDER_WORDS` (`discovery`) | The screen MorphoRelate's `init` ran (`project_image_names`), now discovery's warnings `image_named_as_mask` and `mask_folder_in_experiment` (§1) for every brick. Names are compared in lower case across the whole experiment; the longest `position_id` owns a file, an image of the same folder first. `mask_names` adds the caller's expected mask names (`cells` for `f1_cells.tif`) to `MASK_NAME_SUFFIXES`; `discover_experiment` gives none. A folder counts as a masks folder when a word of its name (`CellLabels` is `Cell` and `Labels`) is in `MASK_FOLDER_WORDS`. `images_named_as_masks` returns the `(image, owner)` pairs of that detection for the caller's own name endings (`["_cells"]`), for a brick that acts on the files itself (MorphoRelate leaves out the images named with a declared mask name). |
 | Pixel sizes (D-a) | `same_pixel_size(a, b, roles=("X", "Y"))`, `pixel_size_groups(sizes, roles=)`, `PIXEL_SIZE_TOLERANCE` (`physical_sizes`) | Two pixel sizes are the same when `|a - b| <= 1e-3 * max(a, b)` on every compared axis, in micrometres; one rule for a declared size against a header and for two headers. `same_pixel_size` takes `PhysicalSizes` (converted with `micrometers`) or sizes already in micrometres, one per role, and returns None when either is unknown (no size, or a unit that cannot be converted), never True. The rule is not transitive (1.0 ≈ 1.0009 ≈ 1.0018, but 1.0 ≠ 1.0018), so there is no per-image grouping key: `pixel_size_groups` links images by single linkage, so images of two groups always differ while the extremes of one group may drift; unknown sizes are left out; the largest group first. A size that cannot be converted to a positive finite length in micrometres (`1e308 m`) is unknown, never infinite, and an infinite or NaN value is never the same as any other (issue #93; before, `inf` matched every size). The folder check's `different_pixel_sizes` uses both (§11). |
 | Experiment design | `read_confirmed_design(path, expected_sha256=)`, `ConfirmedDesign`, `DesignCheckError` (`design`) | The one reader of `experiment_design.csv` (§12) for a brick that pools biological repeats (formerly CellPatcher `load_experiment_design` and MorphoFeature `ExperimentDesign.read`). It takes the file as this brick writes it and fills nothing in: UTF-8 without a byte-order mark; the header exactly the §12 columns in order (a column named twice or without a name is refused naming it, as by the lenient reader, §12); one value per column in every row (an unclosed quote, a missing or extra value, or a blank line is refused); no value with outer whitespace; `control` and `confirmed` exactly `true` or `false`; counts written as whole numbers of at least 0; `schema_version` 1; `confirmed_on` empty or `YYYY-MM-DD`; `condition`, `biological_repeat` and `repeat_group` pass `check_identity_text` and do not differ only in case (`check_identity_case`); no folder twice, one `control` per condition, at least one row; every group confirmed with one folder per condition (`design_issues`); with `expected_sha256` (the manifest's `experiment_design_sha256`) the file must be that one. The SHA-256 is of the bytes parsed. `ConfirmedDesign` gives `path`, `sha256`, the `design`, read-only `repeat_groups` (`{(condition, biological_repeat): repeat_group}`, the normalization brick's input), `group_members()` (`{repeat_group: {condition: biological_repeat}}`), `conditions`, `controls` and `groups`. Refusals raise `DesignCheckError` (a `ValueError`) with its `issues`: `missing_experiment_design`, `unreadable_experiment_design`, `invalid_experiment_design`, `repeat_group_conflict`, `design_not_confirmed` or `changed_experiment_design`, each an error naming the file (or the group's folders). This brick's step 1 and `check` read the file being edited with the lenient `load_design(output_folder, repeats)` (§12), which returns `(design, changes)`: the saved design brought up to date with the folders, or the default design when there is no file. A saved file that cannot be read raises `DesignCheckError` with the one error `unreadable_experiment_design`, whose message is the same "The experiment design cannot be read: …" as here followed by the lenient reader's refusal, naming the line and column where it can (the `DesignFormatError` is its `__cause__`); no default design is returned and the file is not touched (owner decision of 31 October 2026, plan §12 question 43). **Breaking in v0.4.0:** up to v0.3.4 `load_design` returned a third value, `problem`, the reason as text, together with the default design. |
+| Experiment description (Stage 1) | `read_description(path)`, `description_consume.read_published_description(path)` (the published `morphobricks.toml` only), `run.description_file.publish_description` (the only writer), `draft.draft_from_discovery`, `draft.migrate_legacy_design`, `description_reconcile.reconcile` | A brick reads the **published** `morphobricks.toml` through `RunInputs` (§9.1) and never a draft: a draft is internal to preprocessing, is never named `morphobricks.toml` and is refused with `A_DRAFT_REFUSED` (experiment-v1 §12). The command line shows the draft (`describe`, §12) and confirms nothing: the user's answers are the only way an ID, a repeat or a removal becomes confirmed. The owner's answers Q1 to Q8 of the Stage 1 design are in plan §12, question 78. |
 
 ## 17. Named-axis copies (`convert`, v0.4.0, design v0.4 §5)
 

docs/development.md

@@ -121,6 +121,9 @@ commit SHA in its `PIN.toml`, never a branch.
   modules (`description*.py`) for a second table of codes or names and for lenient, older or
   default-version paths.
 
+The result of the last recorded whole-suite run, with its skips, is in
+[conformance.md](conformance.md).
+
 ## Versioning
 
 - Semantic versioning for the package.

docs/plan.md

@@ -1569,3 +1569,28 @@ Raised during review; to be decided before the milestones that need them:
     written to the copy; it and its source go to the record's `dropped_sizes`. The record
     version is 3. Built in stages: the pure arithmetic (`projection`), the record, then
     the plan and writer, the command, the run side and the notebook card.
+78. Stage 1: the experiment description (`morphobricks.toml`, Spec experiment-v1) and the
+    notebook that writes it. *Decided (owner, 7 November 2026; all eight recommendations of
+    the Stage 1 design, section 6, accepted):* (Q1) the notebook always writes the
+    description; the command line and `run_experiment` without one keep the v0.4 behaviour
+    with a warning, so the NFC and mask-type-name refusals and the raw header-name checks
+    apply only with a description; a description is required from Stage 3. (Q2) The
+    publication records (re-confirmations, removals, image-count changes) live in the
+    append-only `morphobricks.records.jsonl` written under the lock at publication and copied
+    unchanged into the next run's manifest; Spec §4.6 says so. (Q3) Layouts whose units are
+    identical and header-named are one confirmation; unnamed layouts are pre-filled but need
+    one tick per acquisition beside its thumbnails. (Q4) A raw header name is proposed as a
+    channel ID only when it is valid as it stands (never repaired), and nothing is adopted
+    without the user's confirmation. (Q5) A mask has no default representation: `instances`
+    or `binary` is the user's answer, and the dtype and the values seen are evidence only.
+    (Q6) A Spec change before PR 7: a condition gets the `single` rule and the ID proposal
+    rule of §4.6a covers repeat IDs. (Q7) Time-lapses are allowed as before; homogeneity
+    compares axis semantics, not the number of time points; previews show t = 0 with a T
+    slider. (Q8) The added fields are `condition_id` and `technical_repeat_id` beside
+    `repeat_group`, the source folder labels staying in `Fov.key` (contract §9.1, §16).
+    *Implemented in PR 17:* the command `describe` (contract §12) prints the draft and
+    confirms nothing (the owner's rule: confirmations are given only by the user); its
+    `--publish` publishes only a draft with no open question and every decision confirmed.
+    The draft file that would hold the user's answers outside the notebook is not part of the
+    package yet, so `--publish` cannot complete a draft built from folders (open question to
+    the owner, PR 17).