From d023ff91289a2d2204e1ae56f7dee114d1a665e4 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 22:54:33 +0000 Subject: [PATCH 1/6] Prepare v0.5.1: version 0.5.1, the release notes, the shipped notebook for the 7-step app - pyproject, the notebook's current_version/RELEASE_TAG, the version index and the README pins: 0.5.1. CHANGELOG "Unreleased" becomes "v0.5.1 (10 November 2026)" with a user-facing summary and a migration note for users and bricks. - Notebook: cell 3 calls PreprocessingNotebook(); every code cell is a Colab form ("cellView": "form"); a cell that cannot go on raises its own CellStop, shown as one line without a traceback; cell 3 on Colab without Drive opens the app after one line; cell 1 checks the Python version before importing tomllib; the introduction describes the seven-step app. - Tests: the one-line stops in a real kernel (cell 3 alone, a refused checkout), faked Colab without Drive, the cellView metadata, the version check before tomllib. - docs/colab-release-check.md rewritten for the three cells and the seven steps. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- AGENTS.md | 19 +- CHANGELOG.md | 99 +++++++++- README.md | 15 +- docs/colab-release-check.md | 202 +++++++++------------ docs/user_guide.md | 6 +- notebooks/MorphoBricks_Preprocessing.ipynb | 86 ++++++--- notebooks/notebook_latest_versions.yaml | 2 +- pyproject.toml | 2 +- tests/test_notebook_file.py | 48 ++++- tests/test_notebook_setup.py | 98 ++++++++-- 10 files changed, 383 insertions(+), 194 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index c2b97f92..d2ea8e2f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -834,13 +834,14 @@ Technical details go to `logging` (shown in the notebook's "Technical details" p tests skip without zarr; CI's Python 3.11 jobs pin zarr 2.18 and 3.13 jobs get zarr 3. - **The notebook file is tested too:** - valid nbformat 4, with no outputs; - - code cells hidden and titled `# @title . …` (1. Install, 2. Mount Google Drive, 3. Run preprocessing); + - code cells hidden as Colab forms (`"cellView": "form"`) and titled `# @title . …` (1. Install, 2. Mount Google Drive, 3. Run preprocessing); - Colab-only code and `/content` paths only inside the Colab guard; no magics; - the GitHub install pinned to `RELEASE_TAG`, equal to `current_version` and the version index; - exactly three code cells, in order: 1 installs and reports versions, 2 mounts Google Drive on Colab (a one-line note elsewhere), 3 displays only widgets built by `notebook_ui` and, - run out of order, names the cell to run first (`tests/test_notebook_setup.py`); + run before cell 1, names cell 1 (`tests/test_notebook_setup.py`); every stop of a cell is + one line without a traceback (the cell's `CellStop`, tested in a real kernel); - the buttons it names exist; - its cells execute, by `exec` and in a real kernel (`nbclient`, `tests/test_notebook_file.py`; cell 1's alternatives and the cells out of order in `tests/test_notebook_setup.py`). @@ -874,8 +875,12 @@ plainer version "not interactive enough". Descriptive, guided and forgiving is t - Outputs are stripped. - Three code cells (owner, 10 November 2026, as ZeroCostDL4Mic and NucleiSky): **1. Install**, **2. Mount Google Drive**, **3. Run preprocessing**, each under a Markdown heading that - explains it. Cell 3 checks that cell 1 ran (`SETUP_RECORD`) and, on Colab, that Drive is - mounted, and otherwise stops naming the cell to run; no cell runs another. + explains it. Cell 3 checks that cell 1 ran (`SETUP_RECORD`), and otherwise stops naming + cell 1; on Colab without Drive mounted it opens the app after one line saying that only + files on the runtime are visible (owner, v0.5.1). No cell runs another. + - A cell that cannot go on raises its own `CellStop` (a `RuntimeError` whose + `_render_traceback_` returns the message), so Colab and JupyterLab show one line and no + traceback (owner, v0.5.1). Cell 1 checks the Python version before `import tomllib`. - Cell 1 is a standalone standard-library bootstrap: nearby checkout, installed package, or GitHub installation of the release `RELEASE_TAG`, with missing requirements installed using the kernel's Python (`subprocess`, never `!pip`); on Colab with the @@ -1051,14 +1056,14 @@ jupyter lab notebooks/MorphoBricks_Preprocessing.ipynb --- -## 11. Status (current state, v0.4.0) +## 11. Status (current state, v0.5.1) -- **Released:** v0.4.0 (5 November 2026), with the normalization brick pinned at v0.3.4. The +- **Released:** v0.5.1 (10 November 2026), with the normalization brick pinned at v0.4.0. The per-release history is in [CHANGELOG.md](CHANGELOG.md); the milestones PP-0 to PP-13 are in [docs/archive/plan-milestones-v0.1-v0.4.md](docs/archive/plan-milestones-v0.1-v0.4.md); the older status log of this file is in [docs/archive/agents-status-history-v0.1-v0.3.md](docs/archive/agents-status-history-v0.1-v0.3.md). -- **Unreleased:** see the changelog's "Unreleased" section. +- **Unreleased:** none yet; later changes go in a new "Unreleased" section of the changelog. - **Scope:** 2-D per plane (a Z-stack is refused as `z_stack_not_supported`); TIFF and OME-Zarr input read directly (TIFF to OME-Zarr conversion and the `_moved.py` re-exports were removed); optional masks, the mask step and the notebook's **Describe the masks**; named-axis copies diff --git a/CHANGELOG.md b/CHANGELOG.md index 01239fc6..6b2653cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,104 @@ The design is `docs/restart/preprocessing-design-v0.4.md` in MorphoFeature, on b are in [docs/plan.md](docs/plan.md) §12. (The v0.3 to v0.4 migration guide was removed in v0.5.1: no run before v0.5.1 is read.) -## Unreleased (v0.5.1) +## v0.5.1 (10 November 2026) + +The first release in which every run is described, with the shared request resolver for the +other bricks and the seven-step notebook. There is no backward compatibility before the first +pipeline release (MorphoBricks-Spec ADR 0002 AR10): older run folders, records and drafts are +refused, never converted. Normalization is pinned at v0.4.0. + +**Breaking** + +- **Every run needs a published experiment description** (`morphobricks.toml`, saved by the + notebook's *Check and save*). The run, the mask step, `check`/`run`/`masks` and + `RunInputs.open` refuse an output folder without one with `A_DESCRIPTION_REQUIRED`, before + anything is read or written. +- **Older files are refused, not converted**: run manifests of versions 1 to 3 (every run of + v0.5.0 or earlier: run preprocessing again, which reads every image again), conversion + records other than version 3 (convert the originals again), answer drafts saved under the + earlier Spec pin (`spec_commit_mismatch`), and hand-edited manifests missing a key. +- **The legacy `experiment_design.csv` migration is gone**: the file is an output only and + nothing reads it back; the `design` command and `run --design` are removed. +- **Normalization v0.4.0**: `PlaneRecord.position_id` is required, a blank `position_id` in a + table row is refused, and tables of schema versions 1 to 3 are refused. + +**New** + +- **`RunInputs.resolve_request`** (resolver contract version 1): one interpreter of a described + run for every analysis route, channels by canonical ID per position, every refusal at once, + coverage per biological repeat and the provenance. +- **`Fov.image_type`**: each position's declared image type, `"rgb"` or `"channels"`. +- **Lab-tool links**: `run --lab-links FILE` (and `--confirm-repeat-codes`), recorded in the + run manifest only. +- **The seven-step notebook**: Images → Axes → Channels → Masks → Design → Check and save → + Run, with a quick scan and an example picture before the headers are read, progress for every + wait, one column and no box that scrolls on its own. Three code cells shown as Colab forms; + a cell that cannot go on stops with one line, without a traceback. + +**Fixes** + +- The notebook output is no longer flooded with debug messages from ImageJ TIFFs. +- *Is compared against* keeps its dropdown; lists of files draw no empty bullets; every + disabled action says beside it what it waits for. +- On Python 3.10, cell 1 says "Select a Python 3.11 or newer kernel" instead of failing on + `import tomllib`. + +### Migration from v0.5.0 + +**Users.** Open the v0.5.1 notebook, describe the experiment (the seven steps) and run again +into the same output folder: the run reads every image again and replaces the v0.5.0 output. +Nothing in an old `experiment_design.csv` is carried over; describe the design in *Design*. On +the command line, `check`, `run` and `masks` need the published description: write it in the +notebook (`describe` shows what is still open). A `.morphobricks-conversion.json` written before +v0.5.1 is refused: run `convert` again into a new folder. + +**Bricks** (each is unaffected until its pin moves to v0.5.1): + +| v0.5.0 | v0.5.1 | +|---|---| +| A run without a description | Refused (`A_DESCRIPTION_REQUIRED`, `DescriptionRequiredError`); tests use `testing.make_clean_run` (described) or publish a `morphobricks.toml` before `run_experiment` | +| `run_experiment(design=...)`, `run --design`, the `design` command, `edit_design`, `write_check_outputs`/`CheckOutputs` | Removed: the design is derived from the description | +| `read_confirmed_design`, `ConfirmedDesign`, `DesignCheckError`, `RunInputs.design()` | Removed: `Fov.repeat_group` or `RunInputs.resolve_request` | +| `RunInputs.image_type()` | Removed: `Fov.image_type` (or `FovSelection.image_type`) | +| `RunInputs.description()` and its siblings could be None | Never None | +| `preflight`/`remasked`/`masks.find_masks(with_description=...)`, `run.preflight.description_in_use` | Removed: the name refusals always apply | +| `STRICT_MANIFEST_VERSION`, `DESCRIBED_MANIFEST_VERSION`, `LEGACY_MANIFEST_VERSION`, `version_for` | Removed: `MANIFEST_VERSION` is `"4"`, the only version read | +| `source_fingerprint` | Removed: `inventory.source_signature` | +| `GroupSummary.plane_count` | Removed: `xyc_plane_count` or `channel_histogram_count` | +| `discovery.TIFF_SUFFIXES`, `discovery.SYSTEM_NAMES` | Moved to `image_names` (`is_tiff_name` and `is_system_name` still import from `discovery`) | +| `draft.migrate_legacy_design`, `LegacyMigrationRefused`, `MigrationNote`, `Draft.origin`, `Draft.legacy_missing`, `Draft.notes`, `Draft.unresolved_controls` | Removed | +| `PreprocessingNotebook(describe=True)` | `PreprocessingNotebook()`: no `describe` argument; an embedding brick must let the user describe and save before the run | +| `PreprocessingNotebook.design`; the widgets `group_confirmations`, `groups_box`, `controls_box`, `control_checks`, `design_change` | `.pairing`; the widgets are removed | +| `welcome()`: a page with a *What next?* box | A one-line `VBox` (name, version, ready) | +| `results_step()` | After `steps()`, an empty hidden box (*Compare groups* is in the run step); alone, the page it was | +| `check_candidate(..., output)` | No `output` argument | +| `PlaneRecord(...)` (normalization v0.3.4) | `PlaneRecord(..., position_id=...)`, keyword-only and required | + +Added for bricks: `Request`, `MaskRequest`, `Capabilities`, `ResolvedRequest`, `FovSelection`, +`MaskSelection`, `PlaneSelection`, `Refusal`, `RepeatCoverage`, `Provenance`, +`RESOLVER_VERSION`, `VolumeNotSupportedError`; `LabLinks`, `LabLinksError`, `read_lab_links`, +`RunMode(lab_links=...)`; `make_described_run(..., mask_images=...)` and +`testing.described_images`; `progress=` callbacks of `discover_experiment`, `masks.find_masks` +and `run.preflight.remasked`. The pinned surface is `tests/test_brick_api_surface.py` (contract +§9.1). The details are in the sections below. + +### Release v0.5.1: the shipped notebook + +- `RELEASE_TAG` `v0.5.1`, `current_version` `0.5.1`, and `RELEASE_REF` the v0.5.1 preparation + commit; cell 3 calls `PreprocessingNotebook()` (`test_notebook_file.RELEASE_CALLS` records + the pair). +- Each code cell is a Colab form with its code hidden (`"cellView": "form"`, AGENTS.md §7). +- A cell that cannot go on (cell 3 before cell 1, a refused checkout, a failed install or Drive + mount, Python 3.10) stops with its message on one line and no traceback: it raises the cell's + own `CellStop`, a `RuntimeError` whose `_render_traceback_` is the message (shown so by + IPython, so in Colab and JupyterLab; tested in a real kernel). +- **Cell 3 on Colab without Drive opens the app** after one line: "Google Drive is not mounted: + only files on this Colab runtime are visible, and they are deleted when the runtime stops." + (It stopped and named cell 2 before.) +- Cell 1 checks the Python version before `import tomllib`, so Python 3.10 gets the "Python + 3.11 or newer" line. +- The introduction describes the seven-step app in short. ### The legacy `experiment_design.csv` migration is removed (inventory PP-03) diff --git a/README.md b/README.md index e74ac92e..876b6b49 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ from there. ## Status -Pre-alpha; the current release is v0.5.0 (what is planned next is in +Pre-alpha; the current release is v0.5.1 (what is planned next is in [docs/roadmap.md](docs/roadmap.md)). Implemented and tested: - Folder discovery and validation, TIFF and OME-Zarr header inspection without reading pixels @@ -57,7 +57,7 @@ v0.5.1 is refused until it is run again; nothing is converted. See the [CHANGELO ## Installation ```bash -pip install "git+https://github.com/CellMigrationLab/morphobricks-preprocessing.git@v0.5.0" +pip install "git+https://github.com/CellMigrationLab/morphobricks-preprocessing.git@v0.5.1" ``` For development, from a clone: @@ -67,7 +67,7 @@ pip install -e ".[dev]" ``` The optional `notebook` extra installs the widget dependencies for the notebook -(`pip install "morphobricks-preprocessing[notebook] @ git+https://github.com/CellMigrationLab/morphobricks-preprocessing.git@v0.5.0"`). +(`pip install "morphobricks-preprocessing[notebook] @ git+https://github.com/CellMigrationLab/morphobricks-preprocessing.git@v0.5.1"`). The notebook's first code cell (**1. Install**) can set this up automatically: it uses a compatible nearby checkout or installed package, and installs missing requirements into the active kernel. A standalone download installs from GitHub if needed (internet access and Git required). @@ -85,9 +85,10 @@ LabConstrictor's code cells, each under a heading that explains it, as in the lab's ZeroCostDL4Mic and NucleiSky notebooks: **1. Install MorphoBricks preprocessing** (prepares the environment and reports versions), **2. Mount Google Drive** (Colab only; elsewhere a one-line note) and - **3. Run preprocessing** (explicitly displays the complete preprocessing UI). Cell 3 run out - of order says which cell to run first and never runs it for the user. Users never edit - code, so LabConstrictor may hide it. + **3. Run preprocessing** (explicitly displays the complete preprocessing UI). Cell 3 run + before cell 1 says in one line to run cell 1 and never runs it for the user; a cell that + cannot go on stops with one line, without a traceback. Users never edit code: each code + cell is a Colab form with its code hidden, and LabConstrictor may hide it. 2. Colab-only setup stays inside `if "google.colab" in sys.modules:` and mounts Google Drive. The UI uses only core ipywidgets supported by both ipywidgets 7.7 and 8; it does not force an ipywidgets upgrade/restart or enable Colab's custom-widget manager. Locally the same notebook @@ -223,7 +224,7 @@ ruff format . pytest ``` -Coding conventions are described in [docs/development.md](docs/development.md). Planned work after v0.5.0 is in [docs/roadmap.md](docs/roadmap.md). +Coding conventions are described in [docs/development.md](docs/development.md). Planned work after v0.5.1 is in [docs/roadmap.md](docs/roadmap.md). ## License diff --git a/docs/colab-release-check.md b/docs/colab-release-check.md index c1b5a703..eabb28db 100644 --- a/docs/colab-release-check.md +++ b/docs/colab-release-check.md @@ -1,124 +1,84 @@ -# Colab and JupyterLab release smoke check - -Automated tests execute the notebook in a local kernel and fake `google.colab`, but they -cannot prove that a browser frontend renders or reacts to widgets correctly. Any release -candidate that changes the notebook, bootstrap, ipywidgets support or notebook UI must be -smoke-tested in both a fresh Google Colab runtime and JupyterLab/LabConstrictor before tagging. - -This is a release gate, not an optional final-major-release check. A frontend failure is a -release blocker even when all headless notebook tests pass. The list collects the hand-check lists of PR #118 -(PP-8d, Colab readiness), PR #123 (PP-10d, Describe the masks) and PR #127 (PP-11b, Make -named-axis copies). Items were updated for later changes: PP-10a replaced the stale-lock code -cell with a button, and PP-12 replaced the `.run-*` backup folders with `.publish` and -`.staging`. - -**The notebook's pin.** The shipped notebook installs the package at its `RELEASE_REF` (set in -cell 1), not main's code, so its cell 3 calls the API of the code at that commit. Between releases main's code -moves on while the cell keeps the released call: after v0.5.1's PP-C, main has no `describe` -argument, while `RELEASE_REF` is still v0.5.0's `fd7afd3`, whose cell calls -`PreprocessingNotebook(describe=True)`. The release PR moves `RELEASE_REF` and cell 3's call -in the same commit and records the new pair in `RELEASE_CALLS` of `tests/test_notebook_file.py`, -which fails for a `RELEASE_REF` without one. - -- [ ] `RELEASE_REF` is the release commit, and cell 3's `PreprocessingNotebook(...)` call is - that commit's API (v0.5.1: `PreprocessingNotebook()`). - -Use a dataset on Google Drive with a masks folder, and a second small experiment of -Y/X-swapped OME-TIFFs for section F. Note the tag, the date and any failure next to each item. -Report a failure as an issue before the final tag. - -## A. Fresh runtime and app rendering - -The notebook has three code cells, run in order: **1. Install MorphoBricks preprocessing**, -**2. Mount Google Drive** and **3. Run preprocessing**. - -1. Open the notebook on a fresh Google Colab runtime and run cell 3 first. - - [ ] It stops at once with "run cell 1 (Install MorphoBricks preprocessing) first", not an - `ImportError`, and installs nothing. -2. Run cell 1 (install). - - [ ] Setup reports the expected notebook, preprocessing, normalization and ipywidgets - versions and the pinned source revision, then names cells 2 and 3. - - [ ] No ipywidgets upgrade or runtime restart is requested, and Drive is not mounted. - - [ ] Run cell 1 again: nothing is installed again and the same versions are reported. -3. Run cell 3 before cell 2. - - [ ] It stops with "Google Drive is not connected … run cell 2 (Mount Google Drive)". -4. Run cell 2 (Drive). - - [ ] Drive authorisation appears, the mount succeeds and the cell says Drive is connected. -5. Run cell 3 (the app). - - [ ] The complete preprocessing app renders inline; Colab does not open the Code snippets - pane instead of the widget. - - [ ] Buttons, dropdowns, accordions, preview image, sliders, progress bars and matplotlib - plots render and react. -6. Repeat cells 1 to 3 in JupyterLab/LabConstrictor. - - [ ] Cell 2 prints its one-line "Not Google Colab" note and does nothing else. - - [ ] The same app renders and uses local paths without executing Colab-only setup. - -## B. Step 1 (PR #118) - -3. - [ ] **Browse…** shows the "could not be opened … Copy path" message. - - [ ] A pasted **/content/drive/MyDrive/…** path checks fine. - - [ ] The default output folder lands on Drive next to the experiment. - -## C. Step 3 on the full dataset (PR #118) - -4. - [ ] The run finishes. - - [ ] On Drive, the output folder has `plane_histograms.csv`, `run_manifest.json`, - `experiment_design.csv` and the report (`discovery_report.txt` and `.json`). - - [ ] There are no "(1)" duplicate files, and no leftover `.publish`, `.staging` or - `...tmp` files. - - [ ] There is no lock file (`.morphobricks-preprocessing.lock`) after the run. -5. - [ ] Run Step 3 again in the same session. Everything is unchanged and nothing is read. -6. - [ ] Disconnect the runtime (Runtime › Disconnect and delete runtime) and open a new one. - Run Steps 0–3 again on the same Drive folders. The rows are reused (unchanged). This - shows the Drive times are stable across mounts. -7. Optional: stop a run part-way (interrupt, or disconnect the runtime). - - [ ] The next check names the lock holder (`output_locked`), and **Remove the lock of a - stopped run** clears it. - - [ ] The next run resumes: the report lists the images read before the stop as - `resumed`. -8. - [ ] Optional: an experiment with OME-Zarr images on Drive checks and runs (the `zarr` - extra is installed). - -## D. Describe the masks (PR #123) - -9. - [ ] In Steps 1–3 with an experiment and a masks folder on Drive, before **Compute - histograms**, the card shows a greyed button: "Available when the output folder holds a - run that recorded masks". -10. - [ ] After the run, the help line says "Describes the run in … written …, of …: N mask - types from …", and the button is enabled. -11. - [ ] Click it: a spinner, a progress bar per image, then a green bar. The tables are - labelled as suggestions, and `mask_relationships.json` appears in the output folder. -12. - [ ] Run Step 3 again: the card clears. -13. - [ ] **Reopen** the notebook (Runtime › Restart session) and choose the same folders in - Step 1 without running. The button is enabled, names the earlier run, and works. -14. - [ ] Move the masks folder on Drive and click: a red card says the masks folder does not - exist, and nothing is written. -15. - [ ] The page scrolls as a whole: no box inside it has its own scroll bar, and there is no - horizontal page scrollbar. - -## E. Make named-axis copies (PR #127) - -16. - [ ] Run Step 0, then Step 1 on an experiment of Y/X-swapped OME-TIFFs on Drive, and - press **Check folder**. -17. - [ ] In Step 2, swap the Y and X roles. The red axis check appears, with the **Make - named-axis copies (optional)** card under it. Its button is disabled. -18. - [ ] Look at the example (drawn by itself) and press **Yes, the example looks right**, then **Yes, write the - copies with these roles**. -19. - [ ] Press **Browse…**: the amber card names **Copy path**. Paste a Drive folder inside - the experiment and check that it is refused. Then paste one next to the experiment. -20. - [ ] Check the plan lists and the space, then press **Make named-axis copies**. The bar - moves, the green card shows the record path, and the pixel-size note lists the copies. -21. - [ ] Press **Check the copies in step 1**. Step 1 shows the copy folder, the same output - folder and a clean check. The roles are suggested from the copies. -22. - [ ] Confirm the labels, preview, roles and image type, then run Step 3. The report says - "named-axis copies of …". -23. - [ ] Optional: disconnect the runtime during a conversion, reconnect, and convert into - the same folder. The lock card appears. Press **Remove the lock of a stopped run**, then - convert again. +# Colab and JupyterLab release check + +The tests run the notebook in a real kernel, with `google.colab` faked, but they cannot show +that a browser draws the widgets and reacts to clicks. So before a release is tagged, the owner +opens the release PR's notebook (its Colab link and its raw download, both pinned to the +`RELEASE_REF` commit) and ticks this list once in a fresh Google Colab runtime and once in +JupyterLab or LabConstrictor. A failure here blocks the tag, even when every test passes. + +Use an experiment on Google Drive (Colab) or on disk (JupyterLab) with two conditions, at least +two biological repeats and a masks folder. Write the date, the commit and any failure (as an +issue number) next to each item. + +**Before opening it.** `RELEASE_REF` in cell 1 is the release preparation commit, and cell 3's +`PreprocessingNotebook(...)` call is that commit's API (v0.5.1: `PreprocessingNotebook()`). The +release PR moves both together and records the pair in `RELEASE_CALLS` of +`tests/test_notebook_file.py`, which fails for a `RELEASE_REF` without one. + +## A. The three cells (Colab) + +1. - [ ] The code of the three cells is hidden: each shows its title (a Colab form). +2. Fresh runtime, run **cell 3** first. + - [ ] One line: "… run cell 1 (Install MorphoBricks preprocessing) first …", no traceback; + nothing is installed. +3. Run **cell 1**. + - [ ] It prints **Ready** with the preprocessing (0.5.1), normalization (0.4.0) and + ipywidgets versions and the pinned source, then names cells 2 and 3. No restart or + ipywidgets upgrade is asked for; Drive is not mounted. + - [ ] Run it again: nothing is installed again, the same versions. +4. Run **cell 3** before cell 2. + - [ ] One line, "Google Drive is not mounted: only files on this Colab runtime are visible, + and they are deleted when the runtime stops.", then the app opens. +5. Run **cell 2**, allow access. + - [ ] "Google Drive is connected: your files are in /content/drive/MyDrive." +6. Run **cell 3** again. + - [ ] No Drive line; the app renders inline (not the Code snippets pane): the one-line + welcome, the strip of seven steps and the first step. + +## B. The seven steps (Colab, the experiment on Drive) + +7. - [ ] **Images:** paste the Drive paths (experiment, masks, output); **Browse…** says it + cannot open here and offers **Copy path**. **Scan the folders** shows the folders and an + example picture with a progress line; **Looks right: read all image headers** shows its + progress (done of total, time left) and the result. +8. - [ ] **Axes:** one row per dimension with its role, the example redrawn when a role changes, + the pixel size in one line, the image type in one line (**Use for all folders**). The roles + are confirmed only by the click. +9. - [ ] **Channels:** name the stains, then tag each folder's channels. +10. - [ ] **Masks:** **Check the masks** runs with progress; each mask type can be described. + **Describe the masks** is greyed out, with its reason, until a run exists. +11. - [ ] **Design:** the grid of conditions by biological repeats; **Looks right: confirm the + design**. +12. - [ ] **Check and save:** the experiment in words; **Yes, save the experiment description** + writes `morphobricks.toml` in the output folder. +13. - [ ] **Run:** **Compute histograms** finishes with progress; the intensity checks, + **Compare groups** and the report are collapsed panels and open. The output folder holds + `plane_histograms.csv`, `run_manifest.json`, `experiment_design.csv` and the report, with + no "(1)" copies, no `.publish`, `.staging` or `.tmp` leftovers and no lock file. +14. - [ ] Back in **Masks**, **Describe the masks** is enabled and writes + `mask_relationships.json`. +15. - [ ] Throughout: one column, the page scrolls as a whole (no box with its own scroll bar, + no horizontal page scroll bar); every greyed-out button says beside it what it waits for; + no flood of debug messages under the cell. + +## C. Again later (Colab) + +16. - [ ] Disconnect and delete the runtime, open a new one, run cells 1-3 on the same Drive + folders: the steps open answered, and **Compute histograms** reuses every row. +17. - [ ] Optional: stop a run part-way. The next check names the lock; **Remove the lock of a + stopped run** clears it, and the next run resumes. + +## D. JupyterLab or LabConstrictor + +18. - [ ] Download the raw notebook; run cell 3 first (the one line naming cell 1), then cells + 1, 2 and 3. Cell 2 prints only "Not Google Colab: there is no Google Drive to mount here, + so nothing was done."; cell 3 prints no Drive line. +19. - [ ] Repeat B (7-15) with local paths; **Browse…** opens a folder chooser on a local + kernel with a display (otherwise it says to paste the path). +20. - [ ] On a Python 3.10 kernel, if one is at hand: cell 1 says in one line to select Python + 3.11 or newer. ## After the check -- Record the result (tag, date, items passed, failures with their issue numbers) in the - release PR of v0.4.0. -- Then update this file's first paragraph, the README's "Notebook" section and plan §10 - (release checklist) to say which release was checked on Colab. +Write the result (date, commit, items passed, failures with their issue numbers) in the release +PR. Then tag. diff --git a/docs/user_guide.md b/docs/user_guide.md index b2ee6882..bca5bafd 100644 --- a/docs/user_guide.md +++ b/docs/user_guide.md @@ -21,8 +21,10 @@ open it and run its three code cells in order: 2. **Mount Google Drive** connects Google Drive on Google Colab. In JupyterLab or LabConstrictor it prints a one-line note and does nothing. 3. **Run preprocessing** displays the complete preprocessing widget app. Run before cell 1, - it stops and says to run cell 1; on Colab without Drive connected, it says to run cell 2. - It never runs an earlier cell for you. + it stops with one line saying to run cell 1. On Colab without Drive mounted it opens the + app after one line: "Google Drive is not mounted: only files on this Colab runtime are + visible, and they are deleted when the runtime stops." It never runs an earlier cell for + you. Cell 1 chooses one source: diff --git a/notebooks/MorphoBricks_Preprocessing.ipynb b/notebooks/MorphoBricks_Preprocessing.ipynb index eedc65ba..717b018c 100644 --- a/notebooks/MorphoBricks_Preprocessing.ipynb +++ b/notebooks/MorphoBricks_Preprocessing.ipynb @@ -6,30 +6,25 @@ "source": [ "# 🔬 MorphoBricks preprocessing\n", "\n", - "Prepare microscopy TIFF or OME-Zarr experiments for the other MorphoBricks bricks. The notebook never changes the source images; results are written to a separate output folder.\n", + "Prepares a microscopy experiment of TIFF or OME-Zarr images for the other MorphoBricks bricks. You describe the experiment once; the notebook then computes one intensity histogram per image plane. Your images are never changed: everything is written to a separate output folder.\n", "\n", "### What it does\n", "\n", - "- **Checks the experiment folder and metadata** and reports problems before pixels are processed.\n", - "- **Confirms the experimental design**: folder labels, biological-repeat pairing and controls.\n", - "- **Confirms image interpretation**: axis roles and fluorescence/RGB image type. Scientific metadata are never silently guessed.\n", - "- **Computes per-plane intensity histograms** used by downstream MorphoBricks.\n", - "- **Runs intensity QC** and, on request, compares groups after preprocessing.\n", + "The app has seven steps: **Images → Axes → Channels → Masks → Design → Check and save → Run**. It scans your folders, shows an example picture, asks what the axes, channels, masks and conditions are, saves your answers as the experiment description and runs. Nothing is guessed: what the app proposes is recorded only when you click.\n", "\n", "### Before you start\n", "\n", - "- Put the experiment in one folder containing TIFF (.tif/.tiff) or OME-Zarr images.\n", - "- Optional masks can be supplied from a separate masks folder.\n", - "- **Google Colab:** keep the experiment and output on Google Drive, under `/content/drive/MyDrive/`.\n", + "- One experiment folder of TIFF (.tif/.tiff) or OME-Zarr images, with folders for the conditions and biological repeats. Masks, if any, in a separate folder.\n", + "- **Google Colab:** keep the experiment and the output folder on Google Drive, under **/content/drive/MyDrive/**.\n", "- **JupyterLab / LabConstrictor:** use normal local paths.\n", "\n", "### How to run\n", "\n", - "Run the three code cells below in order, from top to bottom (click a cell and press **Shift + Enter**, or its ▶ button):\n", + "Run the three code cells in order (click a cell and press **Shift + Enter**, or its ▶ button):\n", "\n", "1. **Install** prepares the environment.\n", "2. **Mount Google Drive** connects your Drive on Google Colab; elsewhere it does nothing.\n", - "3. **Run preprocessing** opens the complete preprocessing app.\n", + "3. **Run preprocessing** opens the app.\n", "\n", "After a restart of the runtime or kernel, run the three cells again: cell 1 then installs nothing that is already there." ] @@ -51,6 +46,7 @@ "cell_type": "code", "execution_count": null, "metadata": { + "cellView": "form", "collapsed": false, "jupyter": { "source_hidden": true @@ -65,18 +61,30 @@ "import platform\n", "import subprocess\n", "import sys\n", - "import tomllib\n", + "\n", + "\n", + "class CellStop(RuntimeError):\n", + " \"\"\"Stop this cell with its message on one line, without a traceback (Colab and Jupyter).\"\"\"\n", + "\n", + " def _render_traceback_(self) -> list:\n", + " return [str(self)]\n", + "\n", + "\n", + "# Checked first, before anything that needs Python 3.11 (tomllib), so 3.10 gets this one line.\n", + "if sys.version_info < (3, 11):\n", + " raise CellStop(\"Select a Python 3.11 or newer kernel, then run this cell again.\")\n", + "\n", "from importlib import import_module, invalidate_caches\n", "from importlib.metadata import PackageNotFoundError, version\n", "from importlib.util import find_spec\n", "from pathlib import Path\n", "\n", "# LabConstrictor version identifiers.\n", - "current_version = \"0.5.0\"\n", + "current_version = \"0.5.1\"\n", "notebook_name = \"MorphoBricks_Preprocessing\"\n", "\n", "# A downloaded/Colab notebook installs this immutable source revision.\n", - "RELEASE_TAG = \"v0.5.0\"\n", + "RELEASE_TAG = \"v0.5.1\"\n", "RELEASE_REF = \"fd7afd339bcaab7579c806a28a65bb7ecebe940c\"\n", "REPOSITORY = \"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/CellMigrationLab/morphobricks-preprocessing.git\"\n", "EXTRAS = \"notebook\"\n", @@ -100,6 +108,8 @@ "\n", "def _checkout_version(checkout: Path) -> str | None:\n", " \"\"\"Return the nearby checkout package version, or None when it cannot be read.\"\"\"\n", + " import tomllib # Python 3.11 or newer, checked at the top of this cell\n", + "\n", " try:\n", " data = tomllib.loads((checkout / \"pyproject.toml\").read_text(encoding=\"utf-8\"))\n", " except (OSError, tomllib.TOMLDecodeError):\n", @@ -146,7 +156,7 @@ " \"it has uncommitted changes\" if dirty else \"its tree state could not be read\"\n", " )\n", " if differences and not DEVELOPMENT_OVERRIDE:\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " f\"The local checkout {checkout} is not the release this notebook pins: \"\n", " + \"; \".join(differences)\n", " + f\". Check out {RELEASE_REF} with a clean tree, or set DEVELOPMENT_OVERRIDE = True \"\n", @@ -207,9 +217,6 @@ "\n", " Running it again installs nothing that is already there.\n", " \"\"\"\n", - " if sys.version_info < (3, 11):\n", - " raise RuntimeError(\"Select a Python 3.11 or newer kernel, then run this cell again.\")\n", - "\n", " current = Path.cwd().resolve()\n", " nearby = next(\n", " (\n", @@ -229,7 +236,7 @@ " source = checkout / \"src\"\n", " loaded = sys.modules.get(\"morphobricks_preprocessing\")\n", " if loaded is not None and not Path(loaded.__file__).resolve().is_relative_to(source):\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " \"This kernel already loaded another MorphoBricks preprocessing copy. \"\n", " \"Restart the kernel, then run this cell again.\"\n", " )\n", @@ -275,7 +282,7 @@ " check=True,\n", " )\n", " except (subprocess.CalledProcessError, OSError) as exc:\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " \"Notebook setup could not install its requirements. Check the installation \"\n", " \"output and internet access, then run this cell again.\"\n", " ) from exc\n", @@ -283,7 +290,7 @@ "\n", " remaining = _missing_requirements()\n", " if remaining:\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " \"Installation finished, but this kernel still cannot import \"\n", " + \", \".join(remaining)\n", " + \". Restart the kernel and run this cell again.\"\n", @@ -292,7 +299,7 @@ " try:\n", " import_module(\"morphobricks_preprocessing.notebook_ui\")\n", " except ImportError as exc:\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " \"MorphoBricks preprocessing was found, but its notebook interface could not load.\"\n", " ) from exc\n", "\n", @@ -332,6 +339,7 @@ "cell_type": "code", "execution_count": null, "metadata": { + "cellView": "form", "collapsed": false, "jupyter": { "source_hidden": true @@ -345,6 +353,14 @@ "# @title 2. Mount Google Drive\n", "import sys\n", "\n", + "\n", + "class CellStop(RuntimeError):\n", + " \"\"\"Stop this cell with its message on one line, without a traceback (Colab and Jupyter).\"\"\"\n", + "\n", + " def _render_traceback_(self) -> list:\n", + " return [str(self)]\n", + "\n", + "\n", "if \"google.colab\" in sys.modules:\n", " from google.colab import drive\n", "\n", @@ -352,7 +368,7 @@ " try:\n", " drive.mount(\"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/content/drive\")\n", " except Exception as exc:\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " \"Google Drive could not be connected. Allow Drive access, then run this cell again.\"\n", " ) from exc\n", " print(\"Google Drive is connected: your files are in /content/drive/MyDrive.\", flush=True)\n", @@ -368,7 +384,7 @@ "\n", "## **3. Run preprocessing**\n", "\n", - "Opens the preprocessing app. Run cells 1 and 2 first: without them this cell stops and says which cell to run.\n", + "Opens the preprocessing app. Run cell 1 first: without it this cell stops with one line naming it. On Google Colab without Drive mounted (cell 2) the app still opens, and a line says that only files on this Colab runtime are visible.\n", "\n", "The app has seven steps, one on screen at a time; move with **Next** or click a step in the strip at the top:\n", "\n", @@ -387,6 +403,7 @@ "cell_type": "code", "execution_count": null, "metadata": { + "cellView": "form", "collapsed": false, "jupyter": { "source_hidden": true @@ -402,26 +419,35 @@ "import sys\n", "from pathlib import Path\n", "\n", + "\n", + "class CellStop(RuntimeError):\n", + " \"\"\"Stop this cell with its message on one line, without a traceback (Colab and Jupyter).\"\"\"\n", + "\n", + " def _render_traceback_(self) -> list:\n", + " return [str(self)]\n", + "\n", + "\n", "if not globals().get(\"SETUP_RECORD\"):\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " \"MorphoBricks preprocessing is not installed in this session yet: run cell 1 \"\n", " \"(Install MorphoBricks preprocessing) first, then run this cell again.\"\n", " )\n", "if \"google.colab\" in sys.modules:\n", " if not Path(\"/content/drive/MyDrive\").is_dir():\n", - " raise RuntimeError(\n", - " \"Google Drive is not connected, so the experiment and output folders on Drive \"\n", - " \"cannot be reached: run cell 2 (Mount Google Drive) first, then run this cell again.\"\n", + " print(\n", + " \"Google Drive is not mounted: only files on this Colab runtime are visible, \"\n", + " \"and they are deleted when the runtime stops.\",\n", + " flush=True,\n", " )\n", "\n", "try:\n", " interface = importlib.import_module(\"morphobricks_preprocessing.notebook_ui\")\n", "except ImportError as exc:\n", - " raise RuntimeError(\n", + " raise CellStop(\n", " \"MorphoBricks preprocessing was found, but its notebook interface could not load. \"\n", " \"Run cell 1 again.\"\n", " ) from exc\n", - "notebook = interface.PreprocessingNotebook(describe=True)\n", + "notebook = interface.PreprocessingNotebook()\n", "\n", "import ipywidgets as widgets\n", "from IPython.display import display\n", diff --git a/notebooks/notebook_latest_versions.yaml b/notebooks/notebook_latest_versions.yaml index 5c6fb1b8..9232b33f 100644 --- a/notebooks/notebook_latest_versions.yaml +++ b/notebooks/notebook_latest_versions.yaml @@ -1,3 +1,3 @@ # Latest version of each notebook, read by LabConstrictor's version check by notebook_name. # Equal to the notebook's current_version and its RELEASE_TAG without the "v" (README, "Notebook"). -"MorphoBricks_Preprocessing": "0.5.0" +"MorphoBricks_Preprocessing": "0.5.1" diff --git a/pyproject.toml b/pyproject.toml index c6b5f148..07b40847 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "morphobricks-preprocessing" -version = "0.5.0" +version = "0.5.1" description = "MorphoBricks preprocessing brick: per-plane intensity histograms and normalization methods for microscopy TIFFs." readme = "README.md" license = "MIT" diff --git a/tests/test_notebook_file.py b/tests/test_notebook_file.py index 56644f54..d8e10773 100644 --- a/tests/test_notebook_file.py +++ b/tests/test_notebook_file.py @@ -74,6 +74,8 @@ def test_notebook_is_an_introduction_and_three_explained_code_cells(): assert code["outputs"] == [] assert code["execution_count"] is None assert code["metadata"].get("jupyter", {}).get("source_hidden") is True + # A Colab form with the code hidden (AGENTS.md section 7, as ZeroCostDL4Mic). + assert code["metadata"].get("cellView") == "form" for cell in cells: if cell["cell_type"] == "markdown": assert "\\n" not in "".join(cell["source"]) @@ -199,11 +201,12 @@ def test_version_metadata_matches_labconstrictor_index(): assert index == {NOTEBOOK.stem: _setting("current_version")} -def _kernel_notebook(nbformat, *, roles=(INSTALL, DRIVE, RUN), colab_drive=None): +def _kernel_notebook(nbformat, *, roles=(INSTALL, DRIVE, RUN), colab_drive=None, override=True): """Return the notebook's code cells ``roles`` as a notebook ready for this checkout's kernel. With ``colab_drive``, a first cell fakes ``google.colab`` whose ``drive.mount`` makes a - ``MyDrive`` folder, and the cells' ``/content/drive`` is ``colab_drive``. + ``MyDrive`` folder, and the cells' ``/content/drive`` is ``colab_drive``. ``override=False`` + leaves cell 1 as shipped, so it refuses this checkout (not the ``RELEASE_REF`` commit). """ notebook = nbformat.read(NOTEBOOK, as_version=4) code = [cell for cell in notebook.cells if cell.cell_type == "code"] @@ -211,9 +214,10 @@ def _kernel_notebook(nbformat, *, roles=(INSTALL, DRIVE, RUN), colab_drive=None) # The kernel runs in this checkout, whose commit is not RELEASE_REF (#103): the one edit a # developer makes to use a checkout is the explicit override. assert install.source.count("DEVELOPMENT_OVERRIDE = False") == 1 - install.source = install.source.replace( - "DEVELOPMENT_OVERRIDE = False", "DEVELOPMENT_OVERRIDE = True" - ) + if override: + install.source = install.source.replace( + "DEVELOPMENT_OVERRIDE = False", "DEVELOPMENT_OVERRIDE = True" + ) # While RELEASE_REF is behind this checkout, the cell calls the released API: the kernel, # running this checkout, is given this checkout's call (the release PR makes them one). assert run.source.count(release_call()) == 1 @@ -306,6 +310,38 @@ def test_the_run_cell_alone_in_a_fresh_kernel_names_cell_1(): _execute(notebook, allow_errors=True) (error,) = _errors(run) - assert error.ename == "RuntimeError" # not an ImportError of the package + assert error.ename == "CellStop" # the cell's own stop, not an ImportError of the package assert "run cell 1 (Install MorphoBricks preprocessing) first" in error.evalue + # One line, without a traceback (owner, v0.5.1): what Colab and JupyterLab show. + assert error.traceback == [error.evalue] assert not _shows_a_widget(run) + + +def test_a_refusal_of_cell_1_in_a_kernel_is_one_line_without_a_traceback(): + nbformat = _kernel_modules() + # As shipped, cell 1 refuses this checkout: its commit is not RELEASE_REF (#103). + notebook, (install,) = _kernel_notebook(nbformat, roles=(INSTALL,), override=False) + _execute(notebook, allow_errors=True) + + (error,) = _errors(install) + assert error.ename == "CellStop" + assert "is not the release this notebook pins" in error.evalue + assert error.traceback == [error.evalue] + assert "\n" not in error.evalue + + +def test_on_a_faked_colab_without_drive_cell_3_opens_the_app_after_one_line(tmp_path): + nbformat = _kernel_modules() + drive_root = tmp_path / "content/drive" + notebook, (install, run) = _kernel_notebook( + nbformat, roles=(INSTALL, RUN), colab_drive=drive_root + ) + _execute(notebook) + + assert not [*_errors(install), *_errors(run)] + assert _text(run).splitlines() == [ + "Google Drive is not mounted: only files on this Colab runtime are visible, " + "and they are deleted when the runtime stops." + ] + assert not drive_root.exists() # nothing mounted for the user + assert _shows_a_widget(run) diff --git a/tests/test_notebook_setup.py b/tests/test_notebook_setup.py index 27b30716..25cfb575 100644 --- a/tests/test_notebook_setup.py +++ b/tests/test_notebook_setup.py @@ -105,8 +105,8 @@ def setup_environment(tmp_path, monkeypatch): drive=tmp_path / "content/drive", namespace={}, versions={ - "morphobricks-preprocessing": "0.5.0", - "morphobricks-normalization": "0.3.4", + "morphobricks-preprocessing": "0.5.1", + "morphobricks-normalization": "0.4.0", "ipywidgets": "7.7.1", }, ) @@ -131,7 +131,7 @@ def install(command, *, check): assert check is True state.installs.append(command) state.available.update({*REQUIRED, ZARR}) - state.versions["morphobricks-preprocessing"] = "0.5.0" + state.versions["morphobricks-preprocessing"] = "0.5.1" monkeypatch.setattr(subprocess, "run", install) @@ -143,7 +143,7 @@ def import_interface(name): import ipywidgets as widgets class FakeNotebook: - def __init__(self, **options): # describe=True, the shipped notebook's run + def __init__(self, **options): # no options: every run is described (v0.5.1) self.options = options state.apps.append(options) @@ -163,7 +163,7 @@ def checkout_at(path): package.mkdir(parents=True) (package / "__init__.py").write_text("", encoding="utf-8") (path / "pyproject.toml").write_text( - '[project]\nname = "morphobricks-preprocessing"\nversion = "0.5.0"\n', + '[project]\nname = "morphobricks-preprocessing"\nversion = "0.5.1"\n', encoding="utf-8", ) return path @@ -179,7 +179,7 @@ def test_setup_reuses_the_matching_installed_release(setup_environment, capsys): "morphobricks_preprocessing.notebook_ui", # cell 3 shows it ] out = capsys.readouterr().out - assert "Ready: preprocessing 0.5.0" in out + assert "Ready: preprocessing 0.5.1" in out assert "ipywidgets 7.7.1" in out @@ -194,7 +194,7 @@ def test_setup_installs_missing_package_from_the_exact_commit(setup_environment, assert target.startswith("morphobricks-preprocessing[notebook] @ git+") assert target.endswith("@" + namespace["RELEASE_REF"]) assert namespace["RELEASE_REF"] == "fd7afd339bcaab7579c806a28a65bb7ecebe940c" - assert "pinned v0.5.0 candidate" in capsys.readouterr().out + assert "pinned v0.5.1 candidate" in capsys.readouterr().out def test_setup_replaces_a_mismatched_installed_release(setup_environment): @@ -285,7 +285,7 @@ def test_an_explicit_development_override_uses_the_checkout_and_says_so( assert sys.path[0] == str(checkout / "src") assert f"commit {OTHER_COMMIT}, uncommitted changes" in out assert "DEVELOPMENT OVERRIDE is on" in out - assert "DEVELOPMENT OVERRIDE on" in out.split("Notebook 0.5.0")[1] + assert "DEVELOPMENT OVERRIDE on" in out.split("Notebook 0.5.1")[1] record = namespace["SETUP_RECORD"] assert record["checkout_commit"] == OTHER_COMMIT assert record["checkout_has_uncommitted_changes"] is True @@ -425,7 +425,7 @@ def test_cell_1_installs_without_mounting_drive_or_showing_the_app( assert state.apps == [] assert namespace["SETUP_RECORD"]["release_ref"] == RELEASE_REF out = capsys.readouterr().out - assert "Ready: preprocessing 0.5.0; normalization 0.3.4; ipywidgets 7.7.1." in out + assert "Ready: preprocessing 0.5.1; normalization 0.4.0; ipywidgets 7.7.1." in out assert "Next: run cell 2 (Mount Google Drive), then cell 3 (Run preprocessing)." in out @@ -441,8 +441,8 @@ def test_running_cell_1_again_installs_nothing_more(setup_environment, capsys): assert len(state.installs) == 1 assert "Installing notebook requirements" in first assert "Installing notebook requirements" not in again - assert "Ready: preprocessing 0.5.0" in again - assert state.namespace["SETUP_RECORD"]["preprocessing_version"] == "0.5.0" + assert "Ready: preprocessing 0.5.1" in again + assert state.namespace["SETUP_RECORD"]["preprocessing_version"] == "0.5.1" assert record["release_ref"] == state.namespace["SETUP_RECORD"]["release_ref"] @@ -486,18 +486,80 @@ def test_cell_3_after_a_failed_cell_1_still_names_cell_1(setup_environment, monk assert state.apps == [] -def test_cell_3_on_colab_without_drive_names_cell_2(setup_environment, monkeypatch): +NO_DRIVE = ( + "Google Drive is not mounted: only files on this Colab runtime are visible, " + "and they are deleted when the runtime stops." +) +"""Cell 3's one line on Colab without Drive (owner, v0.5.1: the app opens anyway).""" + + +def test_cell_3_on_colab_without_drive_opens_the_app_after_one_line( + setup_environment, monkeypatch, capsys +): calls = colab(monkeypatch) state = setup_environment state.start(INSTALL) - imported = list(state.imports) - with pytest.raises(RuntimeError, match=r"Google Drive is not connected.*run cell 2"): - state.start(RUN, fresh=False) + capsys.readouterr() + state.start(RUN, fresh=False) + printed = capsys.readouterr().out.splitlines() + assert printed[0] == NO_DRIVE # one line, then the app (``display`` prints it here) + assert [line for line in printed if "Drive" in line] == [NO_DRIVE] assert calls == [] # never mounted for the user - assert state.imports == imported - assert state.apps == [] - state.start(DRIVE, RUN, fresh=False) assert state.apps == [APP_OPTIONS] + # With Drive mounted the line is not printed. + state.start(DRIVE, fresh=False) + capsys.readouterr() + state.start(RUN, fresh=False) + assert NO_DRIVE not in capsys.readouterr().out + assert state.apps == [APP_OPTIONS, APP_OPTIONS] + + +def test_cell_3_outside_colab_never_prints_the_drive_line(setup_environment, capsys): + setup_environment.start(INSTALL, RUN) + assert NO_DRIVE not in capsys.readouterr().out + + +# A cell that cannot go on stops with one line and no traceback (owner, v0.5.1): it raises the +# cell's own CellStop, a RuntimeError whose ``_render_traceback_`` is its message alone (IPython, +# and so Colab and JupyterLab, shows that instead of the traceback; tests/test_notebook_file.py +# runs it in a real kernel). + + +def _cell_tree(number): + return ast.parse(cell_source(number)) + + +@pytest.mark.parametrize("number", [INSTALL, DRIVE, RUN]) +def test_every_stop_of_a_cell_is_its_one_line_cell_stop(number): + tree = _cell_tree(number) + (stop,) = [node for node in tree.body if isinstance(node, ast.ClassDef)] + assert stop.name == "CellStop" + assert [ast.unparse(base) for base in stop.bases] == ["RuntimeError"] + namespace = {} + exec(compile(ast.Module(body=[stop], type_ignores=[]), "cell", "exec"), namespace) + error = namespace["CellStop"]("one clear message") + assert error._render_traceback_() == ["one clear message"] + raised = [node.exc for node in ast.walk(tree) if isinstance(node, ast.Raise)] + assert raised + for exc in raised: + assert isinstance(exc, ast.Call) and ast.unparse(exc.func) == "CellStop", ast.unparse(exc) + + +def test_cell_1_checks_the_python_version_before_importing_tomllib(): + # tomllib is Python 3.11: on 3.10 an import before the check would be a ModuleNotFoundError + # in place of the "Python 3.11 or newer" line. + tree = _cell_tree(INSTALL) + check = next( + position + for position, node in enumerate(tree.body) + if isinstance(node, ast.If) and "sys.version_info" in ast.unparse(node.test) + ) + for position, node in enumerate(tree.body[:check]): + assert isinstance(node, (ast.Import, ast.ImportFrom, ast.ClassDef, ast.Expr)), position + if isinstance(node, (ast.Import, ast.ImportFrom)): + assert {alias.name for alias in node.names} <= {"platform", "subprocess", "sys"} + for node in ast.walk(ast.Module(body=tree.body[:check], type_ignores=[])): + assert not (isinstance(node, ast.Import) and "tomllib" in ast.unparse(node)) def test_cell_3_outside_colab_needs_no_drive(setup_environment): From a4154ade65c213ce3222b7b1ee1156944cc1ae41 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 22:55:01 +0000 Subject: [PATCH 2/6] Point the notebook at the v0.5.1 preparation commit RELEASE_REF = d023ff91289a2d2204e1ae56f7dee114d1a665e4 (and the tests asserting it), with its PreprocessingNotebook() call recorded in RELEASE_CALLS, the same two-commit pattern as v0.5.0 (3b616a0, c74d717). A commit cannot hold its own SHA, so the notebook installs the preparation commit, whose package code is the release's. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- CHANGELOG.md | 2 +- notebooks/MorphoBricks_Preprocessing.ipynb | 2 +- tests/test_notebook_file.py | 2 ++ tests/test_notebook_setup.py | 2 +- 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b2653cd..8bc5054a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -91,7 +91,7 @@ and `run.preflight.remasked`. The pinned surface is `tests/test_brick_api_surfac ### Release v0.5.1: the shipped notebook - `RELEASE_TAG` `v0.5.1`, `current_version` `0.5.1`, and `RELEASE_REF` the v0.5.1 preparation - commit; cell 3 calls `PreprocessingNotebook()` (`test_notebook_file.RELEASE_CALLS` records + commit `d023ff9`; cell 3 calls `PreprocessingNotebook()` (`test_notebook_file.RELEASE_CALLS` records the pair). - Each code cell is a Colab form with its code hidden (`"cellView": "form"`, AGENTS.md §7). - A cell that cannot go on (cell 3 before cell 1, a refused checkout, a failed install or Drive diff --git a/notebooks/MorphoBricks_Preprocessing.ipynb b/notebooks/MorphoBricks_Preprocessing.ipynb index 717b018c..d540e5f7 100644 --- a/notebooks/MorphoBricks_Preprocessing.ipynb +++ b/notebooks/MorphoBricks_Preprocessing.ipynb @@ -85,7 +85,7 @@ "\n", "# A downloaded/Colab notebook installs this immutable source revision.\n", "RELEASE_TAG = \"v0.5.1\"\n", - "RELEASE_REF = \"fd7afd339bcaab7579c806a28a65bb7ecebe940c\"\n", + "RELEASE_REF = \"d023ff91289a2d2204e1ae56f7dee114d1a665e4\"\n", "REPOSITORY = \"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/CellMigrationLab/morphobricks-preprocessing.git\"\n", "EXTRAS = \"notebook\"\n", "\n", diff --git a/tests/test_notebook_file.py b/tests/test_notebook_file.py index d8e10773..1526dd1f 100644 --- a/tests/test_notebook_file.py +++ b/tests/test_notebook_file.py @@ -120,6 +120,8 @@ def test_notebook_uses_stable_page_api_not_a_new_app_method(): RELEASE_CALLS = { # v0.5.0: the described run is ``describe=True``; the default is the undescribed v0.4 run. "fd7afd339bcaab7579c806a28a65bb7ecebe940c": "PreprocessingNotebook(describe=True)", + # v0.5.1: every run is described; there is no describe argument. + "d023ff91289a2d2204e1ae56f7dee114d1a665e4": "PreprocessingNotebook()", } CHECKOUT_CALL = "PreprocessingNotebook()" """The call of this checkout's code (every run described, no ``describe`` argument).""" diff --git a/tests/test_notebook_setup.py b/tests/test_notebook_setup.py index 25cfb575..17791a98 100644 --- a/tests/test_notebook_setup.py +++ b/tests/test_notebook_setup.py @@ -193,7 +193,7 @@ def test_setup_installs_missing_package_from_the_exact_commit(setup_environment, target = command[5] assert target.startswith("morphobricks-preprocessing[notebook] @ git+") assert target.endswith("@" + namespace["RELEASE_REF"]) - assert namespace["RELEASE_REF"] == "fd7afd339bcaab7579c806a28a65bb7ecebe940c" + assert namespace["RELEASE_REF"] == "d023ff91289a2d2204e1ae56f7dee114d1a665e4" assert "pinned v0.5.1 candidate" in capsys.readouterr().out From 7607617512ca379d2b5c70e81f9f4209663c72a2 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 23:09:19 +0000 Subject: [PATCH 3/6] Release record v0.5.1 in docs/conformance.md (measured on a4154ad) check.py 5270 passed / 4 environment skips (listed as not run); Spec modules 1732 passed, 0 skipped, all 24 expected modules; install from RELEASE_REF d023ff9 verified. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- docs/conformance.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/docs/conformance.md b/docs/conformance.md index 0efeef76..84d5b5ad 100644 --- a/docs/conformance.md +++ b/docs/conformance.md @@ -6,6 +6,46 @@ real run; a skip is never counted as a pass. Owner rulings of 8 November 2026 (i "no skips" applies to the Spec conformance modules (`tests/test_spec_*.py`); a skip elsewhere in the suite is listed as **not run**, with its reason. +## Release record: v0.5.1 (10 November 2026) + +A G1-style record of the v0.5.1 release candidate (as for v0.5.0 below). It does not decide a +gate: the Stage 2 gate (G2) also needs the bricks' items, and deciding a gate is the owner's. + +| | | +|---|---| +| Release candidate | Measured on `a4154ad` (branch `claude/release-0.5.1`: `main` at `9a5da03`, the v0.5.1 preparation `d023ff9` (commit A), then `a4154ad` (commit B), which points the notebook's `RELEASE_REF` at `d023ff91289a2d2204e1ae56f7dee114d1a665e4`). The commit that adds this record changes only this file. `git diff d023ff9 a4154ad -- src pyproject.toml` is empty: the installed package is the one `RELEASE_REF` names | +| Spec pin | `tests/spec_fixtures/PIN.toml`: MorphoBricks-Spec `cc9820f5b84f01bbbb5c8e3f54e70ee13cb5e9eb` (fixtures tree `40fe48c`), checked by `test_spec_pin` (16 passed) | +| Environment | Linux, Python 3.13.18, `pip install -e '.[dev,notebook,zarr]'` in a fresh venv (package version 0.5.1, normalization 0.4.0, numpy 2.5.4, tifffile 2026.11.8, ipywidgets 8.1.9, zarr 3.4.2, pytest 9.2.0); run as root on a case-sensitive file system | +| Commands | `python tools/check.py` (Ruff, Ruff format, pytest over `tests/`, code rules); `python -m pytest -q -p no:cacheprovider -rsxX --junitxml=… tests/test_spec_*.py` | +| `tools/check.py` | Ruff clean, Ruff format 432 files already formatted, pytest **5270 passed, 4 skipped** (not run, below), 0 failed, code rules passed (exit status 0) | +| Spec modules (`tests/test_spec_*.py`) | **1732 passed, 0 skipped, 0 xfailed, 0 failed**, from all 24 modules of `tests/spec_gate.py` `EXPECTED_MODULES` (JUnit: no `skipped`, `failure` or `error` element in any case) | +| Install | `pip install "morphobricks-preprocessing[notebook] @ git+…@d023ff9…"` in a fresh venv reports 0.5.1 (normalization 0.4.0), its package files equal this branch's `src/morphobricks_preprocessing` (`diff -r`), and `PreprocessingNotebook()` builds `welcome()`, `steps()` and `results_step()` | +| Shipped notebook | Its three code cells, unmodified, in a real kernel (`nbclient`) in a fresh venv: cell 3 first stops with one line naming cell 1; cell 1 installs `RELEASE_REF` from GitHub and prints **Ready** (0.5.1, normalization 0.4.0); cell 2 prints its one-line note; cell 3 displays the app. With `google.colab` faked: cell 3 before cell 2 prints the one Drive line and opens the app; after cell 2 it opens without it. Cell 1 on a Python 3.10 kernel stops with the one "Python 3.11 or newer" line | + +Test cases per Spec module: catalogue 376, derived design 80, discovery 17, encoding 58, Fov +image type 14, gate 9, lab links 11, no fallback 15, pin 16, publication 20, reader 330, reader +phases 1-3 287, reader refusal paths 8, reconcile 25, records 22, records goldens 76, refval +cross-check 203, resolve request 43, rules table 14, run aliases 15, run description 9, +RunInputs semantic 8, transitions 73, vendored missing 3. New since v0.5.0: +`test_spec_resolve_request` (PP-0), `test_spec_fov_image_type` (PP-1) and +`test_spec_lab_links` (PP-2, the lab-tool-links part of the section 2 "Provenance" row, which +ADR 0001 GS1 deferred from G1 to the Stage 2 gate). The cases removed since v0.5.0 are the +undescribed and legacy ones (PP-C, SWEEP-PP, PP-03; the P30 legacy case per Spec `cc9820f`). + +**Not run (4 environment skips, outside the Spec modules):** + +| Test | Reason | +|---|---| +| `tests/test_cli_log.py:166` | Case-sensitive file system: the case-only collision cannot be built | +| `tests/test_folder_rules.py:199` | Same | +| `tests/test_folder_rules.py:230` | Same | +| `tests/test_discovery.py:665` | Run as root: file modes are ignored, so an unreadable folder cannot be made | + +These four behaviours are **not verified by this run**. + +**Still the owner's:** the Colab and JupyterLab browser check (`docs/colab-release-check.md`), +before tagging. P24, P27 and P31 stay brick outcomes for the Stage 2 gate. + ## Gate G1 record: release candidate v0.5.0 (10 November 2026) | | | From 7beac12efd7e4cb2f5d0fc4e6991944de7451696 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 23:38:01 +0000 Subject: [PATCH 4/6] Name the cause on a cell's one-line stop; keep other errors' tracebacks The one-line CellStop of cell 1 and cell 3 for an interface that cannot load, and of cell 2 for a failed Drive mount, dropped the error that caused it: the chained ImportError (a missing dependency, a broken import inside the package) or the mount error (for example "Mountpoint must not already contain files") was not shown anywhere, and the advice could be wrong. The stop now names the cause on the same line ("... could not load (ImportError: ...)", "Google Drive could not be connected (ValueError: ...)"). Tests: the interface stop of cells 1 and 3 names its ImportError (exec), the Drive stop names the mount error, and in a real kernel the interface stop is one line naming the ModuleNotFoundError while an unexpected error in cell 3 keeps its full traceback. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- AGENTS.md | 3 +- CHANGELOG.md | 5 ++- docs/plan.md | 4 ++ notebooks/MorphoBricks_Preprocessing.ipynb | 16 ++++++-- tests/test_notebook_file.py | 46 ++++++++++++++++++++++ tests/test_notebook_setup.py | 31 ++++++++++++++- 6 files changed, 98 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index d2ea8e2f..656b878d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -880,7 +880,8 @@ plainer version "not interactive enough". Descriptive, guided and forgiving is t files on the runtime are visible (owner, v0.5.1). No cell runs another. - A cell that cannot go on raises its own `CellStop` (a `RuntimeError` whose `_render_traceback_` returns the message), so Colab and JupyterLab show one line and no - traceback (owner, v0.5.1). Cell 1 checks the Python version before `import tomllib`. + traceback (owner, v0.5.1). A stop caused by another error (the interface import, the Drive + mount) names that error on the same line; an error that is not a stop keeps its traceback. Cell 1 checks the Python version before `import tomllib`. - Cell 1 is a standalone standard-library bootstrap: nearby checkout, installed package, or GitHub installation of the release `RELEASE_TAG`, with missing requirements installed using the kernel's Python (`subprocess`, never `!pip`); on Colab with the diff --git a/CHANGELOG.md b/CHANGELOG.md index 8bc5054a..ba0673e4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -97,7 +97,10 @@ and `run.preflight.remasked`. The pinned surface is `tests/test_brick_api_surfac - A cell that cannot go on (cell 3 before cell 1, a refused checkout, a failed install or Drive mount, Python 3.10) stops with its message on one line and no traceback: it raises the cell's own `CellStop`, a `RuntimeError` whose `_render_traceback_` is the message (shown so by - IPython, so in Colab and JupyterLab; tested in a real kernel). + IPython, so in Colab and JupyterLab; tested in a real kernel). A stop caused by another error + names it on that line, as in "… its notebook interface could not load (ImportError: …)" or + "Google Drive could not be connected (ValueError: …)", so the one line never hides what + failed; any other error keeps its full traceback (tested in a real kernel). - **Cell 3 on Colab without Drive opens the app** after one line: "Google Drive is not mounted: only files on this Colab runtime are visible, and they are deleted when the runtime stops." (It stopped and named cell 2 before.) diff --git a/docs/plan.md b/docs/plan.md index b8d8cb2b..87de8c6d 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -1162,6 +1162,10 @@ Raised during review; to be decided before the milestones that need them: report), 2. Mount Google Drive (Colab only; elsewhere a one-line note), 3. Run preprocessing (the app). Cell 3 run before cell 1, or on Colab without Drive, stops and names the cell to run; no cell runs another.* + *Amended (v0.5.1, owner): the code cells are Colab forms with the code hidden; a cell that + cannot go on stops with one line and no traceback (a stop caused by another error names + that error on the same line; any other error keeps its traceback); cell 3 on Colab without + Drive opens the app after one line saying that only files on the runtime are visible.* 59. How are images with axis errors saved as named-axis copies (design v0.4 §5, PP-11)? *Decided (owner, 2 November 2026; implemented in PP-11) as design v0.4 §5 says: images only, as named-axis OME-TIFF (BigTIFF when large); copies keep the file name and folder diff --git a/notebooks/MorphoBricks_Preprocessing.ipynb b/notebooks/MorphoBricks_Preprocessing.ipynb index d540e5f7..121a69a6 100644 --- a/notebooks/MorphoBricks_Preprocessing.ipynb +++ b/notebooks/MorphoBricks_Preprocessing.ipynb @@ -299,8 +299,11 @@ " try:\n", " import_module(\"morphobricks_preprocessing.notebook_ui\")\n", " except ImportError as exc:\n", + " # The cause stays on the one line: a stop must never hide what failed.\n", + " cause = \" \".join(f\"{type(exc).__name__}: {exc}\".split())\n", " raise CellStop(\n", - " \"MorphoBricks preprocessing was found, but its notebook interface could not load.\"\n", + " \"MorphoBricks preprocessing was found, but its notebook interface could not load \"\n", + " f\"({cause}).\"\n", " ) from exc\n", "\n", " package_version = version(\"morphobricks-preprocessing\")\n", @@ -368,8 +371,11 @@ " try:\n", " drive.mount(\"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/content/drive\")\n", " except Exception as exc:\n", + " # The cause stays on the one line: a stop must never hide what failed.\n", + " cause = \" \".join(f\"{type(exc).__name__}: {exc}\".split())\n", " raise CellStop(\n", - " \"Google Drive could not be connected. Allow Drive access, then run this cell again.\"\n", + " f\"Google Drive could not be connected ({cause}). Allow Drive access, then run \"\n", + " \"this cell again.\"\n", " ) from exc\n", " print(\"Google Drive is connected: your files are in /content/drive/MyDrive.\", flush=True)\n", "else:\n", @@ -443,9 +449,11 @@ "try:\n", " interface = importlib.import_module(\"morphobricks_preprocessing.notebook_ui\")\n", "except ImportError as exc:\n", + " # The cause stays on the one line: a stop must never hide what failed.\n", + " cause = \" \".join(f\"{type(exc).__name__}: {exc}\".split())\n", " raise CellStop(\n", - " \"MorphoBricks preprocessing was found, but its notebook interface could not load. \"\n", - " \"Run cell 1 again.\"\n", + " \"MorphoBricks preprocessing was found, but its notebook interface could not load \"\n", + " f\"({cause}). Run cell 1 again.\"\n", " ) from exc\n", "notebook = interface.PreprocessingNotebook()\n", "\n", diff --git a/tests/test_notebook_file.py b/tests/test_notebook_file.py index 1526dd1f..3d95d526 100644 --- a/tests/test_notebook_file.py +++ b/tests/test_notebook_file.py @@ -347,3 +347,49 @@ def test_on_a_faked_colab_without_drive_cell_3_opens_the_app_after_one_line(tmp_ ] assert not drive_root.exists() # nothing mounted for the user assert _shows_a_widget(run) + + +def _cell_after(nbformat, notebook, source): + """Put a cell running ``source`` in front of the notebook's last cell.""" + notebook.cells.insert(len(notebook.cells) - 1, nbformat.v4.new_code_cell(source)) + + +def test_an_unexpected_error_in_a_cell_keeps_its_traceback(): + # Only the cells' own stops are one line: a real error still shows where it happened. + nbformat = _kernel_modules() + notebook, (install, run) = _kernel_notebook(nbformat, roles=(INSTALL, RUN)) + _cell_after( + nbformat, + notebook, + "import morphobricks_preprocessing.notebook_ui as interface\n" + "def broken(*arguments, **options):\n" + " raise KeyError('a real defect')\n" + "interface.PreprocessingNotebook = broken\n", + ) + _execute(notebook, allow_errors=True) + + assert not _errors(install) + (error,) = _errors(run) + assert error.ename == "KeyError" + assert len(error.traceback) > 1 + assert any("broken" in line for line in error.traceback) + + +def test_the_stop_for_an_interface_that_cannot_load_names_the_import_error(): + # The one-line stop keeps the cause on its line: it never hides what failed. + nbformat = _kernel_modules() + notebook, (install, run) = _kernel_notebook(nbformat, roles=(INSTALL, RUN)) + _cell_after( + nbformat, + notebook, + "import sys\nsys.modules['morphobricks_preprocessing.notebook_ui'] = None\n", + ) + _execute(notebook, allow_errors=True) + + assert not _errors(install) + (error,) = _errors(run) + assert error.ename == "CellStop" + assert "could not load (ModuleNotFoundError: import of " in error.evalue + assert "morphobricks_preprocessing.notebook_ui" in error.evalue + assert error.evalue.endswith("Run cell 1 again.") + assert error.traceback == [error.evalue] diff --git a/tests/test_notebook_setup.py b/tests/test_notebook_setup.py index 17791a98..87f0ea21 100644 --- a/tests/test_notebook_setup.py +++ b/tests/test_notebook_setup.py @@ -380,7 +380,9 @@ def refuse(path): state = setup_environment state.start(INSTALL) imported = list(state.imports) - with pytest.raises(RuntimeError, match="Google Drive could not be connected"): + with pytest.raises( + RuntimeError, match=r"Google Drive could not be connected \(ValueError: mount failed\)\." + ): state.start(DRIVE, fresh=False) assert state.imports == imported assert state.apps == [] @@ -567,3 +569,30 @@ def test_cell_3_outside_colab_needs_no_drive(setup_environment): state.start(INSTALL, RUN) # cell 2 skipped: nothing to mount outside Colab assert state.apps == [APP_OPTIONS] assert "app" in state.namespace + + +@pytest.mark.parametrize("number", [INSTALL, RUN]) +def test_the_interface_stop_names_its_import_error(setup_environment, monkeypatch, number): + # A stop is one line, but it never hides what failed: the ImportError is on that line. + state = setup_environment + if number == RUN: + state.start(INSTALL) + imported = importlib.import_module # the fixture's double + + def broken(name): + if name == "morphobricks_preprocessing.notebook_ui": + raise ImportError("cannot import name 'gone' from 'morphobricks_preprocessing.x'") + return imported(name) + + monkeypatch.setattr(importlib, "import_module", broken) + with pytest.raises(RuntimeError) as stopped: + state.start(number, fresh=number == INSTALL) + stop = stopped.value + assert type(stop).__name__ == "CellStop" + assert ( + "its notebook interface could not load (ImportError: cannot import name 'gone' from " + "'morphobricks_preprocessing.x')." in str(stop) + ) + assert stop._render_traceback_() == [str(stop)] + assert isinstance(stop.__cause__, ImportError) + assert state.apps == [] From 9f61cb9feacc7369e21469c704b525810ba0c29b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 23:38:01 +0000 Subject: [PATCH 5/6] List the removed design editor API in the v0.5.1 migration table for bricks The bricks table named read_confirmed_design and RunInputs.design() but not the rest of the PP-02 removal: design.ExperimentDesign, DesignChanges, default_design, load_design, read_experiment_design, the modules design_edit and cli_design, run.design_file and report.design_summary/design_table (each checked: no longer importable at d023ff9). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index ba0673e4..5dc05c0c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,7 @@ v0.5.1 is refused: run `convert` again into a new folder. | `source_fingerprint` | Removed: `inventory.source_signature` | | `GroupSummary.plane_count` | Removed: `xyc_plane_count` or `channel_histogram_count` | | `discovery.TIFF_SUFFIXES`, `discovery.SYSTEM_NAMES` | Moved to `image_names` (`is_tiff_name` and `is_system_name` still import from `discovery`) | +| The design editor: `design.ExperimentDesign`, `DesignChanges`, `default_design`, `load_design`, `read_experiment_design`; the module `design_edit` (`DesignEdit`, `DesignEditError`, `apply_design_edit`, `parse_repeat_group`); `run.design_file` (`design_for_run`, `write_design`, `write_design_template`); `report.design_summary`, `design_table`; the module `cli_design` | Removed: the design rows are the read-only `DesignTable`, derived from the description | | `draft.migrate_legacy_design`, `LegacyMigrationRefused`, `MigrationNote`, `Draft.origin`, `Draft.legacy_missing`, `Draft.notes`, `Draft.unresolved_controls` | Removed | | `PreprocessingNotebook(describe=True)` | `PreprocessingNotebook()`: no `describe` argument; an embedding brick must let the user describe and save before the run | | `PreprocessingNotebook.design`; the widgets `group_confirmations`, `groups_box`, `controls_box`, `control_checks`, `design_change` | `.pairing`; the widgets are removed | From 849e09d62b95341b285c5e2f1f2d725c5d7bbc5e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 11 Nov 2026 00:04:32 +0000 Subject: [PATCH 6/6] Release record v0.5.1 re-measured on 9f61cb9 after the review fixes Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- docs/conformance.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/conformance.md b/docs/conformance.md index 84d5b5ad..5af8d373 100644 --- a/docs/conformance.md +++ b/docs/conformance.md @@ -13,14 +13,14 @@ gate: the Stage 2 gate (G2) also needs the bricks' items, and deciding a gate is | | | |---|---| -| Release candidate | Measured on `a4154ad` (branch `claude/release-0.5.1`: `main` at `9a5da03`, the v0.5.1 preparation `d023ff9` (commit A), then `a4154ad` (commit B), which points the notebook's `RELEASE_REF` at `d023ff91289a2d2204e1ae56f7dee114d1a665e4`). The commit that adds this record changes only this file. `git diff d023ff9 a4154ad -- src pyproject.toml` is empty: the installed package is the one `RELEASE_REF` names | +| Release candidate | Measured on `9f61cb9` (branch `claude/release-0.5.1`: `main` at `9a5da03`, the v0.5.1 preparation `d023ff9` (commit A), `a4154ad` (commit B), which points the notebook's `RELEASE_REF` at `d023ff91289a2d2204e1ae56f7dee114d1a665e4`, the first record `7607617`, then the review fixes `7beac12` (a cell's one-line stop names the error that caused it; notebook, tests and docs only) and `9f61cb9` (CHANGELOG)). The commit that updates this record changes only this file. `git diff d023ff9 9f61cb9 -- src pyproject.toml` is empty: the installed package is the one `RELEASE_REF` names. Earlier measurement, on `a4154ad` before the review: 5270 passed, 4 skipped; 1732 in the Spec modules, 0 skipped (reproduced by the review) | | Spec pin | `tests/spec_fixtures/PIN.toml`: MorphoBricks-Spec `cc9820f5b84f01bbbb5c8e3f54e70ee13cb5e9eb` (fixtures tree `40fe48c`), checked by `test_spec_pin` (16 passed) | | Environment | Linux, Python 3.13.18, `pip install -e '.[dev,notebook,zarr]'` in a fresh venv (package version 0.5.1, normalization 0.4.0, numpy 2.5.4, tifffile 2026.11.8, ipywidgets 8.1.9, zarr 3.4.2, pytest 9.2.0); run as root on a case-sensitive file system | | Commands | `python tools/check.py` (Ruff, Ruff format, pytest over `tests/`, code rules); `python -m pytest -q -p no:cacheprovider -rsxX --junitxml=… tests/test_spec_*.py` | -| `tools/check.py` | Ruff clean, Ruff format 432 files already formatted, pytest **5270 passed, 4 skipped** (not run, below), 0 failed, code rules passed (exit status 0) | +| `tools/check.py` | Ruff clean, Ruff format 432 files already formatted, pytest **5274 passed, 4 skipped** (not run, below), 0 failed, code rules passed (exit status 0) | | Spec modules (`tests/test_spec_*.py`) | **1732 passed, 0 skipped, 0 xfailed, 0 failed**, from all 24 modules of `tests/spec_gate.py` `EXPECTED_MODULES` (JUnit: no `skipped`, `failure` or `error` element in any case) | | Install | `pip install "morphobricks-preprocessing[notebook] @ git+…@d023ff9…"` in a fresh venv reports 0.5.1 (normalization 0.4.0), its package files equal this branch's `src/morphobricks_preprocessing` (`diff -r`), and `PreprocessingNotebook()` builds `welcome()`, `steps()` and `results_step()` | -| Shipped notebook | Its three code cells, unmodified, in a real kernel (`nbclient`) in a fresh venv: cell 3 first stops with one line naming cell 1; cell 1 installs `RELEASE_REF` from GitHub and prints **Ready** (0.5.1, normalization 0.4.0); cell 2 prints its one-line note; cell 3 displays the app. With `google.colab` faked: cell 3 before cell 2 prints the one Drive line and opens the app; after cell 2 it opens without it. Cell 1 on a Python 3.10 kernel stops with the one "Python 3.11 or newer" line | +| Shipped notebook | Its three code cells, unmodified, in a real kernel (`nbclient`) in a fresh venv: cell 3 first stops with one line naming cell 1; cell 1 installs `RELEASE_REF` from GitHub and prints **Ready** (0.5.1, normalization 0.4.0); cell 2 prints its one-line note; cell 3 displays the app. With `google.colab` faked: cell 3 before cell 2 prints the one Drive line and opens the app; after cell 2 it opens without it. Cell 1 on a Python 3.10 kernel stops with the one "Python 3.11 or newer" line (on `a4154ad`). Re-run by the review on `9f61cb9` in a fresh venv (cell 3 alone, cell 1 twice, cell 2 outside Colab, cell 3; faked Colab: cell 1 adds the `zarr` extra, cell 3 without Drive prints the one line and opens the app), plus an interface that cannot load (one line naming the `ModuleNotFoundError`), a failed Drive mount (one line naming the `ValueError`) and an unexpected error in cell 3 (its full traceback) | Test cases per Spec module: catalogue 376, derived design 80, discovery 17, encoding 58, Fov image type 14, gate 9, lab links 11, no fallback 15, pin 16, publication 20, reader 330, reader