From 30d10d30d73298adf8175f0f6e80b24359cb0671 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 15:06:11 +0000 Subject: [PATCH 1/3] Notebook: three code cells (install, mount Google Drive, run) Owner request of 10 November 2026, as in ZeroCostDL4Mic and NucleiSky: the single code cell of the shipped notebook is split into 1. Install MorphoBricks preprocessing, 2. Mount Google Drive and 3. Run preprocessing, each under a Markdown heading that explains it. RELEASE_REF, the install target and PreprocessingNotebook(describe=True) are unchanged. Cell 2 prints a one-line note outside Colab. Cell 3 run before cell 1 stops naming cell 1 (not an ImportError), and on Colab without Drive it stops naming cell 2; no cell runs another. Tests: the three cells in order and each one's part, Colab-only code inside the guard (ast), cell 1 run twice installs once, the cells out of order, and real-kernel runs with and without a faked google.colab and of cell 3 alone. Docs updated. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- AGENTS.md | 22 +- CHANGELOG.md | 19 ++ README.md | 12 +- docs/colab-release-check.md | 31 ++- docs/plan.md | 5 + docs/user_guide.md | 26 ++- notebooks/MorphoBricks_Preprocessing.ipynb | 159 +++++++++++-- tests/test_notebook_file.py | 252 +++++++++++++++++---- tests/test_notebook_setup.py | 172 ++++++++++++-- 9 files changed, 579 insertions(+), 119 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 24f47b59..b41aaf3e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -732,7 +732,7 @@ Technical details go to `logging` (shown in the notebook's "Technical details" p | RGB `S` axis (Q3) | Prefilled as C, confirmed like any role | | Significant bits (Q4) | Manifest only, information only; `bit_depth` and binning unchanged | | Corrupt files (Q6) | Explicit exclusion list, recorded in manifest, rows removed | -| Notebook target | Google Colab and JupyterLab/LabConstrictor from one file, following LabConstrictor's notebook portability guide: one `# @title` runnable cell, explicit `display()` of the existing notebook pages, Colab-only Drive mount inside `if "google.colab" in sys.modules:`, no magics, core ipywidgets 7.7/8 compatibility, no forced widget restart/custom manager, remote install pinned to immutable `RELEASE_REF` | +| Notebook target | Google Colab and JupyterLab/LabConstrictor from one file, following LabConstrictor's notebook portability guide: three `# @title` code cells in order (1. Install, 2. Mount Google Drive, 3. Run preprocessing; owner, 10 November 2026), explicit `display()` of the existing notebook pages, Colab-only Drive mount inside `if "google.colab" in sys.modules:`, no magics, core ipywidgets 7.7/8 compatibility, no forced widget restart/custom manager, remote install pinned to immutable `RELEASE_REF` | | CLI | `check ` (summary, label tree, issues, suggested `--axes`) and `run --axes ROLES` with `--output`, `--exclude`, `--bins`, `--histogram-range`, `--robust-quantiles`, `--memory-limit 2GiB`, `--missing-as-singleton` (also on `check`, F10), `--mode restart|full`, `--no-masks`, `--no-exclude`, `--remove-stale-lock` and `--log`; also `masks`, `calibrate`, `design` and `convert` (the TIFF conversion options of v0.3, `--convert-tiff` and `--chunk-size`, were removed in v0.4.0). Axis roles are always stated explicitly, never taken silently from metadata (amended 3 November 2026, plan §12 question 73: when every image names all its axes, `run` uses those roles without `--axes` and prints them as the images name them; plain stacks still need `--axes`). Exit 1 on errors | | Large planes (Q15) | Planes over the memory limit are read in blocks automatically, announced by the warning `planes_read_in_blocks` and `read_in_blocks` in the manifest. Block size derived from the limit (whole strips/tiles/chunks, uncompressed pieces and chunks also by rows). Compressed single-strip/tile planes keep the `plane_over_memory_limit` error. Two-pass histograms identical to whole planes. (Up to v0.3, OME-Zarr copies were written block by block; conversion was removed in v0.4.0.) Axis preview reduced (every n-th pixel) with a caption saying so | | Normalization brick (P1) | The modules `dtypes`, `histogram`, `records`, `csv_io`, `pooling` and `normalization` moved unchanged to `morphobricks_normalization` (normalization plan N1, N2, N6) and are imported from there. Up to v0.3 their public names stayed importable from `morphobricks_preprocessing` and the old submodules, with a `DeprecationWarning`; those stand-ins (`_moved.py`) were removed in v0.4.0 (plan Q46). The package's own code never uses the old paths. Outputs unchanged (CSV schema 2); the manifest's `versions` adds `morphobricks-normalization`; Step 0 checks for it; contract §3–§7 text moved to the normalization contract (§1.2, now §6.2 for the CSV) | @@ -817,14 +817,16 @@ Technical details go to `logging` (shown in the notebook's "Technical details" p tests skip without zarr; CI's Python 3.11 jobs pin zarr 2.18 and 3.13 jobs get zarr 3. - **The notebook file is tested too:** - valid nbformat 4, with no outputs; - - code cells hidden and titled `# @title Run to …`; + - code cells hidden and titled `# @title . …` (1. Install, 2. Mount Google Drive, 3. Run preprocessing); - Colab-only code and `/content` paths only inside the Colab guard; no magics; - the GitHub install pinned to `RELEASE_TAG`, equal to `current_version` and the version index; - - The single runnable cell bootstraps the environment and then displays only widgets built by `notebook_ui`; + - exactly three code cells, in order: 1 installs and reports versions, 2 mounts Google Drive + on Colab (a one-line note elsewhere), 3 displays only widgets built by `notebook_ui` and, + run out of order, names the cell to run first (`tests/test_notebook_setup.py`); - the buttons it names exist; - its cells execute, by `exec` and in a real kernel (`nbclient`, `tests/test_notebook_file.py`; - Step 0's alternatives in `tests/test_notebook_setup.py`). + cell 1's alternatives and the cells out of order in `tests/test_notebook_setup.py`). --- @@ -850,14 +852,18 @@ plainer version "not interactive enough". Descriptive, guided and forgiving is t - **Code cells:** - Hidden, with metadata `{"cellView": "form", "collapsed": false, "jupyter": {"source_hidden": true}, "jupyterlabHideCode": {"locked": true}}`. - - The first line is a Colab title: `# @title Run to …` (LabConstrictor portability - guide; every code cell, all hidden, plan Q58). Longer explanations go in Markdown cells. + - The first line is a Colab title: `# @title . …`, numbered as the headings (LabConstrictor + portability guide; every code cell, all hidden, plan Q58). Longer explanations go in Markdown cells. - Outputs are stripped. - - Step 0 is a standalone standard-library bootstrap: nearby checkout, installed package, + - Three code cells (owner, 10 November 2026, as ZeroCostDL4Mic and NucleiSky): **1. Install**, + **2. Mount Google Drive**, **3. Run preprocessing**, each under a Markdown heading that + explains it. Cell 3 checks that cell 1 ran (`SETUP_RECORD`) and, on Colab, that Drive is + mounted, and otherwise stops naming the cell to run; no cell runs another. + - Cell 1 is a standalone standard-library bootstrap: nearby checkout, installed package, or GitHub installation of the release `RELEASE_TAG`, with missing requirements installed using the kernel's Python (`subprocess`, never `!pip`); on Colab with the `zarr` extra too. Colab-only code - (Drive mount) stays inside `if "google.colab" in sys.modules:`; no `/content` path + (cell 2's Drive mount) stays inside `if "google.colab" in sys.modules:`; no `/content` path outside it, and no custom widget manager is enabled. `current_version` and `notebook_name` are LabConstrictor's names; at each release the pin moves (plan §10, release checklist). diff --git a/CHANGELOG.md b/CHANGELOG.md index 0e974a1a..e83d233d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,25 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the ## Unreleased (v0.5.1) +### The notebook has three code cells: install, mount Google Drive, run + +Owner request of 10 November 2026, as in the lab's ZeroCostDL4Mic and NucleiSky notebooks. The +single code cell of `notebooks/MorphoBricks_Preprocessing.ipynb` is split into three, each under +a Markdown heading that explains it; what the cells do together is unchanged (`RELEASE_REF`, +the install target and `PreprocessingNotebook(describe=True)` stay as they were). + +- **1. Install MorphoBricks preprocessing**: the pinned install with its extras, the version + report and `SETUP_RECORD`, as before; it no longer mounts Drive. Running it again installs + nothing that is already there; no restart is requested. +- **2. Mount Google Drive**: mounts Drive on Colab (the same refusal when the mount fails); in + JupyterLab or LabConstrictor it prints a one-line note and does nothing. +- **3. Run preprocessing**: shows the app. Run before cell 1 it stops with "run cell 1 (Install + MorphoBricks preprocessing) first", not an `ImportError`; on Colab without Drive at + `/content/drive/MyDrive` it stops naming cell 2. It never runs an earlier cell. +- Tests: exactly these three cells in order and each one's part (`test_notebook_file.py`), the + cells out of order and cell 1 run twice (`test_notebook_setup.py`), and the notebook in a + real kernel with and without a faked `google.colab`, and cell 3 alone. + ### PP-C: every run is described; undescribed runs and their compatibility code removed No backward compatibility before the first pipeline release (owner policy of 10 November 2026, diff --git a/README.md b/README.md index 5919ab74..db377a0c 100644 --- a/README.md +++ b/README.md @@ -68,7 +68,7 @@ pip install -e ".[dev]" The optional `notebook` extra installs the widget dependencies for the notebook (`pip install "morphobricks-preprocessing[notebook] @ git+https://github.com/CellMigrationLab/morphobricks-preprocessing.git@v0.5.0"`). -The notebook's single runnable cell can set this up automatically: it uses a compatible nearby +The notebook's first code cell (**1. Install**) can set this up automatically: it uses a compatible nearby checkout or installed package, and installs missing requirements into the active kernel. A standalone download installs from GitHub if needed (internet access and Git required). @@ -81,9 +81,13 @@ is one portable widget app for Google Colab, JupyterLab and LabConstrictor. It f LabConstrictor's [notebook portability guide](https://github.com/CellMigrationLab/LabConstrictor/blob/main/.tools/docs/notebook_portability.md): -1. The notebook is intentionally small: one short Markdown introduction and one `# @title …` - runnable cell. That cell prepares the environment and explicitly displays the complete - preprocessing UI; users never edit code, so LabConstrictor may hide it. +1. The notebook is intentionally small: a short Markdown introduction and three `# @title …` + code cells, each under a heading that explains it, as in the lab's ZeroCostDL4Mic and + NucleiSky notebooks: **1. Install MorphoBricks preprocessing** (prepares the environment and + reports versions), **2. Mount Google Drive** (Colab only; elsewhere a one-line note) and + **3. Run preprocessing** (explicitly displays the complete preprocessing UI). Cell 3 run out + of order says which cell to run first and never runs it for the user. Users never edit + code, so LabConstrictor may hide it. 2. Colab-only setup stays inside `if "google.colab" in sys.modules:` and mounts Google Drive. The UI uses only core ipywidgets supported by both ipywidgets 7.7 and 8; it does not force an ipywidgets upgrade/restart or enable Colab's custom-widget manager. Locally the same notebook diff --git a/docs/colab-release-check.md b/docs/colab-release-check.md index f4b92b14..63666788 100644 --- a/docs/colab-release-check.md +++ b/docs/colab-release-check.md @@ -12,15 +12,15 @@ named-axis copies). Items were updated for later changes: PP-10a replaced the st cell with a button, and PP-12 replaced the `.run-*` backup folders with `.publish` and `.staging`. -**The notebook's pin.** The shipped notebook installs the package at its `RELEASE_REF`, not -main's code, so its cell calls the API of the code at that commit. Between releases main's code +**The notebook's pin.** The shipped notebook installs the package at its `RELEASE_REF` (set in +cell 1), not main's code, so its cell 3 calls the API of the code at that commit. Between releases main's code moves on while the cell keeps the released call: after v0.5.1's PP-C, main has no `describe` argument, while `RELEASE_REF` is still v0.5.0's `fd7afd3`, whose cell calls -`PreprocessingNotebook(describe=True)`. The release PR moves `RELEASE_REF` and the cell's call +`PreprocessingNotebook(describe=True)`. The release PR moves `RELEASE_REF` and cell 3's call in the same commit and records the new pair in `RELEASE_CALLS` of `tests/test_notebook_file.py`, which fails for a `RELEASE_REF` without one. -- [ ] `RELEASE_REF` is the release commit, and the cell's `PreprocessingNotebook(...)` call is +- [ ] `RELEASE_REF` is the release commit, and cell 3's `PreprocessingNotebook(...)` call is that commit's API (v0.5.1: `PreprocessingNotebook()`). Use a dataset on Google Drive with a masks folder, and a second small experiment of @@ -29,17 +29,28 @@ Report a failure as an issue before the final tag. ## A. Fresh runtime and app rendering -1. Open the notebook on a fresh Google Colab runtime and run its single code cell. - - [ ] Drive authorisation appears and the mount succeeds. +The notebook has three code cells, run in order: **1. Install MorphoBricks preprocessing**, +**2. Mount Google Drive** and **3. Run preprocessing**. + +1. Open the notebook on a fresh Google Colab runtime and run cell 3 first. + - [ ] It stops at once with "run cell 1 (Install MorphoBricks preprocessing) first", not an + `ImportError`, and installs nothing. +2. Run cell 1 (install). - [ ] Setup reports the expected notebook, preprocessing, normalization and ipywidgets - versions and the pinned source revision. - - [ ] No ipywidgets upgrade or runtime restart is requested. -2. In the same cell output: + versions and the pinned source revision, then names cells 2 and 3. + - [ ] No ipywidgets upgrade or runtime restart is requested, and Drive is not mounted. + - [ ] Run cell 1 again: nothing is installed again and the same versions are reported. +3. Run cell 3 before cell 2. + - [ ] It stops with "Google Drive is not connected … run cell 2 (Mount Google Drive)". +4. Run cell 2 (Drive). + - [ ] Drive authorisation appears, the mount succeeds and the cell says Drive is connected. +5. Run cell 3 (the app). - [ ] The complete preprocessing app renders inline; Colab does not open the Code snippets pane instead of the widget. - [ ] Buttons, dropdowns, accordions, preview image, sliders, progress bars and matplotlib plots render and react. -3. Repeat the same single-cell smoke test in JupyterLab/LabConstrictor. +6. Repeat cells 1 to 3 in JupyterLab/LabConstrictor. + - [ ] Cell 2 prints its one-line "Not Google Colab" note and does nothing else. - [ ] The same app renders and uses local paths without executing Colab-only setup. ## B. Step 1 (PR #118) diff --git a/docs/plan.md b/docs/plan.md index 0f53e474..b8d8cb2b 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -1157,6 +1157,11 @@ Raised during review; to be decided before the milestones that need them: Colab, Step 0 installs the `notebook` and `zarr` extras (locally `notebook`, unchanged), so OME-Zarr images can be read there; (5) `RELEASE_TAG` stays `v0.4.0rc2`; the owner tags it after PP-8c and PP-8d are merged.* + *Amended (10 November 2026, owner): the notebook has three code cells, as the lab's + ZeroCostDL4Mic and NucleiSky notebooks: 1. Install (the pinned install and the version + report), 2. Mount Google Drive (Colab only; elsewhere a one-line note), 3. Run + preprocessing (the app). Cell 3 run before cell 1, or on Colab without Drive, stops and + names the cell to run; no cell runs another.* 59. How are images with axis errors saved as named-axis copies (design v0.4 §5, PP-11)? *Decided (owner, 2 November 2026; implemented in PP-11) as design v0.4 §5 says: images only, as named-axis OME-TIFF (BigTIFF when large); copies keep the file name and folder diff --git a/docs/user_guide.md b/docs/user_guide.md index f777549f..2a3ed3a5 100644 --- a/docs/user_guide.md +++ b/docs/user_guide.md @@ -14,16 +14,23 @@ its own output folder, which is never inside the experiment folder. You need Python 3.11 or newer and JupyterLab (or Google Colab). Download [`notebooks/MorphoBricks_Preprocessing.ipynb`](../notebooks/MorphoBricks_Preprocessing.ipynb), -open it and run its single code cell. That cell prepares the environment and displays the -complete preprocessing widget app. +open it and run its three code cells in order: -The setup chooses one source: +1. **Install MorphoBricks preprocessing** prepares the environment and prints the versions. + Running it again installs nothing that is already there; no restart is needed. +2. **Mount Google Drive** connects Google Drive on Google Colab. In JupyterLab or + LabConstrictor it prints a one-line note and does nothing. +3. **Run preprocessing** displays the complete preprocessing widget app. Run before cell 1, + it stops and says to run cell 1; on Colab without Drive connected, it says to run cell 2. + It never runs an earlier cell for you. + +Cell 1 chooses one source: 1. **Nearby checkout:** when the kernel is running in this repository or one of its subfolders, use that checkout's source code. The checkout must be the commit the notebook pins (`RELEASE_REF`) with no uncommitted changes, and setup prints its commit and tree state. Any other commit or edited tree is refused; developers set `DEVELOPMENT_OVERRIDE = True` at the - top of the cell to use it, and the printed header then says so. + top of cell 1 to use it, and the printed header then says so. 2. **Matching installed package:** otherwise, reuse an installed preprocessing package only when its version matches the notebook. 3. **Pinned GitHub source:** otherwise install the notebook's immutable `RELEASE_REF`, never @@ -53,12 +60,13 @@ paste the path instead. ### Google Colab Open the notebook in Google Colab (**File › Open notebook › GitHub**, or upload the file) and -run the notebook's single code cell. On Colab it: +run the notebook's three code cells in order. On Colab: -- connects your Google Drive: allow access in the window that opens. Your Drive is then the - folder `/content/drive/MyDrive`; -- installs the notebook's pinned preprocessing revision, with its notebook dependencies and - the `zarr` extra when needed; +- cell 1 installs the notebook's pinned preprocessing revision, with its notebook + dependencies and the `zarr` extra when needed; +- cell 2 connects your Google Drive: allow access in the window that opens. Your Drive is then + the folder `/content/drive/MyDrive`. Cell 3 refuses to open the app until Drive is + connected, so the data and the output are never silently put on the runtime's own disk; - uses Colab's native core ipywidgets support. The notebook accepts ipywidgets 7.7 or 8 and does not force a widget upgrade, runtime restart or custom-widget manager. diff --git a/notebooks/MorphoBricks_Preprocessing.ipynb b/notebooks/MorphoBricks_Preprocessing.ipynb index 3644a17c..f27261b8 100644 --- a/notebooks/MorphoBricks_Preprocessing.ipynb +++ b/notebooks/MorphoBricks_Preprocessing.ipynb @@ -23,18 +23,28 @@ "- **Google Colab:** keep the experiment and output on Google Drive, under `/content/drive/MyDrive/`.\n", "- **JupyterLab / LabConstrictor:** use normal local paths.\n", "\n", - "### Run\n", + "### How to run\n", "\n", - "Run the single code cell below. It prepares the environment and opens the complete preprocessing app.\n", + "Run the three code cells below in order, from top to bottom (click a cell and press **Shift + Enter**, or its ▶ button):\n", "\n", - "In the app:\n", + "1. **Install** prepares the environment.\n", + "2. **Mount Google Drive** connects your Drive on Google Colab; elsewhere it does nothing.\n", + "3. **Run preprocessing** opens the complete preprocessing app.\n", "\n", - "1. Give the **experiment folder** and optional **output/masks folders**, then click **Check folder**.\n", - "2. Go through the steps one at a time with **Next** (or click a step in the strip at the top): axes, pixel sizes, conditions, replicates, comparisons, image type, stains, channel tags and masks.\n", - "3. On **Check and save**, read your experiment in words and save it; then **Run** computes the histograms.\n", - "4. Review the QC results; pooled statistics are optional.\n", + "After a restart of the runtime or kernel, run the three cells again: cell 1 then installs nothing that is already there." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "___\n", + "\n", + "## **1. Install MorphoBricks preprocessing**\n", + "\n", + "Installs the preprocessing release this notebook is made for, with its notebook dependencies (on Google Colab also the OME-Zarr reader), into the Python environment of this notebook. Nothing is installed when it is already there, so running the cell again is safe.\n", "\n", - "Detailed explanations appear inside the app only where a decision is required.\n" + "When it finishes, it prints **Ready** with the versions of preprocessing, normalization and ipywidgets, the notebook version and where the package was loaded from. No restart of the runtime is needed." ] }, { @@ -51,7 +61,7 @@ }, "outputs": [], "source": [ - "# @title Start MorphoBricks preprocessing\n", + "# @title 1. Install MorphoBricks preprocessing\n", "import platform\n", "import subprocess\n", "import sys\n", @@ -78,15 +88,7 @@ "SETUP_RECORD: dict = {}\n", "\n", "if \"google.colab\" in sys.modules:\n", - " from google.colab import drive\n", - "\n", - " print(\"Google Colab: connecting Google Drive...\", flush=True)\n", - " try:\n", - " drive.mount(\"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/content/drive\")\n", - " except Exception as exc:\n", - " raise RuntimeError(\n", - " \"Google Drive could not be connected. Allow Drive access, then run this cell again.\"\n", - " ) from exc\n", + " print(\"Google Colab: preparing MorphoBricks preprocessing...\", flush=True)\n", " EXTRAS = \"notebook,zarr\"\n", "else:\n", " print(\n", @@ -200,8 +202,11 @@ " return [name for name in required if find_spec(name) is None]\n", "\n", "\n", - "def _start_preprocessing_notebook() -> object:\n", - " \"\"\"Prepare this kernel and return the preprocessing notebook controller.\"\"\"\n", + "def _install_preprocessing() -> None:\n", + " \"\"\"Prepare this kernel: install what is missing, check the interface loads, print versions.\n", + "\n", + " Running it again installs nothing that is already there.\n", + " \"\"\"\n", " if sys.version_info < (3, 11):\n", " raise RuntimeError(\"Select a Python 3.11 or newer kernel, then run this cell again.\")\n", "\n", @@ -285,7 +290,7 @@ " )\n", "\n", " try:\n", - " interface = import_module(\"morphobricks_preprocessing.notebook_ui\")\n", + " import_module(\"morphobricks_preprocessing.notebook_ui\")\n", " except ImportError as exc:\n", " raise RuntimeError(\n", " \"MorphoBricks preprocessing was found, but its notebook interface could not load.\"\n", @@ -301,13 +306,119 @@ " f\"ipywidgets {widget_version}.\\n\"\n", " f\"Notebook {current_version}; source: {origin}.\\n\"\n", " f\"{checkout_line}\"\n", - " f\"Loaded from: {package_file}\",\n", + " f\"Loaded from: {package_file}\\n\"\n", + " \"Next: run cell 2 (Mount Google Drive), then cell 3 (Run preprocessing).\",\n", " flush=True,\n", " )\n", - " return interface.PreprocessingNotebook(describe=True)\n", "\n", "\n", - "notebook = _start_preprocessing_notebook()\n", + "_install_preprocessing()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "___\n", + "\n", + "## **2. Mount Google Drive**\n", + "\n", + "**Google Colab:** connects your Google Drive. Allow access in the window that opens; your Drive is then the folder **/content/drive/MyDrive**. Keep the experiment and the output folder there: everything else on a Colab runtime is wiped when it disconnects.\n", + "\n", + "**JupyterLab / LabConstrictor:** there is no Drive to mount. The cell only prints a note, and you use normal local paths in the app." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false, + "jupyter": { + "source_hidden": true + }, + "jupyterlabHideCode": { + "locked": true + } + }, + "outputs": [], + "source": [ + "# @title 2. Mount Google Drive\n", + "import sys\n", + "\n", + "if \"google.colab\" in sys.modules:\n", + " from google.colab import drive\n", + "\n", + " print(\"Google Colab: connecting Google Drive...\", flush=True)\n", + " try:\n", + " drive.mount(\"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/content/drive\")\n", + " except Exception as exc:\n", + " raise RuntimeError(\n", + " \"Google Drive could not be connected. Allow Drive access, then run this cell again.\"\n", + " ) from exc\n", + " print(\"Google Drive is connected: your files are in /content/drive/MyDrive.\", flush=True)\n", + "else:\n", + " print(\"Not Google Colab: there is no Google Drive to mount here, so nothing was done.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "___\n", + "\n", + "## **3. Run preprocessing**\n", + "\n", + "Opens the preprocessing app. Run cells 1 and 2 first: without them this cell stops and says which cell to run.\n", + "\n", + "In the app:\n", + "\n", + "1. Give the **experiment folder** and optional **output/masks folders**, then click **Check folder**.\n", + "2. Go through the steps one at a time with **Next** (or click a step in the strip at the top): axes, pixel sizes, conditions, replicates, comparisons, image type, stains, channel tags and masks.\n", + "3. On **Check and save**, read your experiment in words and save it; then **Run** computes the histograms.\n", + "4. Review the QC results; pooled statistics are optional.\n", + "\n", + "Detailed explanations appear inside the app only where a decision is required." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false, + "jupyter": { + "source_hidden": true + }, + "jupyterlabHideCode": { + "locked": true + } + }, + "outputs": [], + "source": [ + "# @title 3. Run preprocessing\n", + "import importlib\n", + "import sys\n", + "from pathlib import Path\n", + "\n", + "if not globals().get(\"SETUP_RECORD\"):\n", + " raise RuntimeError(\n", + " \"MorphoBricks preprocessing is not installed in this session yet: run cell 1 \"\n", + " \"(Install MorphoBricks preprocessing) first, then run this cell again.\"\n", + " )\n", + "if \"google.colab\" in sys.modules:\n", + " if not Path(\"/content/drive/MyDrive\").is_dir():\n", + " raise RuntimeError(\n", + " \"Google Drive is not connected, so the experiment and output folders on Drive \"\n", + " \"cannot be reached: run cell 2 (Mount Google Drive) first, then run this cell again.\"\n", + " )\n", + "\n", + "try:\n", + " interface = importlib.import_module(\"morphobricks_preprocessing.notebook_ui\")\n", + "except ImportError as exc:\n", + " raise RuntimeError(\n", + " \"MorphoBricks preprocessing was found, but its notebook interface could not load. \"\n", + " \"Run cell 1 again.\"\n", + " ) from exc\n", + "notebook = interface.PreprocessingNotebook(describe=True)\n", "\n", "import ipywidgets as widgets\n", "from IPython.display import display\n", diff --git a/tests/test_notebook_file.py b/tests/test_notebook_file.py index 0141f7d8..84e2dae1 100644 --- a/tests/test_notebook_file.py +++ b/tests/test_notebook_file.py @@ -1,5 +1,6 @@ """The standalone notebook file: portable, readable and executable in a real kernel.""" +import ast import json import re from pathlib import Path @@ -10,54 +11,102 @@ NOTEBOOK = ROOT / "notebooks/MorphoBricks_Preprocessing.ipynb" VERSIONS = ROOT / "notebooks/notebook_latest_versions.yaml" +# The three code cells, in order, as in the lab's ZeroCostDL4Mic and NucleiSky notebooks +# (owner, 10 November 2026): install, mount Google Drive, run the app. +INSTALL, DRIVE, RUN = 0, 1, 2 +CELL_TITLES = ( + "# @title 1. Install MorphoBricks preprocessing", + "# @title 2. Mount Google Drive", + "# @title 3. Run preprocessing", +) +HEADINGS = ( + "## **1. Install MorphoBricks preprocessing**", + "## **2. Mount Google Drive**", + "## **3. Run preprocessing**", +) +COLAB_GUARD = 'if "google.colab" in sys.modules:' + def _notebook(): return json.loads(NOTEBOOK.read_text(encoding="utf-8")) -def _code_cell(): - cells = [cell for cell in _notebook()["cells"] if cell["cell_type"] == "code"] - assert len(cells) == 1 - return cells[0] +def _code_cells(): + return [cell for cell in _notebook()["cells"] if cell["cell_type"] == "code"] + +def _source(role): + return "".join(_code_cells()[role]["source"]) -def _source(): - return "".join(_code_cell()["source"]) + +def _all_code(): + return "\n".join(_source(role) for role in (INSTALL, DRIVE, RUN)) def _setting(name): - match = re.search(rf'^{name} = "([^"]*)"$', _source(), re.M) - assert match is not None, f"the setup cell sets no {name}" + match = re.search(rf'^{name} = "([^"]*)"$', _source(INSTALL), re.M) + assert match is not None, f"the install cell sets no {name}" return match.group(1) -def test_notebook_is_one_readable_markdown_cell_and_one_runnable_cell(): +def test_notebook_is_an_introduction_and_three_explained_code_cells(): notebook = _notebook() assert notebook["nbformat"] == 4 - assert [cell["cell_type"] for cell in notebook["cells"]] == ["markdown", "code"] - - markdown = "".join(notebook["cells"][0]["source"]) - assert "What it does" in markdown - assert "Before you start" in markdown - assert "Google Colab" in markdown - assert "JupyterLab / LabConstrictor" in markdown - assert "\\n" not in markdown - - code = _code_cell() - assert code["outputs"] == [] - assert code["execution_count"] is None - assert _source().splitlines()[0].startswith("# @title ") - assert code["metadata"].get("jupyter", {}).get("source_hidden") is True + cells = notebook["cells"] + assert [cell["cell_type"] for cell in cells] == [ + "markdown", + *("markdown", "code") * 3, + ] + + introduction = "".join(cells[0]["source"]) + for text in ("What it does", "Before you start", "Google Colab", "JupyterLab / LabConstrictor"): + assert text in introduction + for step in ("1. **Install**", "2. **Mount Google Drive**", "3. **Run preprocessing**"): + assert step in introduction + assert "single code cell" not in introduction + + code_cells = [cell for cell in cells if cell["cell_type"] == "code"] + assert [cell["source"][0].rstrip("\n") for cell in code_cells] == list(CELL_TITLES) + for position, (heading, code) in enumerate(zip(HEADINGS, code_cells, strict=True)): + explanation = "".join(cells[1 + 2 * position]["source"]) + assert heading in explanation.splitlines() + assert cells[2 + 2 * position] is code # each heading sits right above its own cell + assert code["outputs"] == [] + assert code["execution_count"] is None + assert code["metadata"].get("jupyter", {}).get("source_hidden") is True + for cell in cells: + if cell["cell_type"] == "markdown": + assert "\\n" not in "".join(cell["source"]) + + +def test_each_code_cell_does_only_its_own_part(): + install, drive, run = (_source(role) for role in (INSTALL, DRIVE, RUN)) + # 1: the pinned install and the version report; no Drive, no app. + assert "RELEASE_REF = " in install + assert '"pip",' in install + assert "Ready: preprocessing" in install + assert "drive.mount" not in install + assert "PreprocessingNotebook" not in install + assert "display(" not in install + # 2: the Drive mount only. + assert 'drive.mount("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/content/drive")' in drive + for other in ("pip", "subprocess", "morphobricks", "PreprocessingNotebook", "display("): + assert other not in drive + # 3: the app only. It never installs, mounts, or runs an earlier cell's code for the user. + assert "display(app)" in run + for other in ("pip", "subprocess", "drive.mount", "RELEASE_REF", "_install_preprocessing"): + assert other not in run + assert 'globals().get("SETUP_RECORD")' in run def test_notebook_uses_stable_page_api_not_a_new_app_method(): - source = _source() + source = _source(RUN) assert "notebook.app(" not in source # One stepper holds steps 1-13 (owner, 2026-11-09); the statistics stay a page of their own. for call in ("notebook.welcome()", "notebook.steps()", "notebook.results_step()"): assert call in source for page in ("folder_step(", "axis_step(", "channels_step(", "check_save_step(", "run_step("): - assert f"notebook.{page}" not in source # never shown twice beside the stepper + assert f"notebook.{page}" not in _all_code() # never shown twice beside the stepper assert release_call() in source assert "display(app)" in source @@ -85,21 +134,47 @@ def release_call(): def test_the_shipped_cell_calls_the_api_of_its_release_ref(): - source = _source() - calls = re.findall(r"PreprocessingNotebook\([^)]*\)", source) + calls = re.findall(r"PreprocessingNotebook\([^)]*\)", _all_code()) assert calls == [release_call()] # one call, the released code's: never this checkout's + assert release_call() in _source(RUN) def test_colab_setup_mounts_drive_without_custom_widget_manager_or_restart(): - source = _source() - assert 'if "google.colab" in sys.modules:' in source + source = _source(DRIVE) + assert COLAB_GUARD in source assert 'drive.mount("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/content/drive")' in source - assert "enable_custom_widget_manager" not in source - assert "Restart session" not in source + for role in (INSTALL, DRIVE, RUN): + assert "enable_custom_widget_manager" not in _source(role) + assert "Restart session" not in _source(role) + + +GUARD_TEST = ast.unparse(ast.parse(COLAB_GUARD.removeprefix("if ").removesuffix(":"))) + + +def _colab_only_nodes(tree): + """Yield the nodes inside an ``if "google.colab" in sys.modules:`` block of ``tree``.""" + for node in ast.walk(tree): + if isinstance(node, ast.If) and ast.unparse(node.test) == GUARD_TEST: + for statement in node.body: + yield from ast.walk(statement) + + +@pytest.mark.parametrize("role", [INSTALL, DRIVE, RUN]) +def test_colab_only_code_and_content_paths_stay_inside_the_colab_guard(role): + tree = ast.parse(_source(role)) + inside = {id(node) for node in _colab_only_nodes(tree)} + for node in ast.walk(tree): + if isinstance(node, ast.Constant) and isinstance(node.value, str): + if node.value.startswith("/content"): + assert id(node) in inside, node.value + if isinstance(node, ast.ImportFrom) and (node.module or "").startswith("google"): + assert id(node) in inside + assert "!pip" not in _source(role) + assert not re.search(r"^\s*[%!]", _source(role), re.M) # no magics def test_remote_install_is_pinned_to_one_exact_commit(): - source = _source() + source = _source(INSTALL) tag = _setting("RELEASE_TAG") ref = _setting("RELEASE_REF") assert re.fullmatch(r"v\d+\.\d+\.\d+(rc\d+)?", tag), tag @@ -109,6 +184,9 @@ def test_remote_install_is_pinned_to_one_exact_commit(): assert _setting("REPOSITORY") == ( "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/CellMigrationLab/morphobricks-preprocessing.git" ) + for role in (DRIVE, RUN): # one place sets the pin + assert "RELEASE_TAG" not in _source(role) + assert "current_version" not in _source(role) def test_version_metadata_matches_labconstrictor_index(): @@ -121,35 +199,113 @@ def test_version_metadata_matches_labconstrictor_index(): assert index == {NOTEBOOK.stem: _setting("current_version")} -def test_notebook_runs_in_a_kernel(): - pytest.importorskip("ipywidgets") - pytest.importorskip("matplotlib") - pytest.importorskip("ipykernel", reason="the dev extra installs ipykernel") - nbclient = pytest.importorskip("nbclient", reason="the dev extra installs nbclient") - nbformat = pytest.importorskip("nbformat") +def _kernel_notebook(nbformat, *, roles=(INSTALL, DRIVE, RUN), colab_drive=None): + """Return the notebook's code cells ``roles`` as a notebook ready for this checkout's kernel. + With ``colab_drive``, a first cell fakes ``google.colab`` whose ``drive.mount`` makes a + ``MyDrive`` folder, and the cells' ``/content/drive`` is ``colab_drive``. + """ notebook = nbformat.read(NOTEBOOK, as_version=4) - code = next(cell for cell in notebook.cells if cell.cell_type == "code") + code = [cell for cell in notebook.cells if cell.cell_type == "code"] + install, run = code[INSTALL], code[RUN] # The kernel runs in this checkout, whose commit is not RELEASE_REF (#103): the one edit a # developer makes to use a checkout is the explicit override. - assert code.source.count("DEVELOPMENT_OVERRIDE = False") == 1 - code.source = code.source.replace("DEVELOPMENT_OVERRIDE = False", "DEVELOPMENT_OVERRIDE = True") + assert install.source.count("DEVELOPMENT_OVERRIDE = False") == 1 + install.source = install.source.replace( + "DEVELOPMENT_OVERRIDE = False", "DEVELOPMENT_OVERRIDE = True" + ) # While RELEASE_REF is behind this checkout, the cell calls the released API: the kernel, # running this checkout, is given this checkout's call (the release PR makes them one). - assert code.source.count(release_call()) == 1 - code.source = code.source.replace(release_call(), CHECKOUT_CALL) + assert run.source.count(release_call()) == 1 + run.source = run.source.replace(release_call(), CHECKOUT_CALL) + cells = [code[role] for role in roles] + if colab_drive is not None: + for cell in cells: + cell.source = cell.source.replace("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/content/drive", str(colab_drive)) + fake = nbformat.v4.new_code_cell( + "import sys, types\n" + "from pathlib import Path\n" + "drive = types.SimpleNamespace(\n" + " mount=lambda path: (Path(path) / 'MyDrive').mkdir(parents=True, exist_ok=True)\n" + ")\n" + "sys.modules['google.colab'] = types.SimpleNamespace(drive=drive)\n" + ) + cells = [fake, *cells] + notebook.cells = cells + return notebook, cells[-len(roles) :] + + +def _execute(notebook, **options): + nbclient = pytest.importorskip("nbclient", reason="the dev extra installs nbclient") client = nbclient.NotebookClient( notebook, timeout=300, kernel_name="python3", resources={"metadata": {"path": str(NOTEBOOK.parent)}}, + **options, ) client.execute() - assert not [output for output in code.outputs if output.output_type == "error"] - text = "".join(output.get("text", "") for output in code.outputs) - assert "Ready: preprocessing" in text - assert any( + +def _text(cell): + return "".join(output.get("text", "") for output in cell.outputs) + + +def _errors(cell): + return [output for output in cell.outputs if output.output_type == "error"] + + +def _shows_a_widget(cell): + return any( "application/vnd.jupyter.widget-view+json" in output.get("data", {}) - for output in code.outputs + for output in cell.outputs ) + + +def _kernel_modules(): + pytest.importorskip("ipywidgets") + pytest.importorskip("matplotlib") + pytest.importorskip("ipykernel", reason="the dev extra installs ipykernel") + pytest.importorskip("nbclient", reason="the dev extra installs nbclient") + return pytest.importorskip("nbformat") + + +def test_notebook_runs_in_a_kernel(): + nbformat = _kernel_modules() + notebook, (install, drive, run) = _kernel_notebook(nbformat) + _execute(notebook) + + assert not [*_errors(install), *_errors(drive), *_errors(run)] + assert "Ready: preprocessing" in _text(install) + assert "Next: run cell 2 (Mount Google Drive), then cell 3" in _text(install) + # Outside Colab the Drive cell prints one line and does nothing else. + assert _text(drive).strip().splitlines() == [ + "Not Google Colab: there is no Google Drive to mount here, so nothing was done." + ] + assert not _shows_a_widget(install) + assert _shows_a_widget(run) + + +def test_notebook_runs_in_a_kernel_with_a_faked_colab(tmp_path): + nbformat = _kernel_modules() + drive_root = tmp_path / "content/drive" + notebook, (install, drive, run) = _kernel_notebook(nbformat, colab_drive=drive_root) + _execute(notebook) + + assert not [*_errors(install), *_errors(drive), *_errors(run)] + assert "Google Colab: preparing MorphoBricks preprocessing" in _text(install) + assert "Google Colab: connecting Google Drive" in _text(drive) + assert (drive_root / "MyDrive").is_dir() + assert _shows_a_widget(run) + + +def test_the_run_cell_alone_in_a_fresh_kernel_names_cell_1(): + pytest.importorskip("ipykernel", reason="the dev extra installs ipykernel") + nbformat = _kernel_modules() + notebook, (run,) = _kernel_notebook(nbformat, roles=(RUN,)) + _execute(notebook, allow_errors=True) + + (error,) = _errors(run) + assert error.ename == "RuntimeError" # not an ImportError of the package + assert "run cell 1 (Install MorphoBricks preprocessing) first" in error.evalue + assert not _shows_a_widget(run) diff --git a/tests/test_notebook_setup.py b/tests/test_notebook_setup.py index 126c9068..ab58b092 100644 --- a/tests/test_notebook_setup.py +++ b/tests/test_notebook_setup.py @@ -1,5 +1,10 @@ -"""Exercise the standalone setup cell without installing into the test environment.""" +"""Exercise the notebook's three code cells without installing into the test environment. +Cell 1 installs, cell 2 mounts Google Drive on Colab, cell 3 shows the app; the cells run in +one namespace, as in a kernel. +""" + +import ast import importlib import importlib.metadata import importlib.util @@ -26,12 +31,23 @@ ZARR = "zarr" -def cell_source(): +INSTALL, DRIVE, RUN = 1, 2, 3 +"""The cells' numbers, as their titles and the notebook's text name them.""" +COLAB_DRIVE = "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/content/drive" + + +def cell_source(number=INSTALL): notebook = json.loads(NOTEBOOK.read_text(encoding="utf-8")) - return "".join(next(c["source"] for c in notebook["cells"] if c["cell_type"] == "code")) + code = [cell for cell in notebook["cells"] if cell["cell_type"] == "code"] + return "".join(code[number - 1]["source"]) RELEASE_REF = re.search(r'RELEASE_REF = "([0-9a-f]{40})"', cell_source()).group(1) +_APP_CALL = ast.parse( + re.search(r"PreprocessingNotebook\([^)]*\)", cell_source(RUN)).group(0), mode="eval" +).body +APP_OPTIONS = {keyword.arg: ast.literal_eval(keyword.value) for keyword in _APP_CALL.keywords} +"""The options cell 3 passes, as its call names them (``test_notebook_file.RELEASE_CALLS``).""" def install_git_double(monkeypatch, state): @@ -69,6 +85,9 @@ def setup_environment(tmp_path, monkeypatch): available={*REQUIRED, ZARR}, installs=[], imports=[], + apps=[], + drive=tmp_path / "content/drive", + namespace={}, versions={ "morphobricks-preprocessing": "0.5.0", "morphobricks-normalization": "0.3.4", @@ -110,6 +129,7 @@ def import_interface(name): class FakeNotebook: def __init__(self, **options): # describe=True, the shipped notebook's run self.options = options + state.apps.append(options) def __getattr__(self, name): # welcome, folder_step, axis_step, channels_step, ... return lambda: widgets.HTML(name) @@ -118,13 +138,18 @@ def __getattr__(self, name): # welcome, folder_step, axis_step, channels_step, monkeypatch.setattr(importlib, "import_module", import_interface) - def start(): - notebook = json.loads(NOTEBOOK.read_text(encoding="utf-8")) - source = next(c["source"] for c in notebook["cells"] if c["cell_type"] == "code") - namespace = {} - exec("".join(source), namespace) - state.namespace = namespace - return namespace + def start(*numbers, fresh=True): + """Run the code cells ``numbers`` (all three by default) in order, in one namespace. + + ``fresh`` starts a new kernel namespace; ``/content/drive`` is ``state.drive`` (the + fake Drive's mount point). + """ + if fresh: + state.namespace = {} + for number in numbers or (INSTALL, DRIVE, RUN): + source = cell_source(number).replace(COLAB_DRIVE, str(state.drive)) + exec(source, state.namespace) + return state.namespace state.start = start return state @@ -146,8 +171,9 @@ def test_setup_reuses_the_matching_installed_release(setup_environment, capsys): state.start() assert state.installs == [] assert state.imports == [ - "morphobricks_preprocessing.notebook_ui", + "morphobricks_preprocessing.notebook_ui", # cell 1 checks the interface loads "morphobricks_preprocessing", + "morphobricks_preprocessing.notebook_ui", # cell 3 shows it ] out = capsys.readouterr().out assert "Ready: preprocessing 0.5.0" in out @@ -297,8 +323,14 @@ def test_setup_does_not_mix_an_already_loaded_package_with_another_checkout( def colab(monkeypatch): + """Fake ``google.colab``: its ``drive.mount`` records the mount point and makes MyDrive.""" calls = [] - fake = SimpleNamespace(drive=SimpleNamespace(mount=lambda path: calls.append(path))) + + def mount(path): + calls.append(path) + (Path(path) / "MyDrive").mkdir(parents=True, exist_ok=True) + + fake = SimpleNamespace(drive=SimpleNamespace(mount=mount)) monkeypatch.setitem(sys.modules, "google.colab", fake) return calls @@ -306,8 +338,12 @@ def colab(monkeypatch): def test_colab_mounts_drive_without_a_custom_widget_manager(setup_environment, monkeypatch, capsys): calls = colab(monkeypatch) setup_environment.start() - assert calls == ["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/content/drive"] - assert "Google Colab: connecting Google Drive" in capsys.readouterr().out + assert f'drive.mount("{COLAB_DRIVE}")' in cell_source(DRIVE) + assert calls == [str(setup_environment.drive)] # the cells' /content/drive, faked + out = capsys.readouterr().out + assert "Google Colab: connecting Google Drive" in out + assert "Google Drive is connected" in out + assert setup_environment.apps == [APP_OPTIONS] def test_colab_installs_the_zarr_extra(setup_environment, monkeypatch): @@ -338,9 +374,13 @@ def refuse(path): "google.colab", SimpleNamespace(drive=SimpleNamespace(mount=refuse)), ) + state = setup_environment + state.start(INSTALL) + imported = list(state.imports) with pytest.raises(RuntimeError, match="Google Drive could not be connected"): - setup_environment.start() - assert setup_environment.imports == [] + state.start(DRIVE, fresh=False) + assert state.imports == imported + assert state.apps == [] def test_setup_rejects_old_python_before_installing(setup_environment, monkeypatch): @@ -362,3 +402,103 @@ def fail(command, *, check): with pytest.raises(RuntimeError, match="could not install"): state.start() assert state.imports == [] + + +# The three cells (owner, 10 November 2026): each does its own part, and a cell run out of order +# says which cell to run; no cell runs another for the user. + + +def test_cell_1_installs_without_mounting_drive_or_showing_the_app( + setup_environment, monkeypatch, capsys +): + calls = colab(monkeypatch) + state = setup_environment + state.available.clear() + state.versions.pop("morphobricks-preprocessing") + namespace = state.start(INSTALL) + assert len(state.installs) == 1 + assert "morphobricks-preprocessing[notebook,zarr]" in state.installs[0][-1] + assert calls == [] + assert state.apps == [] + assert namespace["SETUP_RECORD"]["release_ref"] == RELEASE_REF + out = capsys.readouterr().out + assert "Ready: preprocessing 0.5.0; normalization 0.3.4; ipywidgets 7.7.1." in out + assert "Next: run cell 2 (Mount Google Drive), then cell 3 (Run preprocessing)." in out + + +def test_running_cell_1_again_installs_nothing_more(setup_environment, capsys): + state = setup_environment + state.available.clear() + state.versions.pop("morphobricks-preprocessing") + state.start(INSTALL) + first = capsys.readouterr().out + record = dict(state.namespace["SETUP_RECORD"]) + state.start(INSTALL, fresh=False) + again = capsys.readouterr().out + assert len(state.installs) == 1 + assert "Installing notebook requirements" in first + assert "Installing notebook requirements" not in again + assert "Ready: preprocessing 0.5.0" in again + assert state.namespace["SETUP_RECORD"]["preprocessing_version"] == "0.5.0" + assert record["release_ref"] == state.namespace["SETUP_RECORD"]["release_ref"] + + +def test_cell_2_outside_colab_prints_one_note_and_does_nothing(setup_environment, capsys): + state = setup_environment + assert "google.colab" not in sys.modules + state.start(DRIVE) + out = capsys.readouterr().out + assert out.strip().splitlines() == [ + "Not Google Colab: there is no Google Drive to mount here, so nothing was done." + ] + assert state.installs == state.imports == state.apps == [] + assert not state.drive.exists() + + +def test_cell_2_alone_on_colab_only_mounts_drive(setup_environment, monkeypatch): + calls = colab(monkeypatch) + state = setup_environment + state.start(DRIVE) + assert calls == [str(state.drive)] + assert state.installs == state.imports == state.apps == [] + + +@pytest.mark.parametrize("on_colab", [False, True]) +def test_cell_3_without_cell_1_names_cell_1(setup_environment, monkeypatch, on_colab): + state = setup_environment + if on_colab: + colab(monkeypatch) + with pytest.raises(RuntimeError, match=r"run cell 1 \(Install MorphoBricks preprocessing\)"): + state.start(RUN) + assert state.installs == state.imports == state.apps == [] + + +def test_cell_3_after_a_failed_cell_1_still_names_cell_1(setup_environment, monkeypatch): + state = setup_environment + monkeypatch.setattr(sys, "version_info", (3, 10, 0)) + with pytest.raises(RuntimeError, match="Python 3.11 or newer"): + state.start(INSTALL) + with pytest.raises(RuntimeError, match="run cell 1"): + state.start(RUN, fresh=False) + assert state.apps == [] + + +def test_cell_3_on_colab_without_drive_names_cell_2(setup_environment, monkeypatch): + calls = colab(monkeypatch) + state = setup_environment + state.start(INSTALL) + imported = list(state.imports) + with pytest.raises(RuntimeError, match=r"Google Drive is not connected.*run cell 2"): + state.start(RUN, fresh=False) + assert calls == [] # never mounted for the user + assert state.imports == imported + assert state.apps == [] + state.start(DRIVE, RUN, fresh=False) + assert state.apps == [APP_OPTIONS] + + +def test_cell_3_outside_colab_needs_no_drive(setup_environment): + state = setup_environment + state.start(INSTALL, RUN) # cell 2 skipped: nothing to mount outside Colab + assert state.apps == [APP_OPTIONS] + assert "app" in state.namespace From f67bea0958224e2a09fb2796087fc85959bd1bcf Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 15:17:10 +0000 Subject: [PATCH 2/3] Notebook setup tests: run the cells from a module-level helper The fixture stays within the code rules' 60 executable lines. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- tests/test_notebook_setup.py | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/tests/test_notebook_setup.py b/tests/test_notebook_setup.py index ab58b092..69f37d20 100644 --- a/tests/test_notebook_setup.py +++ b/tests/test_notebook_setup.py @@ -5,6 +5,7 @@ """ import ast +import functools import importlib import importlib.metadata import importlib.util @@ -75,6 +76,20 @@ def check_output(command, **kwargs): monkeypatch.setattr(subprocess, "check_output", check_output) +def run_cells(state, *numbers, fresh=True): + """Run the code cells ``numbers`` (all three by default) in order, in one namespace. + + ``fresh`` starts a new kernel namespace; ``/content/drive`` is ``state.drive`` (the fake + Drive's mount point). + """ + if fresh: + state.namespace = {} + for number in numbers or (INSTALL, DRIVE, RUN): + source = cell_source(number).replace(COLAB_DRIVE, str(state.drive)) + exec(source, state.namespace) + return state.namespace + + @pytest.fixture def setup_environment(tmp_path, monkeypatch): platform.platform() @@ -138,20 +153,7 @@ def __getattr__(self, name): # welcome, folder_step, axis_step, channels_step, monkeypatch.setattr(importlib, "import_module", import_interface) - def start(*numbers, fresh=True): - """Run the code cells ``numbers`` (all three by default) in order, in one namespace. - - ``fresh`` starts a new kernel namespace; ``/content/drive`` is ``state.drive`` (the - fake Drive's mount point). - """ - if fresh: - state.namespace = {} - for number in numbers or (INSTALL, DRIVE, RUN): - source = cell_source(number).replace(COLAB_DRIVE, str(state.drive)) - exec(source, state.namespace) - return state.namespace - - state.start = start + state.start = functools.partial(run_cells, state) return state From 6a2f37d5b408278f620022594f0f14b223e867f7 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 15:38:26 +0000 Subject: [PATCH 3/3] Notebook tests: put the faked Drive path into the cells in POSIX form A Windows temporary path in a Python string literal holds backslash escapes; the forward-slash form is the same folder there. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc --- tests/test_notebook_file.py | 4 ++-- tests/test_notebook_setup.py | 7 ++++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/tests/test_notebook_file.py b/tests/test_notebook_file.py index 84e2dae1..56644f54 100644 --- a/tests/test_notebook_file.py +++ b/tests/test_notebook_file.py @@ -220,8 +220,8 @@ def _kernel_notebook(nbformat, *, roles=(INSTALL, DRIVE, RUN), colab_drive=None) run.source = run.source.replace(release_call(), CHECKOUT_CALL) cells = [code[role] for role in roles] if colab_drive is not None: - for cell in cells: - cell.source = cell.source.replace("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/content/drive", str(colab_drive)) + for cell in cells: # the path goes into a Python string: no backslashes (Windows) + cell.source = cell.source.replace("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/content/drive", colab_drive.as_posix()) fake = nbformat.v4.new_code_cell( "import sys, types\n" "from pathlib import Path\n" diff --git a/tests/test_notebook_setup.py b/tests/test_notebook_setup.py index 69f37d20..27b30716 100644 --- a/tests/test_notebook_setup.py +++ b/tests/test_notebook_setup.py @@ -85,7 +85,8 @@ def run_cells(state, *numbers, fresh=True): if fresh: state.namespace = {} for number in numbers or (INSTALL, DRIVE, RUN): - source = cell_source(number).replace(COLAB_DRIVE, str(state.drive)) + # The path goes into a Python string: no backslashes (Windows). + source = cell_source(number).replace(COLAB_DRIVE, state.drive.as_posix()) exec(source, state.namespace) return state.namespace @@ -341,7 +342,7 @@ def test_colab_mounts_drive_without_a_custom_widget_manager(setup_environment, m calls = colab(monkeypatch) setup_environment.start() assert f'drive.mount("{COLAB_DRIVE}")' in cell_source(DRIVE) - assert calls == [str(setup_environment.drive)] # the cells' /content/drive, faked + assert calls == [setup_environment.drive.as_posix()] # the cells' /content/drive, faked out = capsys.readouterr().out assert "Google Colab: connecting Google Drive" in out assert "Google Drive is connected" in out @@ -461,7 +462,7 @@ def test_cell_2_alone_on_colab_only_mounts_drive(setup_environment, monkeypatch) calls = colab(monkeypatch) state = setup_environment state.start(DRIVE) - assert calls == [str(state.drive)] + assert calls == [state.drive.as_posix()] assert state.installs == state.imports == state.apps == []