From c5143a0057f1b58068e1af061e4b76d352d322b3 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 27 Oct 2026 17:45:52 +0000 Subject: [PATCH] Let other bricks embed the folder step without the experiment design MorphoRelate's "your data" notebook embeds PreprocessingNotebook.folder_step. From v0.1.0, confirming the labels opens the "Experiment design" card, writes experiment_design.csv into a new _MorphoBricks_Preprocessing folder beside the image folder, and says that step 3 runs once every group is confirmed. MorphoRelate does not use the file, and its step 3 is another step. folder_step takes show_design (default True). With False the card is left out of the page; confirming the labels asks nothing, design stays None and nothing is written (no file, no output folder). The labels question drops "Step 3 can only run once the labels are confirmed", and the "What next?" boxes of steps 1 and 2 no longer send the user to the design. Step 3, if shown, does not wait for a design and runs without one, as run_experiment(design=None). Building the page again with the other value opens or closes the design for labels already confirmed. The default page, its texts and its files are unchanged, pinned by a test. ConfirmToggle gains explain() to replace the text under its question. Nothing else in step 1 writes files: the folder check and header inspection only read. Tested headless (pages built, driven by clicks); not looked at in a browser or a real JupyterLab. Decision recorded as plan section 12 question 29; contract section 12 and AGENTS.md updated. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_017TP44aBCGzyksTpPS2FFmQ --- AGENTS.md | 15 +++- docs/contract.md | 2 +- docs/plan.md | 28 +++++- .../notebook_style.py | 14 ++- src/morphobricks_preprocessing/notebook_ui.py | 47 ++++++++-- tests/test_notebook_ui.py | 87 +++++++++++++++++++ 6 files changed, 173 insertions(+), 20 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 14b9a4e5..9299f8ad 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -100,7 +100,8 @@ src/morphobricks_preprocessing/ notebook_style.py notebook look and feel: cards, status cards, tiles, tables, ConfirmToggle, Browse dialogs notebook_ui.py PreprocessingNotebook: the four notebook steps (needs the `notebook` extra); folder_step/axis_step take title, purpose, intro and show_next (and - show_output for step 1) so other bricks can embed them (plan §12 Q26) + show_output and show_design for step 1) so other bricks can embed them + (plan §12 Q26, Q29) notebooks/MorphoBricks_Preprocessing.ipynb thin notebook: every code cell calls notebook_ui only tests/ pytest with simulated TIFFs (tests/tiff_tree.py, tests/simulation.py); the schema-version-2 fixture (tests/v2_fixture.py, tests/data) @@ -364,7 +365,8 @@ functions the tests use. gain a folder, drops removed folders. - **Notebook** writes the file after every answer once the labels are confirmed; a confirmed design for the same folders is folded away behind **Change the experiment - design**; step 3 is gated on it. **CLI** `run` reads it (or `--design`), else writes the + design**; step 3 is gated on it. `folder_step(show_design=False)` (embedding, Q29) asks + and writes nothing, and step 3 runs without a design. **CLI** `run` reads it (or `--design`), else writes the reconciled template and exits 1. **API** `run_experiment(design=None)` is optional; with a design it is written with the table and `experiment_design_sha256` goes in the manifest. @@ -525,11 +527,14 @@ plainer version "not interactive enough". Descriptive, guided and forgiving is t ### Page layout (`notebook_style`) -- **Embedding steps 1 and 2 (Q26):** `folder_step` and `axis_step` keep this notebook's +- **Embedding steps 1 and 2 (Q26, Q29):** `folder_step` and `axis_step` keep this notebook's words by default; another brick (MorphoRelate) passes its own `title`, `purpose` and `intro` (HTML; `""` leaves the box out), `show_output=False` to hide the output folder card (shown again when the check reports `output_inside_experiment`), and `show_next=False` to hide the "What next?" box naming this notebook's - steps. The widgets and their behaviour are the same either way. + steps. The widgets and their behaviour are the same either way. `show_design=False` + leaves out the "Experiment design" card: confirming the labels then writes nothing + (no `experiment_design.csv`, no output folder), the step's texts name neither the design + nor step 3, and step 3 runs without a design. - **Page and cards:** `ui.page(title, subtitle, children)` wraps a step in a grey page (`mb-root`) with a title and a one-sentence purpose. `ui.card(header, children)` gives a white card with an upper-case header. `ui.intro(html)` adds a blue "what happens here" @@ -695,6 +700,8 @@ jupyter lab notebooks/MorphoBricks_Preprocessing.ipynb - **Released as v0.1.0** (27 October 2026), with everything above; the normalization brick is pinned to its v0.2.0 (question 28). The next tag, v0.2.0, follows follow-ups P4 and P5. +- **After v0.1.0 (not released yet):** step 1 embeddable without the experiment design, + `folder_step(show_design=False)` (question 29). - **Next (not started):** follow-ups P4–P5 of the normalization plan (image type, intensity checks and plots). - **Not yet verified:** diff --git a/docs/contract.md b/docs/contract.md index 0a469a1d..b21770fb 100644 --- a/docs/contract.md +++ b/docs/contract.md @@ -236,6 +236,6 @@ folder. | Controls | None, one or several conditions; every row of a condition has the same value. Recorded only. | | Reruns | The file in the output folder is read and brought up to date with the folders found: folders in it keep their group, control mark and confirmation (added or removed images within a folder keep it too, with the counts recorded at confirmation); a new folder joins the group named after it and inherits its condition's control mark, and that group is asked again; folders no longer found are dropped without asking again. | | Reading | `read_experiment_design` reads labels as text; `control` and `confirmed` accept `true`/`false` in any case, also `yes`/`no` and `1`/`0`, and empty as false; an empty `repeat_group` means the folder name; a byte-order mark is ignored. A missing column, an invalid value, a folder listed twice, rows of one condition disagreeing on `control` or another `schema_version` raise `DesignFormatError`. `ExperimentDesign.repeat_groups()` gives the `{(condition, biological_repeat): repeat_group}` mapping the normalization brick takes. | -| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. Step 3 runs only once every group is confirmed. | +| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. Step 3 runs only once every group is confirmed. A brick embedding step 1 with `folder_step(show_design=False)` (plan §12 question 29) gets no design: nothing is asked or written, and step 3 runs without one, as `run_experiment(design=None)`. | | Command line | `run` reads the file from the output folder, or `--design FILE`. Without a fully confirmed design it writes the design brought up to date with the folders (a template with `confirmed=false` for the groups still to confirm, keeping the answers already given) to the output folder, explains the columns, and stops with status 1. Confirmed rows without a date get the date of the run. | | Run | `run_experiment(..., design=...)` brings the design up to date with the folders and stops with `design_not_confirmed` or `repeat_group_conflict` if it is not fully confirmed. Otherwise the file is written with the table and published before the manifest, which records its SHA-256 (§9). Without a design the run is unchanged: no file is written, an existing one is left as it is, and `experiment_design_sha256` is null. `plane_histograms.csv` does not depend on the design. | diff --git a/docs/plan.md b/docs/plan.md index f9e283ef..3689aa8e 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -333,7 +333,8 @@ API, CLI). The notebook and CLI only call the API. read as a one-member level, physical sizes with a unit per axis in the image header (TIFF and OME-Zarr, from the OME-NGFF scale), and steps 1 and 2 of the notebook embeddable by another brick. Outputs are unchanged. *Implemented, see questions 23 to - 26.* + 26.* After v0.1.0 (not released yet), step 1 embeds without the experiment design + (`show_design=False`). *Implemented, see question 29.* - **Channels across columns (follow-up P3b of the normalization plan):** `plane_histograms.csv` schema version 3, written with the normalization brick's Milestone 2b (now pinned to its v0.2.0, question 28): one row per XYC plane with each channel in its own @@ -656,7 +657,8 @@ Raised during review; to be decided before the milestones that need them: (optional)" card; the check then uses the folder in its box, the default when empty, and the card is shown again when the check finds that folder inside the experiment, so the box to fix it is never hidden). The defaults give the pages as before, and this notebook calls them - without arguments; the widgets, the checks and the gating are the same either way.* + without arguments; the widgets, the checks and the gating are the same either way. + Later, `show_design` leaves out the experiment design (question 29).* 27. How does this brick follow the channels moving into columns (normalization plan N41–N44, its Milestone 2b, follow-up P3b)? *Decided (October 2026): `plane_histograms.csv` is written in schema version 3 with @@ -691,3 +693,25 @@ Raised during review; to be decided before the milestones that need them: depends on the normalization brick's release v0.2.0 (tag, not commit) instead of the Milestone 2b merge commit `0e53e67`; v0.2.0 adds only RGB and intensity checks, which this brick does not call yet. The next tag, v0.2.0, follows follow-ups P4 and P5.* +29. Since follow-up P3 (question 22), step 1 asks for the experiment design once the labels + are confirmed: an "Experiment design" card, `experiment_design.csv` written in the + output folder (`_MorphoBricks_Preprocessing/` by default, created beside the + image folder), and texts saying that step 3 runs once every group is confirmed. + MorphoRelate milestone M3's "your data" notebook embeds `folder_step` (question 26) + from tag v0.1.0; it does not use the file, and its step 3 is another step. Can a + brick embedding step 1 leave the design out? + *Decided (27 October 2026, the owner's choice of an option here rather than a + workaround in MorphoRelate): yes. `folder_step` takes `show_design` (default True). + With False the "Experiment design" card is left out of the page; confirming the + labels asks nothing, `design` stays None and nothing is written, so neither + `experiment_design.csv` nor the output folder is created; the texts of the step no + longer name the design or step 3 (the labels question drops "Step 3 can only run once + the labels are confirmed", and the "What next?" box goes from the labels to step 2). + The rest of the notebook follows: step 2's "What next?" box does not send the user + back to the design, and step 3, if shown, does not wait for it and runs without a + design, as `run_experiment(design=None)` does (no file written, an existing one left as + it is). Building the page again with the other value opens or closes the design for + labels already confirmed. With the default the page, its texts and its files are as + before. Nothing else in step 1 writes files: the folder check and the header + inspection only read, and with `show_output=False` the output folder is only named. + The CSV, the manifest and their versions are unchanged.* diff --git a/src/morphobricks_preprocessing/notebook_style.py b/src/morphobricks_preprocessing/notebook_style.py index a7c8d1fe..ca964493 100644 --- a/src/morphobricks_preprocessing/notebook_style.py +++ b/src/morphobricks_preprocessing/notebook_style.py @@ -294,10 +294,9 @@ def __init__( self._confirmed_text = confirmed_text self._value = False self._callbacks: list = [] - self.prompt = widgets.HTML( - f"
❓ {html.escape(question)}" - f"
{explanation}
" - ) + self._question = question + self.prompt = widgets.HTML() + self.explain(explanation) self.button = button(f" {confirm_label}", style="success", icon="check", main=True) self.message = widgets.HTML() self.widget = widgets.VBox([self.prompt, self.button, self.message]) @@ -318,6 +317,13 @@ def disabled(self) -> bool: def disabled(self, disabled: bool) -> None: self.button.disabled = disabled + def explain(self, explanation: str) -> None: + """Replace the text under the question (HTML), keeping the answer.""" + self.prompt.value = ( + f"
❓ {html.escape(self._question)}" + f"
{explanation}
" + ) + def observe(self, callback) -> None: """Call ``callback(value)`` whenever the answer changes.""" self._callbacks.append(callback) diff --git a/src/morphobricks_preprocessing/notebook_ui.py b/src/morphobricks_preprocessing/notebook_ui.py index 937a9c92..cb359e8a 100644 --- a/src/morphobricks_preprocessing/notebook_ui.py +++ b/src/morphobricks_preprocessing/notebook_ui.py @@ -188,6 +188,7 @@ def __init__(self, experiment_folder: str = "", output_folder: str = ""): self._checked_folder = "" self._preview_shown = False self._show_output = True + self._show_design = True self._layouts_cache: tuple[tuple, dict[str, PlaneLayout]] | None = None self._build_folder_step(str(experiment_folder), str(output_folder)) self._build_axis_step() @@ -229,12 +230,14 @@ def folder_step( intro: str | None = None, show_output: bool = True, show_next: bool = True, + show_design: bool = True, ) -> widgets.Widget: """Step 1: choose and check the experiment folder, confirm the labels. The defaults give this notebook's step. Another brick embedding the step (it checks the folder and the image headers, nothing more) can give its - own words and leave out the output folder of this brick's histograms. + own words and leave out the output folder of this brick's histograms and + its experiment design. Parameters ---------- @@ -254,6 +257,11 @@ def folder_step( show_next False hides the "What next?" box, which names this notebook's next steps. + show_design + False leaves out the "Experiment design" card: once the labels are + confirmed nothing is asked and no ``experiment_design.csv`` (nor its + output folder) is written, the step's texts no longer name the design + or this notebook's step 3, and step 3 runs without a design. Returns ------- @@ -263,6 +271,11 @@ def folder_step( ui.set_visible(self.folder_next, show_next) self._show_output = show_output ui.set_visible(self.output_card, show_output) + self.labels_confirmed.explain(_labels_explanation(show_design)) + if show_design != self._show_design: + self._show_design = show_design + # Labels confirmed on an earlier page open or close the design to match. + self._labels_changed() if intro is None: intro = ( "What happens here: the folders are scanned, the labels are taken " @@ -286,7 +299,7 @@ def folder_step( self.folder_status, self.folder_tiles, self.labels_card, - self.design_card, + *([self.design_card] if show_design else []), self.consistency_card, self.issues_card, self.folder_next, @@ -456,9 +469,7 @@ def _build_folder_step(self, folder: str, output: str) -> None: self.labels_table = widgets.HTML() self.labels_confirmed = ui.ConfirmToggle( "Do the labels in the table match your experiment?", - "Check that every image folder has the right condition, biological repeat and " - "technical repeat. If a label is wrong, rename the folder and click Check " - "folder again. Step 3 can only run once the labels are confirmed.", + _labels_explanation(show_design=True), "Yes, the labels are correct", "Labels confirmed.", "Undo: the labels are not right", @@ -685,7 +696,7 @@ def _show_issue_summary(self) -> None: self._refresh_folder_next() def _labels_changed(self) -> None: - if self.labels_confirmed.value and self.preflight is not None: + if self.labels_confirmed.value and self.preflight is not None and self._show_design: self._open_design() else: self._close_design() @@ -934,11 +945,15 @@ def _refresh_design_status(self) -> None: def _design_confirmed(self) -> bool: return self.design is not None and self.design.confirmed + def _design_pending(self) -> bool: + """Return True while the design shown in step 1 still has questions to answer.""" + return self._show_design and not self._design_confirmed() + def _refresh_folder_next(self) -> None: self.folder_next.value = "" if self.preflight is None or not self.preflight.ok: return - if self.labels_confirmed.value and not self._design_confirmed(): + if self.labels_confirmed.value and self._design_pending(): self.folder_next.value = ui.next_steps( "Confirm the biological repeats in Step 1.", "Under Experiment design, pair the repeats if needed and answer each yellow " @@ -1400,7 +1415,7 @@ def _refresh_axis_next(self) -> None: "Go to Step 1 and confirm the folder labels.", "Review the labels and click Yes, the labels are correct before running Step 3.", ) - elif not self._design_confirmed(): + elif self._design_pending(): self.axis_next.value = ui.next_steps( "Go to Step 1 and confirm the biological repeats.", "Under Experiment design, answer each yellow question before running Step 3.", @@ -1760,7 +1775,11 @@ def _refresh_run_step(self) -> None: items = [ (folder_ok, "Check the experiment folder and fix any errors (step 1)"), (pre is not None and self.labels_confirmed.value, "Confirm the labels (step 1)"), - (self._design_confirmed(), "Confirm the biological repeats (step 1)"), + *( + [(self._design_confirmed(), "Confirm the biological repeats (step 1)")] + if self._show_design + else [] + ), (self.confirmed_mapping is not None, "Confirm the axis roles (step 2)"), ] missing = [text for done, text in items if not done] @@ -2507,6 +2526,16 @@ def __exit__(self, *exc): return False +def _labels_explanation(show_design: bool) -> str: + """Return what to check before confirming the labels; step 3 is named only with the design.""" + text = ( + "Check that every image folder has the right condition, biological repeat and " + "technical repeat. If a label is wrong, rename the folder and click Check " + "folder again." + ) + return (text + " Step 3 can only run once the labels are confirmed.") if show_design else text + + def _group_question(design: ExperimentDesign, group: str) -> str: """Return the question asked for a repeat group, or for the whole experiment.""" if design.single_repeat: diff --git a/tests/test_notebook_ui.py b/tests/test_notebook_ui.py index a52d949f..a4419961 100644 --- a/tests/test_notebook_ui.py +++ b/tests/test_notebook_ui.py @@ -708,6 +708,93 @@ def test_hidden_output_card_comes_back_when_its_folder_is_inside_the_experiment( assert ui.is_visible(nb.output_card) +def files_under(folder): + return sorted(str(path.relative_to(folder)) for path in folder.rglob("*")) + + +def page_text(found, *paths): + """Return the text of every HTML widget of a page, with the given paths left out.""" + text = " ".join(w.value for w in found if isinstance(w, notebook_ui.widgets.HTML)) + for path in paths: + text = text.replace(str(path), "") + return text + + +def test_folder_step_shows_the_design_by_default(tmp_path): + for name, kwargs in [("default", {}), ("explicit", {"show_design": True})]: + root = design_tree(tmp_path / name / "exp", ["ctrl/rep1", "drug/rep1"]) + nb = PreprocessingNotebook(str(root)) + page = nb.folder_step(**kwargs) + assert page.children[1].children[3:] == ( + nb.folder_card, + nb.output_card, + nb.exclude_box, + nb.check_button, + nb.folder_status, + nb.folder_tiles, + nb.labels_card, + nb.design_card, + nb.consistency_card, + nb.issues_card, + nb.folder_next, + ) + assert "Step 3 can only run once the labels are confirmed." in ( + nb.labels_confirmed.prompt.value + ) + nb.check_button.click() + nb.labels_confirmed.button.click() + assert ui.is_visible(nb.design_card) and nb.design is not None + assert "Step 3 runs once every group is confirmed." in nb.design_status.value + assert "Confirm the biological repeats in Step 1" in nb.folder_next.value + assert "Still to do: confirm the biological repeats" in nb.readiness.value + path = tmp_path / name / "exp_MorphoBricks_Preprocessing" / "experiment_design.csv" + assert f"Saved in {path}" in nb.design_status.value + assert not read_experiment_design(path).confirmed + + +def test_folder_step_without_the_design_writes_nothing(tmp_path): + root = design_tree(tmp_path / "exp", ["ctrl/rep1", "drug/rep1"]) + before = files_under(tmp_path) + nb = PreprocessingNotebook(str(root)) + nb.folder_step(show_output=False, show_next=False, show_design=False) + nb.axis_step(show_next=False) + nb.check_button.click() + nb.labels_confirmed.button.click() + nb.preview_button.click() + nb.preview_checked.button.click() + nb.confirm_button.click() + assert nb.labels_confirmed.value and nb.confirmed_mapping is not None + assert nb.design is None + assert files_under(tmp_path) == before + assert not (tmp_path / "exp_MorphoBricks_Preprocessing").exists() + + +def test_folder_step_without_the_design_card_or_its_words(experiment): + nb = PreprocessingNotebook(str(experiment)) + page = nb.folder_step(show_design=False) + nb.axis_step() + nb.run_step() + assert nb.design_card not in page_parts(page)[2] + nb.check_button.click() + nb.labels_confirmed.button.click() + found = page_parts(page)[2] + assert nb.design_card not in found and not ui.is_visible(nb.design_card) + text = page_text(found, experiment.parent) + assert "Step 3" not in text and "design" not in text.lower() + assert "biological repeats" not in nb.folder_next.value + assert "Go to Step 2" in nb.folder_next.value + nb.preview_button.click() + nb.preview_checked.button.click() + nb.confirm_button.click() + assert "Experiment design" not in nb.axis_next.value + # This notebook's own step 3 then runs without a design. + assert "biological repeats" not in nb.readiness.value and not nb.run_button.disabled + # Building the default page again asks for the design of the confirmed labels. + nb.folder_step() + assert ui.is_visible(nb.design_card) and nb.design is not None + assert "Step 3 can only run" in nb.labels_confirmed.prompt.value + + def test_steps_without_an_intro_box(): nb = PreprocessingNotebook() assert intros(page_parts(nb.folder_step(intro=""))[2]) == []