diff --git a/AGENTS.md b/AGENTS.md index 584b614d..bbcf1ac7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,7 +106,8 @@ src/morphobricks_preprocessing/ notebook_style.py notebook look and feel: cards, status cards, tiles, tables, ConfirmToggle, Browse dialogs notebook_ui.py PreprocessingNotebook: the four notebook steps (needs the `notebook` extra); folder_step/axis_step take title, purpose, intro and show_next (and - show_output for step 1) so other bricks can embed them (plan §12 Q26) + show_output and show_design for step 1) so other bricks can embed them + (plan §12 Q26, Q31) notebooks/MorphoBricks_Preprocessing.ipynb thin notebook: every code cell calls notebook_ui only tests/ pytest with simulated TIFFs (tests/tiff_tree.py, tests/simulation.py); the schema-version-2 fixture (tests/v2_fixture.py, tests/data) @@ -370,7 +371,8 @@ functions the tests use. gain a folder, drops removed folders. - **Notebook** writes the file after every answer once the labels are confirmed; a confirmed design for the same folders is folded away behind **Change the experiment - design**; step 3 is gated on it. **CLI** `run` reads it (or `--design`), else writes the + design**; step 3 is gated on it. `folder_step(show_design=False)` (embedding, Q31) asks + and writes nothing, and step 3 runs without a design. **CLI** `run` reads it (or `--design`), else writes the reconciled template and exits 1. **API** `run_experiment(design=None)` is optional; with a design it is written with the table and `experiment_design_sha256` goes in the manifest. @@ -560,11 +562,14 @@ plainer version "not interactive enough". Descriptive, guided and forgiving is t ### Page layout (`notebook_style`) -- **Embedding steps 1 and 2 (Q26):** `folder_step` and `axis_step` keep this notebook's +- **Embedding steps 1 and 2 (Q26, Q31):** `folder_step` and `axis_step` keep this notebook's words by default; another brick (MorphoRelate) passes its own `title`, `purpose` and `intro` (HTML; `""` leaves the box out), `show_output=False` to hide the output folder card (shown again when the check reports `output_inside_experiment`), and `show_next=False` to hide the "What next?" box naming this notebook's - steps. The widgets and their behaviour are the same either way. + steps. The widgets and their behaviour are the same either way. `show_design=False` + leaves out the "Experiment design" card: confirming the labels then writes nothing + (no `experiment_design.csv`, no output folder), the step's texts name neither the design + nor step 3, and step 3 runs without a design. - **Page and cards:** `ui.page(title, subtitle, children)` wraps a step in a grey page (`mb-root`) with a title and a one-sentence purpose. `ui.card(header, children)` gives a white card with an upper-case header. `ui.intro(html)` adds a blue "what happens here" @@ -731,7 +736,9 @@ jupyter lab notebooks/MorphoBricks_Preprocessing.ipynb is pinned to its v0.2.0 (question 28). The next tag, v0.2.0, follows follow-ups P4 and P5. - **Since v0.1.0:** follow-ups P4 (the image type, question 29) and P5 (intensity checks - and plots, question 30) of the normalization plan. The owner tags v0.2.0 next. + and plots, question 30) of the normalization plan, and step 1 embeddable without the + experiment design, `folder_step(show_design=False)` (question 31). The owner tags v0.2.0 + next. - **Not yet verified:** - the exact look of the notebook in a real JupyterLab; - the optional `jl-hidecode` JupyterLab extension, which keeps code cells collapsed. It diff --git a/docs/contract.md b/docs/contract.md index 1a6ad8fd..8e3767e2 100644 --- a/docs/contract.md +++ b/docs/contract.md @@ -239,7 +239,7 @@ folder. | Controls | None, one or several conditions; every row of a condition has the same value. Recorded only. | | Reruns | The file in the output folder is read and brought up to date with the folders found: folders in it keep their group, control mark and confirmation (added or removed images within a folder keep it too, with the counts recorded at confirmation); a new folder joins the group named after it and inherits its condition's control mark, and that group is asked again; folders no longer found are dropped without asking again. | | Reading | `read_experiment_design` reads labels as text; `control` and `confirmed` accept `true`/`false` in any case, also `yes`/`no` and `1`/`0`, and empty as false; an empty `repeat_group` means the folder name; a byte-order mark is ignored. A missing column, an invalid value, a folder listed twice, rows of one condition disagreeing on `control` or another `schema_version` raise `DesignFormatError`. `ExperimentDesign.repeat_groups()` gives the `{(condition, biological_repeat): repeat_group}` mapping the normalization brick takes. | -| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. Step 3 runs only once every group is confirmed. | +| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. Step 3 runs only once every group is confirmed. A brick embedding step 1 with `folder_step(show_design=False)` (plan §12 question 31) gets no design: nothing is asked or written, and step 3 runs without one, as `run_experiment(design=None)`. | | Command line | `run` reads the file from the output folder, or `--design FILE`. Without a fully confirmed design it writes the design brought up to date with the folders (a template with `confirmed=false` for the groups still to confirm, keeping the answers already given) to the output folder, explains the columns, and stops with status 1. Confirmed rows without a date get the date of the run. | | Run | `run_experiment(..., design=...)` brings the design up to date with the folders and stops with `design_not_confirmed` or `repeat_group_conflict` if it is not fully confirmed. Otherwise the file is written with the table and published before the manifest, which records its SHA-256 (§9). Without a design the run is unchanged: no file is written, an existing one is left as it is, and `experiment_design_sha256` is null. `plane_histograms.csv` does not depend on the design. | diff --git a/docs/plan.md b/docs/plan.md index 62486590..d03549a3 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -333,7 +333,8 @@ API, CLI). The notebook and CLI only call the API. read as a one-member level, physical sizes with a unit per axis in the image header (TIFF and OME-Zarr, from the OME-NGFF scale), and steps 1 and 2 of the notebook embeddable by another brick. Outputs are unchanged. *Implemented, see questions 23 to - 26.* + 26.* After v0.1.0 (not released yet), step 1 embeds without the experiment design + (`show_design=False`). *Implemented, see question 31.* - **Channels across columns (follow-up P3b of the normalization plan):** `plane_histograms.csv` schema version 3, written with the normalization brick's Milestone 2b (now pinned to its v0.2.0, question 28): one row per XYC plane with each channel in its own @@ -668,7 +669,8 @@ Raised during review; to be decided before the milestones that need them: (optional)" card; the check then uses the folder in its box, the default when empty, and the card is shown again when the check finds that folder inside the experiment, so the box to fix it is never hidden). The defaults give the pages as before, and this notebook calls them - without arguments; the widgets, the checks and the gating are the same either way.* + without arguments; the widgets, the checks and the gating are the same either way. + Later, `show_design` leaves out the experiment design (question 31).* 27. How does this brick follow the channels moving into columns (normalization plan N41–N44, its Milestone 2b, follow-up P3b)? *Decided (October 2026): `plane_histograms.csv` is written in schema version 3 with @@ -748,3 +750,25 @@ Raised during review; to be decided before the milestones that need them: kind with the action once per kind, takes `--outlier-limit`, `--saturation-limit` and `--min-images`, and draws no plots. Comparisons stay within a condition, as the brick does. `plane_histograms.csv` and `experiment_design.csv` do not change.* +31. Since follow-up P3 (question 22), step 1 asks for the experiment design once the labels + are confirmed: an "Experiment design" card, `experiment_design.csv` written in the + output folder (`_MorphoBricks_Preprocessing/` by default, created beside the + image folder), and texts saying that step 3 runs once every group is confirmed. + MorphoRelate milestone M3's "your data" notebook embeds `folder_step` (question 26) + from tag v0.1.0; it does not use the file, and its step 3 is another step. Can a + brick embedding step 1 leave the design out? + *Decided (27 October 2026, the owner's choice of an option here rather than a + workaround in MorphoRelate): yes. `folder_step` takes `show_design` (default True). + With False the "Experiment design" card is left out of the page; confirming the + labels asks nothing, `design` stays None and nothing is written, so neither + `experiment_design.csv` nor the output folder is created; the texts of the step no + longer name the design or step 3 (the labels question drops "Step 3 can only run once + the labels are confirmed", and the "What next?" box goes from the labels to step 2). + The rest of the notebook follows: step 2's "What next?" box does not send the user + back to the design, and step 3, if shown, does not wait for it and runs without a + design, as `run_experiment(design=None)` does (no file written, an existing one left as + it is). Building the page again with the other value opens or closes the design for + labels already confirmed. With the default the page, its texts and its files are as + before. Nothing else in step 1 writes files: the folder check and the header + inspection only read, and with `show_output=False` the output folder is only named. + The CSV, the manifest and their versions are unchanged.* diff --git a/src/morphobricks_preprocessing/notebook_style.py b/src/morphobricks_preprocessing/notebook_style.py index 1ecab568..0dc25348 100644 --- a/src/morphobricks_preprocessing/notebook_style.py +++ b/src/morphobricks_preprocessing/notebook_style.py @@ -294,10 +294,9 @@ def __init__( self._confirmed_text = confirmed_text self._value = False self._callbacks: list = [] - self.prompt = widgets.HTML( - f"
❓ {html.escape(question)}" - f"
{explanation}
" - ) + self._question = question + self.prompt = widgets.HTML() + self.explain(explanation) self.button = button(f" {confirm_label}", style="success", icon="check", main=True) self.message = widgets.HTML() self.widget = widgets.VBox([self.prompt, self.button, self.message]) @@ -330,6 +329,13 @@ def disabled(self) -> bool: def disabled(self, disabled: bool) -> None: self.button.disabled = disabled + def explain(self, explanation: str) -> None: + """Replace the text under the question (HTML), keeping the answer.""" + self.prompt.value = ( + f"
❓ {html.escape(self._question)}" + f"
{explanation}
" + ) + def observe(self, callback) -> None: """Call ``callback(value)`` whenever the answer changes.""" self._callbacks.append(callback) diff --git a/src/morphobricks_preprocessing/notebook_ui.py b/src/morphobricks_preprocessing/notebook_ui.py index 0d477543..b6dbd0aa 100644 --- a/src/morphobricks_preprocessing/notebook_ui.py +++ b/src/morphobricks_preprocessing/notebook_ui.py @@ -224,6 +224,7 @@ def __init__(self, experiment_folder: str = "", output_folder: str = ""): self._checked_folder = "" self._preview_shown = False self._show_output = True + self._show_design = True self._layouts_cache: tuple[tuple, dict[str, PlaneLayout]] | None = None self._build_folder_step(str(experiment_folder), str(output_folder)) self._build_axis_step() @@ -265,12 +266,14 @@ def folder_step( intro: str | None = None, show_output: bool = True, show_next: bool = True, + show_design: bool = True, ) -> widgets.Widget: """Step 1: choose and check the experiment folder, confirm the labels. The defaults give this notebook's step. Another brick embedding the step (it checks the folder and the image headers, nothing more) can give its - own words and leave out the output folder of this brick's histograms. + own words and leave out the output folder of this brick's histograms and + its experiment design. Parameters ---------- @@ -290,6 +293,11 @@ def folder_step( show_next False hides the "What next?" box, which names this notebook's next steps. + show_design + False leaves out the "Experiment design" card: once the labels are + confirmed nothing is asked and no ``experiment_design.csv`` (nor its + output folder) is written, the step's texts no longer name the design + or this notebook's step 3, and step 3 runs without a design. Returns ------- @@ -299,6 +307,11 @@ def folder_step( ui.set_visible(self.folder_next, show_next) self._show_output = show_output ui.set_visible(self.output_card, show_output) + self.labels_confirmed.explain(_labels_explanation(show_design)) + if show_design != self._show_design: + self._show_design = show_design + # Labels confirmed on an earlier page open or close the design to match. + self._labels_changed() if intro is None: intro = ( "What happens here: the folders are scanned, the labels are taken " @@ -322,7 +335,7 @@ def folder_step( self.folder_status, self.folder_tiles, self.labels_card, - self.design_card, + *([self.design_card] if show_design else []), self.consistency_card, self.issues_card, self.folder_next, @@ -496,9 +509,7 @@ def _build_folder_step(self, folder: str, output: str) -> None: self.labels_table = widgets.HTML() self.labels_confirmed = ui.ConfirmToggle( "Do the labels in the table match your experiment?", - "Check that every image folder has the right condition, biological repeat and " - "technical repeat. If a label is wrong, rename the folder and click Check " - "folder again. Step 3 can only run once the labels are confirmed.", + _labels_explanation(show_design=True), "Yes, the labels are correct", "Labels confirmed.", "Undo: the labels are not right", @@ -725,7 +736,7 @@ def _show_issue_summary(self) -> None: self._refresh_folder_next() def _labels_changed(self) -> None: - if self.labels_confirmed.value and self.preflight is not None: + if self.labels_confirmed.value and self.preflight is not None and self._show_design: self._open_design() else: self._close_design() @@ -974,11 +985,15 @@ def _refresh_design_status(self) -> None: def _design_confirmed(self) -> bool: return self.design is not None and self.design.confirmed + def _design_pending(self) -> bool: + """Return True while the design shown in step 1 still has questions to answer.""" + return self._show_design and not self._design_confirmed() + def _refresh_folder_next(self) -> None: self.folder_next.value = "" if self.preflight is None or not self.preflight.ok: return - if self.labels_confirmed.value and not self._design_confirmed(): + if self.labels_confirmed.value and self._design_pending(): self.folder_next.value = ui.next_steps( "Confirm the biological repeats in Step 1.", "Under Experiment design, pair the repeats if needed and answer each yellow " @@ -1541,7 +1556,7 @@ def _refresh_axis_next(self) -> None: "Go to Step 1 and confirm the folder labels.", "Review the labels and click Yes, the labels are correct before running Step 3.", ) - elif not self._design_confirmed(): + elif self._design_pending(): self.axis_next.value = ui.next_steps( "Go to Step 1 and confirm the biological repeats.", "Under Experiment design, answer each yellow question before running Step 3.", @@ -1909,7 +1924,11 @@ def _refresh_run_step(self) -> None: items = [ (folder_ok, "Check the experiment folder and fix any errors (step 1)"), (pre is not None and self.labels_confirmed.value, "Confirm the labels (step 1)"), - (self._design_confirmed(), "Confirm the biological repeats (step 1)"), + *( + [(self._design_confirmed(), "Confirm the biological repeats (step 1)")] + if self._show_design + else [] + ), (self.confirmed_mapping is not None, "Confirm the axis roles (step 2)"), (self.confirmed_image_type is not None, "Confirm the image type (step 2)"), ] @@ -3043,6 +3062,16 @@ def __exit__(self, *exc): return False +def _labels_explanation(show_design: bool) -> str: + """Return what to check before confirming the labels; step 3 is named only with the design.""" + text = ( + "Check that every image folder has the right condition, biological repeat and " + "technical repeat. If a label is wrong, rename the folder and click Check " + "folder again." + ) + return (text + " Step 3 can only run once the labels are confirmed.") if show_design else text + + def _group_question(design: ExperimentDesign, group: str) -> str: """Return the question asked for a repeat group, or for the whole experiment.""" if design.single_repeat: diff --git a/tests/test_notebook_ui.py b/tests/test_notebook_ui.py index 64058bfd..53da4261 100644 --- a/tests/test_notebook_ui.py +++ b/tests/test_notebook_ui.py @@ -883,6 +883,94 @@ def test_hidden_output_card_comes_back_when_its_folder_is_inside_the_experiment( assert ui.is_visible(nb.output_card) +def files_under(folder): + return sorted(str(path.relative_to(folder)) for path in folder.rglob("*")) + + +def page_text(found, *paths): + """Return the text of every HTML widget of a page, with the given paths left out.""" + text = " ".join(w.value for w in found if isinstance(w, notebook_ui.widgets.HTML)) + for path in paths: + text = text.replace(str(path), "") + return text + + +def test_folder_step_shows_the_design_by_default(tmp_path): + for name, kwargs in [("default", {}), ("explicit", {"show_design": True})]: + root = design_tree(tmp_path / name / "exp", ["ctrl/rep1", "drug/rep1"]) + nb = PreprocessingNotebook(str(root)) + page = nb.folder_step(**kwargs) + assert page.children[1].children[3:] == ( + nb.folder_card, + nb.output_card, + nb.exclude_box, + nb.check_button, + nb.folder_status, + nb.folder_tiles, + nb.labels_card, + nb.design_card, + nb.consistency_card, + nb.issues_card, + nb.folder_next, + ) + assert "Step 3 can only run once the labels are confirmed." in ( + nb.labels_confirmed.prompt.value + ) + nb.check_button.click() + nb.labels_confirmed.button.click() + assert ui.is_visible(nb.design_card) and nb.design is not None + assert "Step 3 runs once every group is confirmed." in nb.design_status.value + assert "Confirm the biological repeats in Step 1" in nb.folder_next.value + assert "Still to do: confirm the biological repeats" in nb.readiness.value + path = tmp_path / name / "exp_MorphoBricks_Preprocessing" / "experiment_design.csv" + assert f"Saved in {path}" in nb.design_status.value + assert not read_experiment_design(path).confirmed + + +def test_folder_step_without_the_design_writes_nothing(tmp_path): + root = design_tree(tmp_path / "exp", ["ctrl/rep1", "drug/rep1"]) + before = files_under(tmp_path) + nb = PreprocessingNotebook(str(root)) + nb.folder_step(show_output=False, show_next=False, show_design=False) + nb.axis_step(show_next=False) + nb.check_button.click() + nb.labels_confirmed.button.click() + nb.preview_button.click() + nb.preview_checked.button.click() + nb.confirm_button.click() + assert nb.labels_confirmed.value and nb.confirmed_mapping is not None + assert nb.design is None + assert files_under(tmp_path) == before + assert not (tmp_path / "exp_MorphoBricks_Preprocessing").exists() + + +def test_folder_step_without_the_design_card_or_its_words(experiment): + nb = PreprocessingNotebook(str(experiment)) + page = nb.folder_step(show_design=False) + nb.axis_step() + nb.run_step() + assert nb.design_card not in page_parts(page)[2] + nb.check_button.click() + nb.labels_confirmed.button.click() + found = page_parts(page)[2] + assert nb.design_card not in found and not ui.is_visible(nb.design_card) + text = page_text(found, experiment.parent) + assert "Step 3" not in text and "design" not in text.lower() + assert "biological repeats" not in nb.folder_next.value + assert "Go to Step 2" in nb.folder_next.value + nb.preview_button.click() + nb.preview_checked.button.click() + nb.confirm_button.click() + nb.image_type_checked.button.click() + assert "Experiment design" not in nb.axis_next.value + # This notebook's own step 3 then runs without a design. + assert "biological repeats" not in nb.readiness.value and not nb.run_button.disabled + # Building the default page again asks for the design of the confirmed labels. + nb.folder_step() + assert ui.is_visible(nb.design_card) and nb.design is not None + assert "Step 3 can only run" in nb.labels_confirmed.prompt.value + + def test_steps_without_an_intro_box(): nb = PreprocessingNotebook() assert intros(page_parts(nb.folder_step(intro=""))[2]) == []