Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,8 @@ src/morphobricks_preprocessing/
notebook_style.py notebook look and feel: cards, status cards, tiles, tables, ConfirmToggle, Browse dialogs
notebook_ui.py PreprocessingNotebook: the four notebook steps (needs the `notebook` extra);
folder_step/axis_step take title, purpose, intro and show_next (and
show_output for step 1) so other bricks can embed them (plan §12 Q26)
show_output and show_design for step 1) so other bricks can embed them
(plan §12 Q26, Q31)
notebooks/MorphoBricks_Preprocessing.ipynb thin notebook: every code cell calls notebook_ui only
tests/ pytest with simulated TIFFs (tests/tiff_tree.py, tests/simulation.py); the
schema-version-2 fixture (tests/v2_fixture.py, tests/data)
Expand Down Expand Up @@ -370,7 +371,8 @@ functions the tests use.
gain a folder, drops removed folders.
- **Notebook** writes the file after every answer once the labels are confirmed; a
confirmed design for the same folders is folded away behind **Change the experiment
design**; step 3 is gated on it. **CLI** `run` reads it (or `--design`), else writes the
design**; step 3 is gated on it. `folder_step(show_design=False)` (embedding, Q31) asks
and writes nothing, and step 3 runs without a design. **CLI** `run` reads it (or `--design`), else writes the
reconciled template and exits 1. **API** `run_experiment(design=None)` is optional; with
a design it is written with the table and `experiment_design_sha256` goes in the manifest.

Expand Down Expand Up @@ -560,11 +562,14 @@ plainer version "not interactive enough". Descriptive, guided and forgiving is t

### Page layout (`notebook_style`)

- **Embedding steps 1 and 2 (Q26):** `folder_step` and `axis_step` keep this notebook's
- **Embedding steps 1 and 2 (Q26, Q31):** `folder_step` and `axis_step` keep this notebook's
words by default; another brick (MorphoRelate) passes its own `title`, `purpose` and
`intro` (HTML; `""` leaves the box out), `show_output=False` to hide the output folder
card (shown again when the check reports `output_inside_experiment`), and `show_next=False` to hide the "What next?" box naming this notebook's
steps. The widgets and their behaviour are the same either way.
steps. The widgets and their behaviour are the same either way. `show_design=False`
leaves out the "Experiment design" card: confirming the labels then writes nothing
(no `experiment_design.csv`, no output folder), the step's texts name neither the design
nor step 3, and step 3 runs without a design.
- **Page and cards:** `ui.page(title, subtitle, children)` wraps a step in a grey page
(`mb-root`) with a title and a one-sentence purpose. `ui.card(header, children)` gives a
white card with an upper-case header. `ui.intro(html)` adds a blue "what happens here"
Expand Down Expand Up @@ -731,7 +736,9 @@ jupyter lab notebooks/MorphoBricks_Preprocessing.ipynb
is pinned to its v0.2.0 (question 28). The next tag, v0.2.0, follows follow-ups P4 and
P5.
- **Since v0.1.0:** follow-ups P4 (the image type, question 29) and P5 (intensity checks
and plots, question 30) of the normalization plan. The owner tags v0.2.0 next.
and plots, question 30) of the normalization plan, and step 1 embeddable without the
experiment design, `folder_step(show_design=False)` (question 31). The owner tags v0.2.0
next.
- **Not yet verified:**
- the exact look of the notebook in a real JupyterLab;
- the optional `jl-hidecode` JupyterLab extension, which keeps code cells collapsed. It
Expand Down
2 changes: 1 addition & 1 deletion docs/contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ folder.
| Controls | None, one or several conditions; every row of a condition has the same value. Recorded only. |
| Reruns | The file in the output folder is read and brought up to date with the folders found: folders in it keep their group, control mark and confirmation (added or removed images within a folder keep it too, with the counts recorded at confirmation); a new folder joins the group named after it and inherits its condition's control mark, and that group is asked again; folders no longer found are dropped without asking again. |
| Reading | `read_experiment_design` reads labels as text; `control` and `confirmed` accept `true`/`false` in any case, also `yes`/`no` and `1`/`0`, and empty as false; an empty `repeat_group` means the folder name; a byte-order mark is ignored. A missing column, an invalid value, a folder listed twice, rows of one condition disagreeing on `control` or another `schema_version` raise `DesignFormatError`. `ExperimentDesign.repeat_groups()` gives the `{(condition, biological_repeat): repeat_group}` mapping the normalization brick takes. |
| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. Step 3 runs only once every group is confirmed. |
| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. Step 3 runs only once every group is confirmed. A brick embedding step 1 with `folder_step(show_design=False)` (plan §12 question 31) gets no design: nothing is asked or written, and step 3 runs without one, as `run_experiment(design=None)`. |
| Command line | `run` reads the file from the output folder, or `--design FILE`. Without a fully confirmed design it writes the design brought up to date with the folders (a template with `confirmed=false` for the groups still to confirm, keeping the answers already given) to the output folder, explains the columns, and stops with status 1. Confirmed rows without a date get the date of the run. |
| Run | `run_experiment(..., design=...)` brings the design up to date with the folders and stops with `design_not_confirmed` or `repeat_group_conflict` if it is not fully confirmed. Otherwise the file is written with the table and published before the manifest, which records its SHA-256 (§9). Without a design the run is unchanged: no file is written, an existing one is left as it is, and `experiment_design_sha256` is null. `plane_histograms.csv` does not depend on the design. |

Expand Down
28 changes: 26 additions & 2 deletions docs/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,8 @@ API, CLI). The notebook and CLI only call the API.
read as a one-member level, physical sizes with a unit per axis in the image header
(TIFF and OME-Zarr, from the OME-NGFF scale), and steps 1 and 2 of the notebook
embeddable by another brick. Outputs are unchanged. *Implemented, see questions 23 to
26.*
26.* After v0.1.0 (not released yet), step 1 embeds without the experiment design
(`show_design=False`). *Implemented, see question 31.*
- **Channels across columns (follow-up P3b of the normalization plan):**
`plane_histograms.csv` schema version 3, written with the normalization brick's
Milestone 2b (now pinned to its v0.2.0, question 28): one row per XYC plane with each channel in its own
Expand Down Expand Up @@ -668,7 +669,8 @@ Raised during review; to be decided before the milestones that need them:
(optional)" card; the check then uses the folder in its box, the default when empty,
and the card is shown again when the check finds that folder inside the experiment,
so the box to fix it is never hidden). The defaults give the pages as before, and this notebook calls them
without arguments; the widgets, the checks and the gating are the same either way.*
without arguments; the widgets, the checks and the gating are the same either way.
Later, `show_design` leaves out the experiment design (question 31).*
27. How does this brick follow the channels moving into columns (normalization plan
N41–N44, its Milestone 2b, follow-up P3b)?
*Decided (October 2026): `plane_histograms.csv` is written in schema version 3 with
Expand Down Expand Up @@ -748,3 +750,25 @@ Raised during review; to be decided before the milestones that need them:
kind with the action once per kind, takes `--outlier-limit`, `--saturation-limit` and
`--min-images`, and draws no plots. Comparisons stay within a condition, as the brick
does. `plane_histograms.csv` and `experiment_design.csv` do not change.*
31. Since follow-up P3 (question 22), step 1 asks for the experiment design once the labels
are confirmed: an "Experiment design" card, `experiment_design.csv` written in the
output folder (`<images>_MorphoBricks_Preprocessing/` by default, created beside the
image folder), and texts saying that step 3 runs once every group is confirmed.
MorphoRelate milestone M3's "your data" notebook embeds `folder_step` (question 26)
from tag v0.1.0; it does not use the file, and its step 3 is another step. Can a
brick embedding step 1 leave the design out?
*Decided (27 October 2026, the owner's choice of an option here rather than a
workaround in MorphoRelate): yes. `folder_step` takes `show_design` (default True).
With False the "Experiment design" card is left out of the page; confirming the
labels asks nothing, `design` stays None and nothing is written, so neither
`experiment_design.csv` nor the output folder is created; the texts of the step no
longer name the design or step 3 (the labels question drops "Step 3 can only run once
the labels are confirmed", and the "What next?" box goes from the labels to step 2).
The rest of the notebook follows: step 2's "What next?" box does not send the user
back to the design, and step 3, if shown, does not wait for it and runs without a
design, as `run_experiment(design=None)` does (no file written, an existing one left as
it is). Building the page again with the other value opens or closes the design for
labels already confirmed. With the default the page, its texts and its files are as
before. Nothing else in step 1 writes files: the folder check and the header
inspection only read, and with `show_output=False` the output folder is only named.
The CSV, the manifest and their versions are unchanged.*
14 changes: 10 additions & 4 deletions src/morphobricks_preprocessing/notebook_style.py
Original file line number Diff line number Diff line change
Expand Up @@ -294,10 +294,9 @@ def __init__(
self._confirmed_text = confirmed_text
self._value = False
self._callbacks: list = []
self.prompt = widgets.HTML(
f"<div class='mb-question'><b>❓ {html.escape(question)}</b>"
f"<div class='mb-help' style='margin-bottom:0'>{explanation}</div></div>"
)
self._question = question
self.prompt = widgets.HTML()
self.explain(explanation)
self.button = button(f" {confirm_label}", style="success", icon="check", main=True)
self.message = widgets.HTML()
self.widget = widgets.VBox([self.prompt, self.button, self.message])
Expand Down Expand Up @@ -330,6 +329,13 @@ def disabled(self) -> bool:
def disabled(self, disabled: bool) -> None:
self.button.disabled = disabled

def explain(self, explanation: str) -> None:
"""Replace the text under the question (HTML), keeping the answer."""
self.prompt.value = (
f"<div class='mb-question'><b>❓ {html.escape(self._question)}</b>"
f"<div class='mb-help' style='margin-bottom:0'>{explanation}</div></div>"
)

def observe(self, callback) -> None:
"""Call ``callback(value)`` whenever the answer changes."""
self._callbacks.append(callback)
Expand Down
47 changes: 38 additions & 9 deletions src/morphobricks_preprocessing/notebook_ui.py
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,7 @@ def __init__(self, experiment_folder: str = "", output_folder: str = ""):
self._checked_folder = ""
self._preview_shown = False
self._show_output = True
self._show_design = True
self._layouts_cache: tuple[tuple, dict[str, PlaneLayout]] | None = None
self._build_folder_step(str(experiment_folder), str(output_folder))
self._build_axis_step()
Expand Down Expand Up @@ -265,12 +266,14 @@ def folder_step(
intro: str | None = None,
show_output: bool = True,
show_next: bool = True,
show_design: bool = True,
) -> widgets.Widget:
"""Step 1: choose and check the experiment folder, confirm the labels.

The defaults give this notebook's step. Another brick embedding the step
(it checks the folder and the image headers, nothing more) can give its
own words and leave out the output folder of this brick's histograms.
own words and leave out the output folder of this brick's histograms and
its experiment design.

Parameters
----------
Expand All @@ -290,6 +293,11 @@ def folder_step(
show_next
False hides the "What next?" box, which names this notebook's next
steps.
show_design
False leaves out the "Experiment design" card: once the labels are
confirmed nothing is asked and no ``experiment_design.csv`` (nor its
output folder) is written, the step's texts no longer name the design
or this notebook's step 3, and step 3 runs without a design.

Returns
-------
Expand All @@ -299,6 +307,11 @@ def folder_step(
ui.set_visible(self.folder_next, show_next)
self._show_output = show_output
ui.set_visible(self.output_card, show_output)
self.labels_confirmed.explain(_labels_explanation(show_design))
if show_design != self._show_design:
self._show_design = show_design
# Labels confirmed on an earlier page open or close the design to match.
self._labels_changed()
if intro is None:
intro = (
"<b>What happens here:</b> the folders are scanned, the labels are taken "
Expand All @@ -322,7 +335,7 @@ def folder_step(
self.folder_status,
self.folder_tiles,
self.labels_card,
self.design_card,
*([self.design_card] if show_design else []),
self.consistency_card,
self.issues_card,
self.folder_next,
Expand Down Expand Up @@ -496,9 +509,7 @@ def _build_folder_step(self, folder: str, output: str) -> None:
self.labels_table = widgets.HTML()
self.labels_confirmed = ui.ConfirmToggle(
"Do the labels in the table match your experiment?",
"Check that every image folder has the right condition, biological repeat and "
"technical repeat. If a label is wrong, rename the folder and click <b>Check "
"folder</b> again. Step 3 can only run once the labels are confirmed.",
_labels_explanation(show_design=True),
"Yes, the labels are correct",
"Labels confirmed.",
"Undo: the labels are not right",
Expand Down Expand Up @@ -725,7 +736,7 @@ def _show_issue_summary(self) -> None:
self._refresh_folder_next()

def _labels_changed(self) -> None:
if self.labels_confirmed.value and self.preflight is not None:
if self.labels_confirmed.value and self.preflight is not None and self._show_design:
self._open_design()
else:
self._close_design()
Expand Down Expand Up @@ -974,11 +985,15 @@ def _refresh_design_status(self) -> None:
def _design_confirmed(self) -> bool:
return self.design is not None and self.design.confirmed

def _design_pending(self) -> bool:
"""Return True while the design shown in step 1 still has questions to answer."""
return self._show_design and not self._design_confirmed()

def _refresh_folder_next(self) -> None:
self.folder_next.value = ""
if self.preflight is None or not self.preflight.ok:
return
if self.labels_confirmed.value and not self._design_confirmed():
if self.labels_confirmed.value and self._design_pending():
self.folder_next.value = ui.next_steps(
"Confirm the biological repeats in Step 1.",
"Under Experiment design, pair the repeats if needed and answer each yellow "
Expand Down Expand Up @@ -1541,7 +1556,7 @@ def _refresh_axis_next(self) -> None:
"Go to Step 1 and confirm the folder labels.",
"Review the labels and click Yes, the labels are correct before running Step 3.",
)
elif not self._design_confirmed():
elif self._design_pending():
self.axis_next.value = ui.next_steps(
"Go to Step 1 and confirm the biological repeats.",
"Under Experiment design, answer each yellow question before running Step 3.",
Expand Down Expand Up @@ -1909,7 +1924,11 @@ def _refresh_run_step(self) -> None:
items = [
(folder_ok, "Check the experiment folder and fix any errors (step 1)"),
(pre is not None and self.labels_confirmed.value, "Confirm the labels (step 1)"),
(self._design_confirmed(), "Confirm the biological repeats (step 1)"),
*(
[(self._design_confirmed(), "Confirm the biological repeats (step 1)")]
if self._show_design
else []
),
(self.confirmed_mapping is not None, "Confirm the axis roles (step 2)"),
(self.confirmed_image_type is not None, "Confirm the image type (step 2)"),
]
Expand Down Expand Up @@ -3043,6 +3062,16 @@ def __exit__(self, *exc):
return False


def _labels_explanation(show_design: bool) -> str:
"""Return what to check before confirming the labels; step 3 is named only with the design."""
text = (
"Check that every image folder has the right condition, biological repeat and "
"technical repeat. If a label is wrong, rename the folder and click <b>Check "
"folder</b> again."
)
return (text + " Step 3 can only run once the labels are confirmed.") if show_design else text


def _group_question(design: ExperimentDesign, group: str) -> str:
"""Return the question asked for a repeat group, or for the whole experiment."""
if design.single_repeat:
Expand Down
Loading
Loading