Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,9 +314,21 @@ src/morphobricks_preprocessing/
(plan §12 Q26, Q31). Since PP-9 it holds the page methods, __init__ and
the shared helpers; the step methods live on mixin classes in the modules
below, and it re-exports every public name it had (`__all__`)
notebook_steps.py steps(): the shipped notebook's 13 steps, one at a time (PR 17a; pure
composition of the cards below); notebook_stepper.py the strip, Back/Next
and the overview; notebook_step_states.py each step's done/one-line state
notebook_steps.py steps(): the shipped notebook's 12 steps, one at a time (PR 17a; pure
composition of the cards below); notebook_stepper.py the strip, Back/Next,
the overview, the compact page (one screen per step: the cards scroll in
their own box) and about()/field_rows(); notebook_step_states.py each
step's done/one-line state
notebook_compact.py the Images and Run steps arranged for one screen: one Folders card, the
collapsed panels, and Compare groups (the former statistics page) in the
run step (redesign of 10 November 2026)
notebook_progress.py Wait: one line per wait (done of total, item, elapsed, time left once
measurable); progress_html, time_left (pure parts)
notebook_quick_scan.py the first step's quick scan: the folder walk only, one random image for
looking only, then the explicit click that reads every header
axis_example.py which planes the axes step's example shows (pure): one tile per channel,
a few time points or slices; notebook_plots.example_png draws them
pixel_size_summary.py the pixel size in one line, which images differ and where (pure)
notebook_names.py the names card's rows (conditions on step 4, replicates on step 5)
notebook_axes_quick.py the stepper's one-click Confirm these axes (Q3, PR 17b)
notebook_row_fold.py reopen: unchanged, answered rows fold; new or changed rows orange
Expand Down Expand Up @@ -747,9 +759,12 @@ Technical details go to `logging` (shown in the notebook's "Technical details" p
| Masks (Q32) | Optional masks folder outside the experiment, one subfolder per mask type mirroring the experiment's folders, or the experiment's folders with `<image>_<type>.tif` (both may be mixed; `ambiguous_mask` when a name could fit two images or both layouts), paired by leaf folder + `position_id`; OME-Zarr label images are masks too. Header-only errors (`masks_folder_not_found`, `masks_inside_experiment`, `mask_names_differ_in_case`, `duplicate_mask`, `ambiguous_mask`, `unreadable_mask`, `mask_not_2d`, `mask_size_mismatch`, `mask_pixel_type`) and warnings (`mask_missing`, `mask_without_image`, `mask_outside_type_folder`) in `check`/`run`; manifest key `masks` (optional, version 1). Optional mask step (`masks` command, `scan_masks`): kinds, relations with evidence, direct-parent hierarchy, `mask_relationships.json` (report version 1); default thresholds the user may change (`thresholds=` keyword, one CLI option each, checked by `mask_thresholds`, written in the report); suggestions only, the user names and confirms. SciPy core dependency. Confirmed by the owner: 2D masks at the image's YX, integer/bool pixels, masks folder outside the experiment. Next: notebook step and a file of the user's answers; tag v0.2.0 proposed after merge. Plane identity (issue #36): masks may have Z and/or T named in their own metadata (never guessed: unnamed axes are `mask_not_2d`); the masks of an image share their planes (`mask_planes_differ`) and `run` checks them against the image's confirmed roles (`mask_planes_mismatch`, `mask_plane_issues`), so a 2D mask is refused for a Z-stack or time-lapse; manifest mask entries gain `axes` and `shape` (optional keys, version 1); `read_mask(z=, t=)` reads one plane; `scan_masks` scans plane by plane |
| Notebook style | Follow the lab's NucleiSky and LabChronicle notebooks (see §7) |
| Widget widths | **99%**, never 100% |
| Waits | Every wait over about a second shows `notebook_progress.Wait`: done of total, the item, elapsed time, time left once measurable. Readers take a `progress` callback; no bare spinner (owner, 10 November 2026) |
| Disabled actions | Every disabled button has a visible `notebook_render.blocked_line` beside it saying what blocks it; a tooltip alone is a hidden refusal (owner, 10 November 2026) |
| One screen per step | The shipped notebook's steps fit 1366 x 768: one row of chips, explanations under *About this step*, details in collapsed panels, long lists scrolling in their own box (owner, 10 November 2026) |
| Paths and filenames in widget HTML and notebook text | **`<b>…</b>` / `**…**`, never `<code>` or inline code** |
| Things the user must confirm | **Large buttons** in a highlighted question box (`ConfirmToggle`), never small checkboxes |
| Statistics step | "Step 4 · Calculate the statistics (optional)": clearly optional, writes nothing. A CSV chooser (Browse…) is prefilled with the table just computed, otherwise empty. Not calculated automatically |
| Statistics step | **Shipped notebook since v0.5.1: the collapsed *Compare groups* panel of the run step, for that run's table; no table chooser, no second intensity check (owner, 10 November 2026). The page below remains for `results_step()` without `steps()`.** "Step 4 · Calculate the statistics (optional)": clearly optional, writes nothing. A CSV chooser (Browse…) is prefilled with the table just computed, otherwise empty. Not calculated automatically |

---

Expand Down
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,47 @@ OME-XML description is parsed (which images are marked is unchanged; a malformed
debug line without a traceback), and while the capture is attached the package's DEBUG records
go to the "Technical details" panel only, with WARNING and above still reaching the root handlers.

### Notebook redesign 4: one column, less on screen; Compare groups in the run step

Owner test of v0.5.0 (10 November 2026), items (a) and (b), and the owner's test of the first
redesign the same day, items (m) and (n): "simple and minimalistic"; no box that scrolls inside
the page; always one column. Layout only: no answer, check, refusal, draft, description or run
output changes.

- **One column, and the page scrolls as a whole.** Nothing in a step scrolls on its own (no
fixed-height step body, no table or file list with its own scroll bar), and no cards, panels
or columns sit side by side. A step is short because it says less: its explanation is folded
under **About this step**, a card's help paragraph is one line that opens when pointed at,
details are collapsed panels, and cards, tiles, tables, buttons and pictures are denser. The
strip is one row; a step's title and purpose are one line.
- **Images:** one *Folders* card with three labelled rows (images, masks, output); **Scan the
folders**, what it found, then **Looks right: read all image headers**, one under the other.
The scan shows the conditions in a short table (every folder under *All N folders*). After
the check the scan makes way for the result, and the pictures per folder, the check report
and *Exclude images* are collapsed panels.
- **The quick scan shows every channel of its one image side by side** across the page, each
with its own contrast and its `c=` index, when that image's header names a channel axis
(`header_preview.header_channels`): up to six, and with seven or more it shows six and says
"6 of N channels shown". Display only, read from that one image; a header that names no
channel axis gets the one picture as before (no channel is guessed from the shape).
- **Axes:** one row per dimension (what the file has, its role, what the role does), then the
example, then the pixel size, one under the other. A refusal of the roles is one sentence
with *Files and what to do* on demand (`notebook_render.issues_brief_html`), e.g. "Error ·
different channel counts: The images have different numbers of channels: 3 channels (10
images), 4 channels (4 images)."
- **Run:** *Before you run* is shown only while something is missing; *Settings and mode* and
*What will happen* are collapsed; after a run the intensity checks and plots, **Compare
groups** and the report are collapsed panels.
- **Compare groups replaces the optional statistics page** in the shipped notebook: the
grouping, the percentiles, the pooled statistics and the distribution plot of the table this
run wrote. There is no table to choose and the intensity checks are not repeated.
`results_step()` after `steps()` returns an empty hidden box, so the shipped notebook's cell
needs no change; called without `steps()` it is the page it was.
- **`welcome()` is one line** (name, version, "the notebook is ready"); the *What next?* boxes
are not shown in the stepper.
- The stacked pages other bricks embed (`folder_step`, `axis_step`) get the one-column axes
layout (roles, example, then the named-axis copies card); nothing else changes for them.

### Notebook redesign 3: the axes step shows the data as the roles read it; pixel size in one line

Owner test of v0.5.0 (10 November 2026), items (e) and (f). The shipped notebook has 12 steps:
Expand Down
5 changes: 3 additions & 2 deletions docs/colab-release-check.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,15 +93,16 @@ The notebook has three code cells, run in order: **1. Install MorphoBricks prepr
Step 1 without running. The button is enabled, names the earlier run, and works.
14. - [ ] Move the masks folder on Drive and click: a red card says the masks folder does not
exist, and nothing is written.
15. - [ ] The tables scroll inside the card, with no horizontal page scrollbar.
15. - [ ] The page scrolls as a whole: no box inside it has its own scroll bar, and there is no
horizontal page scrollbar.

## E. Make named-axis copies (PR #127)

16. - [ ] Run Step 0, then Step 1 on an experiment of Y/X-swapped OME-TIFFs on Drive, and
press **Check folder**.
17. - [ ] In Step 2, swap the Y and X roles. The red axis check appears, with the **Make
named-axis copies (optional)** card under it. Its button is disabled.
18. - [ ] Press **Show preview** and **Yes, the preview looks right**, then **Yes, write the
18. - [ ] Look at the example (drawn by itself) and press **Yes, the example looks right**, then **Yes, write the
copies with these roles**.
19. - [ ] Press **Browse…**: the amber card names **Copy path**. Paste a Drive folder inside
the experiment and check that it is refused. Then paste one next to the experiment.
Expand Down
59 changes: 59 additions & 0 deletions docs/user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,65 @@ notebook's step 3 (§4), also tells how they relate.

## 4. The notebook, step by step

### The app in short (v0.5.1)

Cell 3 opens the app: a strip of twelve steps, one on screen at a time, with **Back** and
**Next** under it (**Next** is never blocked; a chip opens its step). The page is one column
and scrolls as a whole: nothing scrolls inside its own box and nothing sits side by side.
Explanations are folded under **About this step**, and details are in collapsed panels.
Every wait shows a bar with what is done of the total, the file being read, the time elapsed
and, once it can be measured, the time left. Every greyed-out button has a line beside it
saying what it waits for.

1. **Images.** Give the folders (images; masks and output are optional). **Scan the folders**
lists the conditions and folders with their image counts without reading any image header,
and shows one image chosen at random, with its path, for looking only: it is not used for
anything. Every channel of it is shown side by side (`c=0`, `c=1`, …, each with its own
contrast; at most six, and then it says *6 of N channels shown*).
If this is your experiment, click **Looks right: read all image headers**; the
headers are then read and checked. Changing a folder asks for a new scan. The pictures of
each folder, the check's report and **Exclude images** are collapsed below.
2. **Axes.** One row per image dimension with its role, and under them an **example of one
image as these roles read it**: one tile per channel (`c=0`, `c=1`, …), a few time points
or slices, or the single plane, each with its height and width. It is drawn by itself and
redrawn whenever you change a role, so a wrong role looks wrong (a strip, stripes). Another
image or other planes are under **Another image, or other planes**. Nothing is confirmed
for you: click **Confirm these axes** (offered when the headers name every axis and
nothing is open), or **Yes, the example looks right** and then **Confirm axis roles**.
- **Pixel size** is one line here: for example *Pixel size: 0.108 × 0.108 µm (from the
headers, same in all 270 images)*. When the headers give none, type it once and click
**Use for all images** (or choose *pixels: deliberately uncalibrated*); leaving it empty
records the images as uncalibrated, nothing is guessed. When the headers do not agree,
the line says how many images have which size and in which folders, and **Set per
imaging unit…** opens: there you state a size for one folder (imaging unit) or image.
**Use for all images** is your statement for every image: it replaces sizes stated per
folder, and a size that differs from an image's header is still listed as a warning.
3. **Conditions**, 4. **Replicates**, 5. **Compared against**, 6. **Image type**, 7. **Stains**,
8. **Tag channels**, 9. **Masks**: one kind of answer per step, saved as you give it.
10. **Check and save.** Your experiment in sentences, one question per replicate, then **Yes,
save the experiment description**. The line beside the button says what is still missing.
11. **Run.** **Compute histograms**; what is still to do is listed only while something is.
The settings and *What will happen* are collapsed. After the run: its result, and collapsed
panels for the **intensity checks and plots**, **Compare groups** and the report.
**Compare groups** replaces the former optional statistics page: choose how to group
(condition, biological repeat, technical repeat or image) and click **Calculate
statistics** for the pooled statistics and the distribution plot of the table this run
wrote. It writes nothing.
12. **Describe the masks** (optional), after a run that recorded masks.

Images with different numbers of channels (for example three in most folders and four in
one) are refused in this version (`different_channel_counts`): the axes step names the error in
one sentence, with the files and what to do one click away, and **Confirm axis roles** says
that this is what it waits for.

The rest of this section describes each card in detail. It is written for the same cards
shown as stacked pages (as other bricks embed them); where the app differs, the list above
is what you see. In particular: **Show preview** and **Yes, the preview looks right** are the
example and **Yes, the example looks right**; **Check folder** is **Looks right: read all
image headers**; and step 4 below is the **Compare groups** panel.

### The cards in detail

The notebook's code cells are hidden: run them from top to bottom (click a cell and press
**Shift + Enter**). Each step is a page of boxes. A blue box at the top says what the step
is for, every control has a grey help line, and one coloured message says what happened
Expand Down
10 changes: 5 additions & 5 deletions notebooks/MorphoBricks_Preprocessing.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"- **Confirms the experimental design**: folder labels, biological-repeat pairing and controls.\n",
"- **Confirms image interpretation**: axis roles and fluorescence/RGB image type. Scientific metadata are never silently guessed.\n",
"- **Computes per-plane intensity histograms** used by downstream MorphoBricks.\n",
"- **Runs intensity QC** and provides optional pooled statistics after preprocessing.\n",
"- **Runs intensity QC** and, on request, compares groups after preprocessing.\n",
"\n",
"### Before you start\n",
"\n",
Expand Down Expand Up @@ -372,12 +372,12 @@
"\n",
"In the app:\n",
"\n",
"1. Give the **experiment folder** and optional **output/masks folders**, then click **Check folder**.\n",
"2. Go through the steps one at a time with **Next** (or click a step in the strip at the top): axes, pixel sizes, conditions, replicates, comparisons, image type, stains, channel tags and masks.\n",
"1. Give the **experiment folder** (masks and output folders are optional) and click **Scan the folders**. Look at the folders and the example image, then click **Looks right: read all image headers**.\n",
"2. Go through the steps one at a time with **Next** (or click a step in the strip at the top): axes and pixel size, conditions, replicates, comparisons, image type, stains, channel tags and masks.\n",
"3. On **Check and save**, read your experiment in words and save it; then **Run** computes the histograms.\n",
"4. Review the QC results; pooled statistics are optional.\n",
"4. After the run, the intensity checks and **Compare groups** are collapsed panels of the run step.\n",
"\n",
"Detailed explanations appear inside the app only where a decision is required."
"Every wait shows its progress, and a greyed-out button says beside it what it waits for."
]
},
{
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ baseline = "tools/code_rules_baseline.json"
"morphobricks_preprocessing.run_labels" = "io"
"morphobricks_preprocessing.notebook_axes" = "ui"
"morphobricks_preprocessing.notebook_progress" = "ui"
"morphobricks_preprocessing.notebook_compact" = "ui"
"morphobricks_preprocessing.notebook_quick_scan" = "ui"
"morphobricks_preprocessing.notebook_axes_quick" = "ui"
"morphobricks_preprocessing.notebook_channel_thumbs" = "ui"
Expand Down
Loading
Loading