Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,26 @@ with `read_experiment_design`. They also ask for the image type, `fluorescence`
`image_type=`. After writing, the run checks the images' intensities from the table
(`result.intensity`); `check_table(path)` checks any table.

## Command line

```bash
morphobricks-preprocessing check /data/exp1 # report, suggested --axes, design template
morphobricks-preprocessing design --output /data/exp1_MorphoBricks_Preprocessing --confirm-all
morphobricks-preprocessing run /data/exp1 --axes ZCYX
morphobricks-preprocessing run --output /data/exp1_MorphoBricks_Preprocessing # later runs
morphobricks-preprocessing masks --output /data/exp1_MorphoBricks_Preprocessing # mask step
morphobricks-preprocessing calibrate --output /data/exp1_MorphoBricks_Preprocessing
```

`check` writes the report and, when the output folder holds no design, an
`experiment_design.csv` prefilled from the folders. `design` prints it and confirms repeat
groups (`--confirm GROUP`, `--confirm-all`), marks controls (`--control`, `--no-control`)
and pairs repeats (`--repeat-group CONDITION/REPEAT=GROUP`) without editing the file;
nothing is confirmed unless asked. `run` and `masks` reuse every input the run in the
output folder recorded. A run that stops writes its report and says so. Every output file
gets the permissions a file you save yourself would (your umask). Details in the
[user guide](docs/user_guide.md) §5.

## Example

```python
Expand Down
26 changes: 21 additions & 5 deletions docs/contract.md

Large diffs are not rendered by default.

77 changes: 77 additions & 0 deletions docs/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -1158,3 +1158,80 @@ Raised during review; to be decided before the milestones that need them:
OME-Zarr image whose `labels` group cannot be read is its own refusal naming the image,
not a silent pass and not an "unrecorded" file; (4) the version stays 0.3.4 until PP-13,
and v0.4.0rc2 is tagged by the owner after this PR.*
52. Which permissions do the output files get (PP-8c, from a run on real data)?
*Decided (2 November 2026, owner): the mode a plain `open()` gives. The atomic write's
temporary file (`tempfile.mkstemp`) is private, so every output was `0600` whatever the
umask, and colleagues could not read the results in a shared lab folder. Now a new file
gets `0666 & ~umask` (the umask read from `/proc/self/status` where the system gives
it, else by setting it under a lock, to the most restrictive value while it is set), and
a file that already exists keeps its own mode, as the user set it: the run never widens
or narrows what the user chose. Files written `0600` by earlier versions therefore stay
so until the user changes them; the user guide says how. The table, written by the
normalization brick's writer into the staging folder, is given its mode when it is
published. Windows: unchanged (no such modes). Contract §9.*
53. Is a pixel size recorded for an axis the image does not have (PP-8c, from a run on
real data)?
*Decided (2 November 2026, owner): no. MorphoRelate's HPMEC max projections (`CYX`)
keep the z-step of their stacks in their ImageJ metadata (0.43 µm), and the run
recorded it as their Z size from the header. A Z or T size is now recorded only for an
image whose roles include that axis (`calibration.fit_to_roles`, after the axis
review; a singleton role of a missing axis does not count); otherwise the axis has no
value and no source. A `calibration.csv` statement for an axis an image lacks is
neither refused nor recorded: the warning `calibration_axis_absent` names the images
(refusing would make a whole-experiment Z statement impossible in an experiment that
mixes stacks and 2D images; the question 49 rule that a statement matching nothing is
an error is about targets, not axes). The report's text counts Z and T sizes for the
images with the axis. The `RowKey` does not include pixel sizes, so earlier rows are
reused (no row depends on a size, question 49): the next `restart` leaves the table
byte-identical and rewrites the manifest without the absent sizes. Contract §9.3.*
54. What do `RunInputs.read_plane` and `read_mask` do with a `z` or `t` that is not a number
(PP-8c, from a run on real data)?
*Decided (2 November 2026, owner): a `TypeError` naming the expected arguments. A
`plane_id` passed as `z` gave the misleading `IndexError` "has no Z …" on a 2D image,
and `read_mask(z=True)` reached the reader. Python and NumPy integers are accepted
(`bool` is not), and `read_mask` keeps None. A read by `plane_id` is not added: the
spec's interface is `read_plane(fov, z, t)` (design v0.4 §6), every table row carries
its `z` and `t`, and a second way to address a plane would be a second rule to keep
in step. Contract §9.1.*
55. What does a run that stops say it wrote (PP-8c, from a run on real data)?
*Decided (2 November 2026, owner): what it wrote. Since PP-7 every run that holds the
lock writes its report, also when it stops, and the CLI `run` writes the design template
when it stops on the design, yet the summary said "Nothing was written". It now says
"The run stopped (…): no table or manifest was written." and names the report files and
the template when written (`RunResult.report_written`, `run_summary(design_template=)`);
the notebook says "The run stopped: no table or manifest was written." Only a run
refused before it starts (the lock) still says "Nothing was written."*
56. Does the `masks` command use the recorded inputs (PP-8c, from a run on real data)?
*Decided (2 November 2026, owner): yes, like `run` (question 48). The experiment
argument is optional when the output folder holds a v2 manifest recording it (else
`experiment_not_recorded`); the masks folder and mask axes are the recorded ones unless
given, and a value given that differs is named with its effect. The exclusions follow
the same rule (with `--no-exclude`), so `masks` never reads a mask of an image the run
left out unless asked. `--no-masks` is not needed. A change is used for that scan only:
`masks` writes only `mask_relationships.json`, never the manifest, so the next `run`
keeps the recorded value until it is given the new one, and then removes a
`mask_relationships.json` written for other masks (question 48). The recorded-experiment
lookup is one helper shared by both commands (`cli_run.use_recorded_experiment`).*
57. Where does the experiment design template come from, and can it be confirmed without
editing the CSV (PP-8c, from a run on real data)?
*Decided (2 November 2026, owner): earlier, and from the command line. `check` also
writes `experiment_design.csv` prefilled from the folders (conditions, biological
repeats, `repeat_group` the folder name, counts; `confirmed=false`, `control=false`)
when the output folder holds none, under the lock, and only when the folders have no
error; it never replaces a design, edited or unreadable. This amends design v0.4 §1
("`check` … writes only the report"). A new command `design --output OUT` prints the
design and what is still unconfirmed, and changes it without editing the file:
`--repeat-group CONDITION/REPEAT=GROUP`, `--control`/`--no-control CONDITION`,
`--confirm GROUP` and `--confirm-all` (a separate flag rather than `--confirm all`, so a
repeat group named `all` stays confirmable). It reads, changes and writes under the lock
with the run's reader (`read_experiment_design`) and validator (`design_issues`), fills
`confirmed_on`, refuses unknown names and a group whose rows break the rules, and
changes nothing when it refuses. It does not rediscover the folders: it works on the
rows `check` or `run` wrote, and the run reconciles them with the folders as before, so
a group that gained a folder since is asked again. Python: `edit_design` with
`DesignEdit` (`apply_design_edit` without files). The confirmation stays an explicit
act of the user: nothing confirms by default. Contract §12.*
*Decided in review (2 November 2026, owner): `design` and `calibrate` also take
`--remove-stale-lock`, as `check`, `run` and `masks` do (the `output_locked` message
tells users to give it to the command). Same rule: the lock is removed only on that
request, and never one of a process still running on the same computer.*
Loading
Loading