Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,9 @@ src/morphobricks_preprocessing/
mask_hierarchy_lines); moved from report.py in PP-10d
discovery_report.py the run report (discovery_report.json and .txt, design v0.4 §4): status,
manifest SHA-256, what was found, reused, recomputed and replaced
cli.py, __main__ `morphobricks-preprocessing check|run|masks|calibrate` (also `python -m morphobricks_preprocessing`);
cli_run.py holds `run`'s options and output; cli_calibrate.py `calibrate`
cli.py, __main__ `morphobricks-preprocessing check|run|masks|calibrate|design|convert` (also `python -m morphobricks_preprocessing`);
cli_run.py holds `run`'s options and output; cli_calibrate.py `calibrate`;
cli_convert.py `convert`
masks/ masks (Q32; a package since PP-5: model, files, header, layout, read,
checks, labels): find_masks pairs every image with its masks (masks folder with one
folder per type mirroring the experiment, or <image>_<type>.tif in the
Expand All @@ -149,6 +150,18 @@ src/morphobricks_preprocessing/
scan_masks -> MaskReport (kinds, relations, direct-parent hierarchy, evidence),
write_mask_report (mask_relationships.json); DEFAULT_THRESHOLDS,
mask_thresholds (checked changes)
copy_axes.py convert (PP-11, contract §17): the axes, shape, pixel sizes (by role; a
relabelled axis gets an unknown scale; equal X and Y kept through a Y/X
swap, question 59) and OME metadata of a named-axis copy
ome_tiff_copy.py write_named_copy: one image as uncompressed OME-TIFF, planes from iter_planes
under the confirmed roles, blocks written in place (BigTIFF when large)
conversion_record.py the copy tree's .morphobricks-conversion.json (fingerprints, roles, sizes),
review_conversion (the run's conversion_source_changed and
copy_changed_after_conversion warnings)
conversion.py convert_experiment/plan_conversion: the copy tree under its lock, .partial
files moved into place together (any failure removes what it wrote),
planes checked against the memory limit first, unchanged copies reused,
record last; conversion_issues.py its messages (pure)
zarr_input.py OME-Zarr input: open_ngff (OME-NGFF 0.4/0.5 metadata -> level-0 array, axis
letters and physical sizes from the scale and axis units),
ngff_label_names/open_ngff_labels (label images, for later bricks),
Expand Down Expand Up @@ -825,6 +838,7 @@ pytest
morphobricks-preprocessing check /data/exp1
morphobricks-preprocessing run /data/exp1 --axes ZCYX
morphobricks-preprocessing masks /data/exp1 --masks /data/exp1_masks
morphobricks-preprocessing convert /data/exp1 --to /data/exp1_named_axes --axes ZXY
jupyter lab notebooks/MorphoBricks_Preprocessing.ipynb
```

Expand Down Expand Up @@ -893,6 +907,9 @@ jupyter lab notebooks/MorphoBricks_Preprocessing.ipynb
- **Notebook masks step (PP-10d):** step 3's **Describe the masks** runs the `masks` step
(`mask_relationships.json`) with the run's recorded inputs, through `run.mask_step`, which
the CLI calls too; see plan §10.
- **Named-axis copies (PP-11):** `convert` / `convert_experiment` writes OME-TIFF copies of
the images with axis errors or unnamed axes with the confirmed roles into a new folder
(contract §17, plan §10 and question 59); the notebook card is PP-11b.
- **Next (not started):** the notebook step for the masks (name each mask, confirm or
correct the relationships) and a file saving those answers for the other bricks
(question 32); 3D and time-lapse masks later.
Expand Down
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ Pre-alpha. Implemented and tested:
recorded per axis with their source; never guessed (`calibration`, command `calibrate`)
- Axis mapping: prefill from metadata, validation, applying one mapping to every
file (`axes`)
- Named-axis copies: images whose metadata name their axes wrongly or not at all saved as
OME-TIFF with the confirmed roles, pixel for pixel, in a new folder with a conversion
record (`conversion`, command `convert`)
- Reading TIFFs one 2D plane at a time, with a memory limit and a preview plane, and the
channels of a table row one after another; planes too large for the limit are read in
blocks with identical results (`planes`)
Expand All @@ -52,7 +55,8 @@ Pre-alpha. Implemented and tested:
refused, projections included (`mask_planes_mismatch`), and the manifest records each
mask's `axes` and `shape`
- A step-by-step notebook ([notebooks/MorphoBricks_Preprocessing.ipynb](notebooks/MorphoBricks_Preprocessing.ipynb))
and a command line (`morphobricks-preprocessing check` / `run` / `masks` / `calibrate`)
and a command line (`morphobricks-preprocessing check` / `run` / `masks` / `calibrate` /
`design` / `convert`)

From the normalization brick (`morphobricks_normalization`, a required dependency, pinned
to its release v0.3.2): per-plane histograms with the robust range and the range check
Expand Down Expand Up @@ -155,6 +159,7 @@ morphobricks-preprocessing run /data/exp1 --axes ZCYX
morphobricks-preprocessing run --output /data/exp1_MorphoBricks_Preprocessing # later runs
morphobricks-preprocessing masks --output /data/exp1_MorphoBricks_Preprocessing # mask step
morphobricks-preprocessing calibrate --output /data/exp1_MorphoBricks_Preprocessing
morphobricks-preprocessing convert /data/exp1 --to /data/exp1_named_axes --axes ZXY # axis errors
```

`check` writes the report and, when the output folder holds no design, an
Expand All @@ -163,7 +168,11 @@ groups (`--confirm GROUP`, `--confirm-all`), marks controls (`--control`, `--no-
and pairs repeats (`--repeat-group CONDITION/REPEAT=GROUP`) without editing the file;
nothing is confirmed unless asked. `run` and `masks` reuse every input the run in the
output folder recorded. A run that stops writes its report and says so. Every output file
gets the permissions a file you save yourself would (your umask). Details in the
gets the permissions a file you save yourself would (your umask). `convert` saves the
images whose metadata name their axes wrongly or not at all as named-axis OME-TIFF copies
with your roles in a new folder (each plane bit for bit the plane read with those roles;
every other image copied as it is; a conversion record written last), then you run
preprocessing on that folder with the same output folder. Details in the
[user guide](docs/user_guide.md) §5.

## Example
Expand Down
Loading
Loading