Skip to content

Z projection 3/6: plan and write projected copies (part of #176) - #213

Merged
guijacquemet merged 2 commits into
mainfrom
claude/z-projection-convert
Nov 8, 2026
Merged

guijacquemet merged 2 commits into
mainfrom
claude/z-projection-convert

Conversation

@guijacquemet

Copy link
Copy Markdown
Contributor

Stacked on #212, which is stacked on #211 (base is claude/z-projection-record, so the diff shows only this PR; GitHub retargets as the earlier ones merge).

What changed

Third PR of the plan for #176 (design: MorphoFeature docs/restart/designs/pp176-z-projection-design-2026-11-07.md; owner answers of 2026-11-07). convert_experiment(..., projection=Projection("max" | "mean" | "sum", z_range=None)) now writes recorded 2-D copies of Z-stacks. There is no command yet (PR 4) and no run-side change (PR 5).

  • Plan. plan_conversion(projection=, masks_folder=) lets z_stack_not_supported through only when projecting. Every non-excluded image is projected (a Z=1 image or one without a Z role is one slice), so a tree has one type and one provenance per image. Excluded images are still copied byte for byte. New modules: projection_plan.py (every refusal in one pass, before any write), projection_layout.py (the converted layout minus Z; the Z size goes to the record's dropped_sizes, X/Y/T sizes and channel names follow copy_layout), projection_issues.py (messages), projection_masks.py, planned_copy.py (PlannedCopy moved out of conversion_plan.py to avoid an import cycle).
  • Write. write_projected_copy (in ome_tiff_copy.py): one page per (t, c); slices z_range read in ascending order with read_plane and added into one running plane (PlaneAccumulator, Z projection 1/6: the pure arithmetic (part of #176) #211); one input plane and the running plane are held. The source's content SHA-256 is taken before and after the projection (a change is refused) between the usual fingerprint checks, and recorded. A sum that overflows to infinity stops the conversion (projection_overflow) and nothing is kept.
  • Refusals, all before anything is written (plain messages with an action): nothing_to_project, projection_z_range_outside (names the shorter images), projection_cannot_be_exact, cannot_convert_pixel_type (64-bit integers), projection_over_memory_limit, projection_masks_3d and projection_masks_axes_unknown (masks folder scanned from headers only; masks are never projected; 2-D masks are left alone), and projection_zarr_not_supported (see below). projection_overflow and projection_undefined come from the pixels, during the write, and still leave the tree as it was.
  • Reuse. Another method or range into the same tree is copy_differs and the message names both ("the folder holds max projections of all slices, and you asked for mean projections of all slices"); an identical rerun reuses every copy and leaves the record as it was.
  • Calibration and provenance. The Z size is never written to the copy. The copy's OME-XML holds an Image/Description (method, slices, source, source SHA-256, types, rounding, NaN rule) and a UUID derived from it, so one projection of one source is the same bytes twice (tifffile otherwise puts a random UUID in every file, which made two conversions differ).
  • Docs: contract §17, user guide (API paragraph and the nine new message codes), AGENTS repo map, CHANGELOG.

Answers to the open points

  • Can tifffile write the OME-XML description the design needs? Yes. metadata={"Description": ...} becomes <Image><Description> (checked with tifffile 2026.10.15, and the key is handled by OmeXml.addimage and UUID by OmeXml.__init__ in the repository's minimum, 2023.7.10, whose source I read). test_the_copy_describes_itself_and_does_not_carry_z pins it.
  • Not done in this PR: OME-Zarr stacks. The design writes them as <name>.ome.tif, but the record, review_conversion, the reuse check and RunInputs all assume the copy has the same relative path as its source. That needs a split between source and copy path throughout the record. Rather than guess, an experiment with a non-excluded OME-Zarr image is refused with projection_zarr_not_supported ("exclude them"); it is a separate bounded change.
  • One signature change. convert_experiment would have had nine parameters (the code rules and Ruff allow eight and the baseline may not grow), so its keyword options other than exclude are now one **options: Unpack[ConversionOptions] group. The keywords are the same, an unknown one is a TypeError, and the CLI and notebook calls are unchanged.
  • Two deviations from the design's bounds are in Z projection 1/6: the pure arithmetic (part of #176) #211; none new here.

Regression test

tests/test_projection_convert.py (35 tests). The regression is test_project_max_makes_a_z_stack_analysable: it writes a named ZCYX uint16 stack with PhysicalSizeZ, asserts that check and convert without a projection still refuse it (z_stack_not_supported), converts with Projection("max"), and asserts the copy is CYX uint16 and equals stack.max(axis=0), the check on the copy tree has no z_stack_not_supported, the record entry has source_content_sha256 == file_sha256(source), dropped_sizes["Z"] is 2.5 micrometer header, and the source SHA-256 is the same before and after. The module also covers the dtype table end to end (uint8/uint16/int16/uint32 x max/sum/mean, values against Python ints and Fraction), the uint16 sum that must not wrap (196605), NaN/inf, overflow to infinity (nothing written), byte-identical copies and an equal copy_sha256 from two conversions, rerun reuse, copy_differs for another method or range, T kept, z_range and a single-slice range, a range no image has, Z=1 / no-Z images, a plain stack under confirmed roles (and none guessed), 64-bit integers, memory limit, OME-Zarr refusal, 3-D / unknown-axes / 2-D masks (2-D masks pair in a run on the copies), and a source whose content changes during the projection.

Checks

python tools/check.py: Ruff, formatting and code rules clean; 4319 passed, 4 skipped, 2 warnings. The base (#212) gives 4284 passed, 4 skipped, 2 warnings; the difference is the 35 new tests. The 4 skips are the pre-existing ones (case-insensitive file system x3, file modes x1).

Series: 1 pure module (#211), 2 record v3 (#212), 3 plan and write (this PR), 4 CLI, 5 run side, 6 notebook card.


Generated by Claude Code

…rt of #176)

plan_conversion(projection=) lets z_stack_not_supported through only when projecting and
refuses, before anything is written, what cannot be projected exactly; a projected copy
drops Z into dropped_sizes, is written page by page with a running plane, binds the source
content SHA-256 between fingerprints, and refuses 3-D masks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
@guijacquemet
guijacquemet changed the base branch from claude/z-projection-record to main November 8, 2026 03:59
An error occurred while trying to automatically change base from claude/z-projection-record to main November 8, 2026 03:59
@guijacquemet
guijacquemet merged commit 695987f into main Nov 8, 2026
7 checks passed
@guijacquemet
guijacquemet deleted the claude/z-projection-convert branch November 14, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants