Repository navigation
Z projection 3/6: plan and write projected copies (part of #176) - #213
Merged
Merged
Conversation
…rt of #176) plan_conversion(projection=) lets z_stack_not_supported through only when projecting and refuses, before anything is written, what cannot be projected exactly; a projected copy drops Z into dropped_sizes, is written page by page with a running plane, binds the source content SHA-256 between fingerprints, and refuses 3-D masks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
This was referenced Nov 8, 2026
An error occurred while trying to automatically change base from
claude/z-projection-record
to
main
November 8, 2026 03:59
…onvert # Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #212, which is stacked on #211 (base is
claude/z-projection-record, so the diff shows only this PR; GitHub retargets as the earlier ones merge).What changed
Third PR of the plan for #176 (design: MorphoFeature
docs/restart/designs/pp176-z-projection-design-2026-11-07.md; owner answers of 2026-11-07).convert_experiment(..., projection=Projection("max" | "mean" | "sum", z_range=None))now writes recorded 2-D copies of Z-stacks. There is no command yet (PR 4) and no run-side change (PR 5).plan_conversion(projection=, masks_folder=)letsz_stack_not_supportedthrough only when projecting. Every non-excluded image is projected (a Z=1 image or one without a Z role is one slice), so a tree has one type and one provenance per image. Excluded images are still copied byte for byte. New modules:projection_plan.py(every refusal in one pass, before any write),projection_layout.py(the converted layout minus Z; the Z size goes to the record'sdropped_sizes, X/Y/T sizes and channel names followcopy_layout),projection_issues.py(messages),projection_masks.py,planned_copy.py(PlannedCopymoved out ofconversion_plan.pyto avoid an import cycle).write_projected_copy(inome_tiff_copy.py): one page per (t, c); slicesz_rangeread in ascending order withread_planeand added into one running plane (PlaneAccumulator, Z projection 1/6: the pure arithmetic (part of #176) #211); one input plane and the running plane are held. The source's content SHA-256 is taken before and after the projection (a change is refused) between the usual fingerprint checks, and recorded. A sum that overflows to infinity stops the conversion (projection_overflow) and nothing is kept.nothing_to_project,projection_z_range_outside(names the shorter images),projection_cannot_be_exact,cannot_convert_pixel_type(64-bit integers),projection_over_memory_limit,projection_masks_3dandprojection_masks_axes_unknown(masks folder scanned from headers only; masks are never projected; 2-D masks are left alone), andprojection_zarr_not_supported(see below).projection_overflowandprojection_undefinedcome from the pixels, during the write, and still leave the tree as it was.copy_differsand the message names both ("the folder holds max projections of all slices, and you asked for mean projections of all slices"); an identical rerun reuses every copy and leaves the record as it was.Image/Description(method, slices, source, source SHA-256, types, rounding, NaN rule) and a UUID derived from it, so one projection of one source is the same bytes twice (tifffileotherwise puts a random UUID in every file, which made two conversions differ).Answers to the open points
tifffilewrite the OME-XML description the design needs? Yes.metadata={"Description": ...}becomes<Image><Description>(checked withtifffile2026.10.15, and the key is handled byOmeXml.addimageandUUIDbyOmeXml.__init__in the repository's minimum, 2023.7.10, whose source I read).test_the_copy_describes_itself_and_does_not_carry_zpins it.<name>.ome.tif, but the record,review_conversion, the reuse check andRunInputsall assume the copy has the same relative path as its source. That needs a split between source and copy path throughout the record. Rather than guess, an experiment with a non-excluded OME-Zarr image is refused withprojection_zarr_not_supported("exclude them"); it is a separate bounded change.convert_experimentwould have had nine parameters (the code rules and Ruff allow eight and the baseline may not grow), so its keyword options other thanexcludeare now one**options: Unpack[ConversionOptions]group. The keywords are the same, an unknown one is aTypeError, and the CLI and notebook calls are unchanged.Regression test
tests/test_projection_convert.py(35 tests). The regression istest_project_max_makes_a_z_stack_analysable: it writes a named ZCYXuint16stack withPhysicalSizeZ, asserts thatcheckandconvertwithout a projection still refuse it (z_stack_not_supported), converts withProjection("max"), and asserts the copy isCYXuint16and equalsstack.max(axis=0), the check on the copy tree has noz_stack_not_supported, the record entry hassource_content_sha256 == file_sha256(source),dropped_sizes["Z"]is2.5 micrometer header, and the source SHA-256 is the same before and after. The module also covers the dtype table end to end (uint8/uint16/int16/uint32x max/sum/mean, values against Python ints andFraction), theuint16sum that must not wrap (196605), NaN/inf, overflow to infinity (nothing written), byte-identical copies and an equalcopy_sha256from two conversions, rerun reuse,copy_differsfor another method or range, T kept,z_rangeand a single-slice range, a range no image has, Z=1 / no-Z images, a plain stack under confirmed roles (and none guessed), 64-bit integers, memory limit, OME-Zarr refusal, 3-D / unknown-axes / 2-D masks (2-D masks pair in a run on the copies), and a source whose content changes during the projection.TypeError: convert_experiment() got an unexpected keyword argument 'projection').Checks
python tools/check.py: Ruff, formatting and code rules clean;4319 passed, 4 skipped, 2 warnings. The base (#212) gives4284 passed, 4 skipped, 2 warnings; the difference is the 35 new tests. The 4 skips are the pre-existing ones (case-insensitive file system x3, file modes x1).Series: 1 pure module (#211), 2 record v3 (#212), 3 plan and write (this PR), 4 CLI, 5 run side, 6 notebook card.
Generated by Claude Code