[guijacquemet]

Redesign iteration 3, part 4 of 4, stacked on #286. Item (t) of "OWNER v0.5.1 NOTEBOOK TEST" (RESTART.md).

Summary

• Core: convert_experiment(..., store="ome-zarr") (default "ome-tiff", unchanged).
  • ome_zarr_copy: write_named_zarr and write_projected_zarr write OME-NGFF 0.4 stores directly (no zarr package needed to write; reading needs the zarr extra as for any OME-Zarr image): one array 0, uncompressed, little-endian, one chunk per plane, sorted JSON metadata, so one copy of one source is the same bytes. Planes are read with iter_planes / read_plane under the confirmed roles and projected with PlaneAccumulator, as the OME-TIFF writers do. Axes, units, level-0 scale (an unknown size is the placeholder 1 named under metadata.unknown_scale, which the reader already reads as unknown), channel names (omero), a projected copy's description (morphobricks.description).
  • Plan: every TIFF image that is read is written (also those whose headers already name their axes); excluded images and OME-Zarr images are copied byte for byte as before. A copy is <position>.ome.zarr in the same folder (same position ID); two images that would share a copy name are refused (copy_names_collide).
  • Record: the entry's relative_path is the copy's path; the new optional key source_relative_path holds the source's path and is written only for such an entry, so every record without OME-Zarr copies is byte for byte as before (tested). significant_bits is null (OME-NGFF 0.4 has no field). The lock, .partial writes, reuse, the copy's SHA-256 and the run's checks (conversion_source_changed names the source by its own path) are those of the other copies.
  • plan_conversion takes projection, masks_folder and store as keywords (the eight-parameter rule).
• Notebook (notebook_zarr_copy): Also save a copy as OME-Zarr (in the output folder) in the Folders card of the first screen, unticked; ticking writes nothing. On Axes, the card OME-Zarr copy shows where the copy goes and Write the OME-Zarr copy and continue with it, greyed with its reason until the axis roles are confirmed (or while a pixel-size statement has an error). The copies go to <output>/ome_zarr and are opened as the experiment (scanned and checked again; the user confirms their axes). With Z-stacks the projection card of Notebook: Z-stacks saved as recorded projections from the Axes step; 3-D stacks kept recorded (redesign 3, part 3) #286 writes the projected copies as OME-Zarr (<output>/projected_<method>_zarr). Without the zarr package the option is greyed with the reason.

Notes for review (owner decisions wanted)

• Record version. source_relative_path is added without raising the record version (3): old records read as before, and a v0.5.1 reader given a tree of OME-Zarr copies finds the copies but would look for the sources under the copies' names (a false conversion_source_changed warning, nothing worse). Raise the version to 4 if a strict refusal is preferred.
• "Pixel sizes validated." The button waits for the confirmed axes and for no error among the pixel-size statements. Differing header sizes do not block it: the copy carries each image's own header size, and sizes stated for . or a folder keep applying (same output folder). A size stated for a single image names the image file and must be stated again for the copy (calibration_target_not_found says so).
• The copy is uncompressed (as the OME-TIFF copies): it takes the size of the pixels. Compression would need a decision on the codec and on byte-reproducibility.
• OME-Zarr sources are not projected (projection_zarr_not_supported, unchanged).

Tests

tests/test_ome_zarr_copy.py (new): bit-for-bit planes read back through the brick's reader, metadata, record, reuse, same bytes twice, projection as OME-Zarr, changed source, name collision, unchanged records, and the notebook flow through save and run on the copies. Gate counts for every head of the stack are in a comment below.

---

Generated by Claude Code

[guijacquemet]

Local gate (python tools/check.py: ruff check, ruff format --check, pytest, code rules) on the final head of every branch of the stack, each with main a775ebf (#288) merged:

PR	Branch	Head	Result
#284	claude/redesign3-1-viewer	e2e160f	5353 passed, 4 skipped, exit 0
#285	claude/redesign3-2-axes	aff9330	5364 passed, 4 skipped, exit 0
#286	claude/redesign3-3-projection	f31ee72	5372 passed, 4 skipped, exit 0
#287	claude/redesign3-4-zarr	351c134	5384 passed, 4 skipped, exit 0

The 4 skips are the same on every head and are environment skips: test_cli_log.py:166, test_folder_rules.py:199 and :230 (this file system tells names apart by case), test_discovery.py:665 (running with permissions that ignore file modes).

Same answers, main a775ebf against 351c134, on 28 images of 2048 × 2048 with 3 channels, masks, pixel sizes differing in the 5th decimal, one size stated for a folder and one for all: morphobricks.toml, plane_histograms.csv, experiment_design.csv, calibration.csv, mask_relationships.json, mask_representation.json are byte-identical; run_manifest.json is identical apart from times and the two output paths. tests/spec_fixtures and the Spec tests are untouched.

Changes made for CI only, in the test helpers: the kernel runs of the notebook file are repeated when the client loses the kernel's output (macOS runners, "Timeout waiting for IOPub output"), and two path assertions compare the folder as the record spells it (Windows).

🤖 Generated with Claude Code

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

---

Generated by Claude Code

[guijacquemet]

Final heads (this replaces the table above; main a775ebf, #288, is merged into each). python tools/check.py exit 0 on every head; CI 7/7 green on every head.

PR	Branch	Head	Local gate
#284	claude/redesign3-1-viewer	3cc0d87	5353 passed, 4 skipped
#285	claude/redesign3-2-axes	4abf0a7	5364 passed, 4 skipped
#286	claude/redesign3-3-projection	786a0e7	5372 passed, 4 skipped
#287	claude/redesign3-4-zarr	1c81f80	5384 passed, 4 skipped

The 4 skips are environment skips, the same on every head (case-sensitive file system ×3, file modes ×1).

The macOS failures of the earlier heads, explained. The app's cell sends one IOPub message per widget model: 1,837 on main, 2,015 with #284 and 2,135 at the top of the stack (counted with nbclient). ZeroMQ keeps 1,000 messages on each side for a subscriber and drops the rest, so above 2,000 a client that reads slowly (nbclient on the macOS runners) lost the end of the stream, with the cell's display message. tests/test_notebook_file.py::_execute now gives the kernel client an unlimited receive queue (and repeats a run that still loses its output, twice at most). JupyterLab and Colab read the stream as it comes; this was seen only with nbclient in CI. Reducing the number of widget models the app creates at start (lazy pages) would remove the cause and is not done here.

Byte comparison (same answers, main a775ebf against the top): morphobricks.toml, plane_histograms.csv, experiment_design.csv, calibration.csv, mask_relationships.json, mask_representation.json identical; run_manifest.json identical apart from times and the two output paths.

🤖 Generated with Claude Code

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

---

Generated by Claude Code

[guijacquemet]

Independent review: #287 (OME-Zarr copy option)

Verdict: approve after one fix (pushed), with owner questions. Merged the projection branch forward (e856f6f, then 5542523 with the #286 fix), with no conflicts. Retargeted to main after #286 merged (be5cead).

Defect found and fixed: 1f5dc17. The copies were not OME-NGFF 0.4 valid. I validated .zattrs against the official https://ngff.openmicroscopy.org/0.4/schemas/image.schema (jsonschema, draft 2020-12). Every copy with channel names failed: 'window' is a required property, 'color' is a required property on each omero.channels[] item. Each channel now carries color: "FFFFFF" and a window equal to the pixel type's whole range (min, max, start, end). Both are display only and never read from the pixels, so one source still gives the same bytes. The brick's reader is unchanged. Contract §17 and the changelog say so. Regression test: test_the_omero_channels_have_what_ome_ngff_0_4_requires. It fails without the fix.

After the fix my script wrote YX, CYX and TCYX (named) and TCZYX/ZCYX (max, mean, sum projections; uint16 and float32). All of them:

• validate against image.schema 0.4;
• have axes in t, c, then space order, with 2 or 3 space axes;
• have NGFF units (micrometer, second), dimension_separator /, and unknown_scale for a missing size;
• have pixels equal bit for bit to the source read in t,c,z,y,x order, and projections equal to numpy's (mean within 1e-9);
• give byte-identical stores on a second write;
• leave the originals' SHA-256 unchanged.

The record stays "3" with source_relative_path. (strict_image.schema additionally wants the SHOULD fields name, type and metadata on the multiscale. They are not written. This is informational, not invalid.)

Also checked:

• Ticking the option writes nothing. Nothing is written before Confirm these axes, and the confirmation itself writes nothing. The card is disabled with "the axis roles are not confirmed" until then. After the copy, the copies' axes are not confirmed for the user.

Owner questions. My recommendation comes first in each.

1. Z-stacks + OME-Zarr write before Confirm axis roles. With the option ticked, the projection card writes projected_<method>_zarr on the example tick plus the method plus the click. Stacks can never be "confirmed" (Z>1 is refused), so this is the same statement of roles as the OME-TIFF projection. The PR body discloses it. Recommendation: keep it. The alternative is to write the projections as OME-TIFF and offer the OME-Zarr copy after the copies' axes are confirmed, which makes a copy of a copy.
2. The option stays ticked when the folder changes. It is a first-screen preference, not an answer about the data: nothing is written without the Axes card's click, which names the destination. Recommendation: keep it. Resetting it would also untick it when the app opens the copies.
3. Record version. source_relative_path is optional at v3. A v0.5.1 reader would ignore it and report a false conversion_source_changed warning, not an error. Recommendation: raise to v4 if strict refusal is the rule (AR10 already refuses other versions). Not changed here.

Byte identity (re-measured on 1f5dc17 against main a685711, same input files, same answers through the stepper):

• Run with masks (Nuclei instances, Mito grouped labels, plus the mask step): morphobricks.toml, plane_histograms.csv, experiment_design.csv, calibration.csv, mask_relationships.json and mask_representation.json are byte-identical. run_manifest.json is identical apart from times and paths.
• Run with 3 channels in rep1 and 4 in rep2 (Stage 3 S3.3: per-unit channel lists, plane_histograms v5 #288 Stage 3): same result.

The only other differences are in discovery_report.*, and they are the manifest's own SHA-256, which follows from its times.

Gates (Ruff check, Ruff format, code rules, pytest -n 2; the skips are the same four everywhere: root-user file modes (one), case-sensitive file system (three)):

Head	Passed	Skipped
#284 e5d7103	5427	4
#285 093843e	5438	4
#286 aa55464	5447	4
#287 1f5dc17	5460	4

CI on 1f5dc17 is 14/14 green.

---

Generated by Claude Code