[guijacquemet]

This is PP-4 of the v0.4.0 plan (preprocessing-design-v0.4.md revision 4, §1, §3, §6 and §8). It is breaking: every v1 run is recomputed. It is based on main, with PP-0, PP-2 and PP-3 merged.

Commits: a move-only commit, the features, the X-before-Y follow-up, a merge of main, and a review fix.

Summary

Move only (commit 1):

• review_axes, check_mapping and channel_count move from axes.py to axis_review.py; axes.py goes from 717 to 434 lines.
• check_sources moves from manifest.py to sources.py. In commit 2, NarrowRange moves to narrow_ranges.py.
• Baseline entries are re-keyed with tools/check.py --rename.

Features (commit 2):

• Manifest version "2":
  • Each file's roles and axes_from (header or confirmed) are required; the reader and the writer both refuse an entry without them.
  • pixel_sizes per file: one entry per role X, Y, Z, T, each with value, unit and source. The source is header for now; PP-8 adds stated and pixels. An uncalibrated axis has value and source null, never a guessed value.
  • Masks gain size, mtime_ns, roles, axes_from and shared (mask_record, B3).
  • Optional conversion_record_sha256.
  • A version 1 manifest is still read, but no row from it is reused. The run reports "the earlier run was written before v0.4.0, without the axis check".
• fingerprint.Fingerprint and fingerprint.RowKey give one rule for "the same file, read the same way" (audit D5). The key is the fingerprint, labels, axes, roles, axes_from, the read path and the row-changing settings.
  • run/entries.py builds the manifest entries and sorts files into new, changed and unchanged by row key.
  • This code moved out of run_experiment, which went from 195 to 172 executable lines.
• axis_check.ConfirmedAxes (audit 2 B-1): a confirmation covers only the files it names, as they were when confirmed: fingerprint.confirm_axes records each file's Fingerprint, and run_experiment drops files changed since (review fix acc6fa7).
  • A plain stack outside it is the error plain_stack_not_confirmed, and is never read with roles confirmed for other files.
  • A plain AxisMapping still confirms every file the run finds.
• X before Y (owner, follow-up e9108d2): a plain stack confirmed with X before Y (TXY) is the error axes_x_before_y; nothing is read transposed.
• Docs: contract §2 and §9, a user guide message, plan §12 Q47, AGENTS.md.

Notes for review

• Not wired in this PR:

  • the notebook does not yet pass the files it showed at confirmation (PP-10, per the plan);
  • the CLI's reuse of the recorded mapping without --axes (PP-7).

  For now, ConfirmedAxes is enforced through the API.

• Read path: it is part of the row key (spec §1), so moving the experiment folder recomputes its rows. A test covers this.

• Settings: settings that change every row are still reported once, as the rebuild reason. Row keys are compared with the current settings on both sides, so the new/changed/unchanged counts stay per file.

• Test edits:

  • test_masks.py now checks only the mask path; the full mask entry is checked in test_manifest_v2.py.
  • The test-only plain() helper is shared as tiff_tree.write_plain.

Tests

• New file tests/test_manifest_v2.py (12 tests):
  • version 2 roles, axes_from and pixel sizes; a plain stack recorded as confirmed and uncalibrated;
  • mask fingerprints;
  • the reader and writer refuse missing roles; recorded-size checks;
  • a v1 manifest gives no reused rows;
  • row reuse by row key (moving the experiment recomputes); the RowKey fields; Fingerprint;
  • per-file confirmation (plain_stack_not_confirmed names only the unconfirmed file); named images need no confirmation.
• On old main the file fails at import: fingerprint and ConfirmedAxes do not exist there.
• tests/test_x_before_y.py (9 tests): X-before-Y confirmations refused.
• Review fix: test_a_plain_stack_changed_since_its_confirmation_is_not_covered fails without the fix.
• Full gate as CI (ruff check, ruff format --check, tools/check.py --rules, pytest with .[dev,notebook,zarr]), at acc6fa7:
  • Python 3.13 with zarr 3: 1366 passed, 1 skipped (running as root ignores file modes).
  • Python 3.11 with zarr 2.18: 1348 passed, 19 skipped (that root skip plus 18 OME-NGFF 0.5 tests that need zarr 3).
  • CI green on all six jobs.

https://claude.ai/code/session_01WX6rdMV1r5jUPUR94kxV5a

---

Generated by Claude Code

[guijacquemet]

Independent review of PP-4 (design v0.4 rev 4 §1, §3, §6, §9).

Finding, fixed in acc6fa7: ConfirmedAxes covered files by path only, so a plain stack rewritten under the same name stayed confirmed and was read with roles confirmed for its old contents (B-1 says new or changed). It now maps each file to its Fingerprint (fingerprint.confirm_axes), and run_experiment drops files changed or gone since (unchanged_confirmation), so they are plain_stack_not_confirmed. The new regression test fails without the fix. run_experiment went from 166 to 161 executable lines (baseline lowered). I also corrected a docstring that said a header naming X before Y is refused; only confirmed roles of unnamed axes are.

Verified:

• Commit 31e3eb6 only moves code (git diff --color-moved).
• Manifest v2: roles, axes_from and pixel_sizes by role with unit and source; uncalibrated axes are null and never guessed. Masks get a fingerprint and roles. The reader and writer refuse entries without roles. A v1 manifest gives no reused row and the run reports it.
• axes_x_before_y (owner answer) is an error with a regression test, and nothing is read.
• Reused rows were checked adversarially: changed roles (TYX→ZYX), bins, histogram range, robust quantiles, the range-check limit, labels (a renamed folder), the read path (a moved experiment) and the file fingerprint each recompute. missing_as_singleton is in the key through singleton_roles.
• §9: the baseline diff, checked by hand, only removes or lowers entries. The two moved functions keep their entries under their new names.
• Gate: ruff, format and tools/check.py --rules pass. Python 3.13 with zarr 3: 1366 passed, 1 skipped. Python 3.11 with zarr 2.18: 1348 passed, 19 skipped. All six CI jobs are green.

For later, not blocking:

• planes._finish still transposes a plane when a header names X before Y. Only plain-stack confirmations are refused. The owner should decide whether named X-before-Y headers are refused too (PP-6 read_plane must never transpose).
• check_sources still accepts v1 manifests; RunInputs (PP-6) refuses them.

---

Generated by Claude Code