[guijacquemet]

Owner decision, 12 November 2026: "Version 4 before v0.5.2".

Why

#287 added the entry key source_relative_path (an OME-Zarr copy's source path) but kept the record at version 3. v0.5.1 reads version 3 and ignores the key, so it looks for each source at the copy's path and wrongly warns conversion_source_changed. Spec ADR 0002 AR10 means no backward compatibility, so the version goes up and version 3 is refused.

Change

• RECORD_VERSION = "4". This is the only version written or read.

• Version 3 is refused the same way PP-06 (7068ec7) refused 1 and 2. The refusal is one message that names the version and the fix: "it is a record of version 3, and this version of preprocessing reads version 4 only (older records are not converted); convert the originals again into a new folder".

• Readers. Every reader goes through conversion_record.read_record → ConversionRecord.from_dict, so they all now refuse version 3. This was checked by grepping RECORD_FILENAME, RECORD_VERSION, read_record and record_version. Each reader is covered by a test:

  • convert (reuse of an earlier copy tree, conversion_record_unreadable)
  • the run's review_conversion (check/run, conversion_record_unreadable)
  • RunInputs.open, through conversion_check.conversion_problem, which is also what RunInputs.projection relies on (RunInputsError)
  • the notebook's earlier_projections, which skips a refused tree and does not offer it

• source_relative_path stays optional at the record level, but is required exactly where it is written. No v4 writer writes it on every entry: only ome_zarr_copy.as_zarr_copy sets it, for entries converted or projected into an OME-Zarr copy. OME-TIFF converted entries, projected entries and byte-for-byte entries (copied, including OME-Zarr sources and excluded images) keep their source's path and never carry it. Making it required on every entry would change every non-Zarr record for no gain. Instead the reader now:

  • requires it, as non-empty text other than relative_path, on a converted/projected entry whose relative_path ends in .ome.zarr;
  • refuses it on any other entry.

  So a v4 record can no longer be read with the key missing. Before this PR that produced the same false warning.

• Contract §9.1/§17, user guide, AGENTS.md and the CHANGELOG Unreleased section are updated. Tests that asserted "3" now assert "4". Docstrings in conversion_record.py were shortened to keep the file within the 600-line code rule.

Tests (all fail on main 682e14e)

tests/test_conversion_record_v4.py:

• a v4 record round-trips, for both an OME-Zarr copy and an OME-Zarr projection: from_dict/to_dict and the same bytes from write_record;
• a v3 record is refused with the message, check gives conversion_record_unreadable and no conversion_source_changed, and convert into the folder is refused;
• an OME-Zarr entry without source_relative_path, or with None, "", its own path or a number, is refused rather than warned as missing;
• the key on a non-Zarr entry is refused;
• the notebook does not offer a v3 projection tree;
• _ZARR_COPY_SUFFIX matches ZARR_COPY_SUFFIX.

tests/test_convert_integrity.py::test_runinputs_refuses_a_version_3_record_naming_the_fix.

On main, 12 of the 12 new tests fail. On this branch they all pass.

Output evidence

Main 682e14e and this branch ran the same script under a frozen clock, on the same source files (mtimes kept) and at the same absolute paths. As a noise baseline, main was also compared against itself.

• No conversion, two fixtures (histogram-golden uint16 tree including an OME-Zarr image, and float32 tree): run_uint16 6/6 files and run_float32 6/6 files are byte-identical to main.
• Conversions (OME-TIFF relabel, Z projection, OME-Zarr copy, OME-Zarr projection): the copied pixel files are byte-identical. Each record differs from main only in record_version "3" → "4"; the other differences are per-run copy mtime_ns/inventory_sha256/copy_sha256, which also differ between two runs of main. source_relative_path is unchanged: it is present on 3/3 and 2/2 OME-Zarr entries and on 0 OME-TIFF entries, as on main.
• Runs on the copies: the only differences beyond the same per-run noise are the record SHA-256 the run carries (conversion_record_sha256, projection.conversion_record.sha256, the report's conversion.sha256) and the manifest_sha256 that follows from it.
• The false warning, with the real v0.5.1 package (tag v0.5.1, 8726cb4), sources in place:
  • main's OME-Zarr copies under the v0.5.1 reader: conversion_source_changed ("3 / 2 source images changed or went missing"). This is the bug.
  • this branch's copies under the v0.5.1 reader: conversion_record_unreadable ("record of version 4"). The record is refused, not misread.
  • main's copies under this branch: conversion_record_unreadable ("record of version 3").
  • this branch's copies under this branch: no issues.

Gate

python tools/check.py on 06df6e6 (Python 3.13, .[dev,notebook,zarr]): ruff check and format pass; 5473 passed, 4 skipped; code rules pass; exit 0.

The 4 skips come from the environment:

• test_cli_log.py:166, test_folder_rules.py:199 and test_folder_rules.py:230: "this file system tells names apart by case";
• test_discovery.py:665: "running with permissions that ignore file modes" (root).

Coordination

This PR is kept minimal so that release PR #290 can be rebuilt after it merges. #290's branch was not touched, and no tag was made.

🤖 Generated with Claude Code

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

[guijacquemet]

Independent review of 06df6e6, with two fixes (head now 51c1c65)

Verdict: approve, with the two fixes below pushed as their own commits.

Verified myself (fresh clone, Python 3.13.18, .[dev,notebook,zarr]):

• RECORD_VERSION = "4". Version 3 (and 1, 2, 5, a missing one, a non-text one) is refused in ConversionRecord.from_dict, with one message naming the version and the fix, as in PP-06 (7068ec7).
• Readers: I grepped RECORD_FILENAME, read_record, from_dict, json.load(s) and .morphobricks-conversion.json over src/. The record is parsed in exactly one place, read_record (json.loads at conversion_record.py:371). Its callers are convert (conversion.py:275), review_conversion, conversion_check.conversion_problem (RunInputs.open / .projection) and notebook_projection.earlier_projections. projection_run.run_projection takes an already-read record, and conversion_plan.py:400 only tests whether the file exists. No other path reads it.
• _check_source_path matches the writer. as_zarr_copy renames only planned copies with a layout (converted or projected), always to *.ome.zarr, and byte-for-byte entries never get the key. A source OME-Zarr image (copied, *.ome.zarr, no key) is accepted.
• 12 new tests: with main's src/ and this branch's tests, all 12 fail (11 in test_conversion_record_v4.py plus test_runinputs_refuses_a_version_3_record_naming_the_fix, along with the 8 updated assertions). All pass on the branch.
• Cross-version check, using the real v0.5.1 package from tag v0.5.1 in its own venv:
  • v0.5.1 reading this branch's OME-Zarr copies and OME-TIFF copies: conversion_record_unreadable.
  • This branch reading v0.5.1 copies: conversion_record_unreadable.
  • OME-TIFF copies written by v0.5.1 and by this branch are byte-identical. The records differ only in record_version, plus the per-run mtimes, source_root, written_at and versions.
• Gate: my serial tools/check.py on 06df6e6 was killed by the container's out-of-memory killer at 59% (the machine is shared), so I have no full count of my own for 06df6e6. On 51c1c65: ruff check and format are clean and the code rules pass. pytest -n 4 tests/ gives 5476 passed, 4 skipped. That is 5473 plus the 3 new test cases. The 4 skips come from the environment (3 for case-sensitive file systems, 1 because the run is root).

Fix 1: 4862550, a projected entry names one source. A projected entry names its source twice: its relative_path (or its source_relative_path on an OME-Zarr copy) and projection.source_relative_path. The writer makes these equal, but the reader accepted two different paths. review_conversion checks the entry's path, while RunProjection copies the projection's path into the manifest, so a run could check one source and record another as projected. CopyEntry.__post_init__ now refuses the difference, which surfaces as conversion_record_unreadable. Regression test: test_a_projected_entry_names_one_source[ome-tiff|ome-zarr], which fails on 06df6e6. conversion_record.py is 598 lines.

Fix 2: 51c1c65, the notebook said nothing about a refused tree. earlier_projections skipped a v3 projected tree without a word, so the user got no Continue button and no reason. Writing the same projection again into that folder is then refused, and the folder name is fixed, so the only way forward is another output folder. Under the no-hidden-refusal rule the user must be told. The Axes step now shows one warning line per such folder: "Not offered: the copies written earlier in . Their conversion record cannot be used (<reason, naming version 3 and the fix>); to write them again, choose another output folder." earlier_projections is unchanged and unusable_projections is new. Regression test: test_copies_written_earlier_that_cannot_be_used_are_said_not_hidden, which fails on 06df6e6. User guide and CHANGELOG updated.

---

Generated by Claude Code