[guijacquemet]

CP-3b of the CellPatcher v0.2.0 plan: the follow-ups of PR #71 that the owner decided on 2026-11-02 (design v0.4 §6 and §9 on MorphoFeature claude/restart-doc). It is based on main at e24243f, which is 3b42093 plus the bot's CHANGELOG commit. This PR does not tag. v0.2.0rc1 is tagged after it.

Commits

1. 1641c2e Installer layout: setup.py now also installs the package from the layout the installer uses. A test installs both layouts and imports cellpatcher.
2. 9a15590 Normalization views: the "Show parameters per group" table and the before/after normalization views come back. They are computed in the package and shown by the notebook.
3. bf78173 Two notebooks rewritten: CellPatcher_Strategy_Examples and CellPatcher_Pytorch are rewritten for v0.2 with the LabConstrictor treatment. All three notebooks now share one Step 0, and the tests run each notebook in a kernel.
4. 432dae6 Docs and CHANGELOG.
5. b876242 Wording in the examples notebook, and an unused test constant removed.
6. 3e15ba3 The kernel tests shut down their kernels. nbclient shuts down only the kernels of managers it creates, so each kernel lived until pytest exited (measured locally). The notebook job of the first CI run ended by terminating an orphan python process; the run on 3e15ba3 has none.
7. Independent review (2026-11-02), small commits on this branch:
   • 2a48e08 Torch adapters' default rescaling was silent (library, pre-existing): see "Review findings".
   • 42c9b3e PyTorch notebook's batch: float32 only when exact; the exported type recorded and shown.
   • 8c90c45 The before/after figure no longer cuts a patch_id at / for its title.
   • 01a0e7d Tests: pip in the test extra; kernel IPC sockets in a short temporary folder.
   • 821470a Docs: the owner decisions of the review (template-sync PR with a CPU-torch CI job before the final v0.2.0).
   • 5ce56fd PyTorch notebook text on the dtype.
   • 5f890f3 AGENTS.md health snapshot.

1. The other two notebooks

What is the same in all three notebooks. Step 0 is the main notebook's cell, made general, and a test checks that the three are identical. Only four things differ: the title, notebook_name, EXTRAS and INTERFACE ("module:class"). Every notebook has:

• # @title Run to … as the first line of every code cell, with hidden code;
• Drive, the widget manager and the zarr extra only inside if "google.colab" in sys.modules:;
• no magics, and no /content outside that guard;
• one git+ install, pinned to RELEASE_TAG = "v0.2.0rc1", with no GITHUB_TOKEN, userdata, git clone or unpinned main;
• current_version = "0.2.0rc1", and notebook_name equal to the file name, matching notebook_latest_versions.yaml (it was 0.0.4/0.0.5, with free-text names);
• Colab and JupyterLab callouts, with the data on Drive;
• cells that only call the package.

Strategy examples (cellpatcher.notebook_examples.StrategyExamplesNotebook):

• It used to make synthetic arrays and draw galleries in its cells.
• Step 1 is now the main notebook's step 1: the preprocessing run is opened through RunInputs, and the user chooses an Example image.
• Then there is one page per strategy (cell, grid, random, whole FOV). Each page shows only that strategy's settings and has a "Show the … example" button.
• Each example is the main notebook's preview_image. It therefore shows the patches that the export gives with the same settings (tested), and it writes nothing.
• Without masks, the cell example shows a red card saying why.

PyTorch (cellpatcher.notebook_torch.TorchNotebook and cellpatcher.torch_session.load_exported_batch):

• It used to be a developer notebook that cloned the repository with a token and ran pytest and parity checks in its cells.
• It now loads the main notebook's export with ExportedPatchTorchDataset and shows the first batch: its shape, dtype, value range and images.
• Two defaults of the public dataset are not used:
  • The axes come from each row's output_axes, not the default axes="YX", which fails on CYX patches.
  • The patches are kept as exported, with channel_policy="keep", dtype_policy="keep", scale_policy="none". The default repeat_to_rgb refuses 2 channels. The default scale_policy="auto" divides integer patches by their dtype's maximum (deprecated).
  • The batch is then converted to float32 only when float32 holds every value exactly (8- and 16-bit integers, normalized float32 patches); otherwise it is refused, never rounded. The exported type is recorded (TorchBatch.exported_dtype) and shown (review fix).
• Patches of different sizes are refused, never resized. Mixed output_axes are refused, never guessed.
• Raw-value exports are normalized on load and equal the normalized export (tested).

2. The normalization views in the main notebook

Show parameters per group (step 2, the Normalization card):

• notebook_session.group_parameters_of_run opens and checks the run as an export does. It then calls the existing normalization.group_parameters, which uses the brick's fit.
• The table lists every group and channel (and time point), with offset, scale and exactness. Groups that would stop a run are shown in a red card.
• The settings' NormalizationWarnings are shown with the table, not dropped. Other warnings are passed on unchanged.
• A test checks that every exported patch's Normalization_Offset/Scale equals its group's row.

Before and after normalization (step 3):

• With normalization on, preview_image patches the image a second time with normalization off. Preview.before_after holds the first four patches both ways.
• cellpatcher.normalization_review is a new pure module. It pairs patches by patch_id, never by order, so a patch that patch-scope normalization dropped is left out.
• It takes the Y/X plane with output_axes and never transposes.
• It bins each plane with the normalization brick's compute_plane_histogram. Nothing in RESERVED_NAMES is defined here.
• The notebook draws each patch beside its histogram.

3. Installer layout

What is wrong.

• This repository's .tools/python/bump_constructor.py is a stale copy of the template.
• The release workflow runs it, and it maps src/cellpatcher/x.py to CellPatcher/src/cellpatcher/cellpatcher/x.py. There, find_packages(where="src") finds nothing, so the installed app has no cellpatcher.
• LabConstrictor already fixed this upstream in a76be61 (2026-07-06, "Fix the setup.py bug"). The current template maps to CellPatcher/src/cellpatcher/x.py.
• I ran the release workflow's bump_constructor.py: it writes this branch's construct.yaml byte for byte (the 4 new modules are listed).

The fix, in CellPatcher only. setup.py reads the package from src/cellpatcher/ when src/cellpatcher/cellpatcher/__init__.py exists, and from src/ otherwise. No .tools/ or .github/ file is changed.

The test. test_the_installed_app_imports_cellpatcher[construct.yaml|template]:

• copies setup.py and the package as each layout lists them;
• runs pip install --target;
• checks that every module is installed;
• imports cellpatcher and cellpatcher.strategies.cell from the install.

On old main, the construct.yaml layout fails: 0 of its 40 modules are installed.

Owner decision (2026-11-02): keep this setup.py fix now; the template sync comes in one PR before the final v0.2.0 (see "Owner decisions").

Review check: mutating SOURCE_ROOT to "src" fails the construct.yaml case, and to "src/cellpatcher" fails the template case. No .tools/ or .github/ file is changed.

Review findings (independent review, 2026-11-02)

1. The torch adapters' default rescaling was silent (2a48e08; library behaviour from before this PR, not the notebook path). PatchTensorConfig(scale_policy="auto"), the default, divides an integer patch by its dtype maximum. Its DeprecationWarning was raised inside cellpatcher.torch_dataset, where Python's default filters hide it: ExportedPatchTorchDataset(csv)[0] of a uint16 patch of 1000 gave 0.0153 (and 3 repeated channels) with no message (measured in a subprocess). Docs said "it warns when it rescales". Fix: the warning is now a ScalePolicyDeprecationWarning, still a DeprecationWarning so existing filters and pytest.warns still match. It is shown under the default filters, once per place, and names the divisor. Each dataset item records scale_divisor (1.0 when the values are not rescaled). The change is additive. The default is unchanged, because contract §15 makes that an owner decision (see "Owner questions").
2. The PyTorch notebook changed the dtype without a check (42c9b3e). dtype_policy defaulted to "float32", so an int32/uint32 value above 2^24 or a float64 value would have been rounded silently. Now: the exported type is read, the batch is converted only when exact and refused otherwise, and the type is recorded and shown.
3. Identity parsed for display (8c90c45). The before/after title used patch_id.rsplit("/", 1)[-1]. A patch without a label is now numbered.
4. Gate in a venv without pip (01a0e7d). test_installers runs python -m pip, and failed in uv venvs (2 failures in every environment). pip is now in the test extra.
5. Kernel sockets on macOS (01a0e7d). IPC sockets were under pytest's tmp_path, which can exceed macOS's 104-byte socket path limit. They now go in a short mkdtemp folder that is removed afterwards.

Verified with no change needed:

• Notebook path: there is no rescale, channel repetition, resize or guessed axes. The axes come from output_axes, and mixed axes are refused. Patches of different sizes are refused.
• normalization_review: it pairs by patch_id without parsing, never transposes, and bins with the brick's compute_plane_histogram; RESERVED_NAMES passes.
• group_parameters_of_run: it calls normalization.group_parameters. NormalizationWarnings are kept as notes and other warnings are re-emitted. Groups with a problem are listed in a red card.
• Step 0: it is identical across the three notebooks (tested). It is pinned to RELEASE_TAG = "v0.2.0rc1", notebook_latest_versions.yaml agrees, Colab code is guarded, and there are no magics, @param, GITHUB_TOKEN, userdata or git clone. Cells only call the package.

Regression tests (they fail on old main)

The new and changed test files were run against origin/main (e24243f) src, notebooks and setup.py, with CPU torch:

• 34 failed: 1 installer layout, 26 notebook file, 6 UI (including the new title test) and 1 deprecation (the new default-filters test);
• 4 modules fail to import on main: test_normalization_review, test_notebook_examples, test_torch_session and test_notebook_session;
• 27 passed.

The new torch-session dtype tests also fail on the PR's original head 3e15ba3: 9 of 14.

Gate (reviewer, own worktree and uv venvs, Python 3.11, preprocessing v0.4.0rc2, head 5ce56fd; tools/check.py --rules exit 0 in every environment, full tools/check.py exit 0 with torch)

Environment	Passed	Skipped
.[test]	1075	41: 21 zarr, 10 notebook extra, 10 torch
.[test,zarr]	1096	20: 10 notebook extra, 10 torch
.[test,zarr,notebook]	1124	11, all torch (7 torch-module skips, 1 torch-session, 1 deprecation, 2 PyTorch-notebook)
.[test,zarr,notebook] + CPU torch 2.15.0	1173	0

• CI on 5ce56fd is green with the same counts: test 1075/41, test,zarr 1096/20, test,zarr,notebook 1124/11, and the rules job.
• No CI job has torch. The PyTorch notebook's kernel run, test_torch_session, the torch tests and the new default-filters test are skipped in CI. They were run locally only, with the results above. A CPU-torch CI job comes with the template-sync PR (owner decision).
• Notebook kernel tests: tests/test_notebook_file.py was run 3 times with torch, and passed 39/39 each time in 12.9, 13.2 and 14.0 s. No ipykernel_launcher process and no cpk-* socket folder was left.
• Before the fixes, head 3e15ba3 gave the PR's counts, except the 2 pip failures above (1162 + 2 failed with torch).

§9 figures (new and changed code, measured by the reviewer)

• Files: normalization_review.py 156 lines, notebook_examples.py 105, notebook_torch.py 90, torch_session.py 181, notebook_session.py 364, notebook_ui.py 452, torch_dataset.py 425. The largest test file touched is test_notebook_file.py, 377. All are within 600.
• Largest functions touched:
  • complexity 10: torch_dataset.patch_array_to_tensor (existing); 7 among the new functions (before_after);
  • executable lines 41: patch_array_to_tensor; 37 among the new functions (load_exported_batch);
  • statements 23: load_exported_batch;
  • parameters 6: before_after.
• Layers: normalization_review is pure, notebook_examples/notebook_torch are ui, and torch_session is orchestration. torch_dataset stays io.
• Ratchet: tools/code_rules_baseline.json is unchanged from main, and the # noqa count is 17 on both. No exemption was added.

Colab hand-check list (after tagging v0.2.0rc1, because the notebooks install that tag)

Main notebook CellPatcher.ipynb:

1. Step 0 mounts Drive and installs cellpatcher[notebook,zarr] @ …@v0.2.0rc1. The card shows 0.2.0rc1.
2. Steps 1 to 4 behave as in the CP-3 list: the run's green card, the settings, the preview and the export to Drive.
3. With normalization at the image or biological-repeat scope, Show parameters per group shows the table, the background note and any red group card.
4. The step 3 preview with normalization shows the before/after figure, with patches and histograms. Patches without a label are titled "patch 1…".

Strategy examples:

1. Step 0 installs cellpatcher[notebook,zarr] @ …@v0.2.0rc1. The card reads "CellPatcher strategy examples are ready", 0.2.0rc1.
2. The widgets render. Browse… shows the paste-the-path card.
3. Step 1 with a run on Drive shows the green card, and Example image is filled.
4. Each strategy page shows only its settings. "Show the … example" draws the rectangles and the gallery.
5. Scan the object sizes of the dataset works on the cell page.
6. With a run without masks, the cell example shows a red card.
7. Nothing is written to Drive.
8. After a disconnect, rerunning Step 0 and Step 1 leaves no stale state.

PyTorch:

1. Step 0 keeps Colab's own torch (its version is unchanged after the install) and installs cellpatcher[notebook,torch,zarr] @ …@v0.2.0rc1.
2. Step 1 with the main notebook's export on Drive shows the green card: tensor (8, C, H, W), torch.float32, "exported as uint16 (or float32), converted exactly", and the range. The images are drawn.
3. An export of cell patches with per-object sizes (not resized) shows the red "different sizes" card.
4. A folder without patches.csv shows a red card.

Owner decisions (2026-11-02), recorded in docs/plan.md and the CHANGELOG "Not yet"

1. Template sync: keep the setup.py layout fix now. Before the final v0.2.0 (not before rc1), one PR syncs the LabConstrictor template. It re-applies CellPatcher's CI rules job, the extras matrix and the rc refusal, and proposes them upstream.
2. A CPU-torch CI job is added in that same PR. Until then the PyTorch notebook's kernel run and the torch tests are skipped in CI and run locally (above: 1173 passed, 0 skipped).
3. The PyTorch notebook becomes a user notebook that loads an export as tensors: accepted.
4. The strategy-examples notebook needs a preprocessing run: accepted, to revisit when preprocessing is embedded after v0.4.0.

Owner questions (none blocks rc1)

• The torch adapters' defaults. PatchTensorConfig() still defaults to scale_policy="auto" (divide integer patches by the dtype maximum) and channel_policy="repeat_to_rgb" (repeat a grayscale patch to 3 channels). The rescaling is now visible and recorded, but a user who leaves the defaults still gets rescaled values. Requiring an explicit scale_policy (or defaulting to "none"), and possibly channel_policy, is a breaking change: contract §15 needs your approval and a CHANGELOG breaking note. Should it go in v0.2.0 final, or later?

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

---

Generated by Claude Code

[guijacquemet]

Independent review: merged as c2008e9 (merge commit, branch kept)

Fixes pushed on the branch:

• 2a48e08: the torch adapters' default rescaling is now shown and recorded.
  • The issue was in the library and predates this PR. scale_policy="auto" warned with a DeprecationWarning that Python's default filters hide inside a library. Through ExportedPatchTorchDataset, a uint16 patch was divided by 65535 and repeated to 3 channels, with no message.
  • The warning is now ScalePolicyDeprecationWarning, a DeprecationWarning subclass, shown under the default filters. Each item gets scale_divisor. This is additive.
  • The default itself is unchanged: changing it is an owner question (contract §15).
• 42c9b3e: the PyTorch notebook's batch is converted to float32 only when that is exact. Otherwise it is refused. The exported dtype is recorded and shown. Before, int32/uint32 values above 2^24 and float64 values were rounded silently.
• 8c90c45: the before/after figure no longer parses patch_id (it used rsplit("/")) for a title.
• 01a0e7d:
  • pip is added to the test extra. test_installers failed in uv venvs without it.
  • The kernel IPC sockets moved to a short temporary folder, for macOS's 104-byte limit.
• 821470a, 5ce56fd, 5f890f3: docs. They cover the owner decisions (template-sync PR with a CPU-torch CI job before the final v0.2.0; torch skipped in CI until then), the notebook text and the AGENTS.md snapshot.

Verified:

• Notebook torch path: there is no rescale, channel repetition, resize or guessed axes. Axes come from output_axes, and mixed axes and mixed sizes are refused.
• normalization_review: it pairs by patch_id without parsing, never transposes, and uses the brick's compute_plane_histogram. RESERVED_NAMES passes.
• group_parameters_of_run: it calls normalization.group_parameters. Its warnings are shown or re-emitted, and groups that would stop a run are shown in red.
• Step 0: it is identical across the three notebooks and pinned to v0.2.0rc1. The version index agrees, Colab code is guarded and there are no magics.
• setup.py mutations: SOURCE_ROOT="src" fails the construct.yaml layout, and "src/cellpatcher" fails the template layout. No .tools/.github file changed.
• Old main: the new tests give 34 failed and 4 modules that do not import.
• Kernel tests: run 3 times, 39/39 each in 13 to 14 s, with no leaked kernels or socket folders.

Gate (reviewer venvs, head 5ce56fd; check.py --rules exit 0 everywhere; full check.py exit 0 with torch):

Environment	Passed	Skipped
.[test]	1075	41
.[test,zarr]	1096	20
.[test,zarr,notebook]	1124	11
.[test,zarr,notebook] + CPU torch 2.15.0	1173	0

CI gave the same counts, green on 5ce56fd and 5f890f3. The torch tests and the PyTorch notebook kernel run are skipped in CI and were run locally only.

§9 figures:

• Largest file touched: notebook_ui.py, 452 lines.
• Largest function by executable lines: torch_dataset.patch_array_to_tensor, 41 (existing). Among new functions, load_exported_batch, 37.
• Highest complexity: 10 in patch_array_to_tensor. Among new functions, 7 in before_after.
• Most statements: 23, in load_exported_batch.
• Most parameters: 6, in before_after.
• Ratchet: the baseline is unchanged and the # noqa count is 17 on both main and the branch.

Owner question (does not block rc1): should PatchTensorConfig require an explicit scale_policy (or default to "none"), and should repeat_to_rgb stop being the default? Either change breaks the API and needs a CHANGELOG note.

Not tagged.

---

Generated by Claude Code