[guijacquemet]

DRAFT, do not merge. This is Spec roadmap Stage 3 S3.3, the preprocessing side. Owner decision of 11 November 2026: enable different channel counts across repeats now. Based on main dd8e86f.

Summary (by module)

• Pin: morphobricks-normalization @ ...@v0.5.0. The tag is on release merge 1291b091fd76e33a33cdc93960aa84d83d6c2cbf, and the pin now uses the tag directly (coordinator, after the owner tagged it).
• description_layouts, run/description_run, description_run_text: R_HETEROGENEOUS_LAYOUT_NOT_ENABLED is now raised only for units of different image types (RGB next to channel units, v29). Units whose channel lists differ go to the per-unit physical checks (R_CHANNEL_COUNT_MISMATCH, R_UNIT_NOT_HOMOGENEOUS, R_HEADER_CONTRADICTION).
• size_checks.channel_count_issues: different_channel_counts now compares only the images inside one folder (one imaging unit), never images across folders. This applies to the folder check and to review_axes, so the notebook's axes step accepts the owner's 3-channel and 4-channel folders. run/steps.header_checks leaves out the replaced channel codes before it decides the bin count and the plane checks (GLOBAL_CHANNEL_CODES moved into run/steps).
• New channel_plan (pure):
  • each image's channel IDs come from its unit's confirmed list;
  • the table header IDs are the description's declared [channels], sorted;
  • channel_coverage classifies each repeat and channel as full, partial or unavailable, and names the units that lack the channel.
• Rows, staging, pipeline:
  • PlaneRecord(..., channels=<unit list>);
  • write_plane_histograms(..., channel_ids=<declared IDs>);
  • each staged file is written with its own IDs;
  • narrow ranges and range_check_issues work by channel ID.
• Manifest version 5 (version 4 is refused: re-run):
  • each files[] entry records channel_ids, which are required and part of the RowKey, so rows recorded under other IDs are recomputed, never relabelled;
  • NarrowRange.channel is an ID;
  • experiment_description gains channel_coverage, and each unit record gains channels.
• New warning partial_channel_coverage: raised when a repeat has a stain in only some of its acquisitions. It never refuses the run.
• intensity, report, notebook_plots, run_labels:
  • every check, box, curve and statistic is keyed by channel ID;
  • group_summaries raises normalization's ChannelScopeIncompleteError (A_SCOPE_INCOMPLETE, "present in k of n planes") for a partly present channel, unless the caller passes partial_channels="planes_with_channel", and the table marks that choice;
  • per-repeat curves are not drawn for a partly present channel;
  • RunLabels no longer maps channel positions.
• RunInputs:
  • reads the table with read_plane_histogram_table;
  • turns a CsvFormatError (a v4 table) into RunInputsError;
  • refuses rows whose channels differ from the IDs the manifest recorded, and refuses an entry whose IDs differ from the unit's current list.
  • The resolver already classified scope per repeat. It is now tested on the extra-stain case and on v17.
• Notebook (minimal):
  • Compare groups refuses partial presence and asks Summarise only the planes that have the channel? (No is the default). It recalculates only after an explicit Yes, marks the rows and writes the choice to Technical details;
  • the intensity panel's channel buttons use the IDs;
  • the axes-step message is reworded for mixed image types.

Decisions / Spec interpretation

• Mixed RGB and channel units stay refused. The Spec (experiment-v1 §4.4b; fixtures README, v29) puts that layout under R_HETEROGENEOUS_LAYOUT_NOT_ENABLED, "lifted at Stage 3". The owner's decision enabled different channel counts, not mixed image types, and Gate G3 says "only then enable heterogeneous processing". This needs an owner decision. If the owner agrees to lift it, one run then holds two pixel types, which needs a design for dtype and bin count.
• P04 conformance tests: they encoded the Stage 1-2 refusal. They are rewritten, citing roadmap S3.3 and the experiment-v1 §8 text "lifted at Stage 3": v04, v05, v06 and v17 are now processed; v29 stays refused. No other Spec test was weakened.
• Partial presence counts per biological repeat over its imaging units (Spec §8). v05 (PHASE only in DrugA/rep1) and v17 give a partial repeat. In the owner's extra-stain case the repeat is full and the other repeats are unavailable.
• HV3 (failed measurement): in preprocessing a plane that cannot be read stops the run (unreadable_planes). Nothing of this run is published, and the earlier generation stays consistent with its own manifest. There is no per-channel ChannelMeasurementFailure path yet (see open questions).

Evidence: homogeneous runs, main dd8e86f (normalization v0.4.0) against this branch (v0.5.0)

I dumped every channel histogram of the same trees from both runs and compared them per plane, with the v4 C index mapped to the v5 ID by the row's channels order. Results:

• v13 (180 channel histograms) and v01 (86): counts, min, max, binning limits, valid, total and non-finite pixel counts are identical. C0 becomes DNA and C1 becomes ACTIN.
• Simulated uint16 TCYX (48) and uint8 CYX (16): same result, all identical.
• Simulated float32 with a robust range, NaN and a hot pixel (32): same result, with 64 non-empty tails. The v5 tail mean and std match the mean and std derived from v4's sums, max relative difference 0.0.

Only the table format changes. The golden science tests (test_histogram_goldens) pass unchanged in substance; only the column names became IDs.

Tests

• python tools/check.py: 5325 passed, 4 skipped, 0 failed; ruff, format and code rules are clean.
• The 4 skips are platform-specific (case-sensitive file system ×3, root ignores file modes ×1). None is in a Spec module, so the Spec modules have 0 skips.
• New tests/test_stage3_channels.py:
  • the owner's extra-stain case end to end: the table is blank, never zero, for EXTRA in repeats 1 and 3; the manifest records channel_ids and coverage; a fit per repeat works; the global fit is refused until opted in; Compare groups refuses; the resolver gives "present in 4 of 12" with repeat 2 full and the others unavailable; reruns and full runs give identical bytes;
  • a single discrepant FOV is refused with R_UNIT_NOT_HOMOGENEOUS; a whole unit with a missing stain gives R_CHANNEL_COUNT_MISMATCH;
  • v04 swapped against unswapped gives identical per-ID histograms and summaries;
  • the conformance plan §4 "Histogram table v5" row on v13, v04, v05, v06 and v17, plus an ID with _ (PAX_488);
  • the v17 partial repeat is recorded;
  • v29 stays refused.
• New tests/test_notebook_extra_stain.py: the notebook goes from the axes step and Channels (4 labels) to save, run and Compare groups, including the partial-presence question.
• New tests/test_channel_counts_per_folder.py.
• Existing tests were updated to IDs, manifest version 5 and the per-folder count rule.
• CI: not yet observed when this PR was opened.

Notebook files touched (to coordinate with the redesign branches claude/redesign3-*)

No .ipynb file was touched. Changed modules:

• notebook_results.py: the partial-presence question and the _group_summaries helper;
• notebook_compact.py: one line, partial_box in Compare groups;
• notebook_plots.py: channel buttons by ID, distribution_png by ID;
• notebook_axis_step.py: one message text.

Notebook tests touched: test_notebook_ui.py, test_notebook_owner_fixes.py, test_notebook_debt.py, plus the new test_notebook_extra_stain.py.

Notes for review / open questions

1. Lifting the RGB and channel-unit mix: owner decision needed (see Decisions).
2. Manifest version bump 4 → 5: consumers that check manifest_version == "4" (MorphoFeature, CellPatcher) now refuse these runs explicitly, as the migration plan wants. Each brick still needs its own S3.3.
3. HV3 per-channel failure reporting: today a failure is reported per file (unreadable_planes). Reporting the plane and the channel via ChannelMeasurementFailure is left for S3.4.
4. Parallel shards: preprocessing has none. Resume and staging use each file's own IDs; byte identity is tested for rerun and full mode.
5. Notebook partial-presence choice: it is recorded in Technical details and the displayed table only. Compare groups writes no file, so nothing is persisted.
6. Check and save does not yet show the partial coverage before the run; the run report and the manifest do.
7. Widgets were tested headless only, not looked at in a real JupyterLab.

---

Generated by Claude Code