[guijacquemet]

Fixes #82, Fixes #83, Fixes #98.

Before-audit batch (triage docs/restart/triage-preprocessing-2026-11-02.md, bucket A). Binding: design v0.4 §3, §6, §9. Each claim below was reproduced by a test that fails on main (6ac81ad, after #126) and passes here.

Summary (by module)

#82: technical-repeat counts (dataset.py, report.py, notebook_render.py, notebook_design.py)

• biological_repeats() counted every technical-repeat folder, also one whose images were all excluded (dataset.py ~180). It now counts only folders with at least one image to analyse, so BiologicalRepeat.technical_repeat_count, the balance table and the design's technical_repeat_count are right (ctrl/rep1/techA + techB with techB's only image excluded: 1 technical repeat, not 2).
• BalanceTable gains excluded_images, excluded_folders and not_counted(). The CLI check balance text, the notebook's consistency card and its design card say how many images were excluded and name the folders not counted.
• The single part follows plan Q23 and is only documented: a technical-repeat folder named single is the level's value single and counts as no technical-repeat folder, like a missing level. There is a test for it, and the behaviour is unchanged.

#83: duplicate design columns (design.py, new design_values.py, run_inputs.py)

• Every design reader refuses a header that names a column twice or has a column with an empty or whitespace-only name. That covers read_experiment_design (and so load_design, notebook step 1, check, run --design and the design command) and read_confirmed_design. Names are compared ignoring case and outer whitespace. The DesignFormatError names the column and both positions, for example: the header names the column 'repeat_group' twice (columns 3 and 10), so it is not clear which value to read. Delete one of the two columns. Before this, the lenient reader kept the last column. The test that asserted that now asserts the refusal, with both conflicting and equal values. Nothing is rewritten on refusal: there are tests for load_design and for design --confirm-all.
• RunInputs.design() now reads with read_confirmed_design(path, expected_sha256=manifest.experiment_design_sha256) instead of the lenient reader. Any refusal becomes a RunInputsError, with the refusal as its __cause__. The SHA-256 is now taken from the bytes that are parsed, not from a separate hash followed by a second read.
• The header check and the value parsers (parse_bool, parse_count, is_date, DATE_FORMAT, DesignFormatError) move to design_values.py (pure). design re-exports them under the same names. The two readers share _check_rows (folder listed twice, control disagreeing). DesignFormatError.__module__ is now morphobricks_preprocessing.design_values.

#98: output folder change (notebook_folder.py, notebook_design.py, notebook_ui.py)

• _output_edited used to keep self.design, so the run wrote the old folder's confirmed design and controls into the new folder. Now, when the design card is open and the output folder resolves to another folder, the design is loaded again with _open_design(output_changed=True):
  • the design saved in the new folder is read, and an unreadable one is refused and kept as it is;
  • when the new folder has no design, the questions are asked again, unanswered and with no controls.
• Nothing is carried over, and nothing is written until the user answers. The field fires on every keystroke, so a path typed one character at a time creates no folder; a test checks this.
• The design card says what happened ("its experiment design was read" or "is asked again ... not carried over"), and "Not saved yet: your next answer saves it in ...". The run is gated as before. Changing back (A to B to A) reads A's design again.
• There is no "copy my design" action (owner, 2 November 2026: accepted). The user answers again or copies the file.

Tests

• New: tests/test_notebook_design_output.py. test_notebook_ui.py is already over the size limit, so the new widget tests go here; design_tree moves to tests/tiff_tree.py and both files share it.
• Extended: tests/test_dataset.py, tests/test_confirmed_design.py, tests/test_run_inputs.py.

Decisions (recorded in plan §12, renumbered 61–63 at the merge of #129)

• Q61 ([Sad Catapella B4][Scientific QC] Technical-repeat counts include excluded-only folders and omit explicit single folders #82): only technical-repeat folders with images to analyse are counted, and the text names what is excluded. single stays as Q23 says, documented only.
• Q62 ([Sad Catapella B4][Scientific identity] Lenient experiment-design reader silently selects last duplicate column #83, owner accepted 2 November 2026): duplicate or unnamed header columns are refused by every reader, compared ignoring case and outer whitespace (and invisible format characters, from the review). RunInputs.design() reads strictly.
• Q63 ([Sad Catapella B8][Scientific identity] Changing notebook output directory retains and can overwrite the wrong confirmed experiment design #98): the design belongs to its output folder. On a change it is read again or asked again, and never carried over.
• Contract §9.1, §11, §12 and §16, the user guide (§4 step 1, §8 messages), plan §10 and AGENTS.md are updated. The version stays 0.4.0rc3.

Review fixes (independent review)

• [Sad Catapella B4][Scientific identity] Lenient experiment-design reader silently selects last duplicate column #83, invisible characters (d4b7b14): the header check compared names ignoring case and outer whitespace only, so a second column written \ufeffcondition or Condition\u200b (a byte-order mark or zero-width space inside the header) looked new and the lenient reader (load_design, notebook step 1, check, run --design, design) silently read the first one. Invisible format characters (Unicode Cf) are now ignored when comparing, and a column of only such characters has no name. Plan Q62 and contract §12 say so; 3 new parametrized cases fail on the previous head.
• Output folder under a file (1b8bf87, pre-existing, owner-requested): somefile/out raised an uncaught NotADirectoryError from the run step's refresh, and a run click showed "unexpected problem". current_fingerprint treats a path under a file as gone; the output hint names the file in red; the design is not saved there; step 3 waits ("choose an output folder that is not a file"). Regression test test_an_output_folder_under_a_file_is_refused_with_a_message (fails on the previous head). User guide §4 step 1.
• 4ac8678: docstring line wrap (E501).
• main merged in twice, no rebase: PP-11b: the notebook's Make named-axis copies card #127 (b1c8157, no conflicts) and OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129 (f7257b7: one conflict in plan §12, resolved by keeping OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129's question 60 and renumbering this PR's 62–64 to 61–63, with every reference in the contract and plan §10).

Tests failing on main

16 regression tests fail with main's src and pass here:

• [Sad Catapella B4][Scientific QC] Technical-repeat counts include excluded-only folders and omit explicit single folders #82: 4 in test_dataset.py, plus the design-card test in test_notebook_design_output.py.
• [Sad Catapella B4][Scientific identity] Lenient experiment-design reader silently selects last duplicate column #83: 6 in test_confirmed_design.py (5 parametrized duplicate or unnamed header cases, plus the design command test), the updated 'condition' twice message case, and test_run_inputs.py::test_the_design_is_read_with_the_strict_reader.
• [Sad Catapella B8][Scientific identity] Changing notebook output directory retains and can overwrite the wrong confirmed experiment design #98: 3 widget tests in test_notebook_design_output.py: A to B to A with conflicting controls, a new empty folder typed character by character, and an unreadable design in the new folder.

The later locked-folder test (save failure) was not run against main.

Full gate (local, Python 3.11, .[dev,notebook,zarr], zarr 3.1.6)

• ruff check .: all checks passed. ruff format --check .: 179 files already formatted.
• pytest: 1797 passed, 1 skipped (implementer); review, after merging main (PP-11b: the notebook's Make named-axis copies card #127, OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129): 1852 passed, 1 skipped (skip as root, expected). The skip is test_discovery.py:665, "running with permissions that ignore file modes", which is expected because the run is as root.
• python tools/check.py --rules: exit 0.

§9 figures (measured)

• Baseline entries: 44 → 43 (design.py _count removed).
• design.py: 964 → 926 lines. test_notebook_ui.py: 1537 → 1529.
• New files under the limits: design_values.py has 105 lines and is declared pure in pyproject. test_notebook_design_output.py has 162 lines.
• Other touched files: dataset.py 377, notebook_design.py 454, notebook_folder.py 406, run_inputs.py 592 lines.
• Ratcheted # noqa counts unchanged (ANN 54, PLR0913 3, C901 0, PLR0915 0). Total noqa in *.py is unchanged at 124: one E402 line was edited, none added.

Notes for review

• Plan question numbers are now 61–63 (after OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129's 60); Links stay inside the experiment and masks folders; --log is checked before it opens (#81, #99) #128 renumbers after them.
• main was merged in (the PP-11c: conversion integrity (copy SHA-256, metadata epochs, exact CLI sizes) #126 conversion PR); the one conflict was in the run_inputs.py import lines. Not rebased.
• Widgets were tested headless only, not looked at in a real JupyterLab.
• The output folder under a file is fixed in this PR (see Review fixes).
• The merge commit carries git's default message, without the session trailers.

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

---

Generated by Claude Code

[guijacquemet]

Independent review: merged as feec007 (merge commit, head f7257b7)

Findings fixed on the branch

• [Sad Catapella B4][Scientific identity] Lenient experiment-design reader silently selects last duplicate column #83: a duplicate column hidden behind an invisible character was read silently (d4b7b14). If a second header column was written condition or Condition, the header check saw it as a new column. The lenient reader then read the first column without saying anything, through load_design, notebook step 1, check, run --design and design. The check now ignores Unicode Cf characters when it compares names, and a column made only of such characters counts as having no name. There are 3 new parametrized cases, and they fail on the previous head.
• Output folder under a file (1b8bf87). This one was already on main and the owner asked for it to be fixed here. Setting the output folder to somefile/out raised NotADirectoryError from the run-step refresh, and a run click showed "unexpected problem". Now current_fingerprint treats a path under a file as gone. The output hint names the file in red, the design is not saved there, and step 3 is blocked with "choose an output folder that is not a file". There is a regression test that fails on the previous head, and a user guide §4 line.
• E501 docstring wrap (4ac8678).
• main was merged twice, with no rebase: PP-11b: the notebook's Make named-axis copies card #127 (b1c8157, no conflicts) and OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129 (f7257b7). In plan §12, OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129's question 60 is kept and this PR's 62–64 become 61–63. Every reference in the contract (§9.1, §11, §12) and in plan §10 now uses the new numbers. Links stay inside the experiment and masks folders; --log is checked before it opens (#81, #99) #128 will need to take its numbers after 63.

Each "Fixes" claim checked in the code with adversarial probes

• [Sad Catapella B4][Scientific QC] Technical-repeat counts include excluded-only folders and omit explicit single folders #82: I built 2 conditions × 3 repeats and excluded some images: one folder only partly, four folders completely, and one whole biological repeat. BiologicalRepeat, the balance cells, excluded_images (6) and excluded_folders (4, named) all agree. They also agree with the design the run wrote, read back with read_confirmed_design, and with RunInputs.design(). A restart that brings back an excluded repeat stops with design_not_confirmed, as it should. single is left as plan question 23 says, documented with a test.
• [Sad Catapella B4][Scientific identity] Lenient experiment-design reader silently selects last duplicate column #83: Every read path refuses a duplicate or unnamed header: read_experiment_design, load_design, read_confirmed_design, the CLI check (reported, file untouched), run --design (exit 1, file untouched), design and design --confirm-all (exit 1, file untouched). I tried the variants: a leading BOM, trailing or leading whitespace, a tab, NBSP, a different case, and with this fix BOM or ZWSP inside the header. In no case was the file rewritten. RunInputs.design() refuses a design that is valid and strict but whose SHA-256 differs from the manifest (changed_experiment_design as __cause__).
  • Imports from other bricks: none of the bricks in the workspace imports DesignFormatError or the moved helpers. Checked: CellPatcher 62d2a73, MorphoFeature 4e6970f, MorphoRelate ea19bb0, MorphoEmbed a8e88df, MorphoObjects 2080ff5, fiberfeatures 362f1ba, MorphoCoverage 0674987, normalization f784c95. Their imports from design (read_confirmed_design, DesignCheckError, DESIGN_FILENAME, DesignRow, ExperimentDesign, write_experiment_design) are unchanged. No brick calls RunInputs.design().
• [Sad Catapella B8][Scientific identity] Changing notebook output directory retains and can overwrite the wrong confirmed experiment design #98: Switching outputs A→B→A gives each folder its own controls. A path typed one character at a time, passing a folder that holds an unreadable design, writes nothing anywhere: I snapshotted the whole tree before and after. Relative paths (a, ./a/) read A's design again. An unreadable design in B is refused and kept, and the next answer replaces it. With a new output folder that is locked, the answer is not saved. Confirmations and controls are never carried over, and the run uses only the new folder's design.

Gate (own worktree, uv venv, Python 3.11, .[dev,notebook,zarr], zarr 3.1.6, after the last merge of main)

• ruff check . passes, and ruff format --check . reports 183 files formatted.
• tools/check.py --rules exits 0.
• pytest: 1852 passed, 1 skipped. The skip is test_discovery.py:665, which is expected when running as root.
• CI is green on f7257b7: 7/7 checks, which are lint plus ubuntu, windows and macOS on 3.11 and 3.13.

§9 figures (measured on the touched functions)

• Largest file touched: design.py at 926 lines. It is in the baseline and went down from 964. test_notebook_ui.py is in the baseline at 1529, down from 1537. The largest file touched that is not in the baseline is run_inputs.py at 592.
• New files: design_values.py (115 lines, pure) and test_notebook_design_output.py (187 lines).
• Largest function: _refresh_design_status with 58 executable lines (limit 60; it was 51 on main). Next is _refresh_run_step with 47.
• Highest complexity: 9, in _parse_strict.
• Most parameters: 5, in _design_load_note.
• Most statements: 31, in _parse_strict.
• Ratchet:
  • Baseline entries went from 44 to 43 (design.py _count removed).
  • # noqa ratchet: ANN 53, PLR0913 3, C901 0, PLR0915 0, the same as main. Lines containing # noqa in *.py: 78 on main and 78 here.
  • No new exemptions.
  • Every module still declares a layer (design_values is pure), and the ANN and docstring checks pass via --rules.

Issues #82, #83 and #98 closed automatically when this merged. The branch was auto-deleted and I restored it at f7257b7.

---

Generated by Claude Code