[guijacquemet]

PP-C2 of 3 (stacked on #259): PP-C, the first change of v0.5.1. Base claude/pp-c1-goldens. Next: PP-C3 (docs: CHANGELOG, contract, README, user guide, AGENTS). Do not merge before the independent review. Retarget to main once #259 merges.

Owner policy of 10 November 2026: no backward compatibility before the first pipeline release. A published morphobricks.toml is required for every run, on every route (Spec ADR 0002 AR10, AR11; Stage 2 plan §0.1, §0.2, the PP-C row).

Summary (by module)

Core and run paths

• required_description (new, io): is_published, description_required_issue; run_inputs_error.DescriptionRequiredError (A_DESCRIPTION_REQUIRED, exported). The message names the fix: the notebook's describe steps (Check and save), or the CLI's describe to see what is open.
• run_experiment: refused before anything is read or written without a published description (one issue, no report, no lock). The run derives experiment_design.csv from the description (derive_design) and writes it with the table; design= is gone.
• prepare_mask_step / CLI masks: refused (MaskStep.refusal). CLI check and run: refused, exit 1, nothing written.
• RunInputs.open: DescriptionRequiredError; description(), section_digests(), images_without_header_channel_names(), references(), mask_semantics() never return None.
• preflight, remasked, masks.find_masks: with_description removed (R_FOLDER_NOT_NFC, R_MASK_TYPE_NAME_INVALID always apply); run.preflight.description_in_use removed; description_run.NO_DESCRIPTION warning removed.
• experiment_design.csv as an input removed: cli_design.py (design command), run/design_file.py (write_design_template, write_design, edit_design, design_for_run), run --design, check's template (write_check_outputs, CheckOutputs; write_check_report stays), recorded.design_change, planned_changes(design=), run_summary(design_template=), run_log_refusal(design=).
• manifest_versions: only "4" is written and read (owner ruling 16). "1", "2", "3" and later versions are refused with "Run preprocessing again" (UnsupportedVersionError → ManifestFormatError); a manifest without experiment_description is refused. Removed STRICT_MANIFEST_VERSION, DESCRIBED_MANIFEST_VERSION, LEGACY_MANIFEST_VERSION, version_for, the manifest_before_v040 source issue and the version-1 branches of reuse, recorded, sources, run_inputs. The next run over a refused manifest reads every image again.

Notebook

• PreprocessingNotebook(experiment_folder, output_folder): no describe argument; every _describe branch removed. The design card no longer reads or writes experiment_design.csv (its undescribed _open_design, _save_design, _design_load_note, _refresh_design_status, _design_pending, _folder_cell and the check's template removed). The shipped notebook calls PreprocessingNotebook(). show_design stays (embedding API).

Test fixtures

• testing.make_clean_run makes a described run of the same tree and FOV keys (ctrl/drug, rep1, a/b): it publishes testing.clean_description(...) (channels ch1..chN, one instances mask per mask type; design=True makes drug reference ctrl), always writes the derived experiment_design.csv, and runs the mask step when there are masks. New: testing_described.publish_and_run.
• tests/described_tree.py (test-only, never in the package): describe_tree publishes the description the user would confirm for a tree a test wrote, every proposal accepted (IDs from the draft rules, a fixed confirmation date, header names as aliases, instances masks); run_described, main_described, check_masks, without_mask_step, description_entry. 70 test files run described trees now instead of undescribed ones (not deleted).
• Undescribed-only and legacy-only tests deleted (below); test_strict_manifest.py + test_described_manifest_version.py → test_manifest_version.py (7 tests); released_code.py, tests/data/rc4_run_manifest.json, make_rc4_manifest.py deleted; CI's fetch-depth: 0 removed.
• position_id stays the file stem (owner ruling 2): no test or code changed for it.

What was removed (counts, PP-C2)

• Source: 2 modules (cli_design.py, run/design_file.py), src diff +520/−1440 lines; 1 module added.
• Tests: 116 test functions removed or replaced: 4 whole files (test_design_command.py 10, test_strict_manifest.py 9, test_described_manifest_version.py 6, test_notebook_design_output.py 6) plus released_code.py, and 85 functions in 29 other files that checked only undescribed or legacy behaviour (v0.4 notebook design/labels/image-type cards, the design command and --design, the undescribed-run warning and None answers, version 1/2/3 and rc4 manifests). 22 tests are new or rewritten as the described equivalent (refusals on every route, the derived design, test_manifest_version.py, the unit-level checks). 70 test files were converted to described runs.

Deleted test functions, by file

• tests/test_a_error_classes.py: test_section_digests_are_none_without_a_description
• tests/test_check_sources.py: test_a_version_1_manifest_fails_every_image
• tests/test_cli_exit_status.py: test_check_exits_with_0_on_warnings_and_an_unconfirmed_design, test_check_exits_with_1_on_a_refused_design, test_check_exits_with_1_on_a_repeat_group_conflict, test_design_exits_with_1_when_it_reports_a_conflict
• tests/test_cli_log.py: test_a_log_over_the_design_given_leaves_it_unchanged
• tests/test_confirmed_design.py: test_a_spreadsheet_edit_is_read_strictly_once_the_run_has_written_it, test_the_design_command_does_not_rewrite_a_design_with_a_column_named_twice
• whole file tests/test_described_manifest_version.py: 6 tests
• tests/test_described_unit_checks.py: test_the_undescribed_notebook_keeps_the_folder_refusal
• tests/test_design.py: test_a_run_with_an_invalid_date_stops_before_writing, test_cli_check_refuses_an_unreadable_design_without_writing, test_cli_check_reports_the_design, test_cli_design_option, test_cli_run_writes_a_template_and_stops, test_cli_template_keeps_the_answers_given, test_cli_writes_no_template_while_another_run_holds_the_lock, test_run_needs_a_confirmed_design_when_given_one, test_run_refuses_a_design_for_other_folders, test_run_without_a_design_leaves_an_existing_one, test_run_writes_the_design_and_records_its_hash, test_the_cli_dates_a_hand_confirmation_as_an_iso_date
• whole file tests/test_design_command.py: 10 tests
• tests/test_f10_check_and_named_masks.py: test_a_mask_shared_by_its_header_in_an_f10_manifest_is_refused
• tests/test_make_described_run.py: test_make_clean_run_without_a_description_is_unchanged
• tests/test_manifest.py: test_each_files_axis_roles_round_trip_and_are_none_when_absent, test_position_id_round_trips_and_is_none_when_absent_from_a_v1_manifest
• tests/test_manifest_position_id.py: test_a_version_1_manifest_is_still_read_without_or_with_any_position, test_the_rc4_manifest_still_opens
• tests/test_manifest_v2.py: test_a_version_1_run_gives_no_reusable_row
• tests/test_modes_lock_report.py: test_a_changed_design_is_named_with_its_effect
• tests/test_notebook_cards.py: test_a_design_answer_is_not_saved_in_an_output_folder_inside_the_masks, test_a_design_answer_is_not_saved_while_another_run_holds_the_lock, test_a_design_that_cannot_be_saved_reaches_the_technical_details, test_a_refused_confirmation_is_shown_and_not_taken, test_design_answers_use_the_design_command_rules, test_the_check_writes_its_report_and_the_design_template
• tests/test_notebook_check_save.py: test_without_describe_the_v04_cards_stay
• tests/test_notebook_description.py: test_a_flat_experiment_asks_for_no_condition_or_technical_repeat_names, test_a_saved_draft_resumes_with_its_answers_shown, test_a_saved_pairing_that_differs_from_the_design_follows_the_design, test_pairing_and_unpairing_ask_only_for_the_names_that_are_new, test_the_comparable_date_of_a_repeat_is_the_design_cards_click, test_the_design_summary_follows_the_references, test_the_ids_gate_neither_step_2_nor_run
• whole file tests/test_notebook_design_output.py: 6 tests
• tests/test_notebook_modules.py: test_a_design_that_cannot_be_saved_is_logged_under_the_package_logger
• tests/test_notebook_name_refusals.py: test_p21_p29_without_a_description_the_folder_check_is_as_in_v0_4, test_p21_the_v0_4_folder_check_with_a_published_description_refuses_too
• tests/test_notebook_prefill.py: test_the_recorded_image_type_is_preselected, test_the_recorded_image_type_still_needs_confirming
• tests/test_notebook_ui.py: test_a_confirmed_design_is_not_asked_again, test_a_new_repeat_asks_only_for_its_group, test_an_unreadable_design_is_asked_again, test_an_unreadable_design_is_left_untouched_until_answered, test_answering_again_replaces_an_unreadable_design, test_changing_the_axis_roles_asks_for_the_image_type_again, test_confirmations_are_buttons_that_can_be_undone, test_controls_are_saved, test_design_is_asked_after_the_labels_and_saved, test_float_rgb_images_cannot_be_rgb, test_folder_step_shows_the_design_by_default, test_folder_step_without_the_design_card_or_its_words, test_image_type_is_asked_after_the_axis_roles, test_next_steps_follow_confirmation_and_folder_changes, test_pairing_a_repeat_updates_the_groups_and_the_balance, test_rgb_is_suggested_but_only_used_once_confirmed, test_the_run_writes_the_design, test_two_folders_of_a_condition_cannot_be_confirmed, test_undoing_the_labels_closes_the_design
• tests/test_pp10_api.py: test_write_design_holds_the_lock
• tests/test_run_description.py: test_a_run_without_a_description_is_the_v04_run_with_a_warning
• tests/test_run_inputs.py: test_a_version_1_run_is_refused
• tests/test_run_inputs_semantic.py: test_a_description_published_after_a_run_without_one_changes_nothing, test_a_run_without_a_description_is_unchanged, test_a_run_without_a_description_reads_masks_as_before
• tests/test_run_page_labels.py: test_an_undescribed_run_keeps_the_c_index, test_an_undescribed_run_keeps_the_folder_names
• tests/test_spec_discovery.py: test_description_in_use_looks_only_at_the_published_file, test_p21_p29_an_undescribed_run_of_the_same_trees_is_as_in_v0_4, test_p21_p29_the_command_line_check_without_a_description_is_as_in_v0_4, test_p21_p29_the_mask_step_without_a_description_is_as_in_v0_4, test_p21_without_a_description_discovery_is_as_in_v0_4, test_p29_without_a_description_mask_types_are_accepted_as_in_v0_4
• tests/test_spec_run_aliases.py: test_the_same_folder_without_a_description_is_still_refused_by_the_v0_4_check
• tests/test_stopped_run_text.py: test_the_cli_names_the_report_and_the_design_template
• whole file tests/test_strict_manifest.py: 9 tests
• tests/test_unnamed_header_images.py: test_an_undescribed_run_has_no_new_key_anywhere

Decisions (made here, for review)

• The run derives the design from the description on every run (before: the notebook passed it, make_described_run did not), so experiment_design.csv is always an output and experiment_design_sha256 is always set.
• The A_DESCRIPTION_REQUIRED refusal comes before anything is read (like output_locked), after the output-inside-experiment/masks checks; it writes no report.
• Descriptions declare one arrangement of axes per unit, so tests that mixed TCYX and TYX images in one folder (missing-axes, OME-Zarr + TIFF) now put them in separate unit folders; the science checked is unchanged.
• Left for SWEEP-PP (named in the plan, not removed here): read_experiment_design (used only by draft.migrate_legacy_design, PP-03), ExperimentDesign/default_design/apply_design_edit behind the pairing card (answer 4), read_confirmed_design/RunInputs.design (answer 3, until CP-1/MF-4), the tolerant version-1 field reading in manifest_fields (PP-05), the conversion record version 2 (PP-06), source_fingerprint (PP-08), the global --image-type and the retired notebook widgets (image-type card), docs/migration-0.3-to-0.4.md.
• Not touched: "the 0/1 interim mask labelling in the mask step" named in the plan's PP-C row: preprocessing has none (the 0/1 relabelling is MorphoFeature Bind masks to their fingerprints for the whole run and the relationship scan #180, MF-3).

Notes for review

• Notebook pin window: notebooks/MorphoBricks_Preprocessing.ipynb now calls PreprocessingNotebook() but still installs RELEASE_REF of v0.5.0, where the default is the undescribed mode. Until the v0.5.1 release step moves RELEASE_REF, a user opening main's notebook gets v0.5.0 without the describe steps. The release PR must move the pin.
• make_clean_run's manifest records its masks not yet checked (the mask step runs after the run); RunInputs.read_mask reads the live mask_representation.json, so masks read as checked. A rerun records checked.
• convert's printed next commands (check, run) still need a described output folder; the refusal names the fix.
• Verified headless only (ipywidgets), not in a real JupyterLab.

Tests

• ruff check ., ruff format --check . clean; code rules against the baseline: no violation (baseline lowered: run_experiment parameters 16 → 15 and two test files shorter).
• pytest (Python 3.13, Linux): 4965 passed, 4 skipped (environment skips: case-sensitive-name tests ×3, file modes ignored as root ×1); Spec modules (test_spec_*.py): 1665 passed, 0 skipped.
• Not run here: Windows/macOS and Python 3.11 (CI).

Downstream impact (when each brick moves its pin to v0.5.1)

• Every brick: a run needs a published morphobricks.toml; run_experiment(design=...), PreprocessingNotebook(describe=...), edit_design, write_check_outputs/CheckOutputs, description_in_use, with_description, version_for and the *_MANIFEST_VERSION aliases (except MANIFEST_VERSION, now "4") are gone; run folders written by v0.5.0 or earlier are refused until run again; RunInputs.description() is never None.
• CellPatcher (347c61d): make_clean_run (10 test files) now gives a described run: derived experiment_design.csv always present, masks checked, channel IDs ch1/ch2, manifest v4; design=True still accepted (now a reference). tests/helpers.py:115 calls run_experiment undescribed → refused. CP-1's empty-ID path for undescribed runs has no caller left.
• MorphoFeature (564d6e1): tests/clean_runs.py:82 via make_clean_run (as above); notebooks/MorphoBricks_end_to_end.ipynb:414-422 builds PreprocessingNotebook(...) in the default (formerly undescribed) mode → its run waits for Check and save: the notebook must show the describe steps (steps() or the channels/masks/check-save pages). Undescribed-run tests and the BY_NAME branch lose their input.
• MorphoRelate (12c08a4): notebook_steps.py:42,101,198 embeds folder_step(show_design=False)/axis_step/run_step → its preprocessing run is refused until the experiment is described and saved (it must add the describe steps, X-01); notebook_masks.py:83 and tools/preprocessing_runs.py:90 call run_experiment without a description → refused. test_embedding_api.py's pin of __init__ changed (no describe).
• fiberfeatures (600f388): tests/run_fixtures.py:70 and docs/example_run.py:30 call run_experiment without a description → refused.
• MorphoObjects, MorphoEmbed, MorphoCoverage: no direct use found; affected only through MorphoFeature's fixtures.

---

Generated by Claude Code

[guijacquemet]

Independent review: sound after fixes. Pushed onto this branch:

• 1efc425 Notebook pin window: the shipped notebook goes back to its 3f10545 cell, PreprocessingNotebook(describe=True), which is the API at RELEASE_REF fd7afd3. New test test_the_shipped_cell_calls_the_api_of_its_release_ref records the call for each ref in RELEASE_CALLS and fails for a RELEASE_REF that has no entry. The kernel test runs this checkout, so it swaps in this checkout's call. The release check list gets an item for this. The v0.5.1 release PR moves RELEASE_REF and the call together.
• a7006de A_DESCRIPTION_REQUIRED now comes before anything is read or written:
  • run_experiment used to run the preflight (every image header) and read the recorded run when the output folder was inside the experiment or the masks.
  • prepare_mask_step(None, out) and CLI run/masks used to read the recorded run first.
  • The CLI used to open --log and apply --remove-stale-lock before refusing.
  • Regression tests are in test_run_description.py and test_cli_exit_status.py.
• 80d6f9f Refusal texts: design_not_confirmed, repeat_group_conflict and repeat_missing_conditions still named the removed design command (--confirm) and editing experiment_design.csv. They now name the describe steps, the Replicates page and Check and save (regression test). Stale docstrings are fixed. Two assertions lost in the conversion are restored: the run's missing_axes on the mixed TIFF/OME-Zarr tree, and the error severity of channel_names_differ.
• 5a21cb5 follow-up to one test.

Parity check, main 3f10545 vs this stack: I compared v01, v03, v08, v08 with a bad binary mask, v09, v13, v28, v33, v34, v04, v05, and the P21/P29 trees. Histograms, manifests (apart from the version and file mtimes), the derived CSV, records, mask representation, the RunInputs identities and the refusal codes are all identical. The only differences:

• main's CLI wrote a design template (removed);
• the refusal action texts above.

Gate on 5a21cb5: ruff and code rules clean; 4972 passed, 4 skipped (environment skips); Spec modules 1665 passed, 0 skipped; CI green.

Reported, not changed:

• make_clean_run's manifest records masks as not yet checked. The manifest status is the run-time snapshot that the contract defines, and RunInputs reads the live mask_representation.json. This matches the real flow (run, then mask step) and main's described runs, but anyone reading the manifest directly sees the snapshot.
• The notebook's Check folder still writes discovery_report.json before a description exists. It is the first describe step; CLI check refuses instead.

---

Generated by Claude Code