Skip to content

SWEEP-PP (B): experiment_design.csv is output only, and the pairing card is rebuilt on the draft (PP-02) - #266

Merged
guijacquemet merged 3 commits into
mainfrom
claude/sweep-pp-b
Nov 10, 2026
Merged

guijacquemet merged 3 commits into
mainfrom
claude/sweep-pp-b

Conversation

@guijacquemet

Copy link
Copy Markdown
Contributor

Summary

SWEEP-PP, part B: inventory item PP-02 (owner answers 3 and 4; Spec ADR 0002 AR11 (3)). Stacked on #265 (base claude/sweep-pp). Review and merge that one first.

experiment_design.csv is now an output only and nothing reads it back. The notebook's pairing card is rebuilt on the draft.

Removed

  • Readers for other bricks and for reruns: read_confirmed_design, ConfirmedDesign, DesignCheckError (its codes missing_experiment_design, unreadable_experiment_design and changed_experiment_design are no longer raised here), RunInputs.design(), the lenient read_experiment_design, load_design, DesignChanges.
  • The editor model: ExperimentDesign (with_group, with_confirmed, with_control, dated), default_design, design_edit.py (DesignEdit, DesignEditError, apply_design_edit, parse_repeat_group), and the unused report.design_summary / design_table.
  • Retired parts of the notebook's design card: the hidden comparability questions (group_confirmations, groups_box, _build_group_questions), the hidden control boxes (controls_box, control_checks, _set_control, _sync_controls), the Change the experiment design button, the dead gates _design_pending, _design_items and _design_confirmed, the undescribed _sync_description/_design_pairing of notebook_description, and the described-run overrides that duplicated them.
  • Already removed by PP-C (checked): the design command, run --design, run_experiment(design=), run/design_file.py, the template, design_change.

Kept, as decided

write_experiment_design, DESIGN_COLUMNS, design_derive.py and the manifest's experiment_design_sha256 stay. RunInputs.open still refuses a design file whose SHA-256 is not the manifest's. The rows are now a read-only DesignTable, used by:

  • the run's own check against the folders found: reconcile_design, which now returns the rows only, and design_issues. A folder that is found but not described still gives design_not_confirmed.
  • the strict parse_design_bytes, which the draft's experiment-v1 §7 legacy migration needs (see "Not decided").

Changed

  • Notebook pairing card (notebook_design.py): one row per folder of the draft's base, with a dropdown of the repeat it is paired with. PreprocessingNotebook.pairing replaces .design. A change is a draft answer at once, and the replicates whose folders it changes are asked "same way?" again, as before. The help text says that comparability is confirmed on Check and save.
  • Check and save: check_candidate compares the candidate's derived rows with those of the published description, instead of parsing the file in the output folder. It lost its output argument.
  • Docs updated: contract §9.1, §12 and §16, the user guide's file list and code table, AGENTS.md, conformance, CHANGELOG.

Screenshots

The card was rendered with the real widgets (ipywidgets.embed and headless Chromium), on a synthetic experiment with blob images: ctrl/rep1, ctrl/rep2, drug/rep1, drug/day2, then day2 paired with rep2. Images: main_1_paired_by_name.png, main_2_day2_paired_with_rep2.png (main) and head_1_paired_by_name.png, head_2_day2_paired_with_rep2.png (this head), in the session's scratchpad, not committed. The visible difference: the card's help text now says that comparability is confirmed on Check and save, and the pairing heading has no "1." (there is no second question on the card any more).

Not decided (reported, not changed)

  • PP-03, draft.migrate_legacy_design (and LegacyMigrationRefused, MigrationNote, Draft.origin/legacy_missing, the legacy_folder_missing question, describe's migration of a legacy CSV). The inventory marks it ONLY and plan §3.7 marks it "rule". But the vendored Spec experiment-v1 §7 ("Legacy experiment_design.csv → v1") specifies this migration, and conformance case P30 (AQ12, FR2; tests/test_spec_derived_design.py) tests it, while ADR 0002 says "experiment-v1 unchanged". Removing it drops a Spec conformance case, so it needs a Spec change or an owner ruling. I kept it, so parse_design_bytes and design_issues stay for it.
  • The retired toggles of the described notebook: the labels toggle, ids_confirmed and the global Image type card. They are hidden, not removed (rest of PP-09). This PR removes only the design card's retired parts.

Evidence

Every valid Spec fixture v01 to v37 (44 cases with mask variants, 39 written) was run on main 8776349 and on this head. Compared: the CSV and design bytes, the scrubbed manifest, the refusal codes of the run and of the mask step, the scrubbed mask_representation.json, and RunInputs FOVs with their canonical IDs. 0 differences. v04, v05, v06 and v17 are refused R_HETEROGENEOUS_LAYOUT_NOT_ENABLED on both, and v29 is refused by the helper on both.

Tests

python tools/check.py on this head 245baa7 (Python 3.13, the dev, notebook and zarr extras): ruff check and format clean; pytest 5138 passed, 4 skipped; code rules match their baseline (lowered for design.py and tests/test_notebook_ui.py). Skips, all environment-bound:

  • test_cli_log.py:166, test_folder_rules.py:199 and :230: this file system tells names apart by case.
  • test_discovery.py:665: permissions that ignore file modes.

The count is 11 lower than part A (5149) because the editor-model, lenient-reader and RunInputs.design tests were deleted or merged into test_design.py and test_design_file.py.

Downstream impact

Pins hold; this breaks nobody until a pin moves past it. Callers that must move first (inventory X-03, plus a code search of the default branches):

  • CellPatcher: src/cellpatcher/normalization.py (read_confirmed_design, ConfirmedDesign; inventory normalization.py:115-175), config.py:274-325, setting_checks.py:270, file_runs.py:37,301, and the tests tests/test_normalization.py and tests/test_rgb_normalization.py (mbp.default_design). CP-1 is to read repeat_group from the description.
  • MorphoFeature: experiment_design.py:55-83 and fits.py:79-85 (read_confirmed_design / RunInputs.design(), inventory MF-05). MF-4 is to read repeat_group from the description.
  • MorphoRelate / any embedder of PreprocessingNotebook: .design is now .pairing, and the widgets group_confirmations, groups_box, controls_box, control_checks and design_change are gone. show_design is unchanged.

The shipped notebook and its RELEASE_REF are unchanged.


Generated by Claude Code

… the draft

Owner answers 3 and 4 (Spec ADR 0002 AR11 (3)).

Readers removed: read_confirmed_design, ConfirmedDesign, DesignCheckError,
RunInputs.design(), the lenient read_experiment_design, load_design and
DesignChanges. The derived file is still written by the run and checked by
SHA-256 at RunInputs.open, but nothing reads it back; bricks take repeat_group
from the description (Fov.repeat_group, resolve_request). Check and save now
compares the candidate's derived rows with those of the published description
instead of parsing the file in the output folder (check_candidate loses its
output argument).

Editor model removed: ExperimentDesign (with_group, with_confirmed,
with_control, dated), default_design, design_edit.py (DesignEdit,
DesignEditError, apply_design_edit, parse_repeat_group) and the unused
report.design_summary/design_table. The rows are a read-only DesignTable; the
run's check against the folders (reconcile_design, now returning the rows only,
and design_issues), write_experiment_design, DESIGN_COLUMNS, design_derive and
experiment_design_sha256 stay. parse_design_bytes (strict) stays for the
draft's experiment-v1 section 7 legacy migration.

Notebook: the pairing card is rebuilt on the draft (notebook_design): one row
per folder of the draft's base with its repeat (PreprocessingNotebook.pairing
replaces .design); a change is a draft answer at once and reopens the
replicates whose folders it changes. Removed with it: the hidden
comparability questions, the hidden control boxes and _sync_controls, the
"Change the experiment design" button, the dead _design_pending/_design_items/
_design_confirmed gates and the described-run overrides that duplicated them.
The shipped notebook file is unchanged.

Tests: test_confirmed_design.py becomes test_design_file.py (the run's derived
file parsed with the csv module in the test; the strict parser's refusals);
test_design.py tests DesignTable, the run's check, the writer and the parser;
RunInputs tests read repeat groups from the FOVs; dead design-template setup
removed from CLI tests; the editor-model tests deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
Base automatically changed from claude/sweep-pp to main November 10, 2026 15:04
@guijacquemet
guijacquemet merged commit 78cda50 into main Nov 10, 2026
14 checks passed
@guijacquemet
guijacquemet deleted the claude/sweep-pp-b branch November 10, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants