Skip to content

PP-11b: the notebook's Make named-axis copies card - #127

Merged
guijacquemet merged 4 commits into
mainfrom
claude/pp-11b-convert-card
Nov 2, 2026
Merged

guijacquemet merged 4 commits into
mainfrom
claude/pp-11b-convert-card

Conversation

@guijacquemet

@guijacquemet guijacquemet commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

PP-11b of the v0.4.0 plan (design v0.4 §5: "the notebook offers conversion only for files with errors"; contract §17; plan §10 and question 59 (15)). Version stays 0.4.0rc3.

Summary (by module)

  • notebook_convert (ui, new): _ConvertCard, step 2's Make named-axis copies (optional) card. It sits directly under the axis check and is shown only when that check reports axes_yx_swapped, axes_contradict_header, axes_x_before_y or axes_not_named for the roles being edited. It is hidden with folder_step(show_output=False).
    • Roles (nothing guessed). The copies use the roles confirmed with Confirm axis roles. That is only possible when the only finding is axes_not_named, because the axis errors keep that button disabled. Otherwise they use the roles shown once Yes, the preview looks right and then the card's Yes, write the copies with these roles are pressed. That answer stays disabled until the preview is checked. Any change to a role or to the missing-axes answer takes it back, and the button stays disabled until the roles are confirmed.

    • Folder for the copies. Required, with no default. It can be pasted or chosen with Browse…; when no dialog can open, the card shows the Colab "Copy path" card. It is checked with the command's rule.

    • What will happen comes from preview_conversion and reads headers only. It lists the images written as OME-TIFF (with their axis code) and those copied byte for byte (with excluded), and shows the space needed and the free space. Any error that would stop the conversion, or too little space, keeps the button disabled.

    • Make named-axis copies calls convert_experiment with a progress bar. It holds the copy folder's lock and runs the same checks again first. Refusals use the existing cards: the red issue cards, and _LockCard with Remove the lock of a stopped run, which uses the remove_stale_lock rule. An unexpected exception becomes a red card and its traceback goes to Technical details.

    • After the copies, the card shows:

      • the folder;
      • how many images were written, copied and reused;
      • the path of the conversion record;
      • what to do next.

      Check the copies in step 1 sets step 1's experiment folder to the copies and keeps the output folder: an empty box is filled with the originals' default. It then runs the check again. The card also says that Yes, they have a single plane there must be given again (--missing-as-singleton on the command line). It lists the copies whose relabelled Y/X sizes are unknown, which happens unless the header gave X and Y equal, and says to state them under Pixel sizes in step 1.

  • conversion_plan (orchestration; created by PP-11c PP-11c: conversion integrity (copy SHA-256, metadata epochs, exact CLI sizes) #126, which moved plan_conversion and PlannedCopy here; this PR adds to it after merging main):
    • The folder rule is now public as copy_folder_issues; convert_experiment calls the same function.
    • preview_conversion(...) returns a ConversionPreview with the issues, converted and copied images, needed and free bytes, and earlier_conversion. It writes nothing. It is exported from the package.
    • After the merge with main: conversion_plan 320 lines, conversion 290.
  • conversion: ConversionResult.unknown_sizes (new, read-only) maps each image to the roles whose header sizes the copy does not carry.
  • notebook_output._LockCard: optional header, explanation and who, so the same card can name the folder for the copies. The output folder's defaults are unchanged apart from one help sentence ("…only if you are sure none is going on: …").
  • notebook_ui / notebook_axis_step: the mixin, the card's place in axis_box, and _refresh_convert_card(review) called from _mapping_changed. No public method was added (the pinned embedding API is unchanged).
  • Docs:
    • notebook: step 2 gets a "Make named-axis copies (optional)" section, step 1's pixel-sizes item notes copies, the Quick workflow mentions the card, and Common problems quotes the axis and copy-folder messages;
    • user guide: §4 step 2, and §5 "In the notebook";
    • contract §17 "Notebook (PP-11b)";
    • plan §10 PP-11b (implemented) and question 59 (15);
    • README and AGENTS.md.

Independent review: merge of main (#126) into this PR (73b3efb)

Both PRs created conversion_plan.py, and the conflicts were resolved hunk by hunk:

I found nothing in the card itself that needed changing:

  • It computes nothing: the plan, space and fits all come from preview_conversion.
  • Roles come only from the user's own confirmations.
  • The button stays disabled until those confirmations are given and the plan is clean.
  • The folder is checked with the command's rule.
  • show_output=False hides the card.
  • tests/test_embedding_api.py is unchanged.

Gate after the merge: 1799 passed, 1 skipped (test_discovery.py:665, running as root). The headless nbclient notebook test runs, and ruff and tools/check.py --rules are clean.

Decisions and interpretations (please check)

  • How roles are confirmed while the axis errors block step 2's Confirm button. The card has its own ConfirmToggle. It is enabled only after the preview check and is reset by any change to a role or to the missing-axes answer. Step 3 stays gated as before, because confirmed_mapping is not set by the card.
  • Space check. When the folder holds an earlier conversion, too little space for all copies is only a warning, because unchanged copies are reused. convert_experiment decides under the lock.
  • The card lives in step 2 only. Step 1's check never reports the axis errors: it has no confirmed roles.

Tests

  • Full local gate (Python 3.11, .[dev,notebook,zarr]): ruff check and ruff format --check are clean, tools/check.py --rules passes, and pytest gives 1769 passed, 1 skipped, 1 warning. The skip is test_discovery.py:665, expected as root. The warning is the existing tifffile "closed file" warning in test_planes.py. Main has 1750 passed, 1 skipped.
  • CI is green on lint and on Ubuntu, macOS and Windows with Python 3.11 and 3.13 (head 0835973; the macOS jobs were re-run after the runners cancelled them while queued). The first run failed on Windows only: the kept-lock check needs POSIX, and is now its own POSIX-only test.
  • 19 new tests in tests/test_notebook_convert.py, at widget level. The notebook-file test also checks the 3 new buttons, an addition only. tests/test_embedding_api.py is unchanged.
  • All 19 fail on main (run against main's src), as does test_notebook_names_buttons_that_exist. They cover:
    • the card offered under the axis check (swapped, plain stacks) and hidden when the header's roles agree or with show_output=False;
    • the button waiting for the confirmed roles and the folder;
    • the folder rule: inside the experiment, holding it, or a file;
    • the Colab Copy-path card from Browse…;
    • the plan (converted, copied/excluded, space) equal to preview_conversion;
    • not enough space disabling the button;
    • preview_conversion matching plan_conversion;
    • the copies with a progress bar and the record path, and a rerun that reuses them with the same SHA-256;
    • unknown Y/X sizes stated, and nothing stated when X = Y;
    • copy_folder_not_empty on the card;
    • a lock held by a conversion going on, shown on the lock card; on POSIX, the lock of a process running here is kept on the button (skipped on Windows, which cannot tell whether a process runs, like the other lock tests);
    • a stale lock removed only on the button, followed by a successful conversion;
    • an unexpected failure shown as a red card;
    • the missing-axes note;
    • end to end: Y/X-swapped OME stacks are converted, Check the copies in step 1 keeps the output folder, the roles are suggested as ZYX from the copies' headers, the design is kept, and the run is written with axes_from: "header" on every file, experiment_root = the copies and conversion_record_sha256 = the record's.

§9 figures

  • New files, in lines: notebook_convert 433, conversion_plan 284, tests/test_notebook_convert.py 411.
  • Changed files, in lines: conversion 455 (was 587), notebook_output 249, notebook_axis_step 348, notebook_ui 483.
  • Largest new functions, in executable lines: _build_convert_card 49, _on_convert 33, plan_conversion 23 (moved).
  • Highest new complexity: plan_conversion 5 (moved), _plan_html 4.
  • Most parameters: preview_conversion, 6.
  • The baseline is unchanged, no # noqa was added (125 lines carry one on both main and the branch), and C901/PLR0913/PLR0915/ANN pass.

Not verified

  • Headless widgets only. The card was not rendered in a real JupyterLab or on Colab, and the embed_minimal_html visual check was not done here, since no browser was available.
  • Simulated data only.

Colab hand-check list

  1. Run Step 0, then step 1 on an experiment of Y/X-swapped OME-TIFFs on Drive, and press Check folder.
  2. In step 2, swap the Y and X roles. The red axis check appears, with the Make named-axis copies (optional) card under it. The button is disabled.
  3. Press Show preview and Yes, the preview looks right, then Yes, write the copies with these roles.
  4. Press Browse…: the amber card names Copy path. Paste a Drive folder inside the experiment and check that it is refused; then paste one next to the experiment.
  5. Check the plan lists and the space, then press Make named-axis copies. The bar moves, the green card shows the record path, and the pixel-size note lists the copies.
  6. Press Check the copies in step 1. Step 1 shows the copy folder, the same output folder and a clean check; the roles are suggested from the copies.
  7. Confirm the labels, preview, roles and image type, then run step 3. The report says "named-axis copies of …".
  8. Optional: disconnect the runtime during a conversion, reconnect, and convert into the same folder. The lock card appears; Remove the lock of a stopped run, then convert again.

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc


Generated by Claude Code

Step 2's card under the axis check, shown when it reports axes_yx_swapped,
axes_contradict_header, axes_x_before_y or axes_not_named: a required
folder for the copies (the command's rule), the user's confirmed roles,
the plan from preview_conversion (the command's plan_conversion, moved to
conversion_plan), the space, then convert_experiment with a progress bar
under the copy folder's lock; refusals and the stale-lock button on the
existing cards; afterwards the record and Check the copies in step 1.
ConversionResult.unknown_sizes lists the copies whose relabelled sizes are
unknown. Hidden with folder_step(show_output=False).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
…ween calls

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
… the other lock tests

Windows cannot tell whether a process runs, so remove_stale_lock leaves the decision to the user.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
@guijacquemet
guijacquemet marked this pull request as ready for review November 2, 2026 16:32
Conflicts resolved hunk by hunk:
- conversion_plan.py: PP-11c's plan_conversion (epoch binding, dtype, _copied) kept as
  merged; PP-11b's copy_folder_issues, ConversionPreview and preview_conversion added
  after it, with their imports.
- conversion.py: PP-11c's split kept; PP-11b's ConversionResult.unknown_sizes and the call
  to copy_folder_issues (its _folder_issues removed) applied.
- pyproject.toml: PP-11c's layers plus notebook_convert.
- AGENTS.md, contract §17, plan §10 and question 59: both PRs' text (PP-11c marked
  implemented in #126; PP-11b's notebook row replaces "Not yet").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
@guijacquemet
guijacquemet merged commit c9a54f3 into main Nov 2, 2026
12 of 14 checks passed
@guijacquemet
guijacquemet deleted the claude/pp-11b-convert-card branch November 2, 2026 17:30
@guijacquemet
guijacquemet restored the claude/pp-11b-convert-card branch November 2, 2026 17:31

Copy link
Copy Markdown
Contributor Author

Independent review: merged as c9a54f3 (merge commit, head 73b3efb). The branch was auto-deleted and has been restored on 73b3efb.

Merge of main (#126) into this PR, 73b3efb, no rebase
Both PRs created conversion_plan.py, so the conflicts were resolved hunk by hunk:

Checked in the code

  • The UI computes nothing. The plan, space and fits come from preview_conversion, and the card only shows them (enabled = ok and (fits or earlier_conversion), as decided by the owner).
  • Roles are never taken from the headers. They are either confirmed_mapping or the edited roles after Yes, the preview looks right plus the card's own confirmation. Any role or missing-axes change resets them (_refresh_convert_card).
  • Make named-axis copies stays disabled until the roles and the folder are given and the plan is clean. The card plans again just before converting, and convert_experiment checks again under the lock.
  • The copy folder is checked with copy_folder_issues, the same function the CLI and API use through convert_experiment.
  • Check the copies in step 1 keeps the output folder and checks again. The end-to-end test asserts axes_from: "header" on every file, experiment_root = the copies, and conversion_record_sha256.
  • show_output=False hides the card, and tests/test_embedding_api.py is unchanged.

I found nothing in this PR that needed fixing.

Gate (own worktree, Python 3.12, .[dev,notebook,zarr])

  • ruff check and ruff format: clean. tools/check.py --rules: passes.
  • pytest: 1799 passed, 1 skipped (test_discovery.py:665, running as root) and 1 warning (an existing one). The headless nbclient notebook test and the notebook-file tests ran.
  • CI: green on all 7 jobs. The macOS jobs were cancelled while queued for a runner and passed on re-run.

§9 figures (src files touched)

  • Largest file: notebook_ui.py with 483 lines. notebook_convert.py has 433, conversion_plan.py 320.
  • Largest function: _build_convert_card with 49 executable lines.
  • Highest complexity: 6 (_write_check).
  • Most parameters: 7 (convert_experiment, unchanged). The most for a new function is 6 (preview_conversion).
  • Most statements: 26.
  • The baseline is unchanged, the # noqa count is 94 in both main and this branch (src and tests), and layers are declared (notebook_convert ui).

Generated by Claude Code

guijacquemet pushed a commit that referenced this pull request Nov 2, 2026
guijacquemet pushed a commit that referenced this pull request Nov 2, 2026
No conflicts; plan questions on main end at 59, so 62-64 are renumbered once #128
and #129 land.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
guijacquemet pushed a commit that referenced this pull request Nov 2, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
guijacquemet pushed a commit that referenced this pull request Nov 2, 2026
docs/plan.md: main's PP-11b and PP-11c items first, then this PR's links
item; this PR's questions 60 and 61 become 61 and 62, after main's question
60 (OME-Zarr axes and calibration, #129). docs/contract.md: both additions to
the error row kept (main's OME-Zarr codes, this PR's link codes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc
@guijacquemet
guijacquemet deleted the claude/pp-11b-convert-card branch November 14, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants