Fix singleton-axis readiness and preserve outputs on failed updates

docs/contract.md

@@ -33,6 +33,7 @@ must have a supported role. One confirmed mapping per experiment. Plane coordina
 | Unsupported axes | Position-like axes (tile `R`, mosaic `M`, row `K`, column `J`) cannot have a role. If they hold more than one plane, the file is an error. |
 | Singleton axes | An axis of length 1 needs no role; its index is always 0. |
 | Missing axes | A file lacking axes the mapping has is read with those axes as length 1 only when the user confirmed this and only one arrangement fits; its coordinates for them are 0. |
+| Notebook readiness | Confirming a valid singleton mapping resolves its missing-axis error for running; folder, exclusion and all other metadata errors still block processing. |
 | Series | Only the first image series, at full-resolution level 0. A file with more than one series is an error. |
 | Channel names, significant bits | Recorded in the run manifest when metadata provides them; not part of the CSV. Channel names that differ at the same channel index are a warning. |
 
@@ -140,6 +141,17 @@ suggests an action. Technical details go to the diagnostic log.
 UTF-8 JSON next to `plane_histograms.csv`, written after the table and only when the run
 succeeded.
 
+Both files are prepared in a temporary folder before either earlier output is replaced.
+Earlier files are backed up before publication. A caught replacement failure restores the
+previous state, including the absence of files on a first run. If restoration fails, the
+backup folder is retained and its location is reported for manual recovery. Failures to
+remove temporary files after a successful publication are logged.
+
+The two replacements are not jointly atomic: forced process termination between them can
+leave a mismatched pair, detected by `csv_sha256` on the next run and rebuilt. Only one run
+may write to a given output folder at a time. This write protocol does not change either
+file's schema or statistical interpretation.
+
 | Key | Meaning |
 |---|---|
 | `manifest_version`, `schema_version` | Manifest format (`1`) and CSV format (`1`). |

docs/development.md

@@ -46,6 +46,10 @@ functions the tests use.
 - Floats are written at full precision (`repr`); files are UTF-8.
 - Outputs are written to a temporary file and replace the previous version only after a
   complete successful write.
+- Experiment runs stage both the CSV and manifest before publishing either, back up
+  earlier outputs, and restore their previous state if publication raises. Preserve
+  backups if recovery fails. The CSV hash check still detects forced termination between
+  replacements; run only one writer per output folder.
 
 ## Tests
 

docs/plan.md

@@ -153,8 +153,15 @@ new and changed files without reading unchanged pixels. Provide a force-rebuild
 On rerun: add rows for new planes; replace rows for changed files or changed axis/bin
 settings; remove rows for deleted or renamed files (a rename is a delete plus an add).
 Existing rows of unchanged files are never recomputed, because each row depends only on
-its own plane. Write to a temporary file and replace the previous output only after a
-complete successful run.
+its own plane. Prepare the CSV and manifest in a temporary folder on the output filesystem
+before replacing either output. Back up earlier files before publication; if replacing
+either file fails, restore the earlier pair (or remove newly created files on a first run).
+If restoration itself fails, keep the backups and report their location for manual recovery.
+Temporary-file cleanup failures are logged without treating a completed run as failed.
+
+The two replacements are not a single filesystem transaction. If the process is forcibly
+terminated between them, the CSV hash check on the next run detects the mismatch and
+rebuilds the output. Runs writing to the same output folder must be performed one at a time.
 
 A file is unchanged when its size, modification time, labels and axis layout match the
 manifest and the table still holds all its rows. Every row is recomputed when there is no
@@ -216,11 +223,15 @@ API, CLI). The notebook and CLI only call the API.
   mapping saved to the manifest. *Implemented: axis mapping, prefill and validation
   (`axes`), plane-by-plane reading with a memory limit and the preview plane (`planes`),
   and the axis review step of the notebook.*
+  *Review fix: confirmed singleton axes clear the corresponding readiness error while
+  folder, exclusion and other metadata errors continue to block the run.*
 - **Milestone 3 — Histogram engine:** per-plane histograms, extrema, non-finite and
   constant planes, pooling, normalization library. *Core implemented and tested.*
 - **Milestone 4 — CSV and reruns:** write/update the CSV and manifest, detect
   new/changed/deleted planes, force rebuild. *Implemented: run manifest (`manifest`),
   preflight and incremental runs with exclusions (`run`).*
+  *Review fix: stage both files, preserve earlier output on write/publication failures,
+  and keep backup files with a recovery message if restoration fails.*
 - **Milestone 5 — Integration and docs:** public API, notebook, CLI, user documentation.
   *Implemented: summaries (`report`), notebook widgets (`notebook_ui`,
   `notebooks/preprocessing.ipynb`), `check`/`run` command line (`cli`) and the
@@ -278,3 +289,11 @@ Raised during review; to be decided before the milestones that need them:
    removed.*
 7. Folders deeper than the technical-repeat level: warning or error?
    *Decided (October 2026): error, so that a layer of images is never dropped silently.*
+8. High-priority review fixes: how should confirmed singleton axes and failed output
+   publication behave?
+   *Decided (October 2026): notebook readiness uses the confirmed axis review together
+   with all remaining folder, exclusion and metadata issues. Both output files are staged
+   before publication, and a caught publication failure restores the previous state.
+   Backups are retained if restoration fails. Forced termination remains covered by the
+   existing CSV hash check on rerun. The CSV and manifest schemas stay at version 1;
+   their fields and interpretation are unchanged.*

docs/user_guide.md

@@ -107,6 +107,8 @@ confirmed. If you change the experiment folder, everything after it is reset.
      The confirmed roles are locked until you click **Change axis roles**.
    - If some images lack a dimension the others have (often a single channel), a yellow box
      lists them; press **Yes, they have a single plane there** if that is the case.
+     Once you confirm the axis roles, this resolves the missing-axis error for step 3.
+     Any other errors still need to be fixed before running.
 3. **Compute the histograms.**
    - **Before you run** lists anything still to do.
    - **What will happen** names the output folder and files, and says whether an earlier
@@ -184,6 +186,15 @@ The default output folder is next to the experiment folder:
   modification time. Channel names and significant bits are recorded when the metadata
   provides them. Format in [contract.md](contract.md) §9.
 
+The table and manifest are both prepared before the earlier files are replaced. If a
+replacement fails, the earlier files are restored. This needs temporary disk space for
+the new outputs and copies of the earlier pair. If restoration also fails (for example,
+another program locks an output file), the error names a backup folder: close programs
+using the output files and restore those backups before running again.
+
+Run only one writer per output folder. If the process is forcibly stopped during output
+replacement, rerunning checks the table against the manifest and rebuilds a mismatched pair.
+
 ## 7. Running again, and excluding images
 
 Run the notebook or `run` again after adding, changing or removing images:

src/morphobricks_preprocessing/notebook_ui.py

@@ -914,8 +914,24 @@ def _recommended_changed(self) -> None:
 
     def _refresh_run_step(self) -> None:
         pre = self.preflight
+        folder_ok = pre is not None and pre.ok
+        if pre is not None and self.confirmed_mapping is not None:
+            # Axis confirmation can resolve a preflight missing-axes error, but
+            # folder, exclusion and other metadata errors must still block the run.
+            review = review_axes(
+                pre.metadata,
+                self.confirmed_mapping,
+                missing_as_singleton=self.missing_as_singleton.value,
+            )
+            folder_ok = not has_errors(
+                (
+                    *pre.discovery.issues,
+                    *pre.exclusion_issues,
+                    *review.combined_issues(pre.metadata),
+                )
+            )
         items = [
-            (pre is not None and pre.ok, "Check the experiment folder and fix any errors (step 1)"),
+            (folder_ok, "Check the experiment folder and fix any errors (step 1)"),
             (pre is not None and self.labels_confirmed.value, "Confirm the labels (step 1)"),
             (self.confirmed_mapping is not None, "Confirm the axis roles (step 2)"),
         ]
@@ -995,7 +1011,7 @@ def run(self) -> RunResult | None:
                 "The run stopped because of an unexpected problem.",
                 [
                     f"{type(exc).__name__}: {exc}",
-                    "Nothing was written. Open Technical details below for the full report.",
+                    "Open Technical details below for the full report.",
                 ],
             )
             self._show_log()

src/morphobricks_preprocessing/run.py

@@ -18,7 +18,10 @@
 
 import logging
 import os
-from collections.abc import Callable, Iterable
+import shutil
+import tempfile
+from collections.abc import Callable, Iterable, Iterator
+from contextlib import contextmanager
 from dataclasses import dataclass, replace
 from datetime import datetime, timezone
 from pathlib import Path
@@ -385,20 +388,21 @@ def stop(**counts) -> RunResult:
     for record in records:
         per_file[record.source_relative_path] = per_file.get(record.source_relative_path, 0) + 1
     output.mkdir(parents=True, exist_ok=True)
-    csv_path = write_plane_histograms(records, output / OUTPUT_FILENAME)
-    manifest = RunManifest(
-        experiment_root=root.as_posix(),
-        dtype=dtype.name,
-        settings=settings,
-        mapping=mapping,
-        files=tuple(
-            replace(entry, plane_count=per_file[rel]) for rel, entry in sorted(entries.items())
-        ),
-        csv_sha256=file_sha256(csv_path),
-        versions=package_versions(),
-        written_at=datetime.now(timezone.utc).isoformat(timespec="seconds"),
-    )
-    write_manifest(manifest, output / MANIFEST_FILENAME)
+    with _staged_outputs(output) as staged:
+        csv_path = write_plane_histograms(records, staged / OUTPUT_FILENAME)
+        manifest = RunManifest(
+            experiment_root=root.as_posix(),
+            dtype=dtype.name,
+            settings=settings,
+            mapping=mapping,
+            files=tuple(
+                replace(entry, plane_count=per_file[rel]) for rel, entry in sorted(entries.items())
+            ),
+            csv_sha256=file_sha256(csv_path),
+            versions=package_versions(),
+            written_at=datetime.now(timezone.utc).isoformat(timespec="seconds"),
+        )
+        write_manifest(manifest, staged / MANIFEST_FILENAME)
     logger.info(
         "Wrote %d planes (%d files read, %d kept) to %s",
         len(records),
@@ -417,6 +421,63 @@ def stop(**counts) -> RunResult:
     )
 
 
+class _OutputRollbackError(RuntimeError):
+    """Raised when an output update fails and its earlier files cannot be restored."""
+
+
+@contextmanager
+def _staged_outputs(output: Path) -> Iterator[Path]:
+    """Prepare both outputs before publishing, and retain backups if recovery fails."""
+    staged = Path(tempfile.mkdtemp(prefix=".run-", dir=output))
+    keep_backups = False
+    try:
+        yield staged
+        _publish_outputs(output, staged)
+    except _OutputRollbackError:
+        keep_backups = True
+        raise
+    finally:
+        if not keep_backups:
+            try:
+                shutil.rmtree(staged)
+            except OSError:
+                # A leftover temporary directory does not invalidate completed output.
+                logger.warning("Cannot remove temporary output folder %s", staged, exc_info=True)
+
+
+def _publish_outputs(output: Path, staged: Path) -> None:
+    """Replace the output pair, restoring its previous state on a publication failure."""
+    filenames = (OUTPUT_FILENAME, MANIFEST_FILENAME)
+    backup = staged / "previous"
+    backup.mkdir()
+    # Copy before replacing either file, so even a backup failure leaves the
+    # earlier output intact. Keep backups on the same filesystem for atomic restores.
+    for name in filenames:
+        previous = output / name
+        if previous.exists() or previous.is_symlink():
+            shutil.copy2(previous, backup / name, follow_symlinks=False)
+    published: list[str] = []
+    try:
+        for name in filenames:
+            os.replace(staged / name, output / name)
+            published.append(name)
+    except BaseException:
+        try:
+            for name in reversed(published):
+                previous = backup / name
+                if previous.exists() or previous.is_symlink():
+                    os.replace(previous, output / name)
+                else:
+                    (output / name).unlink()
+        except BaseException as exc:
+            raise _OutputRollbackError(
+                "The output update failed and the earlier files could not be fully restored. "
+                f"Backup files have been kept in {backup}. Close programs using the output "
+                "files and restore the backups before running again."
+            ) from exc
+        raise
+
+
 def _read_file(
     root: Path, rel: str, labels: tuple[str, str, str], mapping: AxisMapping, settings
 ) -> list[PlaneRecord]:

tests/test_notebook_ui.py

@@ -8,7 +8,7 @@
 pytest.importorskip("ipywidgets")
 pytest.importorskip("matplotlib")
 
-from morphobricks_preprocessing import Plane, read_manifest  # noqa: E402
+from morphobricks_preprocessing import Plane, read_manifest, read_plane_histograms  # noqa: E402
 from morphobricks_preprocessing import notebook_style as ui  # noqa: E402
 from morphobricks_preprocessing.axes import AxisMapping  # noqa: E402
 from morphobricks_preprocessing.notebook_ui import (  # noqa: E402
@@ -272,6 +272,49 @@ def test_missing_axes_card(tmp_path):
     assert not nb.confirm_button.disabled
 
 
+@pytest.mark.parametrize("blocking_file", [False, True])
+def test_confirmed_missing_axes_allow_running_unless_other_errors_remain(tmp_path, blocking_file):
+    root = tmp_path / "exp"
+    write_data(root / "a.tif", np.zeros((2, 3, 8, 8), np.uint8), axes="ZCYX")
+    write_data(root / "b.tif", np.zeros((2, 8, 8), np.uint8), axes="ZYX")
+    if blocking_file:
+        write_file(root / "bad.tif", b"garbage")
+    nb = checked(root)
+    nb.labels_confirmed.button.click()
+    nb.preview_button.click()
+    nb.preview_checked.button.click()
+    assert nb.run_button.disabled
+    nb.missing_as_singleton.button.click()
+    nb.confirm_button.click()
+    assert nb.confirmed_mapping is not None
+    assert nb.run_button.disabled == blocking_file
+    if blocking_file:
+        assert nb.run() is None
+        assert "could not be opened" in nb.folder_issues.value
+    else:
+        result = nb.run()
+        assert result.written
+        rows = read_plane_histograms(result.csv_path)
+        assert [(r.z, r.c) for r in rows if r.source_relative_path == "b.tif"] == [(0, 0), (1, 0)]
+
+
+def test_undoing_missing_axes_confirmation_blocks_running_again(tmp_path):
+    root = tmp_path / "exp"
+    write_data(root / "a.tif", np.zeros((2, 3, 8, 8), np.uint8), axes="ZCYX")
+    write_data(root / "b.tif", np.zeros((2, 8, 8), np.uint8), axes="ZYX")
+    nb = checked(root)
+    nb.labels_confirmed.button.click()
+    nb.missing_as_singleton.button.click()
+    nb.preview_button.click()
+    nb.preview_checked.button.click()
+    nb.confirm_button.click()
+    assert not nb.run_button.disabled
+    nb.confirm_button.click()  # unlock the roles before undoing singleton confirmation
+    nb.missing_as_singleton.button.click()
+    assert nb.run_button.disabled and nb.confirm_button.disabled
+    assert nb.run() is None
+
+
 def test_conflicting_axes_explain_why_no_roles_are_offered(tmp_path):
     write_data(tmp_path / "exp/a.tif", np.zeros((2, 8, 8), np.uint8), axes="ZYX")
     write_data(tmp_path / "exp/b.tif", np.zeros((2, 8, 8), np.uint8), axes="CYX")
@@ -369,6 +412,7 @@ def broken(*args, **kwargs):
     assert nb.run() is None
     assert "unexpected problem" in nb.run_status.value
     assert "RuntimeError: disk on fire" in nb.run_status.value
+    assert "Nothing was written" not in nb.run_status.value
     assert "Traceback" in nb.details.value
     assert not nb.run_button.disabled