Keep an unreadable experiment_design.csv untouched in step 1

docs/contract.md

@@ -259,7 +259,7 @@ folder.
 | Controls | None, one or several conditions; every row of a condition has the same value. Recorded only. |
 | Reruns | The file in the output folder is read and brought up to date with the folders found: folders in it keep their group, control mark and confirmation (added or removed images within a folder keep it too, with the counts recorded at confirmation); a new folder joins the group named after it and inherits its condition's control mark, and that group is asked again; folders no longer found are dropped without asking again. |
 | Reading | `read_experiment_design` reads labels as text; `control` and `confirmed` accept `true`/`false` in any case, also `yes`/`no` and `1`/`0`, and empty as false; a byte-order mark is ignored. The user's edits are checked (owner decision of 31 October 2026): `repeat_group` follows the identity rules of §16 (D-e: not empty, `NA`, `nan`, `none`, `null`, `n/a` or `<NA>` in any case, no leading or trailing whitespace, and spelled one way: two groups differing only in case are refused) and `confirmed_on` is empty or a date `YYYY-MM-DD`; a bad edit raises `DesignFormatError` naming the line and column. `design_issues` gives the error `invalid_experiment_design` for such values in a design built in code, so a run stops before writing, and `write_experiment_design` refuses to write them. A missing column, an invalid value, a folder listed twice, rows of one condition disagreeing on `control` or another `schema_version` raise `DesignFormatError`. `ExperimentDesign.repeat_groups()` gives the `{(condition, biological_repeat): repeat_group}` mapping the normalization brick takes. This lenient reader is for the user's own edits; other bricks read the file with the strict `read_confirmed_design` (§16). |
-| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. Step 3 runs only once every group is confirmed. A brick embedding step 1 with `folder_step(show_design=False)` (plan §12 question 31) gets no design: nothing is asked or written, and step 3 runs without one, as `run_experiment(design=None)`. |
+| Notebook | Step 1 asks once the labels are confirmed and writes the file after every answer (unconfirmed groups with `confirmed=false`). A design confirmed for the same folders is loaded with the questions folded away behind **Change the experiment design**. A saved file that cannot be read (`DesignFormatError`, e.g. a `confirmed_on` a spreadsheet program rewrote as `26/10/2026`) is refused and left unchanged (owner decision of 31 October 2026, plan §12 question 42): step 1 shows the refusal, which names the line and column, starts the questions unanswered and writes nothing until the user corrects the file and confirms the labels again, or answers a question, which replaces the file. Nothing falls back to a default design silently, and step 3 does not run until every group is confirmed again. Step 3 runs only once every group is confirmed. A brick embedding step 1 with `folder_step(show_design=False)` (plan §12 question 31) gets no design: nothing is asked or written, and step 3 runs without one, as `run_experiment(design=None)`. |
 | Command line | `run` reads the file from the output folder, or `--design FILE`. Without a fully confirmed design it writes the design brought up to date with the folders (a template with `confirmed=false` for the groups still to confirm, keeping the answers already given) to the output folder, explains the columns, and stops with status 1. Confirmed rows without a date get the date of the run. |
 | Run | `run_experiment(..., design=...)` brings the design up to date with the folders and stops with `design_not_confirmed` or `repeat_group_conflict` if it is not fully confirmed. Otherwise the file is written with the table and published before the manifest, which records its SHA-256 (§9). Without a design the run is unchanged: no file is written, an existing one is left as it is, and `experiment_design_sha256` is null. `plane_histograms.csv` does not depend on the design. |
 

docs/plan.md

@@ -991,3 +991,13 @@ Raised during review; to be decided before the milestones that need them:
     groups differing only in case, or a `confirmed_on` that is not `YYYY-MM-DD`. The
     normalization brick stays at v0.3.2. The version in `pyproject.toml` is 0.3.4 before
     tagging. Every brick pins this tag (D79).*
+42. What does step 1 do with a saved `experiment_design.csv` it cannot read (PR #59
+    follow-up)? Until now it fell back to the default design and wrote it at once, before
+    any answer, so the confirmed answers in a file a spreadsheet program had re-saved (a
+    `confirmed_on` of `26/10/2026`, refused since question 40) were lost.
+    *Decided (31 October 2026): the file is kept untouched and refused. Step 1 shows the
+    refusal, naming the line and column, starts the questions unanswered and writes nothing
+    until the user corrects the file (and confirms the labels again) or answers a question
+    again, which replaces the file. No silent fallback to a default design; step 3 runs
+    only once every group is confirmed again. The command line already refused such a file
+    without writing it.*

docs/user_guide.md

@@ -209,6 +209,11 @@ results or comparing another grouping.
        every answer. Next time, if the folders are the same, they are loaded and the
        questions are folded away; **Change the experiment design** shows them again. A new
        biological-repeat folder asks again only for the group it joins.
+     - If the saved **experiment_design.csv** cannot be read (for example a spreadsheet
+       program rewrote a date such as 2026-10-26 as 26/10/2026), a red box shows why,
+       naming the line and column, and the file is left as it is. Correct the file and
+       undo and confirm the labels again to read it, or answer the questions again: the
+       first answer replaces the file.
    - **Dataset consistency** shows how many images each condition (columns) and
      biological repeat (rows; the repeat groups once paired) has, with the number of technical-repeat folders. A cell is
      amber when it has fewer than half or more than twice the median number of images, or
@@ -622,6 +627,8 @@ nor writes one. To record it as the notebook does:
 from morphobricks_preprocessing import load_design
 
 design, changes, problem = load_design(result.output_folder, check.dataset.repeats)
+if problem:  # an existing file that cannot be read: correct it, do not overwrite it
+    raise SystemExit(problem)
 design = design.with_group(("treated", "rep2_day2"), "rep2")  # pair a repeat
 for group in design.groups:
     design = design.with_confirmed(group, True, "2026-10-26")

src/morphobricks_preprocessing/notebook_ui.py

@@ -221,6 +221,8 @@ class PreprocessingNotebook:
     def __init__(self, experiment_folder: str = "", output_folder: str = ""):
         self.preflight: Preflight | None = None
         self.design: ExperimentDesign | None = None
+        # True while the saved design cannot be read and has not been replaced by an answer.
+        self._design_unreadable = False
         self.dataset = DatasetReport()
         self.mapping: AxisMapping | None = None
         self.confirmed_mapping: AxisMapping | None = None
@@ -763,7 +765,12 @@ def _design_folder(self) -> Path | None:
         return None if check_output_folder(folder, output) else output
 
     def _open_design(self) -> None:
-        """Load the saved design, or start a new one, once the labels are confirmed."""
+        """Load the saved design, or start a new one, once the labels are confirmed.
+
+        A saved file that cannot be read is left as it is (owner decision of 31 October
+        2026): nothing is written until the user corrects it or answers a question again,
+        and the questions start unanswered, so step 3 cannot run on a guessed design.
+        """
         repeats = self.preflight.dataset.repeats
         output = self._design_folder()
         changes, problem, saved = DesignChanges(), None, False
@@ -773,24 +780,35 @@ def _open_design(self) -> None:
             design, changes, problem = load_design(output, repeats)
             saved = problem is None and (output / DESIGN_FILENAME).exists()
         self.design = design
+        self._design_unreadable = problem is not None
         self._build_design_editor()
         # A design confirmed earlier for the same folders is not asked again.
         reopen = not (saved and design.confirmed and not changes.any)
         ui.set_visible(self.design_editor, reopen)
         ui.set_visible(self.design_change, not reopen)
         self._design_note = self._design_load_note(design, changes, problem, saved)
         ui.set_visible(self.design_card, True)
-        self._save_design()
+        if self._design_unreadable:
+            self._design_saved = False
+        else:
+            self._save_design()
         self._design_changed()
 
     def _design_load_note(
         self, design: ExperimentDesign, changes: DesignChanges, problem: str | None, saved: bool
     ) -> str:
         if problem:
             return ui.status(
-                "warn",
-                f"The saved {DESIGN_FILENAME} could not be read, so the questions are asked again.",
-                [problem, "Answering replaces the file."],
+                "error",
+                f"The saved {DESIGN_FILENAME} cannot be read. It was left unchanged.",
+                [
+                    problem,
+                    "Correct the file (a spreadsheet program may have rewritten a date such "
+                    "as 2026-10-26 as 26/10/2026; dates are written YYYY-MM-DD), then undo "
+                    "and confirm the labels again to read it.",
+                    "Or answer the questions below again: your first answer replaces the "
+                    "file with the answers shown here.",
+                ],
             )
         if saved and changes.any:
             lines = []
@@ -928,6 +946,11 @@ def _save_design(self) -> None:
             logging.getLogger(__name__).warning("Cannot save the design", exc_info=True)
             return
         self._design_saved = True
+        if self._design_unreadable:
+            self._design_unreadable = False
+            self._design_note = ui.status(
+                "info", f"Your answers replaced the unreadable {DESIGN_FILENAME}."
+            )
 
     def _design_changed(self) -> None:
         """Update the balance, the messages, the status and the later steps after an answer."""
@@ -949,12 +972,18 @@ def _refresh_design_status(self) -> None:
             return
         note = getattr(self, "_design_note", "")
         saved = getattr(self, "_design_saved", False)
-        where = (
-            f"Saved in <b>{html.escape(str(self._design_folder() / DESIGN_FILENAME))}</b>."
-            if saved
-            else "Not saved: the output folder is not set or cannot be written; the answers "
-            "are kept while the notebook is open and saved with the run."
-        )
+        if saved:
+            where = f"Saved in <b>{html.escape(str(self._design_folder() / DESIGN_FILENAME))}</b>."
+        elif self._design_unreadable:
+            where = (
+                f"Not saved: the unreadable {DESIGN_FILENAME} is kept as it is until you "
+                "correct it or answer a question."
+            )
+        else:
+            where = (
+                "Not saved: the output folder is not set or cannot be written; the answers "
+                "are kept while the notebook is open and saved with the run."
+            )
         controls = ", ".join(html.escape(c) for c in design.controls) or "none"
         if design.confirmed:
             dates = sorted({design.confirmed_on(g) for g in design.groups} - {""})

tests/test_notebook_ui.py

@@ -387,11 +387,68 @@ def test_an_unreadable_design_is_asked_again(tmp_path):
     (tmp_path / "out").mkdir()
     (tmp_path / "out" / "experiment_design.csv").write_bytes(b"not a design")
     nb = labelled(root, tmp_path)
-    assert "could not be read" in nb.design_status.value
+    assert "cannot be read" in nb.design_status.value
+    assert (tmp_path / "out" / "experiment_design.csv").read_bytes() == b"not a design"
     confirm_repeats(nb)
     assert read_experiment_design(tmp_path / "out" / "experiment_design.csv").confirmed
 
 
+def spreadsheet_saved(path):
+    """Rewrite the ISO dates of a saved design as a spreadsheet program may (26/10/2026)."""
+    text = path.read_text(encoding="utf-8")
+    for on in {row.confirmed_on for row in read_experiment_design(path).rows} - {""}:
+        year, month, day = on.split("-")
+        text = text.replace(on, f"{day}/{month}/{year}")
+    path.write_text(text, encoding="utf-8")
+    return path.read_bytes()
+
+
+def test_an_unreadable_design_is_left_untouched_until_answered(tmp_path):
+    # PR #59 follow-up, owner decision of 31 October 2026: the confirmed answers in a file
+    # that cannot be read are never replaced by the default design before the user answers.
+    root = design_tree(tmp_path / "exp", ["ctrl/rep1", "ctrl/rep2", "drug/rep1", "drug/rep2"])
+    nb = labelled(root, tmp_path)
+    nb.control_checks["ctrl"].value = True
+    confirm_repeats(nb)
+    path = tmp_path / "out" / "experiment_design.csv"
+    answers = path.read_bytes()
+    unreadable = spreadsheet_saved(path)
+
+    again = labelled(root, tmp_path)
+    assert path.read_bytes() == unreadable
+    status = again.design_status.value
+    assert "cannot be read" in status and "left unchanged" in status
+    assert "line 2, column confirmed_on" in status
+    assert "mb-error" in status
+    assert not again._design_confirmed()
+    assert ui.is_visible(again.design_editor)
+    # Undoing and confirming the labels again reads the file again: still nothing written.
+    again.labels_confirmed.button.click()
+    again.labels_confirmed.button.click()
+    assert path.read_bytes() == unreadable
+
+    # The corrected file is read with the answers it holds.
+    path.write_bytes(answers)
+    fixed = labelled(root, tmp_path)
+    assert path.read_bytes() == answers
+    assert fixed.design.controls == ("ctrl",)
+    assert fixed.design.confirmed and not ui.is_visible(fixed.design_editor)
+
+
+def test_answering_again_replaces_an_unreadable_design(tmp_path):
+    root = design_tree(tmp_path / "exp", ["ctrl/rep1", "drug/rep1"])
+    nb = labelled(root, tmp_path)
+    confirm_repeats(nb)
+    path = tmp_path / "out" / "experiment_design.csv"
+    spreadsheet_saved(path)
+    again = labelled(root, tmp_path)
+    again.control_checks["drug"].value = True
+    saved = read_experiment_design(path)
+    assert saved.controls == ("drug",) and not saved.confirmed
+    assert "replaced the unreadable" in again.design_status.value
+    assert "mb-error" not in again.design_status.value
+
+
 def test_undoing_the_labels_closes_the_design(tmp_path):
     root = design_tree(tmp_path / "exp", ["ctrl/rep1", "drug/rep1"])
     nb = labelled(root, tmp_path)