[guijacquemet]

Stacked on #187 (the writer, which publication needs): this branch contains #187's commit f71982b, so until #187 merges the diff shows both. The new work is the single commit bf868b2. Independent of #186 (they will conflict only in the CHANGELOG head).

What changed

Design PR 5 (section 3 lifecycle step 2, section 5 row 5) plus owner answer Q2 (records file, now normative in Spec experiment-v1 4.6). All additive; no run, notebook or CLI uses it.

• description_file.read_description(path) -> Description: reads the bytes once, refuses a symbolic link (R_FILE_SYMLINK, also a dangling one; opened with O_NOFOLLOW), then the strict reader. A refusal is a DescriptionError carrying every issue of the first failing phase (phase 0 for the file itself).
• description_transition: every R_* that applies to a candidate offered over the published file, as a set: R_CONFIRMED_ID_CHANGED (4.6 a, b, b2, c), R_USED_ID_DROPPED, R_USED_ID_REUSED (4.9), R_CONFIRMED_ON_CHANGED (4.4a). Reconfirmation(kind, old_id, new_id, confirmed_on) is the user's recorded acceptance; it accepts only the exact change it names, never a used-ID or date refusal, and confirmed_on is required (no default).
• run/description_file.publish_description(folder, candidate, base_sha256=..., confirmations=...), under output_lock: refuse a symlinked file or records file; base None or malformed is R_DRAFT_BASE_MISSING; base differing from the SHA-256 of the file on disk (or absent vs a file that appeared) is R_PUBLISHED_FILE_CHANGED; canonical bytes are re-read by the strict reader; transitions; then atomic write. Every refusal writes nothing. A re-confirmation the candidate does not need is a ValueError, not silently recorded.
• Q2: needed re-confirmations are appended in the same publication to the append-only morphobricks.records.jsonl (one JSON object per line: type, id_kind, old_id, new_id, confirmed_on, old_sha256, new_sha256), O_APPEND plus fsync, never rewritten or truncated; a torn or non-JSON records file refuses before anything is written. If the append fails after the file was replaced, the previous file is restored (or removed for a first publication). read_records(folder) returns the rows for the next run to copy.
• Refusal enum (the R_* codes above) added to description_rules.py, the one table; a test checks each is in Spec section 8. Layers, CHANGELOG, AGENTS.md repo map updated.

Decisions and deviations to review

1. confirmed_on and image counts. Spec 10 says new FOVs in an existing unit do not reopen a confirmation, so a new confirmed_on beside only a changed image_count is R_CONFIRMED_ON_CHANGED here. The Spec's refval.transition treats an image-count change as reopening and accepts it. The oracle cross-check therefore compares with every image_count normalised to 1, and the difference has its own test. This may deserve a sentence in the Spec.
2. Removal confirmations are per entity. By 4.6 (c), removing a condition's last member also removes positions of channels and the folder of a technical repeat, so every such entity needs its own Reconfirmation (P20's removal of DrugA from v01 needs five). The UI will batch them; the records stay one per entity. This follows the text and the oracle.
3. Records format and crash window. The Spec fixes what a record holds (old ID, new ID, date, both SHA-256) but not the encoding; the JSON keys are preprocessing's choice. A process killed between the file replace and the append can leave a published file without its records (documented in the module).
4. Not in this PR: the draft file and its base_sha256 field (design PR 7), the physical checks and derived experiment_design.csv (PRs 8, 10), and copying the records into the run manifest (PR 11). publish_description takes the base as an argument.

Tests

• tests/test_spec_transitions.py (70): t01-t12 each give exactly their listed set (t06 none); every ordered pair of the 48 valid and transition fixtures agrees with the Spec's oracle (with decision 1); a re-confirmation accepts only the change it names.
• tests/test_spec_publication.py (20): P16 (t01/t02 refused, t01 accepted only with records, v16 accepted with unchanged digests), P17 (second draft refused with R_PUBLISHED_FILE_CHANGED, published bytes stay the first's, re-derived draft accepted), P18 (appeared file, six unusable bases), P19 (including the returning DrugA folder), P20 (dropped ID; removal accepted with the ID kept in used_ids), P22 (date byte-identical, t05 refused, accepted after a reopening).
• tests/test_description_file.py (21): P09 (symlink and dangling symlink refused by reader and by publication, link untouched), held lock, records append-only, torn and symlinked records file, failed append restores, invalid candidate or published file.

Result on the old code: none of the modules exist on main, so the new test modules fail at collection (ImportError).

Checks

python tools/check.py: 3791 passed, 4 skipped (case-insensitive file system x3, file-mode permissions x1; none are Spec tests), 2 warnings; Ruff and code rules clean. Symlink tests skip only where the system forbids symbolic links (not the case here, they ran).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm