[guijacquemet]

Fixes #81, fixes #99

Before-audit fixes from the 2 November triage (design v0.4 §1 "sources are never touched", §2, §9).

Summary

#81: links stay inside the folder walked (owner policy: contain)

• links.py (new, io): LinkGuard follows a link (file or folder) only when its real path (os.path.realpath) is inside the real path of the folder walked. Three errors:
  • link_outside_folder: the link leads outside. It names each link and its target (ctrl/rep1/b.tif -> /data/other/far.tif), broken links included. The link is not followed.
  • link_loop: the link leads to a folder on its own walk path. It is not followed.
  • linked_twice: one file is reached by two paths: a link and its target, two links, a linked folder and its target, or two hard links (the same device and inode, from the one stat taken per image or mask). Both paths are named.
• What counts as a link: whatever os.DirEntry.is_symlink() reports, plus, on Windows, directory junctions (mount-point reparse tag).
• One helper serves every walk:
  • discovery._Walker.visit: images and folders, plus the links inside each OME-Zarr store (check_store);
  • masks/files._folder_masks: replaces rglob;
  • unrecorded.unrecorded_files: raises LinkRefusedError, which RunInputs.open turns into a RunInputsError;
  • convert: it plans from discovery.
• Hidden and system names are skipped before any link is looked at, so a hidden link is never followed. A contained link to a hidden folder is followed: the hidden folder itself is never walked, so its files are found once, under the name of the link the user made.
• A broken link that stays inside is kept and reported by its reader (unreadable_tiff, unreadable_mask), never skipped.

#99: --log is checked before it is opened

• log_target.py (new, io) refuses a log:
  • inside the experiment or masks folder, given or recorded by the earlier run in the output folder;
  • inside convert's --to;
  • as, or inside, a protected output: plane_histograms.csv, run_manifest.json, the report files, experiment_design.csv, calibration.csv, mask_relationships.json, the lock, .morphobricks-conversion.json, or the .run- staging folders;
  • as the design file given with run --design (an input of the run; review);
  • as a folder, or in a folder that does not exist;
  • as an existing file that has a second name (a hard link).
• Paths are compared resolved. Folders are also compared by device and inode, so another case on macOS or Windows is caught. Output names are compared ignoring case.
• cli.main checks --mask-axes and the recorded experiment first, then the log path, and only then opens the log. An invalid command writes nothing. This applies to every command with --log: run, masks and convert (check, design and calibrate have none).

Owner decisions (2 November 2026)

• Plan §12 question 64 (links: contain; hard links detected; the store scan cost accepted) and 65 (where --log may write). The PR numbered them 60 and 61; after OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129 (question 60) and Experiment design identity and counts: excluded folders, duplicate columns, output folder (#82, #83, #98) #130 (61 to 63) they are 64 and 65, renumbered in the merge of main (2aebb97), with the plan's done-list entry.
• Docs: contract §1 ("Links" row) and §8 (new codes, the log rule); user guide §2, §5 (--log) and §8 (codes); AGENTS.md (repository map).

Review (independent reviewers)

First review: f4381fb, 65e963c (merge of main: #126, #127, #129), 9b372da.

• Owner decision implemented: hard links → linked_twice, both paths named. Detected in the experiment folder (discovery and run) and in the masks folder. 2 regression tests, both failing on fbe6829.
• Finding (fixed): a broken link in the masks folder was skipped, so its image only showed mask_missing. It is now unreadable_mask, naming the link. Regression test fails on fbe6829.
• Finding (fixed): --log compared resolved paths only (case on macOS and Windows; a hard link to a source outside every folder). Regression tests: the hard-link and output-case tests fail on fbe6829; the folder-case test runs only where the file system ignores case (macOS and Windows CI) and is skipped on Linux.
• Contract §8 corrected for OME-Zarr axes and calibration: nothing guessed, nothing silent (#72, #73, #94) #129's 0.1/0.2 refusal (9b372da).

Second review: 610aba6, 2aebb97 (merge of main: #130), 795faab.

• Finding (fixed, 610aba6): run --design my_design.csv --log my_design.csv, with the design outside the output folder, appended the log to the confirmed design and the run went on (exit 0, 207 → 315 bytes). The design file given is now refused (resolved, or the same device and inode). Regression test fails before the fix. Contract §8, user guide §5 and plan question 65 updated.
• §9 (fixed, 795faab): the rewritten _Walker.visit (36 lines) had no docstring; __init__'s # noqa: ANN204 replaced by its annotation.
• Verified by probes (discovery, masks, RunInputs.open, convert, masks command): containment by real path for files and folders; links inside OME-Zarr stores (outside, loop to the store, a link up to an ancestor folder); hard links across leaves; broken links; hidden links and links to hidden folders; the root named through a link; the log checked before it opens and before any write for run, masks and convert; valid logs still written. No other walk of the experiment or masks tree (grep for rglob, os.walk, iterdir, glob, scandir, listdir).
• Not changed: a link inside the experiment to an output folder placed inside it is followed, but that layout is already the error output_inside_experiment.
• Windows junctions: detected in code; not exercised by a test (no junction can be made on Linux CI).

Store scan cost (measurement only, not a performance claim)

discover_experiment, best of 7, on 500 TIFFs plus two stores of 10,000 files each (first review, this container): main (7f55ba1) 34 to 37 ms, this branch 50 to 57 ms, about 20 ms per 20,000 store files plus one stat per image; the implementer measured 84 ms on its machine. Results are identical. The owner accepted this cost.

Tests and gate

• tests/test_links.py: 20 tests. tests/test_cli_log.py: 30 cases.
• Full gate on 795faab (uv venv, Python 3.11, .[dev,notebook,zarr], root): ruff check clean; ruff format: 187 files already formatted; pytest 1901 passed, 2 skipped (test_discovery.py:665, permissions ignored as root; test_cli_log.py:164, a file system that tells case apart, runs on macOS and Windows); tools/check.py --rules passed.

Code standards (§9), measured

• Largest file touched: discovery.py, 609 lines (baselined at 610 on main; 609 here). New links.py 247, log_target.py 182.
• Largest function: _folder_masks, 39 executable lines. Then LinkGuard.issues 37, visit 36 and cli.main 32.
• Highest complexity: visit 12, main 11, _folder_masks 10, log_refusal 8.
• Parameters: at most 5 (run_log_refusal).
• # noqa baseline: ANN 53 → 52; nothing grows; no new exemption.

https://claude.ai/code/session_01G3qp9AxjpGPw9xfWAUJNqc

🤖 Generated with Claude Code

[guijacquemet]

Independent review (second pass, after the first reviewer was cut off): merged as 950f642 (merge commit; parents feec007 and 795faab). The branch was deleted automatically and has been restored at 795faab.

Findings, fixed on the branch

• 610aba6: run --design FILE --log FILE, with the design outside the output folder, appended the log to the confirmed design and the run went on (exit 0; the file grew from 207 to 315 bytes). The design file given is now refused, compared resolved or by device and inode. Regression test test_a_log_over_the_design_given_leaves_it_unchanged fails before the fix. Contract §8, user guide §5 and plan question 65 are updated.
• 795faab (§9): the rewritten _Walker.visit (36 lines) had no docstring, so I added one. __init__'s # noqa: ANN204 is replaced by a return annotation, so the noqa ANN baseline goes from 53 to 52.
• 2aebb97: merge of main (Experiment design identity and counts: excluded folders, duplicate columns, output folder (#82, #83, #98) #130). Plan §12 keeps main's questions 60 to 63. This PR's questions are renumbered 64 (links) and 65 (--log), and the plan's done-list entry with them. No other file cites them.

Verified (probes and code)

• Each "Fixes" claim checked in the code:
  • [Sad Catapella B4][Discovery] Symbolic links can silently import images outside the experiment root #81: LinkGuard checks containment by real path in discovery, in the masks pairing, in unrecorded_files (which RunInputs.open raises as RunInputsError) and in convert (exit 1, nothing written). No other walk of either tree was found by searching for rglob, os.walk, iterdir, glob, scandir and listdir.
  • [Sad Catapella B8][Data safety] CLI --log opens arbitrary files before validating experiment and output paths #99: the log is checked before it is opened for run, masks and convert.
• Links inside OME-Zarr stores behave as follows:
  • a link that leads outside is link_outside_folder;
  • a link to the store itself, or to a folder above it, is link_loop.
• Other link cases behave as follows:
  • hard links across leaves are linked_twice, naming both;
  • broken links are reported, never skipped;
  • hidden links are skipped before they are looked at;
  • a link to a hidden folder is found once;
  • an experiment folder named through a link works.
• Not changed: a link to an output folder placed inside the experiment is followed. That layout is already the error output_inside_experiment.
• Windows junctions are detected in code but not tested.

Gate on 795faab (own worktree, uv venv, Python 3.11, .[dev,notebook,zarr], root)

• ruff check: clean.
• ruff format: 187 files already formatted.
• pytest: 1901 passed, 2 skipped. The skips are test_discovery.py:665, where file permissions are ignored as root, and test_cli_log.py:164, which runs only on a case-insensitive file system.
• tools/check.py --rules: passed.
• CI: all 7 checks green (lint; ubuntu, macOS and Windows on Python 3.11 and 3.13).

§9 figures

• Largest file touched: discovery.py, 609 lines. It was baselined at 610 on main. New files: links.py 247 lines, log_target.py 182.
• Largest function: _folder_masks, 39 executable lines.
• Highest complexity: visit, 12. Next are main 11, _folder_masks 10 and log_refusal 8.
• Most parameters: 5 (run_log_refusal).
• # noqa and the baseline only shrink (ANN 53 → 52). No new exemption.

Store scan cost (first review's measurement, accepted by the owner): about 20 ms per 20,000 store files on this container, against the implementer's 84 ms. Results are identical.

---

Generated by Claude Code