Phase 2, theme 1: retention markers + prune subprocess + Tools run record, and run identity (RUN-01..07)

.github/workflows/lint.yml

@@ -1,4 +1,4 @@
-# Lints the QuPath script (LabConstrictorTools.groovy) with npm-groovy-lint (CodeNarc rules); the rules are set in .groovylintrc.json.
+# Lints the QuPath script (the files listed in parts.txt, joined into one) with npm-groovy-lint (CodeNarc rules); the rules are set in .groovylintrc.json.
 #
 # The GUI tests in tests/ run in their own workflow, qupath-gui.yml.
 name: lint
@@ -20,9 +20,27 @@ jobs:
       - uses: actions/setup-node@v4
         with:
           node-version: "22"
+      - name: Part files stay below 600 lines (LabConstrictorTools.groovy is the grandfathered main script, from before the split)
+        run: |
+          RES=src/main/resources/org/cellmigrationlab/labconstrictor/qupath
+          status=0
+          while IFS= read -r part || [ -n "$part" ]; do
+            [ -z "$part" ] || [ "$part" = "LabConstrictorTools.groovy" ] && continue
+            lines=$(wc -l < "$RES/$part")
+            echo "$part: $lines lines"
+            [ "$lines" -lt 600 ] || { echo "::error file=$RES/$part::$part has $lines lines (the limit is 600): split it into a new file and list it in parts.txt"; status=1; }
+          done < "$RES/parts.txt"
+          exit $status
+      - name: Join the script files (parts.txt) into the one script the extension runs
+        # CodeNarc reads one file at a time, so a class used from another file would not resolve: the joined text is what is linted
+        run: |
+          RES=src/main/resources/org/cellmigrationlab/labconstrictor/qupath
+          mkdir lint-input
+          { while IFS= read -r part || [ -n "$part" ]; do [ -z "$part" ] || { cat "$RES/$part"; echo; }; done < "$RES/parts.txt"; } > lint-input/LabConstrictorTools-all.groovy
       - name: Lint the script (fails on any finding)
         run: >
           npx --yes npm-groovy-lint@18.0.0
-          --path src/main/resources/org/cellmigrationlab/labconstrictor/qupath
+          --path lint-input
           --files "**/*.groovy"
+          --config .groovylintrc.json
           --failon info

README.md

@@ -10,7 +10,9 @@ so QuPath never imports the app's packages and needs no Python of its own. Same
 2. Drop `labconstrictor-qupath-0.1.0.jar` on the QuPath window (or into the QuPath extensions folder) and restart QuPath.
 3. **Extensions > LabConstrictor tools...**
 
-Without the jar: open `src/main/resources/org/cellmigrationlab/labconstrictor/qupath/LabConstrictorTools.groovy` in QuPath's Script Editor and run it.
+Without the jar: the script is several files now (`src/main/resources/org/cellmigrationlab/labconstrictor/qupath/`, joined in the order of `parts.txt`; the first is the main script). `./build.sh` also writes `target/LabConstrictorTools-all.groovy`, all of them in one file: open that one in QuPath's Script Editor and run it. The main script alone no longer runs.
+
+Needs **labconstrictor-tools 0.3.0a1 or newer** in the app's environment (the run record and the clean-up use its helpers and its `prune` command). A run of an app with an older one is refused with `this app's labconstrictor-tools 0.2.0rc1 is older than the 0.3.0a1 this host needs: update it`.
 
 ## What it does
 * Pick an app and a tool. Numbers (range, unit), choices, checkboxes, text, files and folders become fields; parameters of a group sit under a heading, advanced ones in a collapsed section, optional ones without a default have a "set" box (unticked = the tool gets nothing), and `enabled_when` greys fields out.
@@ -21,7 +23,8 @@ Without the jar: open `src/main/resources/org/cellmigrationlab/labconstrictor/qu
 * **Points** found in the image that is open in QuPath become a point annotation named `<app>:<output>` (replaced by the next run when the output declares `Replace()`); points of any other image are shown as a table.
 * "No match" is a message, not an error. Progress bar, Cancel (a tool that ignores it is killed after 3 s), worker kept between runs (or not), details of the last run, no worker left behind when the window closes.
 * **Trust checks** on every registry entry (same rules as the Python registry and the Fiji script): the entry names its five text fields; the app name is a plain name; the interpreter lies inside the install prefix (which is not a filesystem root); the entry and its schema file are owned by you (in a shared folder also by root) and not writable by group or others; the schema sits next to the entry; the interpreter, its folder and the prefix are not writable by everybody (a sticky folder like `/tmp` excepted); tool ids and parameter names are plain names, parameter names Python identifiers (a parameter name becomes a file name). An entry that fails is listed under "skipped" with its reason and never started. On Windows the owner and permission checks do not apply.
-* Results are kept in `~/.labconstrictor/results/<time>_<6 hex>_<app>_<tool>` (names made safe), like the command line. The host never deletes a results folder or a run record (retention belongs to the Tools reference code, QuPath issue #14). Every run that reached the tool leaves a record `~/.labconstrictor/runs/<time>_<app>_<tool>/run.json` (none ever deleted by the host, the Tools format plus `"host": "qupath"`), and every log line is also appended to the shared `~/.labconstrictor/logs/labconstrictor.log` (Tools line format, `qupath:` prefix, 1 MB rotation with 5 backups), so `labconstrictor-tools logs` and `support-bundle` see QuPath runs.
+* **Results and run records, and their clean-up** (SPEC_WRONG_SCIENCE section 3). Each run makes a results folder `~/.labconstrictor/results/<time>_<6 hex>_<app>_<tool>` (names made safe, like the command line) and a run record folder `~/.labconstrictor/runs/<time>_<app>_<tool>`. **This host never deletes anything there.** It writes the two marker files of the spec: `.lc-owner.json` first, when the folder is made (a QuPath that dies mid-run leaves an owned, unfinished folder that nobody removes), and `.lc-done.json` last, atomically, when the run ends: `ok` (completed), `cancelled` (the tool honoured Cancel, or the worker had to be killed after Cancel was pressed, or the window was closed during the run), `failed` (the tool failed, or the worker died with no Cancel pressed). The status comes from the Tools helper `labconstrictor_tools.runs.done_status`. Then it starts `labconstrictor-tools prune` (`<the app's Python> -m labconstrictor_tools prune`) in a subprocess and does not wait for it: the newest 20 finished results and the newest 50 run records are kept, and Tools removes the older ones, **only** folders that carry a valid owner marker and a done marker. Folders from older versions (no marker), foreign folders, folders of runs that are still going or that crashed with QuPath (no done marker), links and files are never removed and do not use up the quota. A prune that cannot start, or that fails, is written to the log (`prune could not be started ...` / `prune ended with exit code N`) and never fails or delays the run; the folders then stay until a `prune` works. The older folders written by earlier versions of this script have no marker, so they are never cleaned up automatically (Tools has a follow-up item for an explicit `clean --legacy`).
+* **Run record** `runs/<...>/run.json`: built by the Tools helper `labconstrictor_tools.runs.build_record` and written with `record_json` (this script keeps no copy of the layout), plus `"host": "qupath"`, the `run_id` and `started_utc` of the run, and the `_calibration` input. A crash after Cancel keeps its crash text there (status `CRASHED`). If the record cannot be built (the app's Python cannot start, the helper fails) the status line says `(the run record could not be written (...); its folders stay as they are and are never removed)`. Every log line is also appended to the shared `~/.labconstrictor/logs/labconstrictor.log` (Tools line format, `qupath:` prefix, 1 MB rotation with 5 backups), so `labconstrictor-tools logs` and `support-bundle` see QuPath runs.
 * **Same rules as the other hosts** (the host-parity inventory of labconstrictor-tools): a form shows parameters in the Napari/Fiji order (a group gathered where it first appears; a Collapsed group is folded where it first appears; advanced last); `clear_after_run` resets every type of field; `enabled_when` follows text, number, choice, yes/no and "set" drivers (`""` counts as unset, numbers compare as numbers); a whole-number box holds ±(2^53 - 1) at most, declared limits or not (the largest whole number a double holds exactly, Tools `notebook.MAX_FORM_INTEGER`; a typed number beyond it stops at the limit), a decimal box without declared limits ±1e15; a choice of integers is sent as an integer; a missing input says `'L' is required`; NaN is refused with `'L' must be a finite number`; a folder input must be a folder; a worker that is idle for 10 minutes is closed; a worker ended after Cancel reads `cancelled (worker stopped)`; a worker that died is explained with the Tools sentences (a missing package first, then the exit code); a tool with a parameter type this host does not know is refused alone (`this tool cannot be shown: ...`); two tools with one label are listed as `label (id)` and the apps are sorted by name; `Copy as command` writes Python text the way `repr` does (a newline cannot break the snippet) and says `there is no clipboard to copy to` when there is none; message results show `- ` items as bullets; a result file that is missing, or a result type this version does not know, is said in one sentence.
 
 ## Pixel size (calibration)
@@ -56,6 +59,15 @@ The tool receives only the area; **results come back on the right place of the s
 
 The downsample of an export is 1 for now and is kept next to the offset (`downsample`, always 1.0), so the later resolution step changes one place.
 
+## Run identity (SPEC_WRONG_SCIENCE section 4)
+A run is placed on the image it was exported from, never on whatever is open when it ends.
+
+* **Frozen at the click.** When Run is pressed (on the FX thread, before any worker call) the host freezes the run: a `run_id` (uuid4), the app and tool, the form values, the plane, the Image areas, the **`ImageData` object** each "Current image" input came from (with its shape: width, height, planes, time points, channels), and the **ROIs of the selected annotations** (ROIs are immutable objects: the copy is by value). The export, the mask of a RegionOf selection, the request, the run record and the placement of the results read this and nothing live: opening another image, selecting other annotations or switching the tool or the app while the tool runs changes nothing that was sent or drawn (RUN-02, RUN-03). The text of the chooser (`Current image: slide A`) is only a name for sentences, never identity.
+* **The chooser follows QuPath.** When the open image changes, the "Current image: ..." entry of every image chooser names the image that is open now (everything else in the form stays). A run whose chooser names another image than the open one is refused (`'Image': 'A' is no longer the image open in QuPath ('B' is): choose the image again`) instead of exporting B under A's name.
+* **Placement checks identity first.** Points and outlines are drawn only on that very `ImageData` (the image the run started on), and only while a viewer still shows it with the shape it was exported with. The image may be in another viewer than the active one: the results go to it, never to the active image. If the image was replaced or closed while the tool ran, nothing is drawn, the files stay in the results folder, the run record is written, and the status line reads (conformance `target_unavailable`): `run <run_id> on 'A': 'A' is no longer open; results are in <path>`. If it is still shown but has another shape (a script replaced its server; practically only a script can do this in QuPath): `run <run_id> on 'A': 'A' changed size while the tool ran; results are in <path>`; the two sentences are never swapped, both are logged with the code `target_unavailable`. A run on a project image or a file has no open image to draw on and says so as before.
+* **What a drawn result carries.** Every annotation a run draws has the metadata `labconstrictor.run_id`, `labconstrictor.app` and `labconstrictor.tool` of **that run** (not of the tool the form shows later); the results window keeps the run id as its `userData`; the run record has `run_id` and `started_utc`. A tool or app switched while a run goes on is never cleared or rebound by the finish of that run (`ClearAfterRun` resets only the form the run was started from).
+* RUN-05 (a Napari layer is moved) does not apply: an `ImageData` has no transform.
+
 ## Fallbacks (intended)
 Each of these is deliberate and is written once to QuPath's log (logger `labconstrictor`) when it happens. Every other failure is logged with its stack trace and shown to the person (status line or Details).
 
@@ -108,6 +120,10 @@ The `gui_test_parity_*` bodies need the registered `interactions` app (its Pytho
 
 `BODY=gui_test_parity_host_body.groovy LC_HOME=<scratch folder named lhome_host> tests/run_gui_test.sh` checks the tool chooser, an unsupported parameter type, folder inputs, the Python literal (also read back by Python), bullets, missing result files, a failing clipboard and the crash sentences (fake interpreters that print and exit); a missing image file is refused with the sentence, a missing table is left to the worker, a clipboard that refuses or throws is said (49 checks).
 
+`BODY=gui_test_retention_body.groovy LC_HOME=<scratch folder named lhome_retention> tests/run_gui_test.sh` runs HRET-01..06, RET-10 and RET-11 of `conformance.json` against the `runs` test app (`tests/runs_app`, registered by `tests/ci_setup_apps.sh`): the markers, their validity judged by Tools itself (`prune --dry-run --json`), the `prune` subprocess replaced by a recorder, by a failing and by a missing command, 60 runs with foreign and old folders that must survive, the REAL `prune` (newest 20 and 50 kept; legacy, foreign, unfinished, renamed and linked folders untouched), a worker killed after Cancel (`cancelled`) and one that dies (`failed`), the Tools-too-old refusals, and a scan that the script has no code that deletes under `results/` or `runs/`.
+
+`BODY=gui_test_identity_body.groovy LC_HOME=<scratch folder named lhome_identity> tests/run_gui_test.sh` runs RUN-01..04, 06 and 07 of `conformance.json` through the real dialog with the delayed tool of the `runs` app (a gate file is the barrier): the image is changed, closed or kept in a second viewer, the selection is changed during the export (a hook on the run thread), the tool and the app are switched while the run goes, and the shape of the image is changed (by reflection: nothing else can do it).
+
 `BODY=gui_test_parity_lifecycle_body.groovy LC_HOME=<scratch folder named lhome_life> tests/run_gui_test.sh` checks results folder names and that no folder is ever deleted, the byte limit, the shapes cap sentence and the idle worker (28 checks).
 
 `BODY=gui_test_parity_cancel_body.groovy tests/run_gui_test.sh` cancels the Playground's "ignores Cancel" tool and checks `cancelled (worker stopped)` (9 checks; needs the Playground in `LC_HOME`).
@@ -120,9 +136,12 @@ The `gui_test_parity_*` bodies need the registered `interactions` app (its Pytho
 
 ### Lint (runs in CI) and what CI does not run
 
-`.github/workflows/lint.yml` lints `LabConstrictorTools.groovy` with `npm-groovy-lint` (pinned version, CodeNarc rules, config in `.groovylintrc.json`; each disabled rule has its reason there). It fails on any finding, including info. Run it locally (needs Node and Java):
+`.github/workflows/lint.yml` lints the script (the files listed in `parts.txt`, joined into one file, which is what the extension runs) with `npm-groovy-lint` (pinned version, CodeNarc rules, config in `.groovylintrc.json`; each disabled rule has its reason there). It fails on any finding, including info. Run it locally (needs Node and Java):
+
+    mkdir -p lint-input && cp target/LabConstrictorTools-all.groovy lint-input/        # ./build.sh writes it (the files of parts.txt joined)
+    npx --yes npm-groovy-lint@18.0.0 --path lint-input --files "**/*.groovy" --config .groovylintrc.json --failon info
 
-    npx --yes npm-groovy-lint@18.0.0 --path src/main/resources/org/cellmigrationlab/labconstrictor/qupath --files "**/*.groovy" --failon info
+The same workflow fails when a part file (all but the older main script) has 600 lines or more: new code goes into a new file listed in `parts.txt`.
 
 ### GUI tests in CI
 

build.sh

@@ -8,3 +8,7 @@ rm -rf target/classes && mkdir -p target/classes
 cp -r src/main/resources/* target/classes/
 "${JAVA_HOME:+$JAVA_HOME/bin/}jar" cf target/labconstrictor-qupath-0.1.0.jar -C target/classes .
 echo "built target/labconstrictor-qupath-0.1.0.jar"
+# one file with the whole script, for QuPath's Script Editor (the jar joins the files listed in parts.txt itself)
+RES=src/main/resources/org/cellmigrationlab/labconstrictor/qupath
+{ while IFS= read -r part || [ -n "$part" ]; do [ -z "$part" ] || { cat "$RES/$part"; echo; }; done < "$RES/parts.txt"; } > target/LabConstrictorTools-all.groovy
+echo "built target/LabConstrictorTools-all.groovy (all parts in one file, for the Script Editor)"

docs/user-guide.md

@@ -32,6 +32,9 @@ Choose **Extensions > LabConstrictor tools...**, pick an app and a tool, and pre
 ## If something does not work
 
 - The status line and **Details** show what failed. The log is `~/.labconstrictor/logs/labconstrictor.log`.
+- Results and run records are kept in `~/.labconstrictor/results` and `~/.labconstrictor/runs`. The extension never deletes them; after each run it starts `labconstrictor-tools prune`, which keeps the newest 20 results and 50 run records and removes only folders LabConstrictor made and finished. Folders from older versions of the extension are not removed automatically.
+- The app's `labconstrictor-tools` must be 0.3.0a1 or newer; with an older one the run is refused and the status line says to update it.
+- If you open another image while a tool runs, the results are not drawn on the new image: the status line says where the files are.
 - `labconstrictor-tools doctor` shows which apps are registered.
 
 Source and issues: [LabConstrictor-QuPath](https://github.com/CellMigrationLab/LabConstrictor-QuPath).