[guijacquemet]

Theme 3 of the wrong-science wave for QuPath: asynchronous ChoicesFrom lookups and who owns the worker (SPEC_WRONG_SCIENCE section 5, ASY-01..08), a Tools compatibility check when the registry is read, and the conformance tests that read conformance.json (docs/CONFORMANCE.md says QuPath has tests/gui_test_conformance_body.groovy; it did not exist). Reference: Napari #54.

Fixes #19.

This PR is based on PR #31 (claude/phase2-qupath-number-limits), which is based on PR #30 (claude/phase2-qupath-retention): it needs the parts.txt split, the runs test app and the new run_gui_ci.sh entries of those two. Merge #30 as a merge commit, retarget #31 to main and merge it the same way, then retarget this one.

What changes

Async lookups and worker ownership (issue #19)

Before: resolveChoices started one thread per ChoicesFrom parameter; each ran ensureWorker (read and overwrite the shared worker / workerApp fields, no lock) and worker.run; a run did the same from its own thread; choiceSeq was reset per form, so a late answer of tool A with number 1 could fill the dropdown of tool B (also number 1); and a lookup could be running on the worker a run was starting on.

• LcLookups (LcLookups.groovy): every question is (form generation, app, tool, parameter, the values it depends on, request id); the generation is process-wide and never reset (it grows when a form is built or destroyed), the request id is per window and never reset. An answer is applied only if all of it still holds when it arrives (the dependencies are recomputed then); otherwise it is dropped whole, with its error and tooltip. A dependency change invalidates at once (the dropdown goes back to the text field with the tooltip "waiting for the choices...") and only the new question waits for the 400 ms pause. One lookup in flight per window on one thread; waiting questions are coalesced (one slot per parameter); at most 8 wait (the oldest is dropped and logged); a lookup past 60 s is ended with its worker (-Dlc.qupath.lookup_timeout_s shortens it for tests, said in the status as did not answer within N s).
• LcWorkers (LcWorkers.groovy): the one owner of the workers of a window, under one lock. A run or a lookup leases a worker when it is submitted and uses only that one. A lookup never uses the worker a run holds and never starts while a run is active (it asks again after 1.5 s); a run waits for a lookup that holds its worker. An app switch during a run does not replace the run's worker (the new app's worker is made for the next job; the old one is closed when its last job ends). A crashed or timed-out worker is discarded. Closing the window drops the waiting lookups, ends the one in flight, stops the thread and closes every worker (also one that was being made at that moment). LcWorker implements LcWorkerLike, counts the processes it starts and takes a time limit; dlg.worker / dlg.workerApp are now read-only views of the controller; Cancel goes to the worker of the run (the run's lease).

Compatibility when the registry is read

LcCompat (introduced in PR #30 for the run-time refusal) now also runs in LcRegistry.load(): an app whose Tools is older than 0.3.0a1, or lacks reserved_inputs_v1 / retention_prune, or has no version, is not offered and is listed under "skipped" with one sentence (... is older than the 0.3.0a1 this host needs: update it); an unparsable version is offered with a warning in the status line. README has a "Tools compatibility" section.

Conformance tests

gui_test_conformance_body.groovy (115 checks): validates the file with Tools' --validate; compares every limit, message template and host-internal code of the file with this script's constants (an unknown key fails: a new rule is a failing test until the host follows it); runs all 7 compat vectors and the version order through LcCompat; runs the done-status rule through the real Tools helper for each way a run can end; and keeps a ledger of every fixture id of the file (83) mapped to the body that runs it or to the reason it does not apply (printed as UNSUPPORTED, never passed): CF-02..06, CF-13 (no transform), RET-01..09, 12, 13 (Tools' own), RET-14, RUN-05.
gui_test_async_body.groovy (64 checks): ASY-01..08 read from conformance.json against gated source tools of the runs test app (the dependency of the dropdown is the gate path, so the test controls both the delay and the dependency): see README. ASY-08 also runs three threads over fake workers at a fixed interleaving (a barrier hook) and at random.

Existing tests changed (spec supersedes them)

• gui_test_area_body.groovy: fx { dlg.worker = null } (it assigned the field) is dlg.stopWorker(true).
• gui_test_parity_trust_body.groovy: its good schema now says tools_version and features (an app with an older Tools is not offered); new cases for the compatibility rules.
• gui_test_retention_body.groovy (from PR Phase 2, theme 1: retention markers + prune subprocess + Tools run record, and run identity (RUN-01..07) #30): its Tools-too-old block registered three apps with an old Tools and ran them; they are now not offered when the window rescans (the body checks the status text and that they are not in dlg.apps), and the run-time refusal is checked by putting such an app into the window's list.
• tests/runs_app gained the gated source tools; tests/run_gui_ci.sh lists the two new bodies.
  Nothing was skipped or loosened.

Sabotage checks

(see the comment on this PR)

Spec gaps and conservative choices

• Section 5 says a lookup that exceeds 60 s is "cancelled and its worker discarded"; here it is ended together with its worker (a cancel request would need a grace period that this thread cannot wait for), and the status says so. The 1 / 8 / 60 s limits are status: default in conformance.json.
• ASY-04's "at most 1 pending per parameter" holds by construction (one slot per parameter); the test checks the total and the dropped count with 12 parameters at once.
• A run waits for a lookup that holds the worker it needs (a lookup is short and limited to 60 s); the spec does not say which of the two waits.
• Closing the window blocks the FX thread only while a worker is being made (a few tens of milliseconds); a worker made while the window closes is closed.
• The compat result "warn" only appears for a version text that is not understood (a missing version is a refusal here because this host sets a minimum).
• RET-01..09, 12, 13 are Tools' own fixtures; RET-14 and RUN-05 do not apply; CF-02..06, 13 do not apply (no transforms).

The other open issues

• Serialize ChoicesFrom lookups and protect QuPath worker ownership across app/tool changes #19 is fixed by this PR (its four acceptance items: ASY-05 two lookups, one process; ASY-01 / ASY-06 app and tool switch and a run's worker; ASY-06 a run next to a lookup; the regression test with a delayed source tool and worker identity and cleanup).
• Protect unrelated and active result folders from QuPath retention cleanup #14 is fixed by PR Phase 2, theme 1: retention markers + prune subprocess + Tools run record, and run identity (RUN-01..07) #30 (markers, prune subprocess, real prune tested against foreign / legacy / unfinished / renamed / linked folders). Keep QuPath image identity and result coordinates tied to the image used at run start #16 (closed) got its end-of-run half in PR Phase 2, theme 1: retention markers + prune subprocess + Tools run record, and run identity (RUN-01..07) #30.
• Parse quoted multiline CSV records correctly in QuPath tables and point annotations #17 (multi-line CSV) is not touched: tables are read with the CSV reader but placePoints still reads points with readLines + splitCsv, so a quoted newline in a points file still breaks. Open.
• Match Toolkit Windows quoting for copied QuPath terminal commands #18 (Windows quoting) is not touched by these PRs; shellQuote already follows the Tools vectors (gui_test_parity_quote_body.groovy), so it looks fixed on main; I did not verify on Windows and left the issue for the maintainers.
• Validate QuPath registry entries and cached manifests to the shared Toolkit/Fiji contract #20 (registry validation) is not touched except for the compatibility rule above; the trust and schema checks of the issue are already on main (gui_test_parity_trust_body.groovy, 26 checks). Left open for the maintainers.

Not verified

• Windows and macOS (process handling, ProcessHandle, the file locks of Tools' prune); no human at a display (private Xvfb, real QuPath 0.7.0 / Java 25). Timing: the retry after a run is 1.5 s and the tests wait for it; on a very slow machine a test could time out, not fail wrongly.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN

---

Generated by Claude Code

[guijacquemet]

Sabotage checks

Each row: production code broken in a copy of the final tree of this PR (a regular expression replaced one place), the PR's own body run on it (gui_test_async_body; gui_test_conformance_body for the limits, messages, compat, done-status and ledger rows; gui_test_parity_trust_body for the registry row). Every row fails the body except the one marked NOT DETECTED, which is an equivalent mutant: a leased worker is always either the kept one or a retired one, and closeAll closes both, so removing the third source of workers changes nothing observable. Three rows exposed a hole in the first version of the tests and the tests were extended (two further commits on this branch): removing only the form-generation check was not caught (the id, tool and dependency checks covered it in every fixture; now each part of the identity is tested alone), and removing the visible half of the pending state was not caught (the dropdown had never been filled in those fixtures; now a filled dropdown is tested).

sabotage (production code broken in a copy)	what the test run said
a reply is applied whatever its identity (no check at all)	4 failed check(s): ASY-01_A_s_reply_was_dropped, ASY-02_the_old_reply_is_dropped, ASY-02_it_is_still_pending_and_shows_no_choices_after_the_old_reply ...
the form-generation check alone is removed (found as NOT detected by the first version of the tests; the identity unit checks were added)	6 failed check(s): ASY-02_a_reply_of_another_form_generation_is_dropped, ASY-02_a_reply_of_another_app_is_dropped, ASY-02_a_reply_of_another_tool_is_dropped ...
a dependency change does not mark the parameter pending	2 failed check(s): ASY-02_the_parameter_is_pending_at_once, ASY-02_it_is_still_pending_and_shows_no_choices_after_the_old_reply
the pending state keeps the old dropdown on screen (dropdown not cleared, text field not shown)	1 failed check(s): ASY-02_a_filled_dropdown_is_pending_and_empty_at_once_when_its_dependency_changes
a dependency change waits for the debounce before invalidating (no immediate invalidation)	2 failed check(s): ASY-02_the_parameter_is_pending_at_once, ASY-02_it_is_still_pending_and_shows_no_choices_after_the_old_reply
the limit of 8 waiting lookups is raised to 100	2 failed check(s): ASY-04_at_most_8_lookups_wait_and_the_oldest_are_dropped, ASY-04_the_drops_are_logged_oldest_first
lookups run in parallel (a thread per lookup)	2 failed check(s): ASY-04_at_most_one_lookup_in_flight, ASY-05_never_more_than_one_lookup_at_a_time
the worker controller gives a lookup lease while a run is active	2 failed check(s): ASY-08_no_lookup_lease_while_a_run_is_active, ASY-08_the_retired_worker_is_closed_when_the_run_ends
an app switch closes the worker a run still holds	2 failed check(s): ASY-08_an_app_switch_does_not_close_the_worker_a_run_holds, ASY-08_stress_nothing_failed_and_every_worker_was_closed_exactly_once
a run does not wait for the lookup that holds its worker	1 failed check(s): ASY-08_a_run_waits_for_the_lookup_that_holds_its_worker
a worker can be closed twice	test aborted with an exception (rc=1 after 35 checks)
closeAll does not close the leased workers (the ones neither kept nor retired)	NOT DETECTED
closing the window does not stop the lookup thread	2 failed check(s): ASY-07_the_lookup_thread_stops, ASY-07_the_lookup_thread_of_that_window_stops_too
leases are still given after the window is closed	1 failed check(s): ASY-08_no_lease_is_given_after_the_close
the time limit of a lookup is ignored	3 failed check(s): lookup_over_the_limit_is_ended_and_said, lookup_over_the_limit_discards_its_worker, lookup_after_the_timeout_works_on_a_new_worker
the pending limit is 9 instead of 8 (the conformance value)	1 failed check(s): limit_lookup_pending_max_is_the_conformance_value
the target_unavailable sentence is reworded	1 failed check(s): message_target_unavailable_is_the_conformance_template
the below-minimum rule of the compatibility check is removed	1 failed check(s): compat_below_the_minimum_is_refuse
the record helper is never told that Cancel was pressed	1 failed check(s): done_status_crashed_after_cancel_is_the_worker_killed_after_the_grace_period_is_cancelled
a fixture id of conformance.json (ASY-08) is missing from the ledger	2 failed check(s): ledger_ASY-08_is_classified, ledger_the_file_has_the_fixture_counts_this_host_expects
the registry accepts apps with an old Tools	4 failed check(s): trust_tools_old_refused, trust_tools_no_feature_refused, trust_tools_no_fields_refused ...

"aborted with an exception" means the body died on a missing marker or on the stress threads (the exception fails the body).

---

Generated by Claude Code