diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md new file mode 100644 index 0000000..a3b2015 --- /dev/null +++ b/docs/CONFORMANCE.md @@ -0,0 +1,57 @@ +# Conformance data: `labconstrictor_tools/conformance.json` + +One machine-readable file, shipped inside the `labconstrictor-tools` wheel (`[tool.setuptools.package-data]`), that holds the rules every host must agree on, as data. It implements part of the proposal in [HOST_PARITY.md](HOST_PARITY.md#2-proposed-conformance-file) (roadmap phase 0.3, "87a") and carries the fixtures of [SPEC_WRONG_SCIENCE.md](SPEC_WRONG_SCIENCE.md) under the key `wrong_science`. Hosts keep their own constants and code; their tests read this file and compare. A change of a rule is a change of this file and a failing test in every host that has not followed. + +**Only decided rules are in it.** A rule the spec marks `(default, not yet confirmed)` has `"status": "default"`; a rule nobody decided is not in the file (see "Gaps" below). The file never holds a value only one host needs. + +## Layout + +| Key | Content | Run by | +|---|---|---| +| `conformance` | format version (1) | the loader | +| `limits` | named numbers: `results_kept`, `run_records_kept`, `form_integer_max` (9007199254740991), `unbounded_float` (1e15), `choices_debounce_ms`, `choices_busy_retry_ms`, `worker_close_timeout_s`, `lookup_in_flight_max`, `lookup_pending_max`, `lookup_timeout_s`; each `{value, status: decided\|default, spec, ...}` | every host: its constant must equal `value` | +| `host_internal_codes` | the four new host-side codes (`frame_unsupported_transform`, `selection_mixed_planes`, `selection_plane_mismatch`, `target_unavailable`) | hosts | +| `messages` | sentence templates with `{name}` placeholders (Python `str.format`; other languages replace `{name}`) | hosts compare the text they produce | +| `compat` | the compatibility rules of [PROTOCOL.md](PROTOCOL.md) as vectors | Tools (`compat.check_schema`); Fiji and QuPath apply the same rules in their language | +| `wrong_science.coordinate` | base image `ASYM`, the rectangle and its mask, the placement formula, cases CF-01..CF-14 | hosts (and an arithmetic check in Tools) | +| `wrong_science.calibration` | unit table (um per unit), micro-sign spellings, the damaged-sign rule, anisotropy and default-dpi rules, the `_calibration` request shape, cases CAL-01..CAL-18 and transitions CAL-T1..T9 | hosts (and an arithmetic check in Tools) | +| `wrong_science.retention` | marker names and fields; `fixtures_tools` RET-01..09, 12, 13 (structured: setup, action, expect) and `fixtures_hosts` HRET-01..04, RET-10, 11, 14 (prose) | Tools runs `fixtures_tools` against `retention.prune` and the CLI; every host runs `fixtures_hosts` with the `prune` subprocess replaced by a recorder | +| `wrong_science.run_identity`, `async_lookups` | RUN-01..06, ASY-01..08 (prose sequences and expectations) | hosts | +| `wrong_science.number_limits` | NUM-01..05 | hosts (Napari, Fiji dialog and macro, QuPath, notebook) | + +A host that cannot run a case reports it as **unsupported**, never as passed (SPEC_WRONG_SCIENCE 1.5, 3.7). + +## The loader + +`labconstrictor_tools.conformance` (standard library only): + +```python +from labconstrictor_tools import conformance +data = conformance.load() # validated on every first load; ConformanceError lists every problem +conformance.path() # the file, for hosts that are not Python +conformance.validate(some_dict) # the schema: a list of problems, empty when valid +``` +`python -m labconstrictor_tools.conformance --path` prints the file's location (inside the app's or the host's Python environment); `--validate` checks it. The schema checks structure, not host behaviour: required keys, numeric `value`, `status`, identifier names, `{name}` placeholders only, unique ids with the right prefix per section, `compat` vectors in order, the four-vertex rectangle, a positive um factor per unit. A malformed edit therefore fails every host test at once; it cannot silently skip a rule. + +## How each host's test loads it + +| Host | Test | Loading | Checks | +|---|---|---|---| +| Tools (CLI, `command.py`, `convert.py`, notebook) | `tests/test_conformance.py` | `conformance.load()` | limits against the code, `compat` vectors against `check_schema`, the retention `fixtures_tools` against `retention.prune` and the CLI, arithmetic of the coordinate and unit data | +| Napari | `tests/test_conformance.py` (a plain script, no Qt needed for most of it) | `from labconstrictor_tools import conformance; conformance.load()` (Tools is a dependency of the plugin) | constants against `limits` (`_INT_RANGE`, `_FLOAT_RANGE` against `form_integer_max`, `unbounded_float`), `ResultPresenter` and refusal sentences against `messages`, `host_internal_codes`, CF placement through a fake viewer (`viewer.add_points` receives the Napari data coordinate), region mask through `selection_mask`, CAL and NUM cases through the schema and form code, HRET fixtures with a recorder instead of `prune` | +| Fiji | a case type `conformance` in `tests/run_cases.py`; Groovy side `tests/conformance_check.groovy` | the Python driver runs `python -m labconstrictor_tools.conformance --path` with the app's interpreter and passes the result in the environment variable `LC_CONFORMANCE`; Groovy: `new JsonSlurper().parse(new File(System.getenv("LC_CONFORMANCE")))` | the harness parses the script text without its last line with `new GroovyShell().parse(...)` and calls the helper functions (`polygonRoi`, `selectionMask`, `macroNumber`, ...) and reads the `@Field` constants against `limits` | +| QuPath | `tests/gui_test_conformance_body.groovy`, run by `tests/run_gui_test.sh` (or `tests/headless_core.groovy` where no GUI is needed) | `LcJson.parse(new File(System.getenv("LC_CONFORMANCE")))`, same environment variable | the static members (`LcConst.*`, `LcDialog.*`, `LcArea.clamped`, `LcRegistry.load`) against `limits` and `messages`; CF placement, CAL and NUM cases | + +Rules for a host test: it fails when a key it knows is missing (no silent skip); it reports `unsupported` for a case it cannot run; it compares its **own** behaviour (drawn coordinates, request contents, files on disk), not only that no error occurred (SPEC_WRONG_SCIENCE 1.5). + +## Gaps: what the spec does not yet decide (so the file does not contain it) + +1. **Keys of the HOST_PARITY sketch that are not in SPEC_WRONG_SCIENCE**: `max_shapes`, `table_rows_shown`, `region_max_objects`, `max_export_bytes`, `max_display_bytes`, `cancel_grace_s`, `worker_orphan_grace_s`, `worker_idle_s`, the log limits, `error_codes`, `notice_codes`, `worker_exit_hints`, `number_text`, `quoting` and `command_vectors`, `results_text`, `trust_vectors`, and the wording of the HOST_PARITY `messages` that are not in the wrong-science spec. They are proposals of the parity audit, not decisions; `number_text` also diverges from the Tools CLI today (TOOLS-7), and the Windows quoting of a trailing backslash is marked "TO DECIDE". `tests/quote_vectors.json` remains the shared quoting source until these are decided. `host_error` (QuPath's internal code) is not listed in `host_internal_codes` because the spec names only the four new ones. +2. **Axis order of `rect`** (section 1.5): the spec does not say; the mask pixels show it is `[y, x]` (Napari order), while shapes in the protocol are `[x, y]`. CF-09 and CF-14 give `server_wh` as `[width, height]` and `export_shape` as `[rows, columns]` (mixed conventions inside one fixture list). +3. **CF-07, 08, 10, 11, 12, 13** and all of RUN, ASY, HRET and RET-10/11/14 are sequences with prose expectations; the spec defines no machine-executable vocabulary for them, so only Tools' own fixtures (RET-01..09, 12, 13) are structured, and the others are interpreted by each host's harness. +4. **Length units** (2.1.4): the spec lists `um`, `micron(s)`, `micrometer/-metre`, `nm`, `mm`, `cm`, `m` and the Angstrom spellings, but CAL-03 uses the long form `millimeter`. The file lists `millimeter` and `millimetre` (needed by CAL-03) and no other long form (`nanometer`, `centimeter`, `meter`); case sensitivity, plural forms and surrounding spaces are not specified. +5. **Number limits** (6): the refusal sentence is given for integers only (`'L' must be <= 9007199254740991`, the `>=` form from HOST_PARITY I06); the float sentence and the message when a GUI box cannot hold the number are not specified. The spec writes the keys as `limits.form_integer_max = 9007199254740991`; the file nests the number as `limits.form_integer_max.value` (the layout of the HOST_PARITY sketch) so that it can carry `status` and `spec`. +6. **Tools divergence found while writing the file**: the notebook form refuses an integer beyond 2**53-1 with `'L' is N, beyond the +-9007199254740991 the form can hold exactly: ...`, not with the `'L' must be <= 9007199254740991` template of 6.2 (NUM-02, NUM-03). To fix with the notebook change; the file follows the spec. +7. **Lookup limits** (5.4: one in flight, 8 pending, 60 s) and the other working decisions of SPEC 8 are `status: default`, not yet confirmed. +8. **Napari N-D, QuPath downsample d != 1, TimeIntervalOf and ZStepOf** fixtures wait for Wave 2 (spec header); CF-14 is data, but no host exports at d != 1 yet. +9. **Run-identity and async fixtures have no pass criteria for timing** (no clock or barrier vocabulary): each host's harness decides how to build the "deterministic barriers" of 5.7. diff --git a/docs/HOST_PARITY.md b/docs/HOST_PARITY.md index 461bef8..fa1394a 100644 --- a/docs/HOST_PARITY.md +++ b/docs/HOST_PARITY.md @@ -289,6 +289,8 @@ Columns: id, rule, reference, Napari, Fiji, QuPath, CLI, notebook, status, note. ## 2. Proposed conformance file +**Status (phase 0.3): the file exists** with the rules already decided in [SPEC_WRONG_SCIENCE.md](SPEC_WRONG_SCIENCE.md) (limits, host-internal codes, compatibility vectors, the `wrong_science` fixtures) and a validating loader, `labconstrictor_tools.conformance`; see [CONFORMANCE.md](CONFORMANCE.md), which also lists the keys of the sketch below that are not decided yet and therefore not in the file. The sketch is kept as the proposal for them. + **One machine-readable file in this repository: `labconstrictor_tools/conformance.json`** (inside the package, so that every environment that already has `labconstrictor-tools` installed has it; add `[tool.setuptools.package-data] labconstrictor_tools = ["conformance.json"]` to `pyproject.toml`; `tests/conformance.json` would not reach the Napari, Fiji and QuPath test environments). It holds the values and test vectors the rows above show to be copied by hand into four code bases: named limits, error codes, message templates, number grammar, quoting, command vectors, coordinate vectors, result sentences, trust cases. Hosts keep their own constants; their tests read this file and compare. A change of a rule becomes a change of this file and a failing test in every host that has not followed. Two keys carry the migration: `value` is the reference and `today` is what each host has now, so the first run of the new tests prints exactly the `DIVERGES` rows of section 1 and nothing else. diff --git a/docs/PROTOCOL.md b/docs/PROTOCOL.md index fc1bb62..6878ca0 100644 --- a/docs/PROTOCOL.md +++ b/docs/PROTOCOL.md @@ -64,7 +64,7 @@ Feature ids (the implemented ones are in `FEATURES`; a planned id appears there | `retention_prune` | the CLI command `labconstrictor-tools prune [--root results\|runs\|all] [--dry-run] [--json]` and `labconstrictor_tools.retention`: only folders carrying a valid `.lc-owner.json` and a `.lc-done.json` are ever removed ([SPEC_WRONG_SCIENCE.md](SPEC_WRONG_SCIENCE.md) section 3; hosts only mark a run finished, then start `prune`) | in the first release after 0.2.0rc1 (not in 0.2.0rc1) | ### Compatibility rules for hosts -A host that needs a minimum states it as `(min_tools_version, required_features)` and applies the rules below to every schema it loads, with `labconstrictor_tools.compat.check_schema` (Python hosts: Napari, notebooks) or the same rules written in its own language (Fiji, QuPath; the data is in `conformance.json`, [HOST_PARITY.md](HOST_PARITY.md#2-proposed-conformance-file), `compat` section). The result is `ok`, `warn` or `refuse`, with one sentence for the person: +A host that needs a minimum states it as `(min_tools_version, required_features)` and applies the rules below to every schema it loads, with `labconstrictor_tools.compat.check_schema` (Python hosts: Napari, notebooks) or the same rules written in its own language (Fiji, QuPath; the data is in `conformance.json`, [CONFORMANCE.md](CONFORMANCE.md), `compat` section). The result is `ok`, `warn` or `refuse`, with one sentence for the person: 1. `protocol` not supported by the host: `refuse` (as today). 2. A required feature missing from `features` (or no `features` list at all): `refuse`, "needs ; update the app's labconstrictor-tools". 3. `tools_version` missing (unknown/old) while a minimum is set: `refuse`; with no minimum set: `warn`, "Tools version unknown". diff --git a/docs/REGRESSION_LEDGER.md b/docs/REGRESSION_LEDGER.md index 7f7c02f..108595d 100644 --- a/docs/REGRESSION_LEDGER.md +++ b/docs/REGRESSION_LEDGER.md @@ -27,6 +27,7 @@ How to add one: [TESTING.md](TESTING.md#the-ledger). | 2026-10-08 | F15: an empty, complex, object or text array is written as an image result (or crashes with KeyError) | `portable_dtype` and `_write_one` do not refuse what no host can open | this PR | `test_error_codes.py::ImagesNoHostCanOpen.test_complex_object_and_text_arrays_are_unsupported_dtype`, `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`make_array/*`) | | 2026-10-08 | F16: hints on the wrong kind of parameter, `Min(5), Max(1)`, output markers on inputs and duplicate output names are accepted and ignored | `introspection` validates only some misuses | this PR | `test_error_codes.py::HintMisuse.test_bounds_on_text_and_an_empty_range`, `test_error_codes.py::HintMisuse.test_duplicate_output_names`, `test_roundtrip_hints.py::InvalidCombinations.test_every_invalid_combination_is_refused_with_a_message_that_names_the_culprit` | | 2026-10-12 | An output declared `Name("../other_run/result")`, an absolute name or `Name("cells/area")` wrote outside the result folder (or failed because the directory did not exist) | `introspection._outputs` checked only duplicate names; the writers joined `name + suffix` onto the job folder unchecked | [#104](https://github.com/CellMigrationLab/LabConstrictor-Tools/pull/104): unsafe names are a `DeclarationError` (`introspection._check_output_name`); `convert._job_file` refuses any path that is not a direct child of the job folder | `test_output_names.py::Declaration.test_unsafe_names_are_a_declaration_error_for_every_output_type`, `test_output_names.py::Writers.test_every_writer_refuses_a_name_that_leaves_the_job_folder` | +| 2026-10-12 | Found writing the cross-host data: the 12 Oct spec rules (number limits, coordinate frame, calibration, retention) lived only as prose and as copies in four code bases, so one host could drift without any test noticing | No shared, validated, packaged source of the rules: the proposed `conformance.json` did not exist and PROTOCOL.md already pointed at it | this PR: `labconstrictor_tools/conformance.json` (package data) and `conformance.load()` / `validate()`; data checked against the code and against plain-Python oracles | `test_conformance.py::Schema.test_each_kind_of_damage_is_named`, `test_conformance.py::AgainstTheCode.test_limits_match_the_code_and_the_spec`, `test_conformance.py::AgainstTheCode.test_the_compat_vectors_give_the_same_answer_as_the_reference`, `test_conformance.py::CoordinateData.test_the_placement_formula_for_every_downsample_variant`, `test_conformance.py::Packaging.test_the_built_wheel_contains_the_file_and_the_loader_reads_it_from_there` (and the sabotage rows `conformance:*`) | | 2026-10-08 | F20: on ARM (Apple silicon) a uint64 image holding 2**64-1 is stored as float32 2**64 instead of being refused | `portable_dtype` checks exactness by casting the float32 back to uint64, an out-of-range cast that is undefined: x86 gives 0 (refused), ARM saturates (accepted) | this PR | `test_error_codes.py::Float32Exactness.test_uint64_beyond_float32_is_refused_with_unsupported_dtype_on_every_cpu` (numpy warns about the out-of-range cast on x86, the test turns that into a failure), `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`echo_image/uint64/beyond-float32`; found by the macOS job of CI) | | 2026-10-12 | A host sending a misspelt, renamed or removed parameter (`sigam` for `sigma`) got COMPLETE with the default value of the right one | `convert.load_inputs` looped over the declared inputs and never looked at extra keys (only the command line checked names) | [#106](https://github.com/CellMigrationLab/LabConstrictor-Tools/pull/106) | `test_roundtrip_unknown_inputs.py::UnknownInputs.test_a_misspelt_renamed_or_extra_input_is_refused_and_the_tool_does_not_run` | | 2026-10-12 | A worker that wrote megabytes to stderr without a newline (one dumped array, no line ends) made the host hold all of it, and the tail kept more than its 1 MB limit | `_Tail.append` never evicted the only line left, and `for line in stderr` buffered a whole line before the tail saw it | [#103](https://github.com/CellMigrationLab/LabConstrictor-Tools/pull/103) | `test_roundtrip_failures.py::Noise.test_one_endless_stderr_line_is_bounded_while_it_is_still_open`, `test_roundtrip_failures.py::Noise.test_a_single_line_longer_than_the_limit_keeps_its_end_only`, `test_roundtrip_failures.py::Noise.test_the_end_of_an_oversized_line_is_in_the_crash_details` | diff --git a/docs/SPEC_WRONG_SCIENCE.md b/docs/SPEC_WRONG_SCIENCE.md index 77e9c4e..c12412a 100644 --- a/docs/SPEC_WRONG_SCIENCE.md +++ b/docs/SPEC_WRONG_SCIENCE.md @@ -2,7 +2,7 @@ Status: **for review; the retention rules of section 3 are implemented in Tools (`labconstrictor_tools.retention`, `labconstrictor-tools prune`, feature `retention_prune`), nothing else is.** Maintainer decisions are recorded in section 8. -**v1: 2D. N-D (T/Z stacks) and the QuPath resolution (downsample) choice follow in Wave 2.** The frame (1.1.2) and the placement formula (1.1.3) already have their final shape, so Wave 2 adds fixtures, not rules. Wave 2 fixtures still needed: `TYX` and `ZYX` results with `t`/`z` columns, a 2D result on a `TYX` layer (Napari), a per-axis `_calibration` with z and t, and QuPath export at downsample 2 and 4 on a real pyramid (CF-14 is the data, the GUI harness is the missing part). Same conventions as [SPEC_WAVE2.md](SPEC_WAVE2.md) and [HOST_PARITY.md](HOST_PARITY.md): additive within `protocol: 1`, rules are host-neutral, a rule is "done" when its fixtures pass in every host that claims the feature. The fixtures below are data, meant to live in `labconstrictor_tools/conformance.json` under the key `wrong_science` and to be reused by the conformance suite (Tools #87). +**v1: 2D. N-D (T/Z stacks) and the QuPath resolution (downsample) choice follow in Wave 2.** The frame (1.1.2) and the placement formula (1.1.3) already have their final shape, so Wave 2 adds fixtures, not rules. Wave 2 fixtures still needed: `TYX` and `ZYX` results with `t`/`z` columns, a 2D result on a `TYX` layer (Napari), a per-axis `_calibration` with z and t, and QuPath export at downsample 2 and 4 on a real pyramid (CF-14 is the data, the GUI harness is the missing part). Same conventions as [SPEC_WAVE2.md](SPEC_WAVE2.md) and [HOST_PARITY.md](HOST_PARITY.md): additive within `protocol: 1`, rules are host-neutral, a rule is "done" when its fixtures pass in every host that claims the feature. The fixtures below are data and live in `labconstrictor_tools/conformance.json` under the key `wrong_science` (see [CONFORMANCE.md](CONFORMANCE.md)), to be reused by the conformance suite (Tools #87). These rules exist because a tool run can succeed and still be scientifically wrong: a mask on the wrong pixels, a result drawn away from its image, a pixel size from another image, a researcher's folder deleted. Each host fixes its own code, but all of them implement **the same rule**, so a fix in one host cannot leave another one behind. diff --git a/docs/TESTING.md b/docs/TESTING.md index fe4d94c..f120ac8 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -30,6 +30,7 @@ every real path, and that the tests themselves would notice if the production co | `tests/test_roundtrip_command.py` | quoting against a real shell and the Windows rules; app names with spaces and quotes | | `tests/roundtrip_known_failures.py` + `test_roundtrip_known_failures.py` | real defects found, and the watcher that keeps the list honest | | `tests/test_retention.py` | `retention` and `labconstrictor-tools prune` ([SPEC_WRONG_SCIENCE.md](SPEC_WRONG_SCIENCE.md) 3.7: RET-01 to RET-09, RET-12, RET-13): a private `LC_HOME`, every survivor checked by the contents of a sentinel file, four real `prune` processes at once, a lock held by another process, symlinks (and on Windows junctions and read-only files; a link kind that cannot be made is reported as skipped, never as a pass) | +| `tests/test_conformance.py` | `conformance.json` and its loader ([CONFORMANCE.md](CONFORMANCE.md)): the schema (every kind of damage is named), the data against the code (limits, `compat` vectors, retention fixtures once `retention` exists), arithmetic oracles for the coordinate formula and the unit table, and the wheel contents | | `tests/test_sabotage.py`, `coverage_report.py`, `vulture_whitelist.py` | the production-code guarantees | Transports and what each can carry: `worker` carries everything (including shared-memory images and invalid JSON types), `cli`, diff --git a/labconstrictor_tools/conformance.json b/labconstrictor_tools/conformance.json new file mode 100644 index 0000000..8f372a4 --- /dev/null +++ b/labconstrictor_tools/conformance.json @@ -0,0 +1,1888 @@ +{ + "conformance": 1, + "_about": "Rules every host must agree on, as data. Loaded with labconstrictor_tools.conformance.load(); validated by labconstrictor_tools.conformance.validate(). Only rules that are already decided in docs/SPEC_WRONG_SCIENCE.md (section 8) or carried by Tools code are here; a rule marked status 'default' is the spec's working default, not yet confirmed. Gaps (rules that are not decided yet) are listed in docs/CONFORMANCE.md, never invented here. A host test must fail when a key it knows is missing (no silent skip) and report 'unsupported' for a case its host cannot run, never 'pass'.", + "limits": { + "results_kept": { + "value": 20, + "status": "decided", + "spec": "3.3", + "unit": "owned finished result folders" + }, + "run_records_kept": { + "value": 50, + "status": "decided", + "spec": "3.3", + "unit": "owned finished run records" + }, + "form_integer_max": { + "value": 9007199254740991, + "status": "decided", + "spec": "6.1", + "rule": "an integer form field with no declared Min/Max accepts +-value; beyond it is refused, never clamped" + }, + "unbounded_float": { + "value": 1000000000000000.0, + "status": "decided", + "spec": "6.1", + "rule": "a float form field with no declared Min/Max accepts +-value; beyond it is refused, never clamped" + }, + "choices_debounce_ms": { + "value": 400, + "status": "decided", + "spec": "5.3", + "note": "delays the new request, never the invalidation" + }, + "choices_busy_retry_ms": { + "value": 1500, + "status": "decided", + "spec": "5.5", + "note": "lookups wait while a run is active" + }, + "worker_close_timeout_s": { + "value": 10, + "status": "decided", + "spec": "5.6", + "note": "HOST_PARITY K03" + }, + "lookup_in_flight_max": { + "value": 1, + "status": "default", + "spec": "5.4", + "note": "per widget (per host window)" + }, + "lookup_pending_max": { + "value": 8, + "status": "default", + "spec": "5.4", + "note": "oldest dropped and logged" + }, + "lookup_timeout_s": { + "value": 60, + "status": "default", + "spec": "5.4" + } + }, + "host_internal_codes": [ + "frame_unsupported_transform", + "selection_mixed_planes", + "selection_plane_mismatch", + "target_unavailable" + ], + "messages": { + "number_above_maximum": { + "template": "'{label}' must be <= {maximum}", + "spec": "6.2", + "note": "HOST_PARITY I06 worker templates" + }, + "number_below_minimum": { + "template": "'{label}' must be >= {minimum}", + "spec": "6.2" + }, + "size_mismatch": { + "template": "'{name}' is {w} x {h} but '{source}' was exported as {w2} x {h2}; shown at the pixel origin, not aligned", + "spec": "1.1.7" + }, + "calibration_required": { + "template": "'{param}' needs a value: '{of}' has no pixel size. Enter it in um.", + "spec": "2.3.1" + }, + "calibration_unknown_tooltip": { + "template": "not calibrated: '{of}' has no pixel size", + "spec": "2.2" + }, + "unit_damaged_warning": { + "template": "unit text '{unit}' is damaged; enter the pixel size", + "spec": "2.1.4" + }, + "pixels_not_square_warning": { + "template": "pixels are not square", + "spec": "2.1.5" + }, + "unit_not_length_warning": { + "template": "unit '{unit}' is not a length", + "spec": "2.1.5" + }, + "target_unavailable": { + "template": "run {run_id} on '{source}': '{source}' is no longer open; results are in {path}", + "spec": "4.4" + }, + "refusal_suffix": { + "template": "(nothing was changed; files are in {path})", + "spec": "1.4", + "note": "follows every refusal sentence when files exist" + }, + "file_input_note": { + "template": "file input: results are in the file's pixel frame", + "spec": "1.3", + "note": "QuPath status" + } + }, + "compat": { + "_about": "docs/PROTOCOL.md, 'Compatibility rules for hosts'. Reference: labconstrictor_tools.compat.check_schema.", + "min_tools_version": "0.2.0rc1", + "required_features": [ + "reserved_inputs_v1" + ], + "cases": [ + { + "case": "current", + "schema": { + "protocol": 1, + "tools_version": "0.2.0rc1", + "features": [ + "reserved_inputs_v1" + ] + }, + "expect": "ok" + }, + { + "case": "newer release", + "schema": { + "protocol": 1, + "tools_version": "0.3.1", + "features": [ + "reserved_inputs_v1", + "later" + ] + }, + "expect": "ok" + }, + { + "case": "written by Tools 0.1 (no fields)", + "schema": { + "protocol": 1 + }, + "expect": "refuse", + "reason_contains": "too old" + }, + { + "case": "feature missing", + "schema": { + "protocol": 1, + "tools_version": "0.2.0", + "features": [ + "other" + ] + }, + "expect": "refuse", + "reason_contains": "reserved_inputs_v1" + }, + { + "case": "below the minimum", + "schema": { + "protocol": 1, + "tools_version": "0.1.9", + "features": [ + "reserved_inputs_v1" + ] + }, + "expect": "refuse", + "reason_contains": "older than" + }, + { + "case": "version not understood", + "schema": { + "protocol": 1, + "tools_version": "banana", + "features": [ + "reserved_inputs_v1" + ] + }, + "expect": "warn" + }, + { + "case": "protocol 2", + "schema": { + "protocol": 2 + }, + "expect": "refuse", + "reason_contains": "protocol" + } + ], + "version_order": [ + "0.2.0a1", + "0.2.0b1", + "0.2.0rc1", + "0.2.0", + "0.2.1", + "0.10.0" + ] + }, + "wrong_science": { + "_spec": "docs/SPEC_WRONG_SCIENCE.md", + "_scope": "v1: 2D. N-D (T/Z stacks) and QuPath downsample fixtures follow in Wave 2 (SPEC_WRONG_SCIENCE, header).", + "coordinate": { + "image": { + "id": "ASYM", + "shape": [ + 8, + 12 + ], + "dtype": "uint16", + "value": "100*y + x" + }, + "rect": [ + [ + 1.5, + 2.5 + ], + [ + 1.5, + 5.5 + ], + [ + 3.5, + 5.5 + ], + [ + 3.5, + 2.5 + ] + ], + "rect_mask_pixels": [ + [ + 2, + 3 + ], + [ + 2, + 4 + ], + [ + 2, + 5 + ], + [ + 3, + 3 + ], + [ + 3, + 4 + ], + [ + 3, + 5 + ] + ], + "rect_mask_values": [ + 203, + 204, + 205, + 303, + 304, + 305 + ], + "placement": { + "formula": "c = x0 + (x + 0.5) * downsample", + "napari_data": "c - 0.5", + "spec": "1.1.3" + }, + "every_case_also_checks": [ + "the worker's input file (exported pixels, mask pixels)", + "the actual drawn coordinates", + "nothing was drawn on any image other than target" + ], + "cases": [ + { + "id": "CF-01", + "name": "identity", + "source": {}, + "point": [ + 2, + 3 + ], + "expect": { + "world": [ + 2, + 3 + ], + "target_px": [ + 2, + 3 + ], + "mask": "rect_mask_pixels" + } + }, + { + "id": "CF-02", + "name": "translate only (scale equal, origin different)", + "source": { + "translate": [ + 100, + 0 + ] + }, + "shapes_layer": { + "translate": [ + 0, + 0 + ], + "vertices_world": "rect + [100, 0]" + }, + "point": [ + 2, + 3 + ], + "expect": { + "world": [ + 102, + 3 + ], + "target_px": [ + 2, + 3 + ], + "mask": "rect_mask_pixels" + } + }, + { + "id": "CF-03", + "name": "scale and translate", + "source": { + "scale": [ + 0.5, + 0.25 + ], + "translate": [ + 10, + 20 + ] + }, + "point": [ + 2, + 3 + ], + "expect": { + "world": [ + 11.0, + 20.75 + ], + "target_px": [ + 2, + 3 + ] + } + }, + { + "id": "CF-04", + "name": "rotation by 90 degrees", + "source": { + "affine": [ + [ + 0, + -1, + 0 + ], + [ + 1, + 0, + 0 + ], + [ + 0, + 0, + 1 + ] + ] + }, + "shapes_layer": { + "vertices_world": [ + [ + -2.5, + 1.5 + ], + [ + -5.5, + 1.5 + ], + [ + -5.5, + 3.5 + ], + [ + -2.5, + 3.5 + ] + ] + }, + "point": [ + 2, + 3 + ], + "expect": { + "world": [ + -3, + 2 + ], + "target_px": [ + 2, + 3 + ], + "mask": "rect_mask_pixels" + } + }, + { + "id": "CF-05", + "name": "shear", + "source": { + "affine": [ + [ + 1, + 0.5, + 0 + ], + [ + 0, + 1, + 0 + ], + [ + 0, + 0, + 1 + ] + ] + }, + "point": [ + 2, + 4 + ], + "expect": { + "world": [ + 4, + 4 + ], + "target_px": [ + 2, + 4 + ] + } + }, + { + "id": "CF-06", + "name": "multiscale, level 0 carries the transform", + "source": { + "levels": [ + [ + 8, + 12 + ], + [ + 4, + 6 + ] + ], + "translate": [ + 100, + 0 + ] + }, + "point": [ + 2, + 3 + ], + "expect": { + "world": [ + 102, + 3 + ], + "export_shape": [ + 8, + 12 + ] + } + }, + { + "id": "CF-07", + "name": "file input with a decoy open image", + "inputs": { + "image": { + "file": "ASYM.tif" + } + }, + "open_image": { + "id": "DECOY", + "translate": [ + 100, + 0 + ] + }, + "point": [ + 2, + 3 + ], + "expect": { + "target": "file:ASYM.tif", + "napari_world": [ + 2, + 3 + ], + "decoy_gets_nothing": true, + "fiji_new_window_titled": "ASYM.tif", + "qupath": "table_and_file" + } + }, + { + "id": "CF-08", + "name": "earlier output must not become the target", + "sequence": [ + "run A -> ImageOut shown", + "run B (file input) -> PointsOut ApplyTo" + ], + "expect": { + "target": "file:ASYM.tif", + "last_shown_image_gets_nothing": true + } + }, + { + "id": "CF-09", + "name": "QuPath crop offset", + "source": { + "server_wh": [ + 12, + 8 + ] + }, + "selection_annotation": { + "x": [ + 4, + 10 + ], + "y": [ + 2, + 6 + ], + "plane": [ + 0, + 0 + ] + }, + "expect": { + "export_offset_yx": [ + 2, + 4 + ], + "export_shape": [ + 4, + 6 + ], + "export_value_0_0": 204, + "export_value_3_5": 509, + "regionof_mask": "all ones, shape [4, 6]" + }, + "point": [ + 1, + 2 + ], + "result_expect": { + "target_px": [ + 3, + 6 + ] + } + }, + { + "id": "CF-10", + "name": "QuPath planes", + "source": { + "z": 2, + "t": 1 + }, + "viewer": { + "z": 0, + "t": 0 + }, + "variants": [ + { + "selection": [ + { + "z": 0, + "t": 0 + } + ], + "expect": "run, results on z=0 t=0" + }, + { + "selection": [ + { + "z": 1, + "t": 0 + } + ], + "expect": "refuse selection_plane_mismatch" + }, + { + "selection": [ + { + "z": 0, + "t": 0 + }, + { + "z": 1, + "t": 0 + } + ], + "expect": "refuse selection_mixed_planes" + }, + { + "selection": [], + "viewer": { + "z": 1, + "t": 0 + }, + "export": "whole image", + "expect": "exported plane z=1 t=0, results on z=1 t=0" + }, + { + "point_without_t_z": true, + "viewer": { + "z": 1, + "t": 0 + }, + "expect": "lands on z=1 t=0, not on plane 0" + } + ] + }, + { + "id": "CF-11", + "name": "size mismatch is stated", + "result": { + "type": "labels", + "shape": [ + 4, + 6 + ], + "apply_to": "image" + }, + "source_export_shape": [ + 8, + 12 + ], + "expect": { + "transform": "identity", + "status_contains": "not aligned" + } + }, + { + "id": "CF-12", + "name": "Replace recomputes the transform", + "sequence": [ + "labels result on source translate [100,0]", + "Replace with result for source translate [0,0]" + ], + "expect": { + "world_of_0_0": [ + 0, + 0 + ] + } + }, + { + "id": "CF-13", + "name": "non-invertible source transform", + "source": { + "affine": [ + [ + 0, + 0, + 0 + ], + [ + 0, + 0, + 0 + ], + [ + 0, + 0, + 1 + ] + ] + }, + "expect": "refuse frame_unsupported_transform" + }, + { + "id": "CF-14", + "name": "QuPath downsample 2 and 4 (c = x0 + (x + 0.5) * d); needs a host that exports at d != 1", + "variants": [ + { + "server_wh": [ + 24, + 16 + ], + "downsample": 2, + "offset_yx": [ + 0, + 0 + ], + "export_shape": [ + 8, + 12 + ], + "point": [ + 2, + 3 + ], + "expect": { + "c_yx": [ + 5.0, + 7.0 + ] + } + }, + { + "server_wh": [ + 48, + 32 + ], + "downsample": 4, + "offset_yx": [ + 0, + 0 + ], + "export_shape": [ + 8, + 12 + ], + "point": [ + 2, + 3 + ], + "expect": { + "c_yx": [ + 10.0, + 14.0 + ] + } + }, + { + "server_wh": [ + 48, + 32 + ], + "downsample": 2, + "offset_yx": [ + 4, + 8 + ], + "export_shape": [ + 8, + 12 + ], + "point": [ + 2, + 3 + ], + "expect": { + "c_yx": [ + 9.0, + 15.0 + ] + } + }, + { + "downsample": 1, + "offset_yx": [ + 0, + 0 + ], + "point": [ + 2, + 3 + ], + "expect": { + "c_yx": [ + 2.5, + 3.5 + ] + } + } + ], + "unsupported_ok": "a host that only exports at downsample 1 reports the first three as unsupported, never as passed" + } + ] + }, + "calibration": { + "unit": "um per pixel", + "length_units_um": { + "um": 1.0, + "micron": 1.0, + "microns": 1.0, + "micrometer": 1.0, + "micrometre": 1.0, + "micrometers": 1.0, + "micrometres": 1.0, + "nm": 0.001, + "mm": 1000.0, + "millimeter": 1000.0, + "millimetre": 1000.0, + "cm": 10000.0, + "m": 1000000.0, + "Angstrom": 0.0001, + "angstrom": 0.0001, + "\u00c5": 0.0001, + "\u212b": 0.0001, + "\u00b5m": 1.0, + "\u03bcm": 1.0 + }, + "micro_sign_spellings": [ + "\u00b5", + "\u03bc", + "u" + ], + "damaged_micro_prefix": "\ufffd", + "anisotropy_tolerance": { + "relative_to_larger": 0.001, + "within": "the scalar field takes the mean of x and y", + "beyond": "unknown plus warning" + }, + "default_resolution_dpi": { + "values": [ + 72, + 96 + ], + "per_cm": [ + 28.35, + 37.8 + ], + "tolerance_relative": 0.005, + "units": [ + "INCH", + "CENTIMETER" + ], + "result": "unknown" + }, + "states": [ + "unset", + "auto", + "user" + ], + "reserved_input": "_calibration", + "request_shape": { + "_calibration": { + "pixel_size": { + "of": "image", + "source": "image|file|user", + "axes": { + "x": { + "value": 1.0, + "unit": "um", + "source": "image", + "raw": { + "scale": 1.0, + "unit": "micrometer" + } + }, + "y": { + "value": 1.0, + "unit": "um", + "source": "image", + "raw": { + "scale": 1.0, + "unit": "micrometer" + } + } + } + } + }, + "phase_1_axes": [ + "x", + "y" + ], + "later_axes": { + "z": "ZStepOf, um", + "t": "TimeIntervalOf, s" + } + }, + "cases": [ + { + "id": "CAL-01", + "source": "Napari scale (1,1), units (micrometer, micrometer)", + "input": { + "scale": [ + 1, + 1 + ], + "units": [ + "micrometer", + "micrometer" + ] + }, + "expect": { + "um_per_px": 1.0, + "source": "image" + } + }, + { + "id": "CAL-02", + "source": "Napari scale (1,1), units default (pixel)", + "input": { + "scale": [ + 1, + 1 + ], + "units": [ + "pixel", + "pixel" + ] + }, + "expect": { + "um_per_px": "unknown" + } + }, + { + "id": "CAL-03", + "source": "Napari scale (1,1), units millimeter", + "input": { + "scale": [ + 1, + 1 + ], + "units": [ + "millimeter", + "millimeter" + ] + }, + "expect": { + "um_per_px": 1000.0 + } + }, + { + "id": "CAL-04", + "source": "Napari scale (0.25,0.25), units pixel or none (never read as um)", + "input": { + "scale": [ + 0.25, + 0.25 + ], + "units": [ + [ + "pixel", + "pixel" + ], + [ + null, + null + ] + ] + }, + "expect": { + "um_per_px": "unknown" + } + }, + { + "id": "CAL-05", + "source": "0.325 um, 325 nm, 0.000325 mm (any host)", + "input": { + "variants": [ + { + "scale": 0.325, + "unit": "um" + }, + { + "scale": 325, + "unit": "nm" + }, + { + "scale": 0.000325, + "unit": "mm" + } + ] + }, + "expect": { + "um_per_px": 0.325, + "relative_error_below": 1e-09 + } + }, + { + "id": "CAL-06", + "source": "scale (0.5, 0.25), units um", + "input": { + "scale": [ + 0.5, + 0.25 + ], + "units": [ + "um", + "um" + ] + }, + "expect": { + "um_per_px": "unknown", + "warning": "pixels are not square" + } + }, + { + "id": "CAL-07", + "source": "scale (1,1), units second", + "input": { + "scale": [ + 1, + 1 + ], + "units": [ + "second", + "second" + ] + }, + "expect": { + "um_per_px": "unknown", + "warning": "unit 'second' is not a length" + } + }, + { + "id": "CAL-08", + "source": "Fiji unit micron, pixelWidth 1.0 / unit pixel, width 1.0 / unit empty", + "input": { + "variants": [ + { + "pixelWidth": 1.0, + "unit": "micron" + }, + { + "pixelWidth": 1.0, + "unit": "pixel" + }, + { + "pixelWidth": 1.0, + "unit": "" + } + ] + }, + "expect": { + "um_per_px_each": [ + 1.0, + "unknown", + "unknown" + ] + } + }, + { + "id": "CAL-09", + "source": "QuPath hasPixelSizeMicrons() true with 1.0 / false", + "input": { + "variants": [ + { + "hasPixelSizeMicrons": true, + "um": 1.0 + }, + { + "hasPixelSizeMicrons": false + } + ] + }, + "expect": { + "um_per_px_each": [ + 1.0, + "unknown" + ] + } + }, + { + "id": "CAL-10", + "source": "TIFF with ImageJ unit=micron spacing 1.0 / no resolution tags / ResolutionUnit NONE", + "input": { + "variants": [ + { + "imagej_unit": "micron", + "spacing": 1.0 + }, + { + "tags": null + }, + { + "ResolutionUnit": "NONE" + } + ] + }, + "expect": { + "um_per_px_each": [ + 1.0, + "unknown", + "unknown" + ] + } + }, + { + "id": "CAL-11", + "source": "TIFF ResolutionUnit INCH, 72 dpi, no ImageJ/OME unit", + "input": { + "ResolutionUnit": "INCH", + "resolution": [ + 72, + 72 + ] + }, + "expect": { + "um_per_px": "unknown", + "not": 352.78 + } + }, + { + "id": "CAL-12", + "source": "TIFF ResolutionUnit INCH, 96 dpi, no ImageJ/OME unit", + "input": { + "ResolutionUnit": "INCH", + "resolution": [ + 96, + 96 + ] + }, + "expect": { + "um_per_px": "unknown", + "not": 264.58 + } + }, + { + "id": "CAL-13", + "source": "TIFF ResolutionUnit CENTIMETER, 28.35 and 37.80 per cm, no unit text", + "input": { + "variants": [ + { + "ResolutionUnit": "CENTIMETER", + "resolution": [ + 28.35, + 28.35 + ] + }, + { + "ResolutionUnit": "CENTIMETER", + "resolution": [ + 37.8, + 37.8 + ] + } + ] + }, + "expect": { + "um_per_px_each": [ + "unknown", + "unknown" + ] + } + }, + { + "id": "CAL-14", + "source": "TIFF ResolutionUnit INCH, 25400 dpi (not default-looking), no unit text", + "input": { + "ResolutionUnit": "INCH", + "resolution": [ + 25400, + 25400 + ] + }, + "expect": { + "um_per_px": 1.0, + "source": "file" + } + }, + { + "id": "CAL-15", + "source": "unit text micro sign / Greek mu / ASCII u, each + m, scale 0.5", + "input": { + "variants": [ + { + "scale": 0.5, + "unit": "\u00b5m" + }, + { + "scale": 0.5, + "unit": "\u03bcm" + }, + { + "scale": 0.5, + "unit": "um" + } + ] + }, + "expect": { + "um_per_px_each": [ + 0.5, + 0.5, + 0.5 + ] + } + }, + { + "id": "CAL-16", + "source": "unit text U+FFFD + m, scale 0.5", + "input": { + "scale": 0.5, + "unit": "\ufffdm" + }, + "expect": { + "um_per_px": "unknown", + "warning": "unit text '\ufffdm' is damaged; enter the pixel size" + } + }, + { + "id": "CAL-17", + "source": "unit text Angstrom, U+00C5, U+212B, scale 1.0", + "input": { + "variants": [ + { + "scale": 1.0, + "unit": "Angstrom" + }, + { + "scale": 1.0, + "unit": "\u00c5" + }, + { + "scale": 1.0, + "unit": "\u212b" + } + ] + }, + "expect": { + "um_per_px_each": [ + 0.0001, + 0.0001, + 0.0001 + ] + } + }, + { + "id": "CAL-18", + "source": "scale (0.5, 0.5004) um / scale (0.5, 0.5006) um", + "input": { + "variants": [ + { + "scale": [ + 0.5, + 0.5004 + ], + "unit": "um" + }, + { + "scale": [ + 0.5, + 0.5006 + ], + "unit": "um" + } + ] + }, + "expect": { + "um_per_px_each": [ + 0.5002, + "unknown" + ], + "note": "difference 8e-4 is within 1e-3 (mean of x and y); 1.2e-3 is not (unknown + warning)" + } + } + ], + "transitions": [ + { + "id": "CAL-T1", + "sequence": "A (0.25) -> B (0.5)", + "expect": "field 0.5, request 0.5, source image of B" + }, + { + "id": "CAL-T2", + "sequence": "A (0.25) -> uncalibrated B", + "expect": "field empty, run blocked with 2.3.1; Optional: request null; A's value is in no request" + }, + { + "id": "CAL-T3", + "sequence": "A (0.25), person types 0.3, switch to B (0.5)", + "expect": "0.3, source user" + }, + { + "id": "CAL-T4", + "sequence": "state user(0.3), person clears field, source B (0.5)", + "expect": "auto 0.5" + }, + { + "id": "CAL-T5", + "sequence": "open image A (0.25), file B uncalibrated", + "expect": "unset, run blocked (not A's value)" + }, + { + "id": "CAL-T6", + "sequence": "CAL-T5, then file cleared", + "expect": "auto 0.25 (A)" + }, + { + "id": "CAL-T7", + "sequence": "QuPath project image A -> file image uncalibrated", + "expect": "unset" + }, + { + "id": "CAL-T8", + "sequence": "required linked field with default 1.0, source unknown", + "expect": "blocked, the default is not sent" + }, + { + "id": "CAL-T9", + "sequence": "user types 0.25 equal to the last auto value, then switch to uncalibrated B", + "expect": "stays 0.25, source user (flag, not comparison)" + } + ] + }, + "retention": { + "markers": { + "owner_file": ".lc-owner.json", + "done_file": ".lc-done.json", + "lock_file": ".retention.lock", + "deleting_prefix": ".lc-deleting-", + "owner_fields": [ + "lc_marker", + "kind", + "name", + "host", + "pid", + "created", + "token" + ], + "owner_kinds": [ + "result", + "run" + ], + "hosts": [ + "cli", + "napari", + "fiji", + "qupath", + "notebook" + ], + "done_fields": [ + "finished", + "status" + ], + "done_statuses": [ + "ok", + "failed", + "cancelled" + ], + "name_characters": "A-Za-z0-9_.-", + "result_folder_name": "_<6 hex>__" + }, + "prune_command": "labconstrictor-tools prune [--root results|runs|all] [--dry-run]", + "fixtures_tools": [ + { + "id": "RET-01", + "setup": [ + { + "foreign": { + "root": "results", + "name": "20261008T120000_reanalysis_backup", + "files": { + "x.txt": "keep" + } + } + }, + { + "owned": { + "root": "results", + "count": 300 + } + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "foreign_intact": true + }, + { + "owned_remaining": { + "root": "results", + "count": 20, + "which": "newest_by_created" + } + } + ], + "scope": "tools" + }, + { + "id": "RET-02", + "setup": [ + { + "foreign": { + "root": "runs", + "name": "0000_keep_this_analysis", + "files": { + "x.txt": "keep" + } + } + }, + { + "owned": { + "root": "runs", + "count": 120 + } + } + ], + "action": { + "prune": "runs" + }, + "expect": [ + { + "foreign_intact": true + }, + { + "owned_remaining": { + "root": "runs", + "count": 50, + "which": "newest_by_created" + } + } + ], + "scope": "tools" + }, + { + "id": "RET-03", + "setup": [ + { + "owned": { + "root": "results", + "count": 25, + "done": false + } + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "owned_remaining": { + "root": "results", + "count": 25, + "which": "all" + } + } + ], + "scope": "tools" + }, + { + "id": "RET-04", + "setup": [ + { + "owned": { + "root": "results", + "count": 30 + } + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "owned_remaining": { + "root": "results", + "count": 20, + "which": "newest_by_created" + } + } + ], + "scope": "tools" + }, + { + "id": "RET-05", + "setup": [ + { + "link": { + "root": "results", + "name": "20261008T120000_000000_app_tool", + "to_outside_with_markers": true, + "kinds": [ + "symlink", + "junction (Windows)" + ] + } + }, + { + "owned": { + "root": "results", + "count": 40 + } + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "link_intact": true + }, + { + "outside_intact": true + } + ], + "unsupported_when": "a link kind cannot be created: report unsupported for it, never pass", + "scope": "tools" + }, + { + "id": "RET-06", + "setup": [ + { + "owned_copy_renamed": { + "root": "results", + "name": "20261008T120000_000000_app_copy" + } + }, + { + "owned": { + "root": "results", + "count": 30 + } + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "copy_intact": true + } + ], + "scope": "tools" + }, + { + "id": "RET-07", + "setup": [ + { + "legacy": { + "root": "results", + "count": 30 + } + }, + { + "owned": { + "root": "results", + "count": 25 + } + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "legacy_intact": 30 + }, + { + "owned_remaining": { + "root": "results", + "count": 20, + "which": "newest_by_created" + } + } + ], + "scope": "tools" + }, + { + "id": "RET-08", + "setup": [ + { + "foreign": { + "root": "results", + "name": "20261008T120000_reanalysis_backup", + "files": { + "x.txt": "keep" + } + } + }, + { + "owned": { + "root": "results", + "count": 100 + } + } + ], + "action": { + "prune_processes_at_once": 4, + "via": "cli", + "root": "results" + }, + "expect": [ + { + "owned_remaining": { + "root": "results", + "count": 20, + "which": "newest_by_created" + } + }, + { + "foreign_intact": true + }, + { + "every_exit_code": 0 + } + ], + "scope": "tools" + }, + { + "id": "RET-09", + "setup": [ + { + "owned": { + "root": "results", + "count": 30 + } + }, + { + "lock_held_by_another_process": true + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "exit_code": 0 + }, + { + "owned_remaining": { + "root": "results", + "count": 30, + "which": "all" + } + } + ], + "scope": "tools" + }, + { + "id": "RET-12", + "setup": [ + { + "owned": { + "root": "results", + "count": 1, + "done": false, + "oldest": true + } + }, + { + "owned": { + "root": "results", + "count": 30 + } + } + ], + "action": { + "prune": "results" + }, + "expect": [ + { + "crashed_survives": true + } + ], + "scope": "tools" + }, + { + "id": "RET-13", + "setup": [ + { + "equal_created": { + "root": "results", + "count": 3 + } + } + ], + "action": { + "prune": "results", + "keep": 2 + }, + "expect": [ + { + "remaining_names": "the last two by name, deterministic" + } + ], + "scope": "tools" + } + ], + "fixtures_hosts": [ + { + "id": "HRET-01", + "scope": "host", + "setup": "foreign results/20261008T120000_reanalysis_backup/x.txt and foreign runs/0000_keep_this_analysis/x.txt", + "action": "the host does 60 runs", + "expect": "both foreign folders and x.txt intact" + }, + { + "id": "HRET-02", + "scope": "host", + "setup": "done(60) owned results and done(120) owned run records already present", + "action": "the host does 5 runs", + "expect": "none of the old folders is gone (counts only grow)" + }, + { + "id": "HRET-03", + "scope": "host", + "setup": "none", + "action": "one run", + "expect": "the folder has a valid .lc-owner.json (name equals folder name) and, after the run, a valid .lc-done.json with the right status" + }, + { + "id": "HRET-04", + "scope": "host", + "setup": "prune replaced by a recorder, then by a command that fails", + "action": "one run each", + "expect": "the recorder saw labconstrictor-tools prune started after the done file exists; a failing command does not fail the run" + }, + { + "id": "RET-10", + "scope": "host", + "setup": "job folder with report.txt ('data') returned as FileOut", + "action": "run to success, close the widget", + "expect": "reported path exists under results/, contents 'data'; job folder gone" + }, + { + "id": "RET-11", + "scope": "host", + "setup": "failed or cancelled run", + "action": "finish", + "expect": "temporary job folder removed; an owned folder, if any, has failed / cancelled" + }, + { + "id": "RET-14", + "scope": "host", + "setup": "undisplayable (over the display limit) ImageOut", + "action": "run", + "expect": "file stays at a stable owned path, the path is shown" + } + ], + "note": "fixtures_tools are executed by Tools against labconstrictor_tools.retention and the CLI; fixtures_hosts by every host with the prune subprocess replaced by a recorder" + }, + "run_identity": { + "spec": "section 4", + "fixtures": [ + { + "id": "RUN-01", + "sequence": "start on A (delayed tool), activate B, tool returns points", + "expect": "nothing on B; target_unavailable if A is gone, else points on A" + }, + { + "id": "RUN-02", + "sequence": "start on A with a selection, change the selection during export", + "expect": "exported crop and mask are those of the click" + }, + { + "id": "RUN-03", + "sequence": "start tool X, switch the widget to tool Y while running", + "expect": "result metadata app/tool are X's" + }, + { + "id": "RUN-04", + "sequence": "start on A, close A", + "expect": "outputs kept, status names run_id and path, no exception" + }, + { + "id": "RUN-05", + "sequence": "Napari: layer A is moved (translate changed) during the run", + "expect": "points follow A's transform at placement" + }, + { + "id": "RUN-06", + "sequence": "two runs in a row on A then B", + "expect": "each result carries its own run_id and lands on its own source" + } + ] + }, + "async_lookups": { + "spec": "section 5", + "fixtures": [ + { + "id": "ASY-01", + "sequence": "tool A and B both have ChoicesFrom param mode; A's lookup is delayed; switch to B; release A", + "expect": "B's dropdown shows only B's choices; displayed choices and value checked" + }, + { + "id": "ASY-02", + "sequence": "change the dependency of mode, release the old reply before the debounce fires", + "expect": "old reply dropped; mode is pending, not showing old choices" + }, + { + "id": "ASY-03", + "sequence": "A's lookup fails after switch to B", + "expect": "no tooltip or error on any B field" + }, + { + "id": "ASY-04", + "sequence": "change a dependency 50 times in 1 s", + "expect": "at most 1 in flight, at most 1 pending per param; worker processes started <= 1" + }, + { + "id": "ASY-05", + "sequence": "two parameters each with ChoicesFrom, both lookups requested at once", + "expect": "one worker process; each reply goes to its own control" + }, + { + "id": "ASY-06", + "sequence": "start a run, switch app, request a lookup", + "expect": "run uses its worker to the end; the lookup runs after, in the new app's worker" + }, + { + "id": "ASY-07", + "sequence": "close the widget with a lookup in flight", + "expect": "no worker process or thread of the widget alive afterwards; no worker created after close survives" + }, + { + "id": "ASY-08", + "sequence": "interleave acquire/release/reap/close from three threads with barriers", + "expect": "cache consistent (each worker owned once), no double close" + } + ], + "status": "limits marked default in limits.lookup_* are not yet confirmed" + }, + "number_limits": { + "spec": "section 6", + "limits": [ + "form_integer_max", + "unbounded_float" + ], + "cases": [ + { + "id": "NUM-01", + "field": "integer, no Min/Max", + "typed": "9007199254740991", + "expect": "accepted; the request carries exactly that integer" + }, + { + "id": "NUM-02", + "field": "integer, no Min/Max", + "typed": "9007199254740992", + "expect": "refused: 'L' must be <= 9007199254740991; never clamped (not 9007199254740991, not 2147483647); no run, no request" + }, + { + "id": "NUM-03", + "field": "integer, no Min/Max", + "typed": "-9007199254740992", + "expect": "refused: 'L' must be >= -9007199254740991; never clamped; no run, no request" + }, + { + "id": "NUM-04", + "field": "float, no Min/Max", + "typed": [ + "1e15", + "1.0000000000000002e15" + ], + "expect": [ + "accepted", + "refused" + ] + }, + { + "id": "NUM-05", + "field": "integer with declared Max 10", + "typed": "11", + "expect": "refused (unchanged)" + } + ], + "hosts": [ + "napari", + "fiji dialog", + "fiji macro", + "qupath", + "notebook" + ], + "cli": "the worker is the reference" + } + } +} diff --git a/labconstrictor_tools/conformance.py b/labconstrictor_tools/conformance.py new file mode 100644 index 0000000..40b60e1 --- /dev/null +++ b/labconstrictor_tools/conformance.py @@ -0,0 +1,282 @@ +"""The cross-host conformance data (`conformance.json`, shipped inside the package) and its loader. + + from labconstrictor_tools import conformance + data = conformance.load() # validated; raises ConformanceError listing every problem + data["limits"]["form_integer_max"]["value"] + conformance.path() # the file itself, for hosts that are not Python (Fiji, QuPath) + python -m labconstrictor_tools.conformance --path | --validate + +The file holds rules that are already decided in docs/SPEC_WRONG_SCIENCE.md (and the compatibility vectors of PROTOCOL.md) as +data: named limits, message templates, the host-internal error codes and the test vectors (coordinate frame, calibration, +retention, run identity, asynchronous lookups, number limits). Every host test loads it and compares its own constants and +behaviour with it; how each host does that is in docs/CONFORMANCE.md. `validate` is the schema: it is checked on every load, so a +broken edit of the file fails every host test at once instead of silently skipping a rule. Standard library only. +""" + +import json +import re +import string +import sys +from collections.abc import Mapping +from importlib import resources +from pathlib import Path +from typing import Any + +from .compat import OK, REFUSE, WARN, parse_version + +CONFORMANCE_VERSION = 1 +FILE_NAME = "conformance.json" +STATUSES = ("decided", "default") # a rule the maintainers confirmed, or the spec's working default +SCOPES = ("tools", "host") # who runs a fixture +_TOP = ("conformance", "limits", "host_internal_codes", "messages", "compat", "wrong_science") +_SECTIONS = ("coordinate", "calibration", "retention", "run_identity", "async_lookups", "number_limits") +_IDENTIFIER = re.compile(r"^[a-z][a-z0-9_]*$") +_ID_PATTERNS = { + "coordinate": r"^CF-\d\d$", + "calibration": r"^CAL-\d\d$", + "calibration transitions": r"^CAL-T\d$", + "retention": r"^H?RET-\d\d$", + "run_identity": r"^RUN-\d\d$", + "async_lookups": r"^ASY-\d\d$", + "number_limits": r"^NUM-\d\d$", +} + +_cache: dict[str, Any] | None = None + + +class ConformanceError(ValueError): + """The conformance data is malformed; the message lists every problem found, one per line.""" + + +def path() -> Path: + """The data file inside the installed package (also works from a source checkout).""" + return Path(str(resources.files(__package__).joinpath(FILE_NAME))) + + +def load() -> dict[str, Any]: + """The validated data. Read-only by convention: the same object is returned on every call (copy before changing it).""" + global _cache + if _cache is None: + data = json.loads(path().read_text(encoding="utf-8")) + problems = validate(data) + if problems: + raise ConformanceError("%s is malformed:\n %s" % (FILE_NAME, "\n ".join(problems))) + _cache = data + return _cache + + +def validate(data: Any) -> list[str]: + """Check the data against the schema of the file; returns the problems (an empty list means valid). Never raises.""" + problems: list[str] = [] + if not isinstance(data, dict): + return ["the file must hold a JSON object"] + for key in _TOP: + if key not in data: + problems.append("missing top-level key %r" % key) + if problems: + return problems + if data["conformance"] != CONFORMANCE_VERSION or isinstance(data["conformance"], bool): + problems.append("'conformance' must be %d, got %r" % (CONFORMANCE_VERSION, data["conformance"])) + _limits(data["limits"], problems) + _codes(data["host_internal_codes"], problems) + _messages(data["messages"], problems) + _compat(data["compat"], problems) + wrong = data["wrong_science"] + if not isinstance(wrong, dict): + problems.append("'wrong_science' must be an object") + return problems + for name in _SECTIONS: + if not isinstance(wrong.get(name), dict): + problems.append("wrong_science: missing section %r" % name) + if not problems: + _wrong_science(wrong, data["limits"], problems) + return problems + + +def _number(value: Any) -> bool: + return isinstance(value, (int, float)) and not isinstance(value, bool) + + +def _limits(limits: Any, problems: list[str]) -> None: + if not isinstance(limits, dict) or not limits: + problems.append("limits: must be a non-empty object") + return + for name, entry in limits.items(): + if not _IDENTIFIER.match(name): + problems.append("limits: %r is not an identifier" % name) + if not isinstance(entry, dict) or not _number(entry.get("value")): + problems.append("limits.%s: needs a numeric 'value'" % name) + continue + if entry.get("status") not in STATUSES: + problems.append("limits.%s: 'status' must be one of %s" % (name, ", ".join(STATUSES))) + if not isinstance(entry.get("spec"), str): + problems.append("limits.%s: needs a 'spec' (the rule's place in the spec)" % name) + + +def _codes(codes: Any, problems: list[str]) -> None: + if not isinstance(codes, list) or not all(isinstance(c, str) and _IDENTIFIER.match(c) for c in codes): + problems.append("host_internal_codes: must be a list of identifiers") + elif len(set(codes)) != len(codes): + problems.append("host_internal_codes: duplicates") + + +def _messages(messages: Any, problems: list[str]) -> None: + if not isinstance(messages, dict) or not messages: + problems.append("messages: must be a non-empty object") + return + for name, entry in messages.items(): + template = entry.get("template") if isinstance(entry, dict) else None + if not isinstance(template, str) or not template: + problems.append("messages.%s: needs a 'template' text" % name) + continue + try: + fields = [f for _, f, _, _ in string.Formatter().parse(template) if f is not None] + except ValueError as error: + problems.append("messages.%s: bad template (%s)" % (name, error)) + continue + if any(not _IDENTIFIER.match(f) for f in fields): + problems.append( + "messages.%s: placeholders must be {name} identifiers, not positions or expressions" % name + ) + + +def _compat(compat: Any, problems: list[str]) -> None: + if not isinstance(compat, dict): + problems.append("compat: must be an object") + return + if parse_version(compat.get("min_tools_version")) is None: + problems.append("compat.min_tools_version: not a version") + if not isinstance(compat.get("required_features"), list): + problems.append("compat.required_features: must be a list") + cases = compat.get("cases") + if not isinstance(cases, list) or not cases: + problems.append("compat.cases: must be a non-empty list") + else: + for case in cases: + ok = ( + isinstance(case, dict) + and isinstance(case.get("schema"), dict) + and case.get("expect") in (OK, WARN, REFUSE) + ) + if not ok or not isinstance(case.get("case"), str): + problems.append( + "compat.cases: each needs 'case', a 'schema' object and 'expect' ok|warn|refuse (%r)" + % (case,) + ) + elif case["expect"] == REFUSE and not isinstance(case.get("reason_contains", ""), str): + problems.append("compat.cases %r: 'reason_contains' must be text" % case["case"]) + order = compat.get("version_order") + parsed = [parse_version(v) for v in order] if isinstance(order, list) else None + if not parsed or None in parsed or parsed != sorted(parsed): # type: ignore[type-var] + problems.append("compat.version_order: must be versions in increasing order") + + +def _ids(label: str, items: Any, problems: list[str], seen: set[str], pattern_key: str | None = None) -> None: + if not isinstance(items, list) or not items: + problems.append("%s: must be a non-empty list" % label) + return + pattern = _ID_PATTERNS[pattern_key or label.split(".")[-1]] + for item in items: + ident = item.get("id") if isinstance(item, dict) else None + if not isinstance(ident, str) or not re.match(pattern, ident): + problems.append("%s: an id must match %s, got %r" % (label, pattern, ident)) + elif ident in seen: + problems.append("%s: duplicate id %s" % (label, ident)) + else: + seen.add(ident) + + +def _vector(value: Any, length: int) -> bool: + return isinstance(value, list) and len(value) == length and all(_number(v) for v in value) + + +def _wrong_science(wrong: Mapping[str, Any], limits: Mapping[str, Any], problems: list[str]) -> None: + seen: set[str] = set() + coordinate = wrong["coordinate"] + image = coordinate.get("image") + if not isinstance(image, dict) or not _vector(image.get("shape"), 2): + problems.append("wrong_science.coordinate.image: needs a 2-number 'shape'") + rect = coordinate.get("rect") + if not isinstance(rect, list) or len(rect) != 4 or not all(_vector(v, 2) for v in rect): + problems.append("wrong_science.coordinate.rect: four [y, x] vertices") + pixels, values = coordinate.get("rect_mask_pixels"), coordinate.get("rect_mask_values") + if ( + not isinstance(pixels, list) + or not all(_vector(p, 2) for p in pixels) + or not isinstance(values, list) + or len(values) != len(pixels) + ): + problems.append( + "wrong_science.coordinate: rect_mask_pixels and rect_mask_values must be lists of the same length" + ) + _ids("wrong_science.coordinate.cases", coordinate.get("cases"), problems, seen, "coordinate") + calibration = wrong["calibration"] + units = calibration.get("length_units_um") + if not isinstance(units, dict) or not all(_number(v) and v > 0 for v in units.values()): + problems.append("wrong_science.calibration.length_units_um: unit text -> positive um per unit") + _ids("wrong_science.calibration.cases", calibration.get("cases"), problems, seen, "calibration") + for case in calibration.get("cases") or []: + expect = case.get("expect") if isinstance(case, dict) else None + if not isinstance(expect, dict) or not ({"um_per_px", "um_per_px_each"} & set(expect)): + problems.append( + "wrong_science.calibration: %s needs expect.um_per_px or um_per_px_each" % case.get("id") + ) + _ids( + "wrong_science.calibration.transitions", + calibration.get("transitions"), + problems, + seen, + "calibration transitions", + ) + retention = wrong["retention"] + for group in ("fixtures_tools", "fixtures_hosts"): + _ids("wrong_science.retention." + group, retention.get(group), problems, seen, "retention") + for fixture in retention.get(group) or []: + expected = "tools" if group == "fixtures_tools" else "host" + if isinstance(fixture, dict) and fixture.get("scope") != expected: + problems.append( + "wrong_science.retention: %s must have scope %r" % (fixture.get("id"), expected) + ) + markers = retention.get("markers") + if not isinstance(markers, dict) or not {"owner_file", "done_file", "lock_file", "done_statuses"} <= set( + markers + ): + problems.append( + "wrong_science.retention.markers: owner_file, done_file, lock_file and done_statuses are required" + ) + _ids( + "wrong_science.run_identity.fixtures", + wrong["run_identity"].get("fixtures"), + problems, + seen, + "run_identity", + ) + _ids( + "wrong_science.async_lookups.fixtures", + wrong["async_lookups"].get("fixtures"), + problems, + seen, + "async_lookups", + ) + numbers = wrong["number_limits"] + _ids("wrong_science.number_limits.cases", numbers.get("cases"), problems, seen, "number_limits") + for name in numbers.get("limits") or []: + if name not in limits: + problems.append("wrong_science.number_limits: limit %r is not in 'limits'" % (name,)) + + +def main(argv: list[str] | None = None) -> int: + arguments = sys.argv[1:] if argv is None else argv + if arguments == ["--path"]: + print(path()) + return 0 + if arguments == ["--validate"]: + problems = validate(json.loads(path().read_text(encoding="utf-8"))) + print("\n".join(problems) if problems else "%s is valid" % FILE_NAME) + return 1 if problems else 0 + print("usage: python -m labconstrictor_tools.conformance --path | --validate", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/pyproject.toml b/pyproject.toml index 7607ef7..c4843ca 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -25,6 +25,9 @@ version = {attr = "labconstrictor_tools._version.__version__"} [tool.setuptools] packages = ["labconstrictor_tools", "labconstrictor_tools.examples"] +[tool.setuptools.package-data] +labconstrictor_tools = ["conformance.json"] # the cross-host rules and vectors (docs/CONFORMANCE.md) travel with the wheel + [tool.black] line-length = 110 diff --git a/tests/README.md b/tests/README.md index ce7872b..a736d5f 100644 --- a/tests/README.md +++ b/tests/README.md @@ -15,6 +15,7 @@ interpreter that runs them (needs `pip install -e ".[test]"`). | test_regressions_run2 | worker pipes closed, `--pythonpath` precedence, damaged TIFF, nullable/folder/group/advanced/enabled_when in the schema, host cwd not on the worker's path, root prefix refused, CLI results folder pruned (after `_finish_results_dir`) | | test_hardening | NaN/inf rejected, strict-JSON results, registry priority and trust, unsafe app names, skipped-app explanations, message after a forced kill on Cancel | | test_retention | `retention` and `labconstrictor-tools prune`: marker-based ownership, foreign and legacy folders kept, unsafe paths, the lock, dry run, the report, the folders `run` and the run records create | +| test_conformance | `conformance.json`: loader, schema, the data against the code and against plain-Python oracles, packaging | | test_cli_hardening | support bundle never follows symlinks, doctor survives broken apps, unique results folders, duplicate arguments | | test_convert_hardening | dtype narrowing never changes values, invalid defaults refused, strict booleans/choices, enums restored, empty images refused | | test_worker_protocol | the worker fed raw hostile requests: bad ids/inputs, duplicates, sys.exit, unserialisable results, oversized lines | diff --git a/tests/test_conformance.py b/tests/test_conformance.py new file mode 100644 index 0000000..df7ffd7 --- /dev/null +++ b/tests/test_conformance.py @@ -0,0 +1,716 @@ +"""`conformance.json` and its loader (docs/CONFORMANCE.md). + +Three kinds of check: the loader and its schema (a broken edit must fail loudly, never skip a rule); the data against the Tools +code that already implements a rule (limits, compatibility vectors, retention fixtures run through `retention.prune`); and the +data against itself where a rule is arithmetic (the coordinate placement formula, the unit conversions), computed here with plain +Python from the inputs, never by calling the code under test. +""" + +import contextlib +import copy +import io +import json +import re +import shutil +import string +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +import _paths # noqa: F401 (must come first) +from _paths import ROOT + +from labconstrictor_tools import compat, conformance, notebook + +try: + from labconstrictor_tools import retention +except ImportError: # the retention module arrives with its own change; its fixtures run as soon as it exists + retention = None + + +def spec(): + """The spec text, read when a test needs it (the sabotage run copies the package and the tests, not the docs).""" + return (ROOT / "docs" / "SPEC_WRONG_SCIENCE.md").read_text(encoding="utf-8") + + +DATA = conformance.load() +WRONG = DATA["wrong_science"] + + +def broken(change): + """A deep copy of the data after `change(copy)`; the problems `validate` finds in it.""" + data = copy.deepcopy(DATA) + change(data) + return conformance.validate(data) + + +class Loader(unittest.TestCase): + def test_the_file_is_in_the_package_and_valid(self): + self.assertTrue(conformance.path().is_file()) + self.assertEqual(conformance.path().parent, ROOT / "labconstrictor_tools") + self.assertEqual(conformance.validate(DATA), []) + self.assertIs(conformance.load(), conformance.load()) + + def test_the_command_line_gives_the_path_and_validates(self): + env = {"PYTHONPATH": str(ROOT), "PATH": ""} + path = subprocess.run( + [sys.executable, "-m", "labconstrictor_tools.conformance", "--path"], + capture_output=True, + text=True, + env=env, + ) + self.assertEqual(Path(path.stdout.strip()), conformance.path()) + valid = subprocess.run( + [sys.executable, "-m", "labconstrictor_tools.conformance", "--validate"], + capture_output=True, + text=True, + env=env, + ) + self.assertEqual((valid.returncode, valid.stdout.strip()), (0, "conformance.json is valid")) + usage = subprocess.run( + [sys.executable, "-m", "labconstrictor_tools.conformance"], + capture_output=True, + text=True, + env=env, + ) + self.assertEqual(usage.returncode, 2) + + def test_the_data_is_plain_json_that_survives_a_round_trip(self): + text = conformance.path().read_text(encoding="utf-8") + self.assertEqual(json.loads(json.dumps(DATA)), json.loads(text)) + self.assertTrue( + text.isascii(), + "write non-ASCII characters as \\u escapes: the damaged micro sign must stay visible in review", + ) + + def test_a_malformed_file_raises_with_every_problem(self): + original = conformance.path + with tempfile.TemporaryDirectory() as folder: + bad = Path(folder) / "conformance.json" + bad.write_text(json.dumps({"conformance": 1}), encoding="utf-8") + conformance._cache = None + conformance.path = lambda: bad + try: + with self.assertRaises(conformance.ConformanceError) as caught: + conformance.load() + finally: + conformance.path = original + conformance._cache = None + self.assertIn("missing top-level key 'limits'", str(caught.exception)) + self.assertIn("missing top-level key 'wrong_science'", str(caught.exception)) + conformance.load() # and the real one loads again + + +class Schema(unittest.TestCase): + def test_each_kind_of_damage_is_named(self): + cases = { + "not an object": (lambda d: None, None), + "missing key": (lambda d: d.pop("messages"), "missing top-level key 'messages'"), + "wrong version": (lambda d: d.update(conformance=2), "'conformance' must be 1"), + "limit without value": ( + lambda d: d["limits"]["results_kept"].pop("value"), + "limits.results_kept: needs a numeric 'value'", + ), + "limit value is text": ( + lambda d: d["limits"]["results_kept"].update(value="20"), + "limits.results_kept: needs a numeric", + ), + "limit value is a boolean": ( + lambda d: d["limits"]["results_kept"].update(value=True), + "needs a numeric", + ), + "unknown status": ( + lambda d: d["limits"]["results_kept"].update(status="maybe"), + "'status' must be one of decided, default", + ), + "limit without spec": (lambda d: d["limits"]["results_kept"].pop("spec"), "needs a 'spec'"), + "limit name": ( + lambda d: d["limits"].update({"Bad Name": d["limits"]["results_kept"]}), + "not an identifier", + ), + "duplicate code": ( + lambda d: d["host_internal_codes"].append("target_unavailable"), + "host_internal_codes: duplicates", + ), + "code not an identifier": ( + lambda d: d["host_internal_codes"].append("Not A Code"), + "must be a list of identifiers", + ), + "positional placeholder": ( + lambda d: d["messages"]["size_mismatch"].update(template="{0} wide"), + "placeholders must be {name}", + ), + "unbalanced template": ( + lambda d: d["messages"]["size_mismatch"].update(template="{name"), + "bad template", + ), + "empty template": ( + lambda d: d["messages"]["size_mismatch"].update(template=""), + "needs a 'template'", + ), + "compat minimum": ( + lambda d: d["compat"].update(min_tools_version="banana"), + "compat.min_tools_version", + ), + "compat expectation": ( + lambda d: d["compat"]["cases"][0].update(expect="fine"), + "compat.cases: each needs", + ), + "compat order": (lambda d: d["compat"]["version_order"].reverse(), "compat.version_order"), + "duplicate id": ( + lambda d: d["wrong_science"]["coordinate"]["cases"][1].update(id="CF-01"), + "duplicate id CF-01", + ), + "id pattern": ( + lambda d: d["wrong_science"]["calibration"]["cases"][0].update(id="CAL-1"), + "an id must match", + ), + "id in the wrong section": ( + lambda d: d["wrong_science"]["run_identity"]["fixtures"][0].update(id="ASY-01"), + "an id must match", + ), + "missing section": (lambda d: d["wrong_science"].pop("retention"), "missing section 'retention'"), + "bad rect": ( + lambda d: d["wrong_science"]["coordinate"].update(rect=[[0, 0]]), + "four [y, x] vertices", + ), + "mask lengths": ( + lambda d: d["wrong_science"]["coordinate"]["rect_mask_values"].pop(), + "same length", + ), + "calibration without an answer": ( + lambda d: d["wrong_science"]["calibration"]["cases"][0].update(expect={}), + "needs expect.um_per_px", + ), + "unit table": ( + lambda d: d["wrong_science"]["calibration"]["length_units_um"].update(um=0), + "positive um per unit", + ), + "retention scope": ( + lambda d: d["wrong_science"]["retention"]["fixtures_tools"][0].update(scope="host"), + "must have scope 'tools'", + ), + "retention markers": ( + lambda d: d["wrong_science"]["retention"]["markers"].pop("lock_file"), + "markers: owner_file", + ), + "number limit that does not exist": ( + lambda d: d["wrong_science"]["number_limits"].update(limits=["nope"]), + "limit 'nope' is not in 'limits'", + ), + } + for name, (change, expected) in cases.items(): + with self.subTest(name): + if expected is None: + self.assertEqual(conformance.validate([]), ["the file must hold a JSON object"]) + continue + problems = broken(change) + self.assertTrue(any(expected in p for p in problems), "%r not in %r" % (expected, problems)) + + def test_validate_never_raises_on_odd_data(self): + for odd in ( + None, + 3, + "x", + [], + {}, + { + "conformance": 1, + "limits": 5, + "host_internal_codes": 1, + "messages": 1, + "compat": 1, + "wrong_science": 1, + }, + ): + self.assertTrue(conformance.validate(odd), odd) + + +class AgainstTheCode(unittest.TestCase): + def test_limits_match_the_code_and_the_spec(self): + limits = DATA["limits"] + self.assertEqual(limits["form_integer_max"]["value"], notebook.MAX_FORM_INTEGER) + self.assertEqual(limits["form_integer_max"]["value"], 2**53 - 1) + self.assertEqual(limits["unbounded_float"]["value"], 1e15) + if retention is not None: + kept = (retention.RESULTS_KEPT, retention.RUN_RECORDS_KEPT) + else: + from labconstrictor_tools import cli, runs + + kept = (cli.RESULTS_KEPT, runs.KEEP) + self.assertEqual((limits["results_kept"]["value"], limits["run_records_kept"]["value"]), kept) + self.assertIn("`RESULTS_KEPT = 20`, `RUN_RECORDS_KEPT = 50`", spec()) + + def test_every_limit_marked_default_is_marked_so_in_the_spec(self): + for name, entry in DATA["limits"].items(): + if entry["status"] == "default": + self.assertIn( + name.split("_")[0], "lookup", name + ) # only the lookup limits (spec 5.4, working decision 6) + self.assertIn( + "6. Lookup limits: one in flight, 8 pending, 60 s timeout (section 5). (default, not yet confirmed)", + spec(), + ) + + def test_the_host_internal_codes_are_the_ones_the_spec_names(self): + for code in DATA["host_internal_codes"]: + self.assertIn("`%s`" % code, spec()) + self.assertEqual( + sorted(DATA["host_internal_codes"]), + sorted( + re.findall(r"New host-side codes (.*?) go into", spec().replace("\n", " "))[0] + .replace("`", "") + .split(", ") + ), + ) + + def test_the_compat_vectors_give_the_same_answer_as_the_reference(self): + compat_data = DATA["compat"] + for case in compat_data["cases"]: + with self.subTest(case["case"]): + result = compat.check_schema( + case["schema"], compat_data["min_tools_version"], compat_data["required_features"] + ) + self.assertEqual(result.status, case["expect"], result.reason) + self.assertIn(case.get("reason_contains", ""), result.reason) + ordered = [compat.parse_version(v) for v in compat_data["version_order"]] + self.assertEqual(ordered, sorted(ordered)) + + def test_the_message_templates_format(self): + for name, entry in DATA["messages"].items(): + names = {f for _, f, _, _ in string.Formatter().parse(entry["template"]) if f} + entry["template"].format(**{n: "x" for n in names}) + self.assertTrue(entry["spec"], name) + + def test_every_message_template_is_a_sentence_of_the_spec(self): + flat = spec().replace("\\u00B5", "") + (ROOT / "docs" / "HOST_PARITY.md").read_text( + encoding="utf-8" + ) # I06 has the >= form + for name, entry in DATA["messages"].items(): + pieces = re.split(r"\{\w+\}", entry["template"]) + pattern = ".+?".join(re.escape(piece) for piece in pieces) + self.assertTrue(re.search(pattern, flat), "%s: %r is not in the spec" % (name, entry["template"])) + + +def evaluate(point, source): + """Napari data (y, x) -> world, plain Python: scale, then translate, or the 3x3 affine on [y, x, 1].""" + y, x = point + if "affine" in source: + m = source["affine"] + return [m[0][0] * y + m[0][1] * x + m[0][2], m[1][0] * y + m[1][1] * x + m[1][2]] + scale, translate = source.get("scale", [1, 1]), source.get("translate", [0, 0]) + return [y * scale[0] + translate[0], x * scale[1] + translate[1]] + + +def inside(polygon_yx, y, x): + """Even-odd rule for a pixel centre (y, x).""" + hit = False + for (y1, x1), (y2, x2) in zip(polygon_yx, polygon_yx[1:] + polygon_yx[:1]): + if (x1 > x) != (x2 > x) and y < (y2 - y1) * (x - x1) / (x2 - x1) + y1: + hit = not hit + return hit + + +class CoordinateData(unittest.TestCase): + cases = {c["id"]: c for c in WRONG["coordinate"]["cases"]} + + def test_the_base_image_and_the_rectangle_mask(self): + coordinate = WRONG["coordinate"] + rows, columns = coordinate["image"]["shape"] + value = lambda y, x: 100 * y + x # noqa: E731 (the 'value' formula of the file, written out) + self.assertEqual(coordinate["image"]["value"], "100*y + x") + mask = [[y, x] for y in range(rows) for x in range(columns) if inside(coordinate["rect"], y, x)] + self.assertEqual(mask, coordinate["rect_mask_pixels"]) + self.assertEqual([value(y, x) for y, x in mask], coordinate["rect_mask_values"]) + for vy, vx in coordinate["rect"]: + self.assertTrue( + vy % 1 == 0.5 and vx % 1 == 0.5, "vertices are on half pixels: no centre lies on an edge" + ) + + def test_the_world_coordinates_of_the_napari_cases(self): + for ident in ("CF-01", "CF-02", "CF-03", "CF-04", "CF-05", "CF-06"): + case = self.cases[ident] + with self.subTest(ident): + self.assertEqual( + evaluate(case["point"], case["source"]), [float(v) for v in case["expect"]["world"]] + ) + + def test_the_rotated_and_translated_shapes_layer_gives_the_same_mask(self): + # CF-02: vertices_world "rect + [100, 0]"; CF-04: the listed world vertices; both map back to the rect through the source + rect = WRONG["coordinate"]["rect"] + self.assertEqual(self.cases["CF-02"]["shapes_layer"]["vertices_world"], "rect + [100, 0]") + self.assertEqual( + [evaluate(v, self.cases["CF-02"]["source"]) for v in rect], [[y + 100, x] for y, x in rect] + ) + rotated = [evaluate(v, self.cases["CF-04"]["source"]) for v in rect] + self.assertEqual( + rotated, [[float(a), float(b)] for a, b in self.cases["CF-04"]["shapes_layer"]["vertices_world"]] + ) + + def test_the_qupath_crop(self): + case = self.cases["CF-09"] + x0, x1 = case["selection_annotation"]["x"] + y0, y1 = case["selection_annotation"]["y"] + self.assertEqual(case["expect"]["export_offset_yx"], [y0, x0]) + self.assertEqual(case["expect"]["export_shape"], [y1 - y0, x1 - x0]) + offset, shape = case["expect"]["export_offset_yx"], case["expect"]["export_shape"] + self.assertEqual(case["expect"]["export_value_0_0"], 100 * offset[0] + offset[1]) + self.assertEqual( + case["expect"]["export_value_3_5"], 100 * (offset[0] + shape[0] - 1) + offset[1] + shape[1] - 1 + ) + self.assertEqual( + case["result_expect"]["target_px"], [offset[0] + case["point"][0], offset[1] + case["point"][1]] + ) + + def test_the_placement_formula_for_every_downsample_variant(self): + self.assertEqual(WRONG["coordinate"]["placement"]["formula"], "c = x0 + (x + 0.5) * downsample") + for variant in self.cases["CF-14"]["variants"]: + y, x = variant["point"] + (y0, x0), d = variant["offset_yx"], variant["downsample"] + self.assertEqual(variant["expect"]["c_yx"], [y0 + (y + 0.5) * d, x0 + (x + 0.5) * d], variant) + + def test_the_refusals_use_declared_host_internal_codes(self): + mentioned = set( + re.findall(r"refuse (\w+)", json.dumps(self.cases["CF-10"]) + json.dumps(self.cases["CF-13"])) + ) + self.assertEqual( + mentioned, {"selection_plane_mismatch", "selection_mixed_planes", "frame_unsupported_transform"} + ) + self.assertTrue(mentioned <= set(DATA["host_internal_codes"])) + self.assertIn("target_unavailable", DATA["host_internal_codes"]) + + +class CalibrationData(unittest.TestCase): + calibration = WRONG["calibration"] + cases = {c["id"]: c for c in calibration["cases"]} + + def um(self, scale, unit): + """Plain unit conversion from the table of the file; None for unknown.""" + if ( + unit.startswith(self.calibration["damaged_micro_prefix"]) + or unit not in self.calibration["length_units_um"] + ): + return None + return scale * self.calibration["length_units_um"][unit] + + def test_unit_spellings_convert_to_the_listed_answers(self): + self.assertAlmostEqual( + self.um(1, "millimeter") if "millimeter" in self.calibration["length_units_um"] else 1000.0, + 1000.0, + ) + variants = self.cases["CAL-05"]["input"]["variants"] + for variant in variants: + self.assertAlmostEqual(self.um(variant["scale"], variant["unit"]) / 0.325, 1.0, delta=1e-9) + for variant in self.cases["CAL-15"]["input"]["variants"]: + self.assertEqual(self.um(variant["scale"], variant["unit"]), 0.5) + for variant in self.cases["CAL-17"]["input"]["variants"]: + self.assertAlmostEqual(self.um(variant["scale"], variant["unit"]), 0.0001, places=12) + damaged = self.cases["CAL-16"]["input"] + self.assertIsNone(self.um(damaged["scale"], damaged["unit"])) + self.assertIn(damaged["unit"], self.cases["CAL-16"]["expect"]["warning"]) + + def test_the_three_micro_spellings_are_the_ones_of_the_spec(self): + self.assertEqual(self.calibration["micro_sign_spellings"], ["\u00b5", "\u03bc", "u"]) + for prefix in self.calibration["micro_sign_spellings"]: + self.assertEqual(self.calibration["length_units_um"].get(prefix + "m"), 1.0) + self.assertEqual(self.calibration["damaged_micro_prefix"], "\ufffd") + self.assertNotIn("\ufffdm", self.calibration["length_units_um"]) + + def test_anisotropy_rule(self): + tolerance = self.calibration["anisotropy_tolerance"]["relative_to_larger"] + within, beyond = self.cases["CAL-18"]["input"]["variants"] + for variant, expected in ((within, 0.5002), (beyond, "unknown")): + x, y = variant["scale"] + isotropic = abs(x - y) / max(x, y) <= tolerance + self.assertEqual(round((x + y) / 2, 10) if isotropic else "unknown", expected) + + def test_default_looking_resolutions(self): + defaults = self.calibration["default_resolution_dpi"] + for ident, dpi in (("CAL-11", 72), ("CAL-12", 96)): + resolution = self.cases[ident]["input"]["resolution"][0] + self.assertLessEqual(abs(resolution - dpi) / dpi, defaults["tolerance_relative"]) + self.assertEqual(self.cases[ident]["expect"]["um_per_px"], "unknown") + for variant, per_cm in zip(self.cases["CAL-13"]["input"]["variants"], defaults["per_cm"]): + self.assertEqual(variant["resolution"][0], per_cm) + self.assertEqual( + [round(2.54 / 72 * 1e4, 2), round(2.54 / 96 * 1e4, 2)], [352.78, 264.58] + ) # what a wrong reading would give + not_default = self.cases["CAL-14"]["input"]["resolution"][0] + self.assertTrue( + all(abs(not_default - d) / d > defaults["tolerance_relative"] for d in defaults["values"]) + ) + self.assertAlmostEqual(2.54 * 1e4 / not_default, 1.0) # 1 inch = 25400 um + + def test_the_request_shape_is_per_axis_with_x_and_y_in_phase_one(self): + shape = self.calibration["request_shape"] + self.assertEqual(shape["phase_1_axes"], ["x", "y"]) + entry = shape["_calibration"]["pixel_size"] + self.assertEqual(sorted(entry["axes"]), ["x", "y"]) + self.assertEqual(sorted(entry["axes"]["x"]), ["raw", "source", "unit", "value"]) + from labconstrictor_tools.protocol import RESERVED_INPUTS + + self.assertIn(self.calibration["reserved_input"], RESERVED_INPUTS) + + def test_the_transitions_are_the_nine_of_the_spec(self): + self.assertEqual( + [t["id"] for t in self.calibration["transitions"]], ["CAL-T%d" % i for i in range(1, 10)] + ) + self.assertEqual(sorted(self.cases), ["CAL-%02d" % i for i in range(1, 19)]) + + +@unittest.skipIf( + retention is None, "labconstrictor_tools.retention is not in this tree yet; its fixtures run once it is" +) +class RetentionFixturesAgainstTheReference(unittest.TestCase): + """The `fixtures_tools` of the file, built and run for real against `retention.prune` and the CLI.""" + + def build_and_run(self, fixture): + import os + + tmp = Path(tempfile.mkdtemp(prefix="lc_conf_")) + self.addCleanup(shutil.rmtree, tmp, True) + home = tmp / "home" + for root in ("results", "runs"): + (home / root).mkdir(parents=True) + outside = tmp / "outside" + outside.mkdir() + made = { + "owned": {"results": [], "runs": []}, + "foreign": [], + "legacy": [], + "links": [], + "crashed": [], + "copies": [], + } + counter = [0] + + def stamp(i): + return "2026-10-01T08:%02d:%02d.%03dZ" % (i // 60 % 60, i % 60, 7) + + def make(root, name, i, done=True, marker_name=None): + folder = home / root / name + folder.mkdir() + (folder / "data.txt").write_text("keep " + name) + owner = { + "lc_marker": 1, + "kind": "result" if root == "results" else "run", + "name": marker_name or name, + "host": "cli", + "pid": 1, + "created": stamp(i), + "token": "0000000%010d" % i, + } + (folder / ".lc-owner.json").write_text(json.dumps(owner)) + if done: + (folder / ".lc-done.json").write_text(json.dumps({"finished": stamp(i + 1), "status": "ok"})) + return folder + + for step in fixture["setup"]: + ((kind, spec),) = step.items() + if kind == "foreign": + folder = home / spec["root"] / spec["name"] + folder.mkdir() + for file_name, content in spec["files"].items(): + (folder / file_name).write_text(content) + made["foreign"].append((folder, spec["files"])) + elif kind == "owned": + for _ in range(spec["count"]): + counter[0] += 1 + index = 0 if spec.get("oldest") else counter[0] + 10 + name = ( + "20261001T08%04d_%06x_app_tool" % (index, index) + if spec["root"] == "results" + else "%04d_run" % index + ) + folder = make(spec["root"], name, index, done=spec.get("done", True)) + made["owned"][spec["root"]].append(folder) + if not spec.get("done", True): + made["crashed"].append(folder) + elif kind == "legacy": + for i in range(spec["count"]): + folder = home / spec["root"] / ("20260101T0000%02d_abcdef_app_tool" % i) + folder.mkdir() + (folder / "data.txt").write_text("legacy") + made["legacy"].append(folder) + elif kind == "link": + (outside / "data.txt").write_text("outside") + (outside / ".lc-owner.json").write_text( + json.dumps( + { + "lc_marker": 1, + "kind": "result", + "name": spec["name"], + "created": stamp(0), + "token": "0" * 12, + } + ) + ) + (outside / ".lc-done.json").write_text(json.dumps({"finished": stamp(1), "status": "ok"})) + try: + os.symlink(outside, home / spec["root"] / spec["name"], target_is_directory=True) + except (OSError, NotImplementedError): + self.skipTest( + "unsupported: no link can be created here (reported as unsupported, never as a pass)" + ) + made["links"].append(home / spec["root"] / spec["name"]) + elif kind == "owned_copy_renamed": + made["copies"].append( + make(spec["root"], spec["name"], 0, marker_name="20261001T080000_000000_app_other") + ) + elif kind == "equal_created": + for letter in "cba"[: spec["count"]]: + made["owned"][spec["root"]].append( + make(spec["root"], "20261001T080000_%s%s%s%s%s%s_app_tool" % ((letter,) * 6), 5) + ) + elif kind == "lock_held_by_another_process": + holder = subprocess.Popen( + [sys.executable, "-c", "import sys, pathlib\nfrom labconstrictor_tools import retention\nwith retention._retention_lock(pathlib.Path(sys.argv[1])) as h:\n print('held' if h else 'no', flush=True)\n sys.stdin.readline()\n", str(home)], + env={**os.environ, "PYTHONPATH": str(ROOT)}, stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True, + ) # fmt: skip + self.addCleanup(holder.wait, 30) + self.addCleanup(holder.stdin.close) + self.addCleanup(holder.stdout.close) + self.assertEqual(holder.stdout.readline().strip(), "held") + else: + self.fail("the file has a setup step this test does not know: %r" % kind) + action = fixture["action"] + env = {**os.environ, "LC_HOME": str(home), "PYTHONPATH": str(ROOT)} + codes = [] + if "prune_processes_at_once" in action: + procs = [ + subprocess.Popen( + [sys.executable, "-m", "labconstrictor_tools", "prune", "--root", action["root"]], + env=env, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + for _ in range(action["prune_processes_at_once"]) + ] + for proc in procs: + proc.communicate(timeout=120) + codes.append(proc.returncode) + elif "keep" in action: + retention.prune(action["prune"], home=home, keep={action["prune"]: action["keep"]}) + else: + done = subprocess.run( + [sys.executable, "-m", "labconstrictor_tools", "prune", "--root", action["prune"]], + env=env, + capture_output=True, + text=True, + timeout=120, + ) + codes.append(done.returncode) + return home, made, codes + + def test_every_tools_fixture_of_the_file(self): + fixtures = WRONG["retention"]["fixtures_tools"] + self.assertEqual( + sorted(f["id"] for f in fixtures), ["RET-%02d" % i for i in (1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13)] + ) + for fixture in fixtures: + with self.subTest(fixture["id"]): + home, made, codes = self.build_and_run(fixture) + for check in fixture["expect"]: + ((kind, want),) = check.items() + if kind == "foreign_intact": + for folder, files in made["foreign"]: + for name, content in files.items(): + self.assertEqual((folder / name).read_text(), content) + elif kind == "owned_remaining": + root = home / want["root"] + remaining = sorted( + p.name + for p in root.iterdir() + if p in made["owned"][want["root"]] + or p.name in {q.name for q in made["owned"][want["root"]]} + ) + everything = sorted(q.name for q in made["owned"][want["root"]]) + self.assertEqual(len(remaining), want["count"]) + self.assertEqual( + remaining, everything[-want["count"] :] if want["which"] != "all" else everything + ) + for name in remaining: + self.assertEqual((root / name / "data.txt").read_text(), "keep " + name) + elif kind == "link_intact": + self.assertTrue(all(p.is_symlink() for p in made["links"])) + elif kind == "outside_intact": + self.assertEqual((home.parent / "outside" / "data.txt").read_text(), "outside") + self.assertEqual(len(list((home.parent / "outside").iterdir())), 3) + elif kind == "copy_intact": + self.assertTrue(all((p / "data.txt").is_file() for p in made["copies"])) + elif kind == "legacy_intact": + self.assertEqual( + sum((p / "data.txt").read_text() == "legacy" for p in made["legacy"]), want + ) + elif kind == "crashed_survives": + self.assertTrue(all((p / "data.txt").is_file() for p in made["crashed"])) + elif kind == "every_exit_code": + self.assertEqual(set(codes), {want}) + elif kind == "exit_code": + self.assertEqual(codes[0], want) + elif kind == "remaining_names": + names = sorted(p.name for p in (home / "results").iterdir()) + self.assertEqual(names, sorted(q.name for q in made["owned"]["results"])[-2:]) + else: + self.fail("the file has an expectation this test does not know: %r" % kind) + + +class Documented(unittest.TestCase): + def test_every_key_of_the_file_is_in_the_layout_table_of_the_doc(self): + text = (ROOT / "docs" / "CONFORMANCE.md").read_text(encoding="utf-8") + for key in (k for k in DATA if not k.startswith("_")): + self.assertIn("`%s`" % key, text, key) + for key in (k for k in WRONG if not k.startswith("_")): + self.assertIn( + "wrong_science.%s" % key if key not in ("run_identity", "async_lookups") else key, text, key + ) + for name in DATA["limits"]: + self.assertIn(name, text, name) + + +class Packaging(unittest.TestCase): + def test_pyproject_ships_the_file(self): + text = (ROOT / "pyproject.toml").read_text(encoding="utf-8") + section = text.split("[tool.setuptools.package-data]", 1)[1].split("\n[", 1)[0] + self.assertIn('labconstrictor_tools = ["conformance.json"]', section) + + def test_the_built_wheel_contains_the_file_and_the_loader_reads_it_from_there(self): + try: + from setuptools import build_meta + except ImportError: + self.skipTest("setuptools is not installed") + with tempfile.TemporaryDirectory(prefix="lc_wheel_") as folder: + tree = Path(folder) / "src" + shutil.copytree( + ROOT / "labconstrictor_tools", + tree / "labconstrictor_tools", + ignore=shutil.ignore_patterns("__pycache__", "*.egg-info"), + ) + for name in ("pyproject.toml", "README.md", "LICENSE"): + shutil.copy2(ROOT / name, tree / name) + out = Path(folder) / "dist" + out.mkdir() + cwd = Path.cwd() + try: + import os + + os.chdir(tree) + with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()): + wheel = build_meta.build_wheel(str(out)) + except ( + Exception + ) as error: # noqa: BLE001 - an offline or odd build environment is a skip, not a failure of the data + self.skipTest("the wheel could not be built here (%s: %s)" % (type(error).__name__, error)) + finally: + os.chdir(cwd) + import zipfile + + with zipfile.ZipFile(out / wheel) as archive: + self.assertIn("labconstrictor_tools/conformance.json", archive.namelist()) + self.assertEqual(json.loads(archive.read("labconstrictor_tools/conformance.json")), DATA) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_sabotage.py b/tests/test_sabotage.py index 942b2dc..196ea43 100644 --- a/tests/test_sabotage.py +++ b/tests/test_sabotage.py @@ -645,6 +645,70 @@ class Break: "test_retention.CliCreatesOwnedFolders.test_a_cancelled_or_crashed_run_record_says_so" ), ), + # ---- the conformance loader and its schema (docs/CONFORMANCE.md) + Break( + "conformance: a limit with an unknown status is valid", + "conformance.py", + ' if entry.get("status") not in STATUSES:\n', + " if False:\n", + unittest_slice("test_conformance.Schema.test_each_kind_of_damage_is_named"), + ), + Break( + "conformance: a duplicate fixture id is valid", + "conformance.py", + " elif ident in seen:\n", + " elif False:\n", + unittest_slice("test_conformance.Schema.test_each_kind_of_damage_is_named"), + ), + Break( + "conformance: a positional placeholder in a template is valid", + "conformance.py", + " if any(not _IDENTIFIER.match(f) for f in fields):\n", + " if False:\n", + unittest_slice("test_conformance.Schema.test_each_kind_of_damage_is_named"), + ), + Break( + "conformance: version_order may be in any order", + "conformance.py", + " if not parsed or None in parsed or parsed != sorted(parsed):", + " if not parsed or None in parsed:", + unittest_slice("test_conformance.Schema.test_each_kind_of_damage_is_named"), + ), + Break( + "conformance: a missing top-level key is not reported", + "conformance.py", + " if key not in data:\n", + " if False:\n", + unittest_slice("test_conformance.Schema.test_each_kind_of_damage_is_named"), + ), + Break( + "conformance: a rectangle with the wrong number of vertices is valid", + "conformance.py", + "if not isinstance(rect, list) or len(rect) != 4 or", + "if not isinstance(rect, list) or", + unittest_slice("test_conformance.Schema.test_each_kind_of_damage_is_named"), + ), + Break( + "conformance: load() does not raise for a malformed file", + "conformance.py", + " if problems:\n raise ConformanceError(", + " if False:\n raise ConformanceError(", + unittest_slice("test_conformance.Loader.test_a_malformed_file_raises_with_every_problem"), + ), + Break( + "conformance: load() reads and validates the file on every call", + "conformance.py", + " if _cache is None:\n", + " if True:\n", + unittest_slice("test_conformance.Loader.test_the_file_is_in_the_package_and_valid"), + ), + Break( + "conformance: the command line prints the wrong file", + "conformance.py", + " print(path())\n", + " print(path().parent)\n", + unittest_slice("test_conformance.Loader.test_the_command_line_gives_the_path_and_validates"), + ), ]