Fix: error codes and validation (F1 F5 F6 F11 F12 F15 F16 F20)

docs/MANIFEST.md

@@ -57,4 +57,6 @@ A tool declares itself with Python types and a few markers; hosts (Napari, Fiji,
 `ToolError("code", "message")` gives the person a readable message (`no_result` and `no_match` are shown as notices, not failures); `progress(fraction, "text")` updates the progress bar; `check_cancel()` lets Cancel stop the tool between steps.
 
 ## Adding something to the manifest
+Misuse of a hint is refused when the app is registered (`DeclarationError`, a message naming the parameter and the fix): `Min`/`Max` on anything but an int or float, a bound that is not a finite number, `Min` above `Max`, `PixelSizeOf` on anything but a float, `Axes` on anything but an Image/Labels input or an image/labels output, the output markers `Name`, `Replace` and `ApplyTo` on an input, and two outputs given the same `Name`.
+
 A feature joins one family, follows the naming pattern (`XxxOf("param")` for links, `XxxOut` for outputs), and is not merged until it has: a row here, a recipe row in `AUTHORING.md`, a line in `check`, a use in the example app, a clear declaration error for misuse, and a test in every host (or a documented fallback in `HOST_FEATURES.md`).

docs/PROTOCOL.md

@@ -17,6 +17,12 @@ Value rules (enforced by the worker, each violation is a `FAILURE` with code `in
 * A `choice` declared with an `Enum` annotation reaches the tool as the Enum member.
 * An image/labels input with no pixels is refused (`empty_image`), from a file or from shared memory.
 * Table inputs (a CSV file, UTF-8, first line is the header) reach the tool as a DataFrame exactly as written: floating point cells are read with every digit (`float_precision="round_trip"`, about twice as slow as pandas' default on very large tables); column names are kept as written, duplicates (`a,a`) and empty names included (pandas' renaming to `a.1` / `Unnamed: 0` is switched off, so a tool must address such columns by position); an empty cell is missing (NaN); the spellings `NaN`, `nan`, `NAN` mean a missing number (a column of numbers, `NaN` and empty cells, or of only `NaN` and empty cells, is a float column with NaN), and the other words pandas calls missing (`NA`, `N/A`, `NULL`, `None`, `null`, ...) stay text, except in a column whose other cells are all numbers, where they are missing numbers; a text column that happens to contain `nan` keeps it as text. A table file that does not exist is refused with `file_not_found`; one that is empty or not a UTF-8 CSV with `unreadable_table`; both messages name the file.
+* `integer` and `float` values must be JSON numbers: text (even "3"), lists, objects, NaN and infinity give `invalid_parameter` (`'N' must be an integer, got text`, `'N' must be a number, got a list`, `'N' must be a finite number`); an integer-valued float such as `3.0` is accepted for an `integer`. A number beyond the float range is refused for a `float`.
+* `file`, `folder` and `image`/`labels` values must be text paths: anything else gives `invalid_parameter` (`'N' must be a path (text), got a list: the host must send the file or folder path as text`).
+* A shared-memory image descriptor must be `{"appose_type": "ndarray", "shm": {"name"}, "shape", "dtype"}` and the block must still exist and be large enough, with a numeric dtype; a stale or malformed one gives `unreadable_image` (`the shared-memory image for 'N' is gone or malformed (...): the host must keep the block alive until the task ends and send {...}`).
+* What a tool returns must match its declaration, otherwise the task fails with `bad_return` and a message of the form `the <type> output '<name>' must be <what>, got <what arrived>` (an array of numbers for image/labels, a DataFrame / dict of lists / list of dicts for table and points, a dict for values, a path for file, text for message, a 3x3 matrix of numbers for affine). A tool with several outputs must return a tuple with one value per output (`tool returned N values, declared M`). An affine with NaN or infinity is refused (`bad_return`).
+* Table and points outputs are written as UTF-8 CSV. A NUL character (`\\0`) in a column name or a text cell is refused with `bad_return` (`the table output 'X' has a NUL character in a cell of column 'c': remove it`), identically on every Python version; a lone surrogate or other character that is not UTF-8 is refused with `bad_return` (`... cannot be written as a UTF-8 CSV ...`). Line breaks, quotes and commas in cells and names are quoted and arrive unchanged.
+* Image outputs: an empty array is `bad_return` (`image is empty (shape ...)`); complex, object, text and bytes arrays are `unsupported_dtype`; an integer image beyond 16 bits that float32 cannot hold exactly (more than 24 significant bits, for example a uint64 2**64-1) is refused with `unsupported_dtype` on every machine, never stored rounded or kept as 64-bit.
 * Image outputs: integers are never changed in value (wide integers that fit 16 bits are stored as 16-bit; others must be exact as 32-bit float or the result fails with `unsupported_dtype`); floating point values are stored as float32 (about 7 significant digits); float values outside the float32 range fail.
 
 Input `type`: `string integer float boolean choice image labels table file folder`.

docs/REGRESSION_LEDGER.md

@@ -17,6 +17,13 @@ How to add one: [TESTING.md](TESTING.md#the-ledger).
 | 2026-10-08 | F3: duplicate and empty column names of a Table input are renamed (`a.1`, `Unnamed: 0`) | `pd.read_csv` header mangling | this PR | `tests/test_table_input.py::TableInput.test_duplicate_and_empty_names_are_kept`; also `test_roundtrip_matrix.py::Matrix.test_worker_protocol` |
 | 2026-10-08 | F4: a missing, empty or binary table file fails with `FileNotFoundError` / `EmptyDataError` / `UnicodeDecodeError` | no `file_not_found` / unreadable check for tables as for images | this PR | `tests/test_table_input.py::TableInput.test_unreadable_files_are_tool_errors`; also `test_roundtrip_matrix.py::Matrix.test_worker_protocol` |
 | 2026-10-08 | F7: table text cells such as `NA`, `null`, `None` arrive as NaN | pandas default NA values | this PR | `tests/test_table_input.py::TableInput.test_na_words_stay_text_unless_numeric`; also `test_roundtrip_matrix.py::Matrix.test_worker_protocol` |
+| 2026-10-08 | F1: a number parameter given text, a list, NaN or infinity fails with `[ValueError]`/`[TypeError]`/`[OverflowError]` instead of `invalid_parameter` | `convert._load_one` calls `int(value)` / `float(value)` before it checks anything | this PR | `test_error_codes.py::NumbersMustBeNumbers.test_integer_given_something_else_is_invalid_parameter_naming_what_arrived`, `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`echo_int/word` and 12 more), `test_roundtrip_properties.py::Properties.test_only_toolerror_integer`, `test_roundtrip_properties.py::Properties.test_only_toolerror_float` |
+| 2026-10-08 | F5: a tool that returns the wrong kind of value for an output gets a raw Python exception code | `build_results` and its writers trust the shape and type of what a tool returns | this PR | `test_error_codes.py::WrongReturns.test_each_kind_of_wrong_value_names_the_output_and_what_was_expected`, `test_roundtrip_failures.py::WhatAToolReturns.test_the_wrong_type_is_reported_with_a_tool_error_code` |
+| 2026-10-08 | F6: an Affine output with NaN or infinity is delivered as null | `_as_3x3` does not check finiteness (points and shapes do) | this PR | `test_error_codes.py::WrongReturns.test_affine_with_nan_or_infinity_is_refused`, `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`echo_affine/invalid/nan-cell`) |
+| 2026-10-08 | F11, F12: a malformed shared-memory descriptor, or a non-text file/folder value, raises KeyError / FileNotFoundError / TypeError | no validation before `SharedMemory(...)` and `Path(...)` | this PR | `test_error_codes.py::SharedMemoryDescriptors.test_a_block_that_is_gone_or_a_descriptor_with_missing_or_wrong_fields`, `test_error_codes.py::PathsMustBeText.test_file_folder_and_image_given_a_non_text_value`, `test_roundtrip_properties.py::Properties.test_only_toolerror_image_shared_memory`, `test_roundtrip_properties.py::Properties.test_only_toolerror_file_value` |
+| 2026-10-08 | F15: an empty, complex, object or text array is written as an image result (or crashes with KeyError) | `portable_dtype` and `_write_one` do not refuse what no host can open | this PR | `test_error_codes.py::ImagesNoHostCanOpen.test_complex_object_and_text_arrays_are_unsupported_dtype`, `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`make_array/*`) |
+| 2026-10-08 | F16: hints on the wrong kind of parameter, `Min(5), Max(1)`, output markers on inputs and duplicate output names are accepted and ignored | `introspection` validates only some misuses | this PR | `test_error_codes.py::HintMisuse.test_bounds_on_text_and_an_empty_range`, `test_error_codes.py::HintMisuse.test_duplicate_output_names`, `test_roundtrip_hints.py::InvalidCombinations.test_every_invalid_combination_is_refused_with_a_message_that_names_the_culprit` |
+| 2026-10-08 | F20: on ARM (Apple silicon) a uint64 image holding 2**64-1 is stored as float32 2**64 instead of being refused | `portable_dtype` checks exactness by casting the float32 back to uint64, an out-of-range cast that is undefined: x86 gives 0 (refused), ARM saturates (accepted) | this PR | `test_error_codes.py::Float32Exactness.test_uint64_beyond_float32_is_refused_with_unsupported_dtype_on_every_cpu` (numpy warns about the out-of-range cast on x86, the test turns that into a failure), `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`echo_image/uint64/beyond-float32`; found by the macOS job of CI) |
 
 ## Found by the round-trip matrix, fix PRs pending
 
@@ -26,18 +33,11 @@ a finding deletes its entries there and moves its row up to the table above.
 
 | Date | Symptom | Root cause | Fix PR | Guarding test |
 |---|---|---|---|---|
-| 2026-10-08 | F1: a number parameter given text, a list, NaN or infinity fails with `[ValueError]`/`[TypeError]`/`[OverflowError]` instead of `invalid_parameter` | `convert._load_one` calls `int(value)` / `float(value)` before it checks anything | pending | `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`echo_int/word` and 12 more), `test_roundtrip_properties.py::Properties.test_only_toolerror_integer`, `test_roundtrip_properties.py::Properties.test_only_toolerror_float` |
-| 2026-10-08 | F5: a tool that returns the wrong kind of value for an output gets a raw Python exception code | `build_results` and its writers trust the shape and type of what a tool returns | pending | `test_roundtrip_failures.py::WhatAToolReturns.test_the_wrong_type_is_reported_with_a_tool_error_code` |
-| 2026-10-08 | F6: an Affine output with NaN or infinity is delivered as null | `_as_3x3` does not check finiteness (points and shapes do) | pending | `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`echo_affine/invalid/nan-cell`) |
 | 2026-10-08 | F8, F9: the notebook form clamps numbers beyond 10**9 and cannot leave an optional parameter unset (the tool receives 0, empty text, False, the first choice) | `BoundedIntText` / `Text` / `Dropdown` controls always hold a value | this PR | `test_roundtrip_matrix.py::Matrix.test_notebook_form` (`echo_optional_*/omitted`, `echo_int/ok/2147483648`), `test_notebook_unset_and_range.py` |
 | 2026-10-08 | F10: a copied Windows command line changes a text with backslashes before a quote | `command.quote(windows=True)` does not double backslashes in front of a quote or at the end of the text | this PR: `command._quote_windows` follows the C runtime rules; vectors shared with the Groovy hosts in `tests/quote_vectors.json` | `test_quote_vectors.py::QuoteVectors.test_quote_gives_the_expected_text_for_both_shells`, `test_roundtrip_command.py::QuotingHelper.test_windows_quoting_keeps_every_text_one_argument`, `test_roundtrip_command.py::QuotingHelper.test_windows_quoting_survives_the_real_process_start` (the Windows jobs of CI show it on a real process start) |
-| 2026-10-08 | F11, F12: a malformed shared-memory descriptor, or a non-text file/folder value, raises KeyError / FileNotFoundError / TypeError | no validation before `SharedMemory(...)` and `Path(...)` | pending | `test_roundtrip_properties.py::Properties.test_only_toolerror_image_shared_memory`, `test_roundtrip_properties.py::Properties.test_only_toolerror_file_value` |
 | 2026-10-08 | F13: the worker destroys the host's shared-memory block when it exits (and says so in crash messages) | Python's resource tracker unlinks every block the worker attached to | this PR: `convert._attach_shared_memory` attaches without tracking (`track=False` on Python 3.13+, unregister after attaching before) | `test_roundtrip_shared_memory.py::SharedMemoryOwnership.test_the_block_and_the_stderr_are_left_alone`, `test_lifecycle_fixes.py::SharedMemoryAttach.test_a_process_that_only_attached_leaves_the_block_and_its_stderr_alone` |
 | 2026-10-08 | F21: on Windows `labconstrictor-tools register` sometimes fails with `PermissionError [WinError 5]` from `os.replace` (seen once on CI, Windows 3.12) | `registry._write_atomic` replaces the file once; Windows refuses while another process (scanner, indexer) briefly holds the freshly written temporary file | this PR: `registry._replace` retries `os.replace` on Windows (`REPLACE_ATTEMPTS`, `REPLACE_PAUSE_S`, each retry logged, then an error that says why); the test harness no longer retries | `test_lifecycle_fixes.py::ReplaceRetry` (simulated refusals), `test_roundtrip_matrix.py::Matrix.test_command_line` (intermittent on Windows before) |
 | 2026-10-08 | F14: `labels_to_shapes` drops single-pixel objects with the default settings | Douglas-Peucker with 0.5 px collapses a one-pixel diamond to fewer than 4 points, which is then skipped | this PR | `test_roundtrip_geometry.py::Outlines.test_default_simplification_never_drops_a_label`, `test_shapes_every_label.py` |
-| 2026-10-08 | F15: an empty, complex, object or text array is written as an image result (or crashes with KeyError) | `portable_dtype` and `_write_one` do not refuse what no host can open | pending | `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`make_array/*`) |
-| 2026-10-08 | F16: hints on the wrong kind of parameter, `Min(5), Max(1)`, output markers on inputs and duplicate output names are accepted and ignored | `introspection` validates only some misuses | pending | `test_roundtrip_hints.py::InvalidCombinations.test_every_invalid_combination_is_refused_with_a_message_that_names_the_culprit` |
 | 2026-10-08 | F17: `WorkerProcess.task()` on a dead worker raises BrokenPipeError | `_send` writes to a closed pipe and the task is already registered | this PR: `WorkerProcess.task` ends the task as CRASHED with the worker's last words and does not keep it | `test_roundtrip_failures.py::Crashes.test_asking_a_dead_worker_for_a_task_gives_a_crashed_task`, `test_lifecycle_fixes.py::DeadWorker` |
 | 2026-10-08 | F18: POSIX: the processes a tool started survive the worker when the host goes away | the worker leaves without killing its own process group (Windows has the job object) | this PR: `Worker._end_process_group` makes the worker lead its own process group at start (`_lead_own_process_group`: Java hosts cannot start it in one) and kills that group when it leaves (stdin closed) or gives up on an orphaned tool | `test_roundtrip_failures.py::HangsAndCancel.test_children_die_when_the_host_goes_away`, `test_lifecycle_fixes.py::ProcessGroup` |
 | 2026-10-08 | F19: `progress(0.29)` is shown as 28 % | `int(100 * fraction)` truncates | this PR: `worker._whole_percent` rounds half up | `test_roundtrip_failures.py::WhatAToolReturns.test_every_whole_percent_arrives_as_itself`, `test_lifecycle_fixes.py::Progress` |
-| 2026-10-08 | F20: on ARM (Apple silicon) a uint64 image holding 2**64-1 is stored as float32 2**64 instead of being refused | `portable_dtype` checks exactness by casting the float32 back to uint64, an out-of-range cast that is undefined: x86 gives 0 (refused), ARM saturates (accepted) | pending | `test_roundtrip_matrix.py::Matrix.test_worker_protocol` (`echo_image/uint64/beyond-float32`; found by the macOS job of CI) |