Skip to content

Fix: table inputs read floats exactly, keep names as written, NA words stay text, readable errors (F2 F3 F4 F7) - #45

Merged
guijacquemet merged 5 commits into
mainfrom
claude/fix-tables
Oct 8, 2026
Merged

guijacquemet merged 5 commits into
mainfrom
claude/fix-tables

Conversation

@guijacquemet

Copy link
Copy Markdown
Contributor

Fixes the tables theme of the round-trip findings; their skipped matrix cases and properties now run as normal tests and the findings are removed from tests/roundtrip_known_failures.py. The table output writer is untouched. Code: new _read_table in labconstrictor_tools/convert.py (constants PANDAS_NA_WORDS, NAN_SPELLINGS, EMPTY_CELL).

Before -> after

  • F2: floats are read with float_precision="round_trip": 0.30000000000000004 arrives as itself (was 0.3). Cost: about 2x on very large tables (1M rows x 10 float columns: 5.7 s against 2.7 s); documented in PROTOCOL.md.
  • F3: duplicate and empty column names arrive as written (a,a -> ["a","a"], ,b -> ["","b"]; before a.1 and Unnamed: 0). The header row is read separately, the body by position and the names put back; a tool addresses such columns by position (documented).
  • F4: a missing table file is file_not_found naming the file; an empty or non-UTF-8 file is unreadable_table with an instruction to save a UTF-8 CSV with a header row (traceback logged). Before: raw FileNotFoundError, EmptyDataError, UnicodeDecodeError.
  • F7: empty cells are missing; words such as NA, N/A, null, None stay text (a gene called NA, a status None). In a column whose other cells are all numbers they become NaN (a second narrow read). NaN/nan/NAN always mean a missing number: a column of only those spellings (plus empty cells) is float64 NaN, as before. I found and had fixed a regression here before opening this PR: an all-NaN column (what Fiji's ResultsTable, QuPath and numpy/pandas write for an unavailable measurement) had arrived as text. inf/-inf/Infinity columns still come out as floats.

Tests: new tests/test_table_input.py (floats, names, errors, NA words, numeric column with NA, header-only table, all-NaN columns in three spellings, all-NaN plus empty, NaN with numbers, a text column holding nan, an inf column). One existing-test edit: test_roundtrip_known_failures.KnownFailuresDoNotRot.test_the_skip_is_exactly_the_list asserted that an F4 fixture id is a known failure; it now asserts it is not (other fix PRs touch the same line: expect a trivial conflict). Docs: PROTOCOL.md table bullet (states the NaN rule in one sentence), AUTHORING.md, REGRESSION_LEDGER.md (rows moved to Fixed; replace "this PR" with this number).

Host follow-ups: none for reading (hosts only write CSV and send the path); a host writing a table must write UTF-8 with a header row, empty cells for missing, and a text cell such as "NA" is now kept as text; hosts showing unreadable_table or file_not_found should treat them like the image codes.

Verified by the author on Linux, Python 3.11: black, ruff, mypy; the full suite before the last change (only the two environmental sandbox failures; run in groups because a single discover run was killed twice in the sandbox) and, after it, 103 table/properties/known-failure/ledger/unused-code tests and the sabotage test. I re-ran lint and the table tests and checked an all-NaN column (float64), an all-NaN-plus-empty column (float64) and a text column with NA/null (text). Not verified: the full suite and the coverage floor after the last change (CI runs both), Windows, macOS.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN


Generated by Claude Code

claude added 5 commits October 8, 2026 12:41
…dable errors (F2, F3, F4, F7)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
… float)

No existing test edited in this commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
…ns with missing cells are handled like pandas 3

No existing test edited.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2SN8SuEakTdiWy3F2WUqN
@guijacquemet
guijacquemet merged commit c672aca into main Oct 8, 2026
14 checks passed

Copy link
Copy Markdown
Contributor Author

Closed: included in #47 (merged with a merge commit, so these commits are in main).


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants