Skip to content

cowork: schema command error-path hardening + honest sample footer - #49

Draft
Coding-Dev-Tools wants to merge 8 commits into
masterfrom
cowork/improve-datamorph
Draft

cowork: schema command error-path hardening + honest sample footer#49
Coding-Dev-Tools wants to merge 8 commits into
masterfrom
cowork/improve-datamorph

Conversation

@Coding-Dev-Tools

Copy link
Copy Markdown
Owner

Harden the schema subcommand (silent-failure class): unsupported --format and malformed input now exit cleanly with a red ERROR message instead of an unhandled traceback. Fix misleading 'Inferred from N+ rows' footer to 'up to N rows' (honest claim). Adds 3 CLI error-path tests (144 total pass, ruff clean). Branch also carries a merge of origin/master resolving the pyproject self-referential extra conflict in favor of the corrected datamorph-cli name.

cowork-bot and others added 8 commits June 13, 2026 06:03
Three correctness fixes in converters.py:

1. CsvWriter extrasaction='raise' → 'ignore': JSON/YAML→CSV conversions
   with heterogeneous rows (later rows having extra keys not seen in the
   first row) raised ValueError and aborted the conversion. extrasaction=
   'ignore' silently drops extra fields, which is correct CSV behaviour —
   the output schema is fixed at the first row.

2. Remove redundant CSV pre-peek in convert(): the code opened a fresh
   read_stream() to peek the first row and call set_field_order(), then
   opened a second read_stream() for the actual conversion — reading the
   input file twice. CsvWriter.write_stream() already falls back to
   list(row.keys()) when _field_order is unset, so the pre-peek was
   entirely redundant. Removed; one read pass now suffices.

3. Populate ConversionResult.rows_read: the field was declared but never
   set (always 0). Set to rows_written after a successful conversion so
   callers can trust the result object.

5 new regression tests; 119/119 green.
…morph

# Conflicts:
#	.github/workflows/pages.yml
#	.github/workflows/publish.yml
#	.gitignore
#	LICENSE
#	README.md
#	package.json
#	pyproject.toml
#	src/datamorph/__init__.py
#	src/datamorph/cli.py
#	src/datamorph/converters.py
#	tests/test_converters.py
#	tests/test_validate.py
- Add parquet, avro, protobuf, and all optional dependency groups to pyproject.toml
- Wrap optional dependency imports in try/except with user-friendly error messages
  pointing to the correct pip install command
- All 141 tests pass, ruff clean
… bad merge

An earlier merge on cowork/improve-datamorph inadvertently dropped 14 files
that exist on master (ci.yml, auto-code-review.yml, conftest.py, CHANGELOG,
CONTRIBUTING, SECURITY, issue/PR templates, dependabot, FUNDING, cli.js,
.gitattributes). Restored them verbatim from origin/master so this branch's
PR diff contains only the intended improvements (converters.py graceful
optional-import errors + pyproject optional-dependency groups + auto-pr wf).
…ted format and malformed input, no tracebacks); fix misleading 'Inferred from N+ rows' footer to honest sample cap; +3 CLI error-path tests
@github-actions

Copy link
Copy Markdown

🤖 Automated Code Review

✅ Ruff Lint — No issues

⚠️ Ruff Format — Formatting needed

unformatted: File would be reformatted
  --> README.md:28:1
   |
27 | import pandas as pd
   - df = pd.read_csv('huge_file.csv')  # OOM on 5GB+
   - df.to_parquet('output.parquet')
28 +
29 + df = pd.read_csv("huge_file.csv")  # OOM on 5GB+
30 + df.to_parquet("output.parquet")
31 | ```
   |

unformatted: File would be reformatted
 --> conftest.py:2:1
  |
1 | """pytest configuration — add project src to Python path and skip rate limits."""
2 +
3 | import os
  |

unformatted: File would be reformatted
   --> src/datamorph/cli.py:103:15
    |
102 | @click.option("--pattern", default="*", help="File glob pattern (default: all files)")
    - @click.option(
    -     "--recursive", "-r", is_flag=True, help="Search subdirectories recursively"
    - )
103 + @click.option("--recursive", "-r", is_flag=True, help="Search subdirectories recursively")
104 | @click.option("--csv-delimiter", default=",", help="CSV delimiter")
--------------------------------------------------------------------------------

✅ Secret Detection — Clean

✅ Large Files — Within limits

📊 Diff Stats — 2 file(s) changed

 src/datamorph/cli.py          | 16 +++++++++++++---
 tests/test_cli_error_paths.py | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 3 deletions(-)

Verdict: ⚠️ Warnings — Lint/format issues found. Recommend fixing before merge.

Automated by Coding-Dev-Tools/.github reusable workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants