Skip to content

feat(db): persist each import's row counts (#89) - #90

Merged
amar-python merged 1 commit into
mainfrom
feat/persist-import-counts
Oct 26, 2026
Merged

amar-python merged 1 commit into
mainfrom
feat/persist-import-counts

Conversation

@amar-python

Copy link
Copy Markdown
Owner

Fixes #89. This is the last ART Milestone 1 item (BR-34, part).

Stacked on #87. It needs that PR's Alembic setup, so its base is fix/issue-81-84-alembic-registry-unique. Merge #87 first. GitHub then retargets this PR to main.

What changes

  • Migration 0003_import_counts. Adds total_rows, duplicate_rows_skipped and failed_rows to csv_files. row_count already stores the inserted count. Downgrade drops the three columns.
  • Rows registered before 0003 keep NULL. Their counts were never recorded, and NULL says so instead of claiming zero.
  • Both loaders store the counts on registration. For T&E mode, duplicate_rows_skipped is 0, which matches the response.
  • GET /api/csv/files returns the counts. The response only gains fields, so existing clients are unaffected.
  • Fallback bootstrap DDL in api/db.py. Mirrors the new columns.
  • test_migrations. Now expects head 0003.

Out of scope: reconciling edited and excluded rows. That needs the staged import and is planned for M6.

Evidence

Run on local PostgreSQL 16, using the replica of the CI integration job.

Check Result
scripts/test_report.py --strict 249 passed, 0 failed, 0 skipped (247 on #87, plus 2 new)
evals/runner.py p,i,s,e 26/26
tests/test_import_counts.py + tests/test_migrations.py 10 passed
Same tests with the loader changes reverted both count tests fail
scripts/lint.sh clean (new files added to the gate)

Generated by Claude Code

… part)

Migration 0003 adds total_rows, duplicate_rows_skipped and failed_rows to
csv_files (row_count already holds the inserted count). Both loaders write
them at registration and GET /api/csv/files returns them. Rows registered
before 0003 keep NULL: their counts were never recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019YPoXzBSj89ZHgw7nfTmbB
@amar-python amar-python added this to the ART extension milestone Oct 26, 2026 — with Claude
@amar-python amar-python added enhancement New feature or request art ART framework extension work labels Oct 26, 2026 — with Claude
@amar-python
amar-python changed the base branch from fix/issue-81-84-alembic-registry-unique to main October 26, 2026 06:36
@amar-python
amar-python merged commit de8e7b6 into main Oct 26, 2026
3 checks passed
amar-python pushed a commit that referenced this pull request Oct 26, 2026
Brings in #87, #90 and #86. No conflicts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019YPoXzBSj89ZHgw7nfTmbB
@amar-python
amar-python deleted the feat/persist-import-counts branch October 27, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

art ART framework extension work enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Persist each import's row counts (BR-34 part, ART M1)

2 participants