Repository navigation
feat(db): Alembic migrations and registry UNIQUE constraints (#84, #81) - #87
Merged
Merged
Conversation
#84: add alembic.ini and alembic/ with a baseline (0001) matching the fallback bootstrap schema, using IF NOT EXISTS so existing databases adopt it. bootstrap() now finds alembic.ini and runs upgrade head on start. The version table lives in the uploads schema. #81: migration 0002 adds UNIQUE (file_hash) and UNIQUE (file_name) to csv_files, the constraints dynamic_loader's race handling relied on but which never existed. If duplicates already exist, the upgrade stops and lists them instead of deleting anything. The fallback schema gains the same constraints. Loaders: a dynamic upload that loses the race now returns duplicate_file (previously a generic DB error), and a T&E load that loses it returns a structured error and rolls back its rows instead of raising a 500. Tests (tests/test_migrations.py, integration): upgrade/downgrade on a scratch schema, adoption of a fallback-created schema, refusal on existing duplicates, DB-level uniqueness, and deterministic lost-race tests for both loaders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019YPoXzBSj89ZHgw7nfTmbB
This was referenced Oct 26, 2026
amar-python
pushed a commit
that referenced
this pull request
Oct 26, 2026
Brings in #87, #90 and #86. No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019YPoXzBSj89ZHgw7nfTmbB
This was referenced Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #84 and #81. Part of ART Milestone 1 (BR-23, BR-28).
#84: Alembic migrations
alembic.iniandalembic/.api/db.pybootstrap()already ranalembic upgrade headwheneveralembic.iniexisted. It never did, so every start took the fallbackCREATE TABLE IF NOT EXISTSpath. It now migrates on startup.IF NOT EXISTS. Databases created by the old fallback adopt it with no change.#81: registry uniqueness
UNIQUE (file_hash)andUNIQUE (file_name)oncsv_files.dynamic_loader.pyrelied on these to settle concurrent identical uploads, but they never existed. The fallback schema gets the same constraints.duplicate_file. Before, it surfaced as a generic DB error from the concurrentCREATE TABLE.UNIQUE (file_name)matches today's semantics. ART decision 2 (versioned datasets) revisits it in M5.Evidence
Local PostgreSQL 16, replicating the CI integration job step for step:
scripts/test_report.py --strictevals/runner.py --tiers p,i,s,epytest tests/test_migrations.pyscripts/lint.sh(now also coveringapi/db.py, the migrations and the new test)The lost-race tests are deterministic, not timing-based. A competing transaction holds an uncommitted registry row, the loader blocks on the UNIQUE index, then the competitor commits.
Deployment note: after this merges, the next API start creates
csv_uploads.alembic_versionand adds the two constraints. On your localte_mgmt_dev, if startup fails with "Cannot add UNIQUE", the error lists the duplicate rows to remove.Generated by Claude Code