Skip to content

feat(sqlite): generate migrations (#51, fuzzed) - #121

Merged
Makisuo merged 25 commits into
feat/sqlite-ddlfrom
feat/sqlite-generate
Oct 25, 2026
Merged

Makisuo merged 25 commits into
feat/sqlite-ddlfrom
feat/sqlite-generate

Conversation

@Makisuo

@Makisuo Makisuo commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

This is #51 by @tao-io, with review fixes and a migration fuzzer on top. Their commits are unchanged. GitHub doesn't let maintainers push to an organization-owned fork, so the fixes couldn't go onto #51 directly. It's stacked on #120 (DDL), which is stacked on #119 (query dialects). Refs #48.

What #51 adds

See #51: effect-orm generate for dialect: "sqlite", diffSqliteSchemas (ALTER TABLE where SQLite allows it, otherwise a table rebuild), rename hints, drizzle-kit 1.0 SQLite snapshot import, and /migrate refusing SQLite folders.

What this adds

A migration fuzzer (tests/fuzz/migrations.fuzz.test.ts). It generates two random schemas over shared table and column names, covering new, dropped and changed tables, columns, types, nullability, literal and expression defaults, keys, UNIQUE columns and indexes. Then it:

  1. seeds rows that fit both schemas;
  2. runs the generated migration inside a transaction, as D1 does;
  3. checks that the PRAGMA shape equals a fresh create of the new schema, and that every row survives with its values (or the new column's default).

20,000 runs pass. It found two bugs within 100 runs:

Change Before Now
Rebuild that keeps no column ({c2} → {c0}) no INSERT, so every row was deleted, though only the column data was confirmed lost copies rowid; rows keep, new columns take defaults
Replace every column ({c1, c2} → {c3}) dropped both columns first; SQLite can't drop a table's last column, so the migration failed column adds run before column drops

Also, from review:

  • A column made NOT NULL with a default no longer fails the rebuild over existing NULLs. INSERT … SELECT passes an explicit NULL, which doesn't take the default, so the copy is now COALESCE(col, default).
  • docs/migrations.md now has three additions:
    • a warning that under D1, or any transaction, a rebuild's DROP TABLE fires ON DELETE CASCADE in tables that reference the rebuilt one (PRAGMA foreign_keys=OFF does nothing there; reproduced in review);
    • a rebuild renumbers rowids;
    • ADD COLUMN appends, so column order can differ from a fresh create.

The three counterexamples are pinned in tests/fuzz/regressions.ts. With the fixes reverted, the pins fail.

Verification

  • bun run typecheck: pass.
  • bun run test with ClickHouse 26.8.2.7: 68 files, 1,248 tests passed, none skipped. Doc citations, export catalogs and doc examples ok.
  • Migration fuzzer at 100× budget: 20,000 checked.
  • bun scripts/check-package.ts: ok.

Left as-is (from review)

  • A unique column's constraint name comes from table and column names, so renaming one rebuilds the table. Correct, but costly on large D1 tables.
  • A swap (rename a → b while creating a new a) is reported as unsupported.

Second round (after review)

  • A rebuild that changes a kept column's type now needs confirm_data_loss for it. The copy converts values by SQLite's type affinity ('007' → 7, '1e3' → 1000), which happened silently before.
  • Names that differ only in case are renamed with no hint (Url → url, links → Links). SQLite compares names without case, so before this the change read as a drop plus an add that collided. SQLite refuses a case-only table rename in one step, so it goes through a temporary name.
  • More coverage in the migration fuzzer:
    • It runs on D1 too, one batch per migration as Alchemy applies it.
    • Its inputs rename a table and a column through confirm_rename hints (about half of all inputs), or by case only with no hint.
    • A drizzle-kit 1.0 snapshot of any random schema must import with nothing left to generate.
  • Changesets no longer say SQLite migrations are not generated.
  • Verified on the top of the stack against ClickHouse 26.2.19.43 and 26.8.2.7: 69 files, 1,273 tests, and all 118 fuzz properties at FUZZ_SCALE=20. check-package is ok.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

artile and others added 13 commits October 25, 2026 09:55
Found by fuzzing migrations between random schemas on seeded rows:

- A rebuild that keeps no column (every old column dropped, new ones
  added) wrote no INSERT, so it deleted every row though only the
  columns' data was confirmed lost. It now copies the rowids, and each
  row takes the new columns' defaults.
- Replacing every column of a table dropped them before adding the new
  ones, and SQLite cannot drop a table's last column. Column adds now
  run before column drops.
- A column made NOT NULL that has a default failed the rebuild over
  existing NULLs: INSERT ... SELECT passes an explicit NULL, which does
  not take the default. The copy is COALESCE(column, default) now.

docs/migrations.md: a rebuild under D1 or any transaction fires ON DELETE
CASCADE in tables that reference the rebuilt one (PRAGMA foreign_keys=OFF
does nothing there), a rebuild renumbers rowids, and ADD COLUMN appends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…schema and keep rows

Two random schemas over a shared pool of table and column names: new,
dropped and changed tables, columns, types, nullability, literal and
expression defaults, keys, UNIQUE columns and indexes. The migration
diffSqliteSchemas writes runs inside a transaction on rows that fit
both schemas. The PRAGMA shape must equal a fresh create of the new
schema (column order aside) and every row must survive, holding its
values or the new column's default.

Pins the three counterexamples fixed in the previous commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bac77979-401f-4e97-9d30-80337ae60a92
Commits

Reviewing files that changed from the base of the PR and between 5aa2afc and bf88a90.

Files selected for processing: 32
  • .changeset/quiet-sqlite-dialects.md
  • .changeset/sqlite-generate.md
  • .changeset/sqlite-table-ddl.md
  • README.md
  • docs/README.md
  • docs/migrations.md
  • docs/reference.md
  • docs/sqlite.md
  • src/kit/__fixtures__/drizzle-sqlite/20261025075810_init/migration.sql
  • src/kit/__fixtures__/drizzle-sqlite/20261025075810_init/snapshot.json
  • src/kit/__fixtures__/drizzle-sqlite/schema.ts.txt
  • src/kit/cli.ts
  • src/kit/generate.ts
  • src/kit/sqlite-generate.test.ts
  • src/migrate/run.ts
  • src/migrate/source.ts
  • src/migrate/verify.ts
  • src/schema.ts
  • src/schema/diff.ts
  • src/schema/entities.ts
  • src/schema/snapshot.ts
  • src/schema/sqlite-diff.ts
  • src/schema/sqlite-drizzle.test.ts
  • src/schema/sqlite-drizzle.ts
  • src/schema/sqlite-ops.ts
  • src/schema/sqlite-render.ts
  • src/sqlite/ddl.test.ts
  • src/sqlite/shape.test-support.ts
  • tests/fuzz/README.md
  • tests/fuzz/migrations.fuzz.test.ts
  • tests/fuzz/regressions.ts
  • tests/package-consumer.mts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Makisuo and others added 11 commits October 25, 2026 19:02
…ound trip

- The migration property runs on D1 too: each migration as one batch on a
  local workerd binding, as Alchemy's D1 resource applies it, compared
  with a fresh create on a second D1 database.
- Inputs rename a shared table and one of its kept columns through
  confirm_rename hints (about half of all inputs do), so renames combine
  with every other change and with rebuilds.
- A drizzle-kit 1.0 snapshot of any random schema (written as drizzle-kit
  writes it, key columns `notNull: false` included) imports with nothing
  left to generate.
- Changesets no longer say SQLite DDL or migrations are not implemented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	tests/fuzz/regressions.ts
…t a hint

From an independent review:
- A rebuild that changes a kept column's type converts its values by
  SQLite's type affinity ('007' to 7, '1e3' to 1000) with no prompt. It
  now needs confirm_data_loss for that column, like a drop.
- A column or table renamed by case only (Url to url) read as a drop and
  an add, which then collided: SQLite names ignore case. The diff now
  renames it, with no hint, since nothing is lost. SQLite refuses a
  case-only table rename in one step, so it goes through a temporary name.
- The migration fuzzer renames by case without hints too, and confirms
  drops and type changes under old and new names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Makisuo
Makisuo added this pull request to stack #124 October 25, 2026 19:16
@Makisuo
Makisuo merged commit 8eb7200 into main Oct 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants