Skip to content

Migrations have no downstream lane — every release that adds one collides with fork migrations #280

Description

@iceteaSA

What problem does this solve?

schema_migrations is keyed on a single integer owned by upstream. Anything downstream that needs a schema change — a fork, or a sibling plugin sharing context.db — has to pick a number out of the same sequence, and upstream's next release takes it.

We track this repo closely and have renumbered six times: v33 → v41 → v51/52/53 → v70/71/72 → v73/74/75 → v76. Mechanical, but it is not free, because the collision is silent in the worst case.

The failure mode is the part worth fixing. A downstream migration records itself as, say, v41. Upstream then ships its own v41. On the next start the runner sees version 41 already applied and skips upstream's body. No error. The columns simply never exist, and you find out when a query throws much later:

classify chunk failed: SQLiteError "no such column: shareable"

That specific one ran for days before surfacing. We have hit this class three separate times, each needing manual DB surgery to repair (delete the colliding rows, re-run the real bodies). The ensureColumn self-heal net in initializeDatabase catches some of it, but only for tables that have a net — migration-only tables have no backstop at all.

To be clear about scope: this is a downstream problem, not an upstream bug. Nothing here is broken for people running stock. Filing it because the cost lands on anyone extending the plugin, and the fix is cheap if you want it.

Proposed solution

Two options, in the order I would pick them:

Reserve a downstream range. Upstream keeps < 10000; forks and sibling plugins use >= 10000. One documented convention, no schema change, no runner change. Solves collision entirely as long as both sides honor it.

Or key the table on (owner, version). schema_migrations gains an owner TEXT NOT NULL DEFAULT 'upstream', and the runner tracks a high-water mark per owner. Structural, so it cannot be violated by convention drift, at the cost of a migration to the migrations table.

Either way, the fence check in LATEST_SUPPORTED_VERSION would need to compare per-owner rather than against a single max.

Alternatives considered

  • Status quo. Workable, and we have absorbed it six times. The objection is the silent-skip case, not the renaming.
  • Make the runner refuse to skip. Verify a recorded migration's effect before skipping. Robust, but expensive on every open, and it needs a per-migration assertion.

Additional context

Happy to prototype the range convention as a docs-only PR if that is the direction you like — it is the lower-risk of the two and needs no code.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions