Skip to content

fix(server): a migration id clash stops startup with a clear error instead of a crash loop - #14202

Open
shivamhwp wants to merge 2 commits into
t3code/codex-turn-mappingfrom
fix/server-migration-slot-collision
Open

shivamhwp wants to merge 2 commits into
t3code/codex-turn-mappingfrom
fix/server-migration-slot-collision

Conversation

@shivamhwp

Copy link
Copy Markdown
Collaborator

If a database already has a different migration recorded at id 55, for example because main ships its own migration 55 before V2 lands, V2 silently skips its own OrchestrationV2 schema migration. The server then crash-loops on every start with no such column: application_event_version, and the only clue is a warning that already appears in the logs.

This happens because the migrator skips every id up to the highest one it has recorded and never checks the names. runMigrations now checks V2's ids (55 and up) before migrating. If any would be skipped, startup stops with one error that names them, and nothing is migrated. Lower ids keep the existing warning only: #11639 found a real user database with a site-local migration there that still works, and it has to keep booting. A database that a newer V2 build migrated further, such as one with an extra 57, still starts.

Before, on a copy of real data with a fake 55_FutureMainMigration row:

WARN: Database migration history diverges from this build; recorded migration ids are skipped, not reconciled by name.
  divergent: [ '55:FutureMainMigration (this build: OrchestrationV2)' ]
ERROR: EventSinkStreamError: Failed to stream orchestration V2 events.
  ... Error: no such column: application_event_version

Migration 56 had also been applied to the broken database.

After, with the same copy:

ERROR: MigrationError: This database recorded other migrations at ids this build needs for its own schema, so it would skip 55_OrchestrationV2 (recorded: FutureMainMigration). ...

The database was left untouched. An unmodified copy of the same data boots normally.

This doesn't make a clashing database usable. It turns a cryptic crash loop into an error we can act on. The real protection is to number main's migrations above V2's before either ships. Related: #8896 and #9312, which address the same trap for id 45 on main.

Model: Claude Opus 5.5 (Claude Code)

🤖 Generated with Claude Code

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire — 4.9 KiB — 6.8 KiB ✅
Codex Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Codex Live turn WebSocket wire — 1.1 KiB — 2.0 KiB ✅
Codex Live turn WebSocket decoded — 20.4 KiB — 29.3 KiB ✅
Codex Live turn messages — 1 — 8 ✅
Claude Total thread wire — 4.9 KiB — 6.8 KiB ✅
Claude Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Claude Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Claude Live turn WebSocket decoded — 20.8 KiB — 29.3 KiB ✅
Claude Live turn messages — 2 — 8 ✅

Baseline: unavailable · PR result: 269ac28 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 106.1 KiB
  • Claude decoded thread snapshot: 106.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Sep 29, 2026
@macroscopeapp

macroscopeapp Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 269ac28

Macroscope's review found this PR approvable — This is a focused migration-state bug fix that adds startup validation for conflicting V2 IDs while preserving compatible migration paths. Regression tests cover fresh databases, collisions, site-local earlier migrations, and newer V2 histories; the remaining change is documentation.

Notes:

  • Diff unchanged. Approvability was decided on eligibility alone.

You can add or adjust custom eligibility rules. Learn more.

@macroscopeapp
macroscopeapp Bot dismissed their stale review September 29, 2026 02:14

Dismissing prior approval to re-evaluate 872ff2f

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Sep 29, 2026
shivamhwp and others added 2 commits September 30, 2026 13:34
The migrator skips every id up to the highest recorded one without checking
names. If a database recorded another migration at 55 (for example a future
main release), V2 skipped its own schema migration and then crash-looped on
"no such column: application_event_version". Check V2's ids before migrating
and fail with one error naming the clashing ids. Lower ids keep the warning,
since real databases carry site-local migrations there and still work.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Refine pass: one flag gates both preview reconciliation and the V2 id check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@shivamhwp
shivamhwp force-pushed the fix/server-migration-slot-collision branch from 872ff2f to 269ac28 Compare September 30, 2026 13:35
@juliusmarminge juliusmarminge added the macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews label Oct 1, 2026 — with ChatGPT Codex Connector
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 1, 2026 16:02

Dismissing prior approval to re-evaluate 269ac28

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants