fix: bootstrap db schema when migrations are missing

[Nikhil (shadowfax92)]

Summary

• add a temporary SQLite schema bootstrap when Drizzle migration files are missing from packaged resources
• validate migration folders before passing them to Drizzle so empty/partial directories fall back cleanly
• seed the current Drizzle migration history and assert exact hashes/timestamps in DB initialization tests

Verification

• bun test apps/server/tests/lib/db/index.test.ts apps/server/tests/lib/agents/db-agent-store.test.ts apps/server/tests/lib/clients/oauth/token-store.test.ts
• bun run --filter @browseros/server typecheck
• bun run lint
• git diff --check
• bun --env-file=.env.development test --preload=./tests/helpers/test-env.ts ./tests/tools/bookmarks.test.ts ./tests/tools/filesystem/bash.test.ts

Note: a full bun run check hit a transient CDP/test-harness failure in the tools group; after cleanup, the failed bookmark/filesystem bash files passed cleanly.

[Nikhil (shadowfax92)]

Greptile (@greptileai) review

[Nikhil (shadowfax92)]

Claude (@claude) review

[claude]

Claude finished Nikhil (@shadowfax92)'s task in 3m 18s —— View job

---

Code Review

• Gather context and read changed files
• Review client.ts changes
• Review index.test.ts changes
• Post final review feedback

---

Good overall structure. The transaction wrapping, idempotent DDL (CREATE TABLE IF NOT EXISTS), and the journal validation are all solid. A few things worth addressing before merging:

---

client.ts

1. Silent fallback on explicit migrationsDir — behavior regression ⚠️

client.ts:69-73

if (options.migrationsDir) {
  if (hasCompleteMigrationSet(options.migrationsDir)) {
    return options.migrationsDir
  }
  return null   // ← was a thrown error before
}

The old code threw if an explicit path was invalid. Now it silently falls back to bootstrap. If a developer typos the path (or the build step puts migrations in the wrong place), they'll get a silently bootstrapped DB with no signal. At minimum, a logger.warn(...) here would make production failures visible. Without it, this change erases an intentional guard. Fix this →

2. No log on bootstrap path ⚠️

client.ts:52-54

} else {
  bootstrapCurrentSchema(sqlite)
}

When this branch is taken in production there's no trace in the logs. If a user reports a startup issue, there's no way to know the DB was bootstrapped rather than migrated. A single logger.info('Drizzle migrations unavailable; bootstrapping current schema.') here would go a long way. Fix this →

3. SERIAL PRIMARY KEY is not SQLite syntax ⚠️

client.ts:264-270

CREATE TABLE IF NOT EXISTS __drizzle_migrations (
  id SERIAL PRIMARY KEY,   -- ← PostgreSQL type
  hash text NOT NULL,
  created_at numeric
)

SQLite has no SERIAL type — it assigns NUMERIC affinity to unknown type names. This works (tests pass), but INTEGER PRIMARY KEY would be the correct SQLite idiom for an autoincrement rowid alias. More importantly, when migrate() is called against a bootstrapped DB on second run, Drizzle's internal migrate function reads and writes this table. If Drizzle uses INTEGER PRIMARY KEY in its own DDL for this table, the schemas will differ between the bootstrap and migration paths. Worth cross-checking against drizzle-orm/bun-sqlite's migrate source. Fix this →

4. String interpolation in SQL (low risk, style nit)

client.ts:149-156

sqlite.exec(`
  INSERT INTO __drizzle_migrations ("hash", "created_at")
  SELECT '${migration.hash}', ${migration.createdAt}
  ...
`)

currentMigrationHistory is a hardcoded constant so there's no injection risk here. Still, Bun's sqlite.prepare(...).run(...) API would be cleaner and safer by convention, especially since this DDL table may be extended in future.

5. Maintenance hazard: currentSchemaStatements and currentMigrationHistory drift

This is called out in the TODO, so it's acknowledged — but there's no mechanism preventing them from going stale when a new migration is added. The hasCompleteMigrationSet check guards against using an incomplete migrations dir, but nothing guards against forgetting to update the bootstrap constants. A comment in drizzle.config.ts or a test that asserts currentMigrationHistory matches the actual journal entries would catch this. Fix this →

---

index.test.ts

Overall: test coverage is thorough and well-structured. The expectCurrentSchema helper is clean. The four new scenarios cover the important paths.

6. Weak assertion in skips empty packaged migration resources

index.test.ts:82-83

expect(handle.migrationsDir).not.toBe(packagedMigrationsDir)

This passes whether handle.migrationsDir is null (bootstrap) or sourceMigrationsDir (source tree found). Since the intent of this test is to verify the empty packaged dir is skipped in favor of fallback, a more explicit assertion would be:

// If source migrations are available in the test environment:
expect(handle.migrationsDir).not.toBe(packagedMigrationsDir)
// Or if you want to assert bootstrap was used:
expect(handle.migrationsDir).toBe(null)

The current assertion does verify the empty packaged dir is not used, but the intent could be clearer.

7. Removed error test without replacement coverage

The old 'fails clearly when an explicit migration directory is missing' test was removed. The new 'bootstraps the current schema when migration files are unavailable' test covers the happy path, but there's no test asserting that a warning/log is emitted (which ties back to finding #1 above). Once logging is added, a test verifying that the warning fires on explicit-path fallback would be valuable.

---

Summary

The core approach is sound and solves a real packaging gap. The three issues worth addressing before merge are:

1. Log (or warn) when bootstrap is used, especially on the explicit-migrationsDir fallback path
2. Verify SERIAL vs INTEGER PRIMARY KEY against Drizzle's internal schema
3. Add a drift guard for currentMigrationHistory vs the actual journal

[github-actions]

✅ Tests passed — 1314/1320

Suite	Passed	Failed	Skipped
✅ agent	184/184	0	0
✅ build	18/18	0	0
✅ eval	91/91	0	0
✅ server-agent	296/296	0	0
✅ server-api	135/135	0	0
✅ server-browser	9/9	0	0
✅ server-integration	10/10	0	0
✅ server-lib	253/254	0	1
✅ server-root	52/55	0	3
✅ server-tools	266/268	0	2

View workflow run

[greptile-apps]

Greptile Summary

This PR adds a schema-bootstrap fallback for packaged builds where Drizzle migration files are absent, and introduces hasCompleteMigrationSet to validate a migrations directory before handing it to Drizzle. The DbHandle.migrationsDir type widens to string | null to signal which path was taken.

• resolveMigrationsDir now returns null instead of throwing when no complete migration set is found, causing openBrowserOsDatabase to call bootstrapCurrentSchema — an inline DDL sequence that creates all tables and seeds the migration-history table.
• hasCompleteMigrationSet parses _journal.json, verifies every tag in currentMigrationHistory is present, and confirms each .sql file exists — incomplete or empty directories fall back cleanly.
• Four new tests cover missing-dir bootstrap, empty-dir bootstrap, empty-packaged-resources skip, and idempotent re-open after bootstrap.

Confidence Score: 3/5

Safe to merge once the __drizzle_migrations DDL is corrected; the bootstrap logic itself is sound but the wrong column type creates an ongoing schema mismatch.

The bootstrap path correctly guards against re-running migrations and the new tests cover the main scenarios well. However, the __drizzle_migrations table is created with id SERIAL PRIMARY KEY — SQLite does not treat SERIAL as an auto-incrementing integer, so every row inserted by bootstrap and by any subsequent Drizzle migrator run will have a NULL id. This diverges from the table structure Drizzle generates and expects.

The bootstrapCurrentSchema function in client.ts, specifically the __drizzle_migrations DDL at the bottom of currentSchemaStatements.

Important Files Changed

Filename	Overview
packages/browseros-agent/apps/server/src/lib/db/client.ts	Adds schema-bootstrap fallback and migration-set validation; __drizzle_migrations DDL uses SERIAL PRIMARY KEY which is not an auto-incrementing type in SQLite (should be INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL), creating a schema mismatch with what Drizzle itself generates.
packages/browseros-agent/apps/server/tests/lib/db/index.test.ts	Replaces the "throws on missing dir" test with four new scenarios covering bootstrap, empty-dir, empty-packaged-resources, and idempotent re-open; coverage is thorough and the expected migration hashes match the constants in client.ts.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[openBrowserOsDatabase] --> B[resolveMigrationsDir]
    B --> C{options.migrationsDir\nprovided?}
    C -- Yes --> D{hasCompleteMigrationSet?}
    D -- Yes --> E[return migrationsDir]
    D -- No --> F[return null]
    C -- No --> G[iterate candidates:\nresourcesDir / sourceMigrationsDir]
    G --> H{hasCompleteMigrationSet?}
    H -- Yes --> E
    H -- No --> F
    E --> I{runMigrations !== false?}
    F --> I
    I -- Yes + dir not null --> J[migrate via Drizzle]
    I -- Yes + dir null --> K[bootstrapCurrentSchema]
    K --> L[CREATE TABLE IF NOT EXISTS\nfor each table + index]
    L --> M[INSERT migration hashes\nWHERE NOT EXISTS]
    J --> N[DbHandle returned]
    K --> N
    I -- false --> N

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[openBrowserOsDatabase] --> B[resolveMigrationsDir]
    B --> C{options.migrationsDir\nprovided?}
    C -- Yes --> D{hasCompleteMigrationSet?}
    D -- Yes --> E[return migrationsDir]
    D -- No --> F[return null]
    C -- No --> G[iterate candidates:\nresourcesDir / sourceMigrationsDir]
    G --> H{hasCompleteMigrationSet?}
    H -- Yes --> E
    H -- No --> F
    E --> I{runMigrations !== false?}
    F --> I
    I -- Yes + dir not null --> J[migrate via Drizzle]
    I -- Yes + dir null --> K[bootstrapCurrentSchema]
    K --> L[CREATE TABLE IF NOT EXISTS\nfor each table + index]
    L --> M[INSERT migration hashes\nWHERE NOT EXISTS]
    J --> N[DbHandle returned]
    K --> N
    I -- false --> N

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
packages/browseros-agent/apps/server/src/lib/db/client.ts:264-268
The `__drizzle_migrations` DDL uses `SERIAL PRIMARY KEY`, but SQLite has no native `SERIAL` type — it carries NUMERIC affinity and is **not** an alias for the rowid. Drizzle creates this table as `id integer primary key autoincrement not null`; deviating means the `id` column will be NULL for every row inserted by both the bootstrap code and any subsequent Drizzle migrator run. SQLite allows multiple NULLs in a non-INTEGER primary key due to a long-standing quirk, so writes won't error today, but if Drizzle ever issues an `ORDER BY id` or expects non-null ids the results will be wrong. Use the exact DDL Drizzle generates to guarantee schema compatibility.

```suggestion
    CREATE TABLE IF NOT EXISTS __drizzle_migrations (
      id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
      hash text NOT NULL,
      created_at numeric
    )
```

### Issue 2 of 2
packages/browseros-agent/apps/server/src/lib/db/client.ts:69-73
**Silent fallback on explicit `migrationsDir`**

When a caller explicitly passes `migrationsDir`, a missing or incomplete directory now silently falls back to schema bootstrap instead of throwing. The previous behaviour gave a clear diagnostic (`Drizzle migrations directory not found`) — the silent swallow makes it easy for a misconfigured test or deployment to bootstrap an empty DB without any warning, then fail later with opaque "table not found" errors rather than at the point of misconfiguration.

_{Reviews (1): Last reviewed commit: "test: assert db fallback migration histo..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR adds a bootstrap fallback that creates the SQLite schema directly from hardcoded DDL statements when Drizzle migration files are absent from a packaged build, and updates resolveMigrationsDir to return null (instead of throwing) when the migration directory is missing or incomplete.

• client.ts: resolveMigrationsDir now returns string | null; a new hasCompleteMigrationSet validator gates migration folder use; bootstrapCurrentSchema creates all tables and seeds __drizzle_migrations rows in a transaction when no valid folder is found.
• index.test.ts: Replaces the "throws on missing directory" test with four new cases covering missing, empty, and resource-dir migration paths, plus an idempotent re-open check that validates the bootstrap → normal-migration round-trip.

Confidence Score: 3/5

The bootstrap path will silently produce a __drizzle_migrations table whose id column has no auto-increment behaviour, diverging from the schema Drizzle creates normally. Every database that passes through the bootstrap will carry this divergence forward even after packaging is fixed, because Drizzle's later CREATE TABLE IF NOT EXISTS becomes a no-op.

The SERIAL PRIMARY KEY in the __drizzle_migrations DDL is not auto-incrementing in SQLite — only INTEGER PRIMARY KEY triggers that behaviour. All bootstrapped rows get id = NULL, which SQLite's PRIMARY KEY constraint permits (a historical quirk). When the app later transitions to real migrations, Drizzle silently reuses the existing table, so all future migration rows also receive id = NULL. This is a persistent, silent schema divergence on every affected installation.

packages/browseros-agent/apps/server/src/lib/db/client.ts — the __drizzle_migrations DDL and the SQL string-interpolation in bootstrapCurrentSchema both warrant a closer look.

Important Files Changed

Filename	Overview
packages/browseros-agent/apps/server/src/lib/db/client.ts	Adds bootstrap fallback schema + migration history when Drizzle migration files are absent; resolveMigrationsDir now returns null instead of throwing. The __drizzle_migrations DDL uses SERIAL PRIMARY KEY, which SQLite does not recognise as auto-increment, producing a persistent schema divergence from what Drizzle's migrator creates.
packages/browseros-agent/apps/server/tests/lib/db/index.test.ts	Replaces the "throws on missing directory" test with four new scenarios covering bootstrap from missing/empty/resource dirs and idempotent re-open; expectCurrentSchema validates exact tables and migration hashes. Coverage is thorough.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[openBrowserOsDatabase] --> B[resolveMigrationsDir]
    B --> C{options.migrationsDir set?}
    C -->|Yes| D{hasCompleteMigrationSet?}
    D -->|Yes| E[return migrationsDir]
    D -->|No| F[return null]
    C -->|No| G[Build candidate list\nresourcesDir + sourceMigrationsDir]
    G --> H{Any candidate\npasses hasCompleteMigrationSet?}
    H -->|Yes| E
    H -->|No| F
    E --> I[migrate drizzle db\nfrom folder]
    F --> J[bootstrapCurrentSchema\nsqlite exec BEGIN]
    J --> K[CREATE TABLE IF NOT EXISTS\nfor each schema statement]
    K --> L[INSERT INTO __drizzle_migrations\nfor each migration in history]
    L --> M[COMMIT]
    I --> N[return DbHandle\nmigrationsDir = path]
    M --> O[return DbHandle\nmigrationsDir = null]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[openBrowserOsDatabase] --> B[resolveMigrationsDir]
    B --> C{options.migrationsDir set?}
    C -->|Yes| D{hasCompleteMigrationSet?}
    D -->|Yes| E[return migrationsDir]
    D -->|No| F[return null]
    C -->|No| G[Build candidate list\nresourcesDir + sourceMigrationsDir]
    G --> H{Any candidate\npasses hasCompleteMigrationSet?}
    H -->|Yes| E
    H -->|No| F
    E --> I[migrate drizzle db\nfrom folder]
    F --> J[bootstrapCurrentSchema\nsqlite exec BEGIN]
    J --> K[CREATE TABLE IF NOT EXISTS\nfor each schema statement]
    K --> L[INSERT INTO __drizzle_migrations\nfor each migration in history]
    L --> M[COMMIT]
    I --> N[return DbHandle\nmigrationsDir = path]
    M --> O[return DbHandle\nmigrationsDir = null]

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
packages/browseros-agent/apps/server/src/lib/db/client.ts:264-268
**`SERIAL` is not a valid SQLite auto-increment type** — SQLite only treats `INTEGER PRIMARY KEY` as an alias for the rowid (auto-increment). Any other type name, including `SERIAL`, gets numeric affinity but no auto-increment behavior. When this `INSERT` omits `id`, SQLite assigns `NULL` instead of a generated integer. Because SQLite's `PRIMARY KEY` constraint does not imply `NOT NULL` (except for `INTEGER PRIMARY KEY`), multiple rows can each have `id = NULL`, creating an internally inconsistent table. Drizzle's own migrator creates this table as `INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL`; using `SERIAL` here permanently diverges the schema for any database that went through the bootstrap path, since Drizzle's later `CREATE TABLE IF NOT EXISTS` is a no-op once the table already exists.

```suggestion
    CREATE TABLE IF NOT EXISTS __drizzle_migrations (
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      hash text NOT NULL,
      created_at numeric
    )
```

### Issue 2 of 2
packages/browseros-agent/apps/server/src/lib/db/client.ts:148-157
The `hash` value is interpolated directly into the SQL string. Even though these come from the hardcoded `currentMigrationHistory` constant (so there is no injection risk today), `bun:sqlite` supports prepared statements and parameterised queries via `sqlite.prepare()`. Using raw string interpolation for SQL values is fragile if `currentMigrationHistory` is ever extended with values from a less controlled source, and it makes the intent less clear. Prefer `sqlite.prepare()` with bound parameters so the separation between code and data is always explicit.

```suggestion
    const insertMigration = sqlite.prepare(`
      INSERT INTO __drizzle_migrations ("hash", "created_at")
      SELECT ?, ?
      WHERE NOT EXISTS (
        SELECT 1 FROM __drizzle_migrations
        WHERE created_at = ?
      )
    `)
    for (const migration of currentMigrationHistory) {
      insertMigration.run(migration.hash, migration.createdAt, migration.createdAt)
    }
```

_{Reviews (2): Last reviewed commit: "test: assert db fallback migration histo..." | Re-trigger Greptile}