Skip to content

Meetings: load the voiceprint model off the main thread at launch - #1932

Merged
r3dbars merged 1 commit into
mainfrom
claude/voiceprint-load-off-main
Sep 29, 2026
Merged

r3dbars merged 1 commit into
mainfrom
claude/voiceprint-load-off-main

Conversation

@r3dbars

@r3dbars r3dbars commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Problem

MeetingSessionController.init runs on main from applicationDidFinishLaunching. It called SpeakerEmbedderFactory.makeEmbedder, which loaded ReDimNet2 synchronously (MLModel(contentsOf:) via router.preload, plus a GPU compile the first launch after an update). ReDimNet2 is the default in this release, so every launch froze the menubar for that long. On slower Macs that also means Sentry app-hang reports.

Fix

makeEmbedder loads nothing now. It still picks the model, and so the speaker DB, synchronously, from model-file presence. What it returns is Core's new BackgroundLoadedSpeakerSegmentEmbedder. That type knows the model's id, size and thresholds up front, so the controller builds DiarizationService, SpeakerDatabase(thresholds:) and the migration target exactly as before. It loads the real embedder on a utility queue the first time something waits for it. MeetingSessionController and TranscriptedApp only get a comment change.

Who waits:

  • DiarizationService.initialize() (the existing launch warmup in TranscriptedAppState) awaits the load alongside the diarizer models. So areMeetingModelsWarm still means the voiceprint is loaded.
  • DiarizationService.diarizeOffline (pyannote and Nemotron) awaits it again before re-embedding. A meeting that stops before the load finishes gets the right model's vectors against the right DB.
  • SpeakerVoiceprintMigration.run awaits it before moving anyone. The gate still closes at launch in gate.start, so every speaker-DB writer is held through the load plus the move.
  • A sync embed called before the load ends blocks its (background) thread until the load ends. It never returns nil just because it was early.

When the load fails:

  • This launch keeps the model's own DB and gets no voiceprints. embed returns nil, and DiarizationService drops the native vector, so nothing 256-d reaches the 192-d file. Diarization and transcripts still work.
  • The migration fails with embedder_unavailable and writes no ledger rows, so it retries later.
  • SpeakerEmbedderLoadFailureMemory records the failure against app build + macOS version. The next launch then resolves to WeSpeaker + speakers.sqlite with WeSpeaker thresholds, which is the old fallback, and the DB path and embedder stay consistent. A new build or macOS update tries ReDimNet2 again.
  • A loaded model whose id, size or thresholds don't match what was declared is also treated as a failure.

ERes2Net (hidden opt-in) goes through the same path, so it no longer loads on main either.

Tests

New behavior tests:

  • Tests/TranscriptedCoreTests/SpeakerTests/BackgroundLoadedSpeakerSegmentEmbedderTests.swift covers:
    • building the diarizer and DB around it loads nothing
    • the load runs off the waiting (main) actor, and main keeps running mid-load
    • a meeting that re-embeds mid-load gets the loaded vectors, and an early sync embed isn't nil
    • a failed load yields no vectors, so native 256-d vectors are dropped
    • a mismatched model is refused
    • 16 concurrent waiters share one load
  • SpeakerVoiceprintMigrationGateTests adds two tests:
    • a model still loading holds writers from launch, then the move carries everyone
    • a failed load lets writers through with embedder_unavailable, writes no ledger rows and leaves the source DB untouched, and the next launch moves everyone
  • Tests/SpeakerEmbedderLoadFailureMemoryTests.swift (fast suite) covers fallback to speakers.sqlite after a failure on the same build, a retry on a new build or OS, and a success clearing the failure.

Mutation-probed: dropping the migration's wait, dropping the declared-model check, and dropping the failure check each turn a test red.

Run locally:

  • swift test --filter 'BackgroundLoadedSpeakerSegmentEmbedderTests|SpeakerVoiceprintMigrationGateTests|DiarizationReembedTests|SpeakerVoiceprintMigrationTests': 34 pass
  • bash run-tests.sh --filter SpeakerEmbedderLoadFailureMemory: pass
  • bash build-deps.sh --force and bash build.sh --no-open: pass
  • One full bash check.sh on the uncommitted tree:
    • build, full run-tests.sh, integration smoke, full swift test, source pins and test shape all passed.
    • Two checks failed there: a new Sendable warning in Sources/Support, and doc-paths (the new file wasn't tracked yet). Both are fixed in the commit. After the fix I reran check-doc-paths.py and concurrency-census.sh --check, and both pass.
  • No second full check.sh after that fix, per the coordinator. CI runs the full suite.

Not done / risks

  • No real-app launch timing. Launch smoke is blocked on this macOS account. I haven't measured the main-thread stall going away in a running app or confirmed that Sentry hangs stop; that needs a manual launch (ideally the first launch after an update, on a slow Mac).
  • A failing launch has no speaker recognition. It's one launch, and the next falls back fully. Transcript frontmatter on that launch still says voiceprint_model: redimnet2-b4, even though no voiceprints were computed.
  • A transient load failure (say, memory pressure during the first GPU compile) keeps ReDimNet2 off until the next app build or macOS update.
  • The load runs at .utility. A meeting imported in the first seconds after launch may wait a little longer for the voiceprint than it waits for the diarizer.
  • The migration starts the load at launch for upgraders (anyone with speakers.sqlite), before the dictation warmup, now off main. Before this change it ran on main at that same point.
  • An independent review of the full diff hasn't been done yet (AGENTS.md asks for one on meaningful code).

🤖 Generated with Claude Code

MeetingSessionController.init runs on main in applicationDidFinishLaunching,
and it called SpeakerEmbedderFactory.makeEmbedder, which loaded ReDimNet2
(MLModel(contentsOf:), plus a GPU compile the first launch after an update)
right there. With ReDimNet2 the default, every launch froze the menubar for
that long.

makeEmbedder now loads nothing. It picks the model, and so the speaker
database, from model-file presence and returns Core's new
BackgroundLoadedSpeakerSegmentEmbedder, which knows the model's id, size and
thresholds up front and loads the real embedder on a utility queue the first
time something waits for it:

- DiarizationService awaits it during initialize() (the launch warmup), so
  "meeting models warm" still means the voiceprint is loaded, and again
  before re-embedding a meeting, so a meeting that races the load gets the
  right model's vectors.
- SpeakerVoiceprintMigration.run awaits it before moving anyone. The gate
  still closes at launch, so writers stay held through the load.

If the load fails, this launch keeps the model's own database and gets no
voiceprints (nothing 256-d ever reaches the 192-d file), the migration
fails with embedder_unavailable without writing the ledger, and
SpeakerEmbedderLoadFailureMemory makes the next launch on the same app
build and macOS version use WeSpeaker and speakers.sqlite, the old
fallback. A new build or macOS update tries the model again.
@r3dbars

r3dbars commented Sep 29, 2026

Copy link
Copy Markdown
Owner Author

Independent review (Fable reviewer, full diff): SHIP. No deadlocks (continuation waits only; blocking wait unreachable from main), meetings always finish (failed load → no vectors, transcript saves), migration gate race safe, happy path identical, failure memory acceptable. Non-blocking notes: one-launch empty Speakers view after a failed load; no wrapper timeout (CoreML errors instead). Integrated via #1934.

r3dbars added a commit that referenced this pull request Sep 29, 2026
@r3dbars
r3dbars merged commit 541470a into main Sep 29, 2026
4 of 8 checks passed
@r3dbars
r3dbars deleted the claude/voiceprint-load-off-main branch September 29, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant