Keep the meeting mic and system audio going through device switches, hiccups and failed recoveries

[claude]

Requested by Justin · project thread

Before: switching speakers or headphones mid-meeting dropped about 5 seconds of your mic, and each mic restart on a surviving non-Bluetooth mic could flip AirPods into call mode. Pressing Stop during a mic restart threw away the last couple of seconds. If one mic recovery failed, the whole meeting ended. If the chosen mic couldn't start, the meeting failed to start even when the Mac's own mic was right there. On the system-audio side, a third of a second of the Mac being busy ended call audio for the rest of the meeting, a slow wake used up the one reconnect a later real stall needed, and every reconnect threw away its first buffer without padding for it.

After: the mic restarts about a quarter second after an output switch, reusing its engine when the pinned mic survives, so the AirPods aren't touched on that path. Stop mid-restart keeps that audio. A failed recovery keeps the meeting (and system audio) recording and retries, falling back to a built-in mic that can actually hear you (never a lid-closed MacBook mic, at start or later). A mic that won't start falls back the same way instead of failing the meeting. System audio rides out a busy moment by reconnecting and padding the whole hole, slow wakes get their own reconnect, and reconnect pads now line up with the audio clock.

Not part of 1.1.62. Up to date with main (after 1.1.62, #1785, #1772, #1797, #1799, #1790, #1802).

Why

Audio follow-ups from the 1.1.62 review: the ~5s mic gap per output switch, one failed recovery ending the meeting, no built-in fallback at start, clip-removal log noise, the fresh-engine AirPods flip found in the #1771 diagnosis, deep review M3, M4, M5, M6, M7, M8, M11 and M14, plus four independent reviews of this branch (mic side, system-audio side, a full-PR pass, and the next-release deep review in reviews/next-release/1780.md), a verification pass, and a closing review (reviews/next-release/1780-closing.md: ready once CI is green). Every should-fix item is in; the follow-ups are listed under Notes.

Product Impact

• Affects: meetings
• Lane: meeting reliability
• Why this matters: people switch AirPods, speakers and displays mid-call all the time, and losing their own voice, the other side, or the whole meeting is the worst failure we have.

What changed

Mic

• Route change → immediate mic restart. Audio observes AVAudioEngineConfigurationChange. After a 0.25s settle it reads isRecording on main, then checks off main whether the changed engine is the live meeting graph and whether it stopped; if so it runs recoverFromDeviceChange. Pure decision in MicEngineConfigurationChangePolicy: ignores dictation's engine and detached graphs, ignores sleep, waits if another recovery is running, leaves a running engine alone (only a running engine can be "still flowing"; a stopped one can still hand over a last tap block), leaves a route that flaps within 1s of the last recovery ending to the watchdog, and never goes past the watchdog's 5-attempt limit. Will-sleep now sets the sleep mark before releasing the call-audio tap, so no route-change recovery starts while the Mac goes to sleep.
• In-place mic restart. A device-change recovery reuses the engine already bound to the pinned mic when that mic is still alive, is not Bluetooth and VP is off, instead of building a fresh AVAudioEngine whose input node briefly opens the macOS default input. Gate in MicInPlaceRestartPolicy. A failed in-place restart retries once with a fresh graph after 0.2s (not counted as another switch or attempt); after any failed attempt the next one builds fresh, and an in-place restart that comes back on a different input counts as failed. Fresh-graph paths (unplugged mic, retries, fallback, VP on, start) still touch the default input; that's Record the Mac mic directly so AirPods never flip into call mode #1784's job.
• Failed recovery keeps recording (M3). Failure branches no longer call stop(); the watchdog retries and still gives up after 5 in a row. Fixed two latent bugs that made retries no-ops. Recovery bails before touching hardware when the recording no longer owns a mic file. A retry pads from the last frame the recording actually kept (MicRecoveryGapAnchorPolicy), so frames a failed attempt took and then deleted can't leave the mic seconds early.
• Stop mid-recovery keeps the recovery audio (M4). The recovery segment is registered as soon as its writer is installed. A header-only segment (Stop before the first frame) merges as empty instead of "degraded", so raw mic WAVs aren't left on disk. Stop during the first-frame wait is logged as stale.
• Built-in fallback. Start gets a third attempt on the best built-in mic; recovery tries it first after a failed attempt or when the mic never delivered a frame. Reason builtInFallbackAfterFailure.
• Closed lid. With the lid closed (MacLidState, IOKit AppleClamshellState, unreadable = open) the MacBook's own mic (isLidMicrophone) is left out of every built-in pick: the start pick, the stabilization fallback and the post-failure fallback. The filter runs once, in bestBuiltInInput. The headphone-jack mic and display mics still count. MacLidState.swift, isLidMicrophone and the lidClosed: parameters are shared line for line with Record the Mac mic directly so AirPods never flip into call mode #1784, so the two PRs merge without a duplicate type (notes in project files release/merge-notes/1780-vs-1784.md).
• Clip log noise. SpeakerClipExtractor.removeClipFile skips the warning when the file wasn't there.

System audio (CoreAudioSystemAudioCapture, CoreAudioTapBufferRing, Audio)

• M5, overflow no longer ends system audio. Audio queued before the hole is delivered, the tap is rebuilt, and the pad covers the whole hole (the ring counts dropped frames, including when a format change cuts the overflow short); after 3 overflows it still ends cleanly, keeping the queued audio. An overflow found at Stop also keeps its queued audio now. An overflow while an earlier reconnect's pad is pending skips the held audio and keeps one pad. Ring holds 128 callbacks (~1.4s), prefaulted. Overflow reconnects report .fellBehind, which holds writes like a reconnect but doesn't count as a device switch or lower capture_quality. Recording takes a .latencyCritical activity (idle sleep allowed, ended on failure) so App Nap can't coalesce the 10ms drain timer into an overflow.
• M6, slow wake keeps the stall budget. A wake/route/overflow reconnect that never delivers gets one noFirstBuffer reconnect of its own (warns like a stall, reports the same event kind as the reconnect it retries, own on-device count).
• M7, one silent-tap rebuild per wake instead of three.
• M8, reconnect pads line up. The ring records the HAL's input host time per callback; the pad is firstNewSample - lastKeptSampleEnd when both are stamped and plausible. The host now queues the pad and releases the write-hold on the thread that sends .gap, before the reconnect's first buffer arrives, so that buffer is no longer thrown away uncounted. The pad write is gated by the file queue's generation check, not a main-thread flag.
• M11, faster rate-change detection. 50ms poll, plus an immediate check when the callback size changes.
• M14, sleep release finishes before the sleep notice returns (waits up to 1s on the capture queue).
• Telemetry (Telemetry for 1.1.62: call-audio failures and reconnects, permission prompt, warmup, crash builds #1781) seam: overflow and no-first-buffer reconnects get their own on-device counts; nothing new is forwarded off-device.

How I checked it

• python3 scripts/dev/check-build-source-lists.py
• Trial merges with Record the Mac mic directly so AirPods never flip into call mode #1784 (c906bd9) and Mic-only meetings don't start the call-audio tap #1787 (0c1fb4a): one MacLidState, conflicts only in keep-both hunks, resolutions written to project files release/merge-notes/.
• CI (app-build, checks, spm-tests) on the current head: running now. There's no Swift toolchain in the authoring session, so CI is the first compile of the system-audio half and the new tests.
• Tests: MicRecoveryFallbackTests (route-change decisions incl. stopped-engine tap block, attempt limit, running engine; fallback policy incl. lid closed at start and after failure, the jack mic and a localized MacBook name; in-place gate; gap anchor; writer handoff), AudioInitializationTests (segment register/unregister/Stop wins), MicRecordingFileMergerTests (header-only segment), CoreAudioSystemAudioCaptureTests (overflow keeps queued audio, pad covers dropped audio, long stall, overflow during a pending pad, repeated overflows, overflow at Stop keeps queued audio, only route reconnects report a device switch, host-clock pad, buffer-size format check, slow wake keeps the stall reconnect and warns, sleep release), SystemAudioRecoveryParityTests (.gap releases the hold before main runs; falling behind pads without a route change), CoreAudioTapBufferRingTests, ClipRemovalPolicyTests.
• Manual check on hardware (plan below). No CI job exercises real CoreAudio.

Risk Review

• Privacy / local-first behavior reviewed: no new off-device fields beyond the builtInFallbackAfterFailure category string for the existing selection_reason; new logs carry transport classes, counts and scratch file basenames only.
• Storage path or migration impact reviewed: none.
• Release/update impact reviewed: not in 1.1.62. Adds IOKit to the app, fast-test and package link lists.
• Agent PRs stay draft until human review
• No private transcripts, audio, tokens, personal paths, or customer data are included

Notes

• Not done here, on purpose:
  • When the mic is gone for good (e.g. a Mac mini's only USB mic unplugged), the watchdog still stops the meeting after 5 tries. Keeping call audio going with a slow mic retry needs a visible "your mic stopped" cue first, so it's a separate change.
  • There's no on-screen cue yet when the meeting falls back to a built-in mic. It's logged only. The existing route warning says "Bluetooth mic is unstable", which is wrong for a USB mic, so a proper cue gets its own small UI PR.
  • If the lid-closed MacBook mic is itself the macOS input, automatic mode still keeps it (same as main today). Fixing that changes selection(), which Record the Mac mic directly so AirPods never flip into call mode #1784 also rewrites, so it's a follow-up.
• .latencyCritical stays on for the recording: overflow from a coalesced drain timer is the failure M5 is about. Revisit if battery data says otherwise.
• If Stop lands after a recovery's first frame but before its gap is finalized, the last segment keeps a pre-start gap estimate (tail up to ~2s early). Narrow window, last seconds only.
• Ring memory is ~8 MB per system-audio tap (was ~2 MB); a HAL teardown failure leaks that much.
• Merging with Mic-only meetings don't start the call-audio tap #1787: Audio.swift gap and sleep hunks plus one test rename git won't flag; see project files release/merge-notes/1787-vs-1780.md.

Hardware test (Justin's Mac; step-by-step copy in project files release/after-1.1.62-mic-and-call-audio-test.md):

1. Meeting on Mac speakers, switch output to AirPods mid-sentence and back. No ~5s hole in your voice; log "Audio route change stopped the microphone; recovering now". (Also answers: does the config-change notice fire on an output-only switch?)
2. AirPods as the macOS default input, meeting pinned to the MacBook mic: switch output. Expect "Restarting mic engine in place" and clean AirPods playback.
3. Stop within ~2s of switching output. Last words in the transcript; no leftover _mic_recovery.wav.
4. USB mic unplugged mid-meeting. Meeting keeps going with your voice (the new default mic, or "Falling back to the built-in microphone" if that fails).
5. Two lid-closes with a video playing: both sides survive; no "could not reconnect".
6. Lid closed on an external display, macOS input set to the USB or display mic: the meeting never records from the silent MacBook mic, at start or after a failed mic.
7. New speaker clip: no "Failed to remove clip file" warning.

Agent handoff

COORD_DONE: BRIEF | this PR | mic + system audio recovery; M3-M8, M11, M14; all reviews addressed, closing review ready once CI green; follow-ups: mic-lost-for-good, fallback cue, lid-closed default input | none | build source lists; CI running | merge when CI green, then hardware test

🤖 Generated with Claude Code

https://claude.ai/code/session_01NZkqPnF73GpVNv9N5LYX9b