Skip to content

Move a mic back to the engine after two empty pinned takes - #1880

Merged
r3dbars merged 3 commits into
mainfrom
claude/transcription-speed-percentiles-yyabey
Sep 27, 2026
Merged

r3dbars merged 3 commits into
mainfrom
claude/transcription-speed-percentiles-yyabey

Conversation

@r3dbars

@r3dbars r3dbars commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Why

Before: in 1.1.66, #1857 moved built-in and wired mics onto the pinned Core Audio recorder, only to make dictation start faster. On one M1 MacBook Air (macOS 27), every held dictation on 1.1.66 came back empty: six were "no speech" and one was "signal but no words". The same Mac dictated fine on the engine path in 1.1.64, minutes before it updated. The person kept pressing the key, and those short taps showed up in Sentry as APPLE-MACOS-2J ("Dictation ended before the microphone finished opening"). PostHog shows the same Mac going 0 for 37. On the "no speech" takes the words are lost, because nothing is pasted and the audio isn't kept.

After: a mic that is on the pinned recorder only for speed, and gives two empty held takes in a row, goes back to the engine path it used in 1.1.65 for the rest of that app version. An update tries the recorder again. The switch sends one dictation_pinned_microphone_fell_back_to_engine event with stage empty_takes, plus the input's format and bucketed health counts, so we can see what the recorder saw on a Mac like this.

The recorder's own health checks found nothing wrong on that Mac: no restart, no silent-input flag, no fallback. So the cause inside the recorder is still unknown (n=1). This PR is the safety net, plus the data to find the cause.

Product Impact

  • Affects: dictation
  • Lane: dictation reliability
  • Why this matters: on the one Mac we know about, 1.1.66 turned dictation into silent failures. With this change, a Mac like that loses at most two takes per version before it's back on the path that works.

What changed

  • New Sources/Speech/PinnedDictationSpeedPath.swift scores each held take on a speed-only pinned mic as words or empty. recordingTooShort, modelFailure and cancelled takes aren't scored. It stores a per-mic, per-app-version count in UserDefaults, keyed by UID.
  • PinnedDictationInputPolicy.recorderIsNeeded skips the speed path for a mic that was moved back. Bluetooth-default and user-chosen overrides still always use the recorder. New recorderIsSpeedOnly.
  • DictationStartCuePolicy plays the click after recording starts for a moved mic, because the engine start is slower again.
  • ParakeetPinnedMicrophone keeps the take's format and health counts. STTRouter.transcribe(preparedRecording:) scores the take after transcription. Starting a new take clears any stale marker.
  • Analytics: new empty_takes stage. dictation_pinned_microphone_fell_back_to_engine also carries input_channels, input_rate_hz, pinned_mic_restart_bucket, pinned_mic_gap_bucket and pinned_mic_dropped_callback_bucket, and analytics-events.psv is updated. No device names or UIDs leave the Mac.
  • New tests in PinnedDictationSpeedPathTests, plus new suites in the input-policy, start-cue and analytics-forwarding tests.
  • Also updated: the run-tests.sh and scripts/dev/test-parakeet-lifecycle.sh source lists, the Speech/UI CLAUDE.md files, and the pinned-mic bullet in the privacy doc.

How I checked it

  • scripts/dev/agent-preflight.sh
  • bash scripts/dev/linux-checks.sh (runs without Swift): 48 passed, 0 failed
  • Selected checks from .agents/test-matrix.yml for the files changed (CI green on 1381c6e)
  • bash build.sh --no-open (CI app-build)
  • bash run-tests.sh (CI checks)
  • Performance budget passed
  • bash run-integration-smoke.sh (CI spm-tests; its Parakeet lifecycle smoke compiles the input policy)
  • swift test: not needed
  • bash run-e2e-smoke.sh / swift test --package-path Tools/<Package>: not mapped
  • Manual check:

Checks I could not run, and why:

  • This session has no Swift toolchain, so the build and tests ran in CI only. check-analytics-emitters.py, the taxonomy normalizer and check-telemetry-keys.py passed as part of linux-checks.

Mac or hardware test still needed? If yes: the steps, the log line or UI change each step should show, and whether each step fails on main:

  • An M1 Air is best, but any Mac with a built-in mic works for the fallback path. Hold the dictation key twice while staying silent, then hold it a third time and speak.
    • The second silent take should log the empty_takes fallback warning.
    • The third take should use the engine path (the backend isn't pinned_ioproc) and paste the words, with the click after recording starts.
    • On main, the third take stays on the pinned recorder.
  • Restarting with a different CFBundleShortVersionString should use the pinned recorder again.

Risk Review

  • Privacy / local-first behavior reviewed (the store is local; the event carries only counts and format)
  • New analytics properties or Sentry tags avoid the sanitizer's drop fragments
  • Checked the text-pin tests for every file I edited (check-source-pins.py --changed-only: PASS)
  • Storage path or migration impact reviewed (one new UserDefaults key, pinned-dictation-speed-path)
  • Public-facing copy stays concrete and matches current product scope (no copy changes)
  • Release/update impact reviewed (the move resets on every app version by design)
  • Agent PRs got an independent deep review of the full diff (READY at 1381c6e; its one finding, in the privacy doc, is fixed)
  • UI changes include sanitized .agent-review/visuals/ evidence: no UI changes
  • No private transcripts, audio, tokens, personal paths, or customer data are included

Notes

  • A user whose built-in mic works fine can also get moved, if they hold the key and say nothing twice in a row. They lose about 0.2 s of start speed for that version, and nothing else.
  • Engine takes aren't scored, so a moved mic stays moved until the next version.

Agent handoff

COORD_DONE: GREEN | this PR | adaptive fallback from the pinned recorder after 2 empty speed-only takes + format/health telemetry | none | none | linux-checks, source pins, CI | ship in 1.1.67

🤖 Generated with Claude Code

https://claude.ai/code/session_016DYGa1i8HDv497ewpWgaCM

1.1.66 put plain built-in and wired mics on the pinned recorder for a
faster start. On one M1 MacBook Air every held dictation it recorded came
back with no words (six "no speech", one kept for recovery), while the same
Mac dictated through the engine minutes earlier on 1.1.64. The recorder
reported no restart, fallback or silent input, so it could not catch this.

Each take the recorder made only for speed is now scored once
STTRouter.transcribe knows the result. Two empty takes in a row move that
mic back to the engine path, and the start click back to after recording
starts, for the rest of the app version. Takes the recorder is required for
(a skipped Bluetooth input, a picked mic) are never scored.

The move reports pinned_microphone_fell_back_to_engine with stage
empty_takes plus the take's channels, rate and bucketed gaps, dropped
callbacks and restarts, so the next report can tell a wrong layout from
lost buffers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016DYGa1i8HDv497ewpWgaCM
The privacy doc still called the pinned_mic_*_bucket fields meeting-only
and didn't list input_channels or input_rate_hz on the dictation fallback.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016DYGa1i8HDv497ewpWgaCM
DictationInputDeviceSelectionPolicy now defaults to
PinnedDictationSpeedPath, which needs DictationEmptyTranscriptionReason
from ParakeetShortAudioGate. The smoke's hand-kept source list had
neither, so spm-tests failed with "cannot find in scope".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016DYGa1i8HDv497ewpWgaCM
@r3dbars
r3dbars marked this pull request as ready for review September 27, 2026 13:12
@r3dbars
r3dbars merged commit 4837f50 into main Sep 27, 2026
8 checks passed
@r3dbars
r3dbars deleted the claude/transcription-speed-percentiles-yyabey branch September 27, 2026 13:12
r3dbars added a commit that referenced this pull request Sep 28, 2026
Keeps #1880's speed-path fallback and this branch's warmup rule. Warmup
now asks the same question the recorder does, including whether empty
takes moved the mic back to the engine, so that mic's engine stays warm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants