Skip to content

feat: add reliable local voice dictation - #14

Merged
yaacovcorcos merged 5 commits into
mainfrom
feat/scient-voice-local-20260807
Aug 8, 2026
Merged

yaacovcorcos merged 5 commits into
mainfrom
feat/scient-voice-local-20260807

Conversation

@yaacovcorcos

Copy link
Copy Markdown
Contributor

Summary

  • add private, offline voice dictation to the Scient composer using a packaged whisper.cpp runtime
  • keep recording, model management, transcription, IPC, and UI responsibilities separated behind Scient-owned seams
  • preserve Claude's original prototype commit, followed by the reliability hardening and final targeted fixes
  • provide record, cancel, transcribe-and-insert, and transcribe-and-send interactions with a responsive waveform and concise tooltips

Why

Scientists should be able to dictate prompts without sending microphone audio or credentials to a cloud transcription service. The implementation also needs to remain narrowly integrated with the T3-derived composer so future upstream updates stay manageable.

Reliability and safety

  • guards recorder lifecycle ownership so stale cleanup cannot stop a newer recording
  • flushes final audio, aborts stale work, and waits deterministically for the native helper to exit
  • stages a pinned and checksummed whisper.cpp runtime without committing generated binaries or models
  • gives the child process an explicit environment allowlist
  • makes model setup cancellable and resumable
  • replaces model receipts atomically and uses bounded user-facing errors
  • keeps live transcription out of this first reliable release by explicit product decision

User impact

Users can record speech locally, cancel safely, and either insert the transcription into the composer or send it directly. The composer remains editable while recording. The first model setup can be cancelled and resumed later.

Validation

  • manual macOS microphone and visual review approved
  • formatting and focused lint passed
  • typechecks passed for scient-voice, contracts, web, and desktop
  • 2,615 affected-package tests passed
  • web production build passed
  • git diff --check passed

Limitations

  • this first release transcribes after recording stops; it does not stream live partial transcription
  • transcription quality follows the bundled small multilingual model and can be revisited independently

Preserve Claude's source implementation as a discrete historical checkpoint before architectural hardening. Local wedged runtime state and the unverified loose Whisper executable are intentionally excluded; reproducible runtime staging will replace the latter.
Preserve Claude's prototype as the parent commit while simplifying the product path, tightening capture and cancellation lifecycles, bounding IPC and audio validation, and packaging a pinned verified whisper.cpp runtime behind narrow Scient-owned adapters.
@yaacovcorcos
yaacovcorcos force-pushed the feat/scient-voice-local-20260807 branch from 66598e7 to f1fcd23 Compare August 8, 2026 08:56
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL labels Aug 8, 2026
@github-actions

github-actions Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 11.3 KiB 11.3 KiB −17 B (−0.1%) 15.1 KiB ✅
Codex Thread snapshot wire 5.5 KiB 5.5 KiB +3 B (+0.1%) 7.3 KiB ✅
Codex Live turn WebSocket wire 5.9 KiB 5.9 KiB −20 B (−0.3%) 7.8 KiB ✅
Codex Live turn WebSocket decoded 49.7 KiB 49.7 KiB 0 B (0.0%) 66.4 KiB ✅
Codex Live turn messages 16 16 0 (0.0%) 21 ✅
Claude Total thread wire 11.3 KiB 11.3 KiB +5 B (+0.0%) 15.1 KiB ✅
Claude Thread snapshot wire 5.5 KiB 5.5 KiB −3 B (−0.1%) 7.3 KiB ✅
Claude Live turn WebSocket wire 5.8 KiB 5.9 KiB +8 B (+0.1%) 7.8 KiB ✅
Claude Live turn WebSocket decoded 50.6 KiB 50.6 KiB 0 B (0.0%) 66.4 KiB ✅
Claude Live turn messages 16 16 0 (0.0%) 21 ✅

Baseline: 96de3c3 · PR result: 324b183 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 94.6 KiB
  • Claude decoded thread snapshot: 95.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@yaacovcorcos
yaacovcorcos marked this pull request as ready for review August 8, 2026 10:47
@yaacovcorcos
yaacovcorcos merged commit 3118448 into main Aug 8, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant