Skip to content

feat(spike): voice dictation + photo capture integration for iOS 26 - #8

Merged
obvious-autobuild[bot] merged 1 commit into
masterfrom
spikes/voice-photo
Sep 17, 2026
Merged

obvious-autobuild[bot] merged 1 commit into
masterfrom
spikes/voice-photo

Conversation

@obvious-autobuild

@obvious-autobuild obvious-autobuild Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Human author: Gilbert Polanco (gilbertpolanco42@gmail.com)

Why

The T3 dictation findings (art_tXjksHOh) recommended adopting @react-native-ai/apple for on-device record-then-transcribe and expo-image-picker + Convex storage for photos, but only after proving the dependency set installs coherently, the native surface is shaped correctly, and the flow is unit-testable without an iOS 26 device. This spike delivers that proof — isolated under spikes/voice-photo/ so arena candidates are untouched.

What

  • Pinned dependency set (all versions retrieved from npm this session): @react-native-ai/apple@0.12.0 (verified MIT TurboModule, peer react-native >= 0.76), Expo SDK 57 line (expo@57.0.23, expo-audio@57.0.5, expo-file-system@57.0.7, expo-image-picker@57.0.18), react-native@0.86.3 + react@19.2.3 (New Architecture), effect@4.0.0-rc.115 (the contract's pin), convex@1.46.0. Exact pins and rationale in the README table. t3code's 194-line pnpm patch is deliberately not adopted (test unpatched first, per the findings).
  • API-surface correction: the findings described isAvailable() as no-arg; the actual 0.12.0 surface (read from the package's own .d.ts) is isAvailable(language: string): boolean, prepare(language), transcribe(ArrayBufferLike, language) → { segments, duration }. The adapter mirrors this structurally.
  • VoiceTranscriber port (src/transcription/): typed error taxonomy (unavailable / unsupported-locale / preparation-failed / transcription-failed / cancelled), AbortSignal cancellation, live Apple adapter with preserved cause wrapping, deterministic FakeVoiceTranscriber, and platform wiring where Android/web resolve to null (UI hides dictation) — the graceful-degradation shape from the findings.
  • Photo → Convex flow (src/photo/): the 3-request pattern from docs.convex.dev (generateUploadUrl mutation → POST bytes → commit mutation) with an injected transport, so the whole flow is unit-testable without a backend. A reference convex/photoStorage.ts module shows contract-mapped validators; production tables/auth scoping belong to the future app scaffold.
  • Contract alignment: src/contract/schema.ts is a spike-local mock of the published contract art_I2TCG08V (the contract's own swap-the-import rule; packages/domain does not exist yet). Wire conversion goes through the contract's decodeUnknownSync/encodeSync. No photo↔Entry linkage field was added — the v0.1 contract has none and the spike may not extend it (flagged for the contract thread).
  • Docs: Expo config wiring (plugin props verified from the pinned packages' own plugin sources), custom dev-client build note (Expo Go will not work — Swift pod), MIT attribution in NOTICE.md for adapted t3code shapes, and a 10-item device-proof checklist.

How to Review

Key files: src/transcription/apple.ts (adapter), src/photo/photoStorage.ts (upload flow), convex/photoStorage.ts (reference validators), src/contract/schema.ts (contract mock), README.md (pins, config, checklist).

Intentionally excluded: live streaming transcription (batch-only MVP per the findings), the contract extension proposal (goes to the contract thread), the monorepo app scaffold, and any CI changes outside spikes/voice-photo/ — CI here is the repo's existing PR pipeline running typecheck/tests on the spike package.

Test Evidence

  • vitest run: 38/38 green (transcription adapter + fake + cancellation, contract decode/encode, photo upload ordering, wire conversion, failure wrapping).
  • tsc --noEmit: strict-mode clean, including the live adapter compiled against the real package types and the convex reference module.
  • npm ci from the committed lockfile: 520 packages, no conflicts (the CI-equivalent local gate).

Device-proof gap (no device evidence claimed)

The sandbox cannot run an iOS 26 device, a custom dev-client build, real microphone/photo permission prompts, or a live Convex deployment. The README carries an explicit 10-item device-proof checklist stating what must be verified on hardware before this architecture ships — none of it is claimed as verified here.

🔗 Obvious Project · 🧵 Obvious Thread

CI note (pre-existing, not introduced here)

The Typecheck, test, build check fails at Install dependencies with ERR_PNPM_NO_LOCKFILE. This is structural on the base branch, not caused by this PR: the workflow's own header says the package-level steps "require the monorepo scaffold (root package.json, pnpm-lock.yaml, turbo tasks) which lands via feat/monorepo-scaffold. Until that PR merges and this branch is rebased, those steps are expected to fail here." Verified this session: the base tip (ecf4018) has no root pnpm-lock.yaml, and this PR's diff touches zero files outside spikes/voice-photo/ (the spike carries its own npm lockfile and passes its own gates locally: npm ci → 520 packages, 38/38 tests, strict tsc clean). Once the scaffold merges, this branch will rebase so the check runs green here too.

@obvious-autobuild
obvious-autobuild Bot marked this pull request as ready for review September 17, 2026 17:53
Integration spike from the T3 dictation findings (art_tXjksHOh):

- Pinned SDK 57-line dependency set (expo 57.0.23, expo-audio 57.0.5,
  expo-file-system 57.0.7, expo-image-picker 57.0.18, react-native
  0.86.3, @react-native-ai/apple 0.12.0, effect 4.0.0-rc.115,
  convex 1.46.0)
- VoiceTranscriber port with typed error taxonomy, AbortSignal
  cancellation, live AppleSpeechAnalyzer adapter, and a deterministic
  device-free fake; Android/web degrade to null
- Photo capture -> Convex 3-request storage flow with injected
  transport, validated against the published Effect v4 schema contract
  (spike-local mock of art_I2TCG08V until packages/domain lands)
- Reference Convex module with contract-mapped validators
- Expo config wiring, dev-client build note, NOTICE attribution
  (MIT), and a device-proof checklist — no device evidence claimed

38/38 vitest tests green; strict tsc clean on the committed tree.

Co-authored-by: Gilbert Polanco <gilbertpolanco42@gmail.com>
@obvious-autobuild
obvious-autobuild Bot merged commit 01c7c0d into master Sep 17, 2026
1 check passed
obvious-autobuild Bot pushed a commit that referenced this pull request Sep 17, 2026
… driver, and macOS runbook

- apps/mobile: pinned native deps (react-native-ai/apple 0.12.0, expo-audio 57.0.5,
  expo-file-system 57.0.7, expo-image-picker 57.0.18), permission plugins +
  speech usage string, EXPO_PUBLIC_QA_PROOF proof-screen toggle
- apps/mobile/src/proof: VoicePhotoProof checklist surface + Convex HTTP transport
  reusing the spike's real storePhotoCapture flow
- spikes/voice-photo/qa-convex: throwaway schema + reference-module deployment set
- spikes/voice-photo/qa: real-backend evidence driver (3-request flow, SHA-256
  read-back, entry-commit negative probe)
- scripts/qa-ios-sim-proof.sh: repeatable macOS simulator runbook
- spike types.ts: AbortSignal.reason lib-compat cast (38/38 tests pass)
- verified: app strict typecheck clean, vitest 38/38, expo export ios 896 modules
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants