feat(spike): voice dictation + photo capture integration for iOS 26 - #8
Merged
Merged
Conversation
Integration spike from the T3 dictation findings (art_tXjksHOh): - Pinned SDK 57-line dependency set (expo 57.0.23, expo-audio 57.0.5, expo-file-system 57.0.7, expo-image-picker 57.0.18, react-native 0.86.3, @react-native-ai/apple 0.12.0, effect 4.0.0-rc.115, convex 1.46.0) - VoiceTranscriber port with typed error taxonomy, AbortSignal cancellation, live AppleSpeechAnalyzer adapter, and a deterministic device-free fake; Android/web degrade to null - Photo capture -> Convex 3-request storage flow with injected transport, validated against the published Effect v4 schema contract (spike-local mock of art_I2TCG08V until packages/domain lands) - Reference Convex module with contract-mapped validators - Expo config wiring, dev-client build note, NOTICE attribution (MIT), and a device-proof checklist — no device evidence claimed 38/38 vitest tests green; strict tsc clean on the committed tree. Co-authored-by: Gilbert Polanco <gilbertpolanco42@gmail.com>
obvious-autobuild
Bot
force-pushed
the
spikes/voice-photo
branch
from
September 17, 2026 18:06
8e413e8 to
59ff6cc
Compare
obvious-autobuild Bot
pushed a commit
that referenced
this pull request
Sep 17, 2026
… driver, and macOS runbook - apps/mobile: pinned native deps (react-native-ai/apple 0.12.0, expo-audio 57.0.5, expo-file-system 57.0.7, expo-image-picker 57.0.18), permission plugins + speech usage string, EXPO_PUBLIC_QA_PROOF proof-screen toggle - apps/mobile/src/proof: VoicePhotoProof checklist surface + Convex HTTP transport reusing the spike's real storePhotoCapture flow - spikes/voice-photo/qa-convex: throwaway schema + reference-module deployment set - spikes/voice-photo/qa: real-backend evidence driver (3-request flow, SHA-256 read-back, entry-commit negative probe) - scripts/qa-ios-sim-proof.sh: repeatable macOS simulator runbook - spike types.ts: AbortSignal.reason lib-compat cast (38/38 tests pass) - verified: app strict typecheck clean, vitest 38/38, expo export ios 896 modules
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human author: Gilbert Polanco (gilbertpolanco42@gmail.com)
Why
The T3 dictation findings (
art_tXjksHOh) recommended adopting@react-native-ai/applefor on-device record-then-transcribe and expo-image-picker + Convex storage for photos, but only after proving the dependency set installs coherently, the native surface is shaped correctly, and the flow is unit-testable without an iOS 26 device. This spike delivers that proof — isolated underspikes/voice-photo/so arena candidates are untouched.What
@react-native-ai/apple@0.12.0(verified MIT TurboModule, peerreact-native >= 0.76), Expo SDK 57 line (expo@57.0.23,expo-audio@57.0.5,expo-file-system@57.0.7,expo-image-picker@57.0.18),react-native@0.86.3+react@19.2.3(New Architecture),effect@4.0.0-rc.115(the contract's pin),convex@1.46.0. Exact pins and rationale in the README table. t3code's 194-line pnpm patch is deliberately not adopted (test unpatched first, per the findings).isAvailable()as no-arg; the actual0.12.0surface (read from the package's own.d.ts) isisAvailable(language: string): boolean,prepare(language),transcribe(ArrayBufferLike, language) → { segments, duration }. The adapter mirrors this structurally.src/transcription/): typed error taxonomy (unavailable/unsupported-locale/preparation-failed/transcription-failed/cancelled), AbortSignal cancellation, live Apple adapter with preservedcausewrapping, deterministicFakeVoiceTranscriber, and platform wiring where Android/web resolve tonull(UI hides dictation) — the graceful-degradation shape from the findings.src/photo/): the 3-request pattern from docs.convex.dev (generateUploadUrl mutation → POST bytes → commit mutation) with an injected transport, so the whole flow is unit-testable without a backend. A referenceconvex/photoStorage.tsmodule shows contract-mapped validators; production tables/auth scoping belong to the future app scaffold.src/contract/schema.tsis a spike-local mock of the published contractart_I2TCG08V(the contract's own swap-the-import rule;packages/domaindoes not exist yet). Wire conversion goes through the contract'sdecodeUnknownSync/encodeSync. No photo↔Entry linkage field was added — the v0.1 contract has none and the spike may not extend it (flagged for the contract thread).NOTICE.mdfor adapted t3code shapes, and a 10-item device-proof checklist.How to Review
Key files:
src/transcription/apple.ts(adapter),src/photo/photoStorage.ts(upload flow),convex/photoStorage.ts(reference validators),src/contract/schema.ts(contract mock),README.md(pins, config, checklist).Intentionally excluded: live streaming transcription (batch-only MVP per the findings), the contract extension proposal (goes to the contract thread), the monorepo app scaffold, and any CI changes outside
spikes/voice-photo/— CI here is the repo's existing PR pipeline running typecheck/tests on the spike package.Test Evidence
vitest run: 38/38 green (transcription adapter + fake + cancellation, contract decode/encode, photo upload ordering, wire conversion, failure wrapping).tsc --noEmit: strict-mode clean, including the live adapter compiled against the real package types and the convex reference module.npm cifrom the committed lockfile: 520 packages, no conflicts (the CI-equivalent local gate).Device-proof gap (no device evidence claimed)
The sandbox cannot run an iOS 26 device, a custom dev-client build, real microphone/photo permission prompts, or a live Convex deployment. The README carries an explicit 10-item device-proof checklist stating what must be verified on hardware before this architecture ships — none of it is claimed as verified here.
🔗 Obvious Project · 🧵 Obvious Thread
CI note (pre-existing, not introduced here)
The
Typecheck, test, buildcheck fails at Install dependencies withERR_PNPM_NO_LOCKFILE. This is structural on the base branch, not caused by this PR: the workflow's own header says the package-level steps "require the monorepo scaffold (root package.json, pnpm-lock.yaml, turbo tasks) which lands via feat/monorepo-scaffold. Until that PR merges and this branch is rebased, those steps are expected to fail here." Verified this session: the base tip (ecf4018) has no rootpnpm-lock.yaml, and this PR's diff touches zero files outsidespikes/voice-photo/(the spike carries its own npm lockfile and passes its own gates locally:npm ci→ 520 packages, 38/38 tests, strict tsc clean). Once the scaffold merges, this branch will rebase so the check runs green here too.