feat(voice): GPT Live voice agent on Orchestrator V2

[saphid]

What Changed

Adds GPT Live voice to Orchestrator V2. You talk to T3 Code, and the voice agent lists, opens, reads, starts and follows up threads through the V2 orchestration APIs. The agent runs on OpenAI Live (Realtime over WebRTC).

• Server: a voice broker under /api/voice mints Live sessions with the environment's stored OpenAI key, and also handles session close, usage, and the Responses delegation backend. Every route requires orchestration:operate, and settings also require access:write. The key never leaves the server. The environment descriptor advertises voiceLive only while a key is configured, so clients show voice only where it works.
• Contracts: packages/contracts/src/voice.ts holds the broker and tool schemas.
• Web and desktop: the Live client, and a tool executor built on V2 (message.dispatch with a deterministic commandId, run-status readouts from thread projections). Also navigation, a voice history, an overlay panel (corner button, expand, minimize), and a Settings → Integrations → Voice section for the key and models.
• Docs: docs/user/voice*.md and docs/internals/voice-live.md.

This is the V2 counterpart of #12034, which targets main.

Why

Voice lets you steer threads hands-free: check on a worker, open a thread, or start a follow-up without leaving what you're doing. Porting it to V2 meant replacing the V1 session-status and turn model with V2 runs. Examples: delivery is keyed by run id, a follow-up readback detects a stale run by run identity, and dispatch mode comes from the active run's provider capabilities.

Surfaces: web and desktop. Mobile is not supported yet; the overlay only mounts in the web client. The voice agent drives threads through orchestration commands, so it works with every provider.

Relation to #13168

#13168 adds per-thread voice conversations on Codex's built-in realtime voice. The two are complementary: #13168 lets you talk to one thread's Codex session, and this PR adds an app-level agent that works across threads, projects and environments. Codex realtime can't host this agent's tools, because app-server realtime has no custom tool registry, so the agent stays on OpenAI Live.

Once #13168 lands, I plan to rebuild this PR on top of its transport. The agent would become a second session kind on its streaming voice-session RPC and shared client controller. That gives one voice stack, one entry point and one Settings section, and drops this PR's separate HTTP session endpoints and WebRTC handling. Until then, this PR stands on its own.

UI Changes

Before (V2 base without this PR): no voice entry point, and no Voice settings.

After (this PR): a Voice button appears in the corner and opens the panel, which minimizes back to the button. Settings → Integrations gains a Voice section with activation, fast commands, the key status, and the models.

Still captures

Before: landing view without a voice button.

After: landing view with the Voice button in the bottom-right corner.

After: the expanded panel, idle.

Before: Settings → Integrations.

After: Settings → Integrations with the Voice section.

Capture conditions: the GIFs and stills were recorded on the previous V2 head d6dcd10, before the branch was rebased. At that verification point, the voice diff was byte-identical on the rebased branch, and I rechecked it there: fresh panel and Settings stills on f876a47 match the ones below. Headless Chrome at 1280×800, both revisions against their own isolated dev home seeded with two synthetic projects. The candidate home holds an OpenAI key, which is what makes voiceLive true. GIFs are real-time recordings at 10 fps. Each includes a full dev-mode page load when navigating to Settings, and a provider update toast dismissed with its own close button. The landing project differs between the two runs because of prior navigation state, which is unrelated to this change.

Verification

• Live session (ea97483 on base f876a47): I used headless Chrome with a fake microphone. Connect went live in 2.9 s, End reached "Ended" 0.8 s later, and Connect was re-enabled, with no page errors. POST /api/voice/sessions and POST /api/voice/sessions/close both returned 200 against the real OpenAI Live API. The same check on the previous base took 5.9 s to go live. The historical head 05d8dde differed from ea97483 only by the knip, import-order and test-dependency cleanup below, with no runtime behavior change.
• Settings persistence: changed Activation to "Hold to talk" and toggled Fast commands, reloaded, and both values held. I restored them afterward.
• Head 05d8dde on base 67a2be0: knip:check, covering files, dependencies and all workspace exports, is clean. The broker's secret name moved into its own leaf module (voice/secretNames.ts), so the environment descriptor no longer imports the broker; that import created a module cycle that broke the server memory tests. The PR adds no dependencies: the only jsdom-based voice test was removed, since nothing else in the repo uses a DOM test environment. The DOM control adapter keeps its logic tests in domControls.test.ts.
• Typecheck: contracts, server and web → 0 errors.
• Web: vp test run src/voice src/missingThreadRedirects.test.ts src/components/settings → 865 passed. The full web suite (437 files, 5,858 tests) also passed locally on the previous head.
• Server: vp test run src/voice src/environment/ServerEnvironment.test.ts src/orchestration-v2/ProviderTurnStartService.memory.test.ts src/rpcInitialItems.memory.test.ts → 41 passed.
• Contracts: vp test run → 524 passed.
• Lint and format: no lint errors in the touched voice files; vp fmt --check is clean.
• Review: two rounds of cross-vendor review (GPT-6 Astra, high). No security findings. The correctness findings were fixed with regression tests: releasing the mic and transport when a session ends on its own, a bounded wait for session.closed with local WebRTC closed before broker accounting, stale-readback detection by run identity, restored DOM UI controls, and ending the session when the selected broker changes.

Not exercised: a spoken conversation that drives tools (the fake mic only sends a tone), the Electron shell (desktop wraps the same web UI), and remote or tunnel connections, where the browser talks to OpenAI directly and the broker only exchanges SDP.

Known follow-ups: the research bridge's delegate() has no production caller yet, so "start research and report back" launches the thread but records no delivery. Its polling stays dormant until that is wired.

Checklist

• This PR is small and focused. It is one feature, but large: about 21.5k lines across 61 files, most of it apps/web/src/voice with its tests, plus docs. It adds no dependencies.
• I explained what changed and why
• I included before/after screenshots for any UI changes
• I included a video for animation/interaction changes (GIF recordings above)

Model: Claude Opus 5.5 in T3 Code via Claude Code (V2 port fixes, review fixes, verification, PR); GPT-6 Astra via Codex (review); initial V2 port by GLM 5.3 Flash via OpenCode.

🤖 Generated with Claude Code

Latest review-fix verification (6fa193b): vp test run apps/server/src/voice/broker.test.ts apps/web/src/voice/ui/VoiceControls.test.tsx apps/web/src/voice/ui/VoiceHistory.test.tsx passed 44 tests; both UI files were rerun after the final control adjustment (18 passed). pnpm run typecheck passed in apps/server and apps/web. Earlier screenshots and videos predate these control changes; no new browser verification was performed. Earlier head-specific test statements describe their historical runs.

Final settings-error follow-up (fa0ad2da83): vp test run apps/server/src/voice/broker.test.ts passed 27 tests; pnpm run typecheck in apps/server passed. Direct SWE-2 Max review exited 0 with no actionable findings.

[macroscopeapp]

Macroscope skipped reviewing this pull request. Per-review cost limit exceeded (workspace setting).

This review would cost an estimated $17.93, which exceeds your per-review limit of $15.00.

The top 3 files driving up this estimate:

File	Diff Size	Estimate
apps/web/src/voice/tools.ts	76.79KB	$3.07
apps/server/src/voice/broker.ts	38.22KB	$1.53
apps/web/src/voice/history.ts	37.45KB	$1.50

Tip

To get this pull request reviewed, you can:

1. Comment @macroscope-app on this PR to request a manual review (monthly spend limits still apply).
2. Exclude the file(s) above from review by adding a pattern to your .macroscope/ignore.md — note that creating this file replaces Macroscope's built-in default ignores rather than extending them.
3. Raise your cost limit in your workspace billing settings.

^{Turn off this reminder going forward}

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR adds a large GPT Live capability spanning server brokering, secrets, billed OpenAI sessions, WebRTC, authenticated orchestration actions, UI automation, and persistent client history. Its broad runtime and security-sensitive surface, plus product-default changes and a diagnostic suppression, require human review.

Not approved because:

• Per-review cost limit exceeded (workspace setting). Approvability relies on correctness review in order to determine eligibility

^{Review your spending limits in Billing settings, or comment @macroscope-app review this PR to bypass the limit and review now. You can add or adjust custom eligibility rules. Learn more.}

[macroscopeapp]

Additional UI consistency finding in apps/web/src/voice/ui/VoiceHistory.tsx:39: Shared primitives over custom controls: the styled History disclosure bypasses apps/web/src/components/ui/Button. Use that primitive instead of a raw button with local typography/color styling.

Suggested fix: No single-hunk diff; this requires adding the Button import and replacing this control.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Additional UI consistency finding in apps/web/src/voice/ui/VoiceHistory.tsx:55: Shared primitives over custom controls: the styled Delete action bypasses apps/web/src/components/ui/Button. Use that primitive rather than local padding, color, and disabled-state classes.

Suggested fix: No single-hunk diff; this requires adding the Button import and replacing this control.

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

Effect service convention follow-up: apps/server/src/voice/broker.ts:369 and the corresponding layer export use implementation-specific names (makeBroker / VoiceLiveBrokerLive) even though this module owns the sole implementation. Please expose canonical make and layer names.

Suggested fix: No diff — both declarations and consumers need coordinated renaming.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

Effect Service Conventions found 3 blocking issues in apps/server/src/voice/broker.ts: explicit service injection, unsafe/lost error cause handling, and non-canonical service constructor/layer names. Details and expected fixes are provided in the review comments above.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

Correction to the summary above: the detailed review contains two inline findings, while the constructor/layer naming finding is in the immediately preceding PR-level comment.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

Effect Service Conventions found one new blocking issue in apps/server/src/voice/broker.ts: the local ServerSecretStore service boundary is imported directly rather than through its module namespace. The prior unsafe error-detail/cause handling and non-canonical constructor/layer naming findings also remain unresolved.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

Correction: the prior explicit ServerSecretStore injection finding is fixed at the current head; only the prior unsafe error-detail/cause handling and constructor/layer naming findings remain, alongside the new namespace-import finding.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

All clear

^{Posted via Macroscope — UI Consistency}

[macroscopeapp]

The latest change preserves the underlying settings error as cause; no new Effect service convention issues were found. The previously reported makeBroker / VoiceLiveBrokerLive naming issue remains open in apps/server/src/voice/broker.ts. Please use canonical make / layer exports and update consumers.

^{Posted via Macroscope — Effect Service Conventions}