Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 52 additions & 15 deletions docs/desktop-roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,37 +104,42 @@ audio delivery after TCP loss and wrong-process rejection with zero secret bytes
received. This checks the originally attributed process and current path/file
identity; it is not historical loaded-image attestation.

There is no app entry point. Actual OBS enrollment, the original server plugin's
There is no live-capture app entry point. Actual OBS enrollment, the original server plugin's
restrictive DACL/client authentication, atomic arming/start coordination,
controller/UI, live recognition and streaming-load acceptance remain open.
The original C-only [development module](../native/obs-plugin/README.md) now
builds against 39 verified OBS 32.2.2 public resources. It opens, initializes and
unloads through the installed libobs runtime in an isolated fixture without
starting the OBS application or creating audio sources. This establishes only
the inert build/load prerequisite, merged through PR #32 at `e61c7dd`.
The original C-only [development module](../native/obs-plugin/README.md) first
established the inert build/load prerequisite through PR #32 at `e61c7dd`.
The current linked module builds against 41 pinned OBS/frontend/vendor public
resources and deliberately refuses headless initialization before opening its
pairing store. The isolated fixture starts no OBS application or audio sources.
The separate [native session components](plans/active/obs-native-session.md) now
pass fixed Hello/ACK and CNG-failure checks, 11 actual child-process transport
tests, current-logon DACL/noninheritance checks and 64 create/destroy handle
balance cycles. A focused desktop boundary/pipe regression passes 54 tests.
Independent admission source/test/evidence review is clear; PR #33 merged at
`006d482` after all five exact-source desktop CI jobs passed. The next
[enrollment step](plans/active/obs-native-enrollment.md) accounts for the public
vendor API's lack of caller authentication context. These components are not
linked into the module; vendor authorization/enrollment, atomic arming and PCM capture remain
unimplemented. The enrollment branch now implements independent capability
vendor API's lack of caller authentication context. That earlier increment kept
the session components separate; the current linked increment is described below.
The enrollment branch implements independent capability
proof and native admission ownership: 30 client tests, 10 native child-process
cases and six state/fault groups pass, including expiry, replay, revocation and
concurrent preparation. The focused desktop regression passes 177 tests; final
independent source/evidence review is clear. Pairing-file UI and the vendor
adapter remain to be implemented before any app entry point is exposed.
independent source/evidence review is clear. These historical component checks
precede the native pairing/vendor integration described below.
PR #34 merged at `9260e6b` after all five
[exact-source desktop CI jobs](https://github.com/RioPlay/utterleaf/actions/runs/34754090586)
passed at `d44126a`. The `feat/obs-pairing-store` increment now implements the
passed at `d44126a`. The `feat/obs-pairing-store` increment, merged through
PR #35 at `3bd072d` from exact source head `c7f86f6`, now implements the
selected CurrentUser DPAPI package, separate native/desktop stores, native export,
explicit desktop import/replacement and separate forget behavior. The first
local regression passes 292 tests, including 48 pairing cases, plus the native
store state/fault and five cross-language checks, including an actual junction.
Independent source/test and final receipt review is clear.
local regression passes 292 tests, including 48 pairing cases, plus 22 native
verification commands, 57 source/artifact hashes, five interop cases and five
store state/fault groups, including an actual junction. Exact-head CI
[34755748029](https://github.com/RioPlay/utterleaf/actions/runs/34755748029)
passes all five desktop jobs; release publication was skipped. Independent
source/test and final receipt review is clear. Three known warnings remain
isolated to the test heap shim.
The enrollment plan distinguishes these
storage results from pending UI, live revocation, vendor and audio integration.
Cross-user/logon, remote clients, forced PID reuse and kernel
Expand All @@ -145,6 +150,38 @@ binary is included in desktop or Android releases. The
[control dependency record](desktop-obs-control-resource.md) records the pinned
library, reviewed full license and development-wheel provenance.

Continue on `feat/obs-enrollment-flow` with frontend acceptance and the desktop
connection/controller after the pairing setup increment. Arm/PCM and live
recognition remain later gates. No live-capture app entry point, audio endpoint, plugin binary
publication or OBS capture integration exists yet.

The linked native pairing Tools flow, exclusive per-user owner and strict
vendor Issue/Prepare dispatch are now present in the development module.
All 28 native driver commands pass, including lifecycle/dispatch races, actual
owner competition, shimmed bridge lifecycle and six real-libobs parser cases.
The linked DLL builds and headless refusal passes with unchanged fixed-store
metadata. Frontend UI interaction and real OBS acceptance remain unverified.
The typed desktop adapter sends only the defined Utterleaf Issue/Prepare
requests after explicit invocation, verifies challenge binding and the OBS peer
before proof, and rejects concurrent operations. The focused bundle passes
**341 tests with no skips**, including **66** enrollment tests and an ephemeral
loopback exchange. Independent source review is clear; native identity is stubbed
in that new exchange fixture. This prepares a session ID and does not connect
the audio pipe or arm capture. Native owner/UI/vendor integration remains
unverified for real OBS/device acceptance.

The [desktop pairing dialog](plans/active/obs-desktop-pairing-ui.md) now opens from
Speech & privacy on Windows, with explicit import/replace/forget, a separate
per-user store owner and asynchronous cancellation/teardown. Saved status never
claims an OBS connection. The related bundle passes 204 tests; all 23 UI cases
pass after the final status-card adjustment. Native TaskDialog activation,
Escape, modal cleanup and zero-mutation checks also pass in an isolated fixture;
all 29 native driver commands pass with `--ui`. Normal/compact/enlarged desktop
renders were inspected. Independent final desktop setup review is clear, with
29 focused and 187 related tests rerun on the final source. Its own CI remains
pending; no existing release changes. Earlier linked checkpoint `15e1c77` passed
all five desktop CI jobs in [34758907354](https://github.com/RioPlay/utterleaf/actions/runs/34758907354).

Current integrated desktop regression: **1,432 passed, 13 skipped in 39.47 seconds**,
including the reviewed OBS control, native identity, pipe and audio handshake.
The combined focused OBS/pipe/privacy/configuration/boundary/packaging bundle
Expand Down
13 changes: 11 additions & 2 deletions docs/plans/active/obs-audio-implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ in source; its physical microphone and release gates remain separate.

## Current increment

The [enrollment plan](obs-native-enrollment.md) now records the integrated private
pairing stores and the typed desktop Issue/Prepare adapter on
`feat/obs-enrollment-flow`. Its focused regression passes 341 tests with no skips,
including 66 enrollment cases. Native owner, Tools and vendor dispatch are still
pending; no app entry point or capture activation is exposed.

The bounded audio protocol, consent/session receiver and read-only loopback
WebSocket control client are reviewed. The Windows TCP peer identity gate is now
implemented and integrated, as recorded below. The separate original C module now
Expand Down Expand Up @@ -176,8 +182,11 @@ not a hard real-time scheduling guarantee.
`obs_control.py` requires a password-protected Hello, computes OBS's documented
challenge response and waits for RPC 1 identification before requesting version
information. The response must advertise the required read-only requests and a
5.x WebSocket version. Only `GetVersion` and `GetStreamStatus` are allowed; there
is no arbitrary request passthrough. The General/Outputs subscription mask is 65.
5.x WebSocket version. That initial component allowed only `GetVersion` and
`GetStreamStatus`. The later enrollment adapter also permits the exact Utterleaf
Issue/Prepare schemas on explicit invocation, gated by advertised
`CallVendorRequest` support. There is no arbitrary request passthrough. The
General/Outputs subscription mask is 65.
Only stream lifecycle events are retained, with a 32-event limit; shutdown closes
the control connection. Unrelated event payloads, including recording paths, are
discarded. Events arriving after Identify but before Identified stay quarantined
Expand Down
126 changes: 126 additions & 0 deletions docs/plans/active/obs-desktop-pairing-ui.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Desktop OBS pairing setup

September 13, 2026. Continue `feat/obs-enrollment-flow` after the reviewed native
checkpoint `15e1c77` in draft PR #36. The full OBS audio goal remains in the
[enrollment](obs-native-enrollment.md) and [audio](obs-audio-design.md) plans.

## Goal and area

Let Windows users explicitly import, replace, inspect and forget their desktop
OBS pairing from Speech & privacy, in one separate compact dialog. Preserve the
five Settings destinations. Display a saved local pairing as saved, never as a
verified connection or live transcription readiness. The native plugin remains
an unpublished development component; the dialog must state that live OBS
transcription is still in development.

Area: new `utterleaf/obs_pairing_ui.py`, the existing Settings entry/close path,
desktop store owner support, focused store/UI tests and corresponding docs.
Native TaskDialog acceptance runs independently against synthetic state, with
no consumer OBS instance or store.

## Constraints and observable acceptance

- Opening Settings does not open a pairing store. Only explicitly opening the
Windows pairing dialog may create its private directories/owner sentinel.
The dialog worker claims a protected noninheritable share-zero owner file in
the fixed desktop pairing directory, independent of configuration profiles.
No automatic repair, key generation, network, capture or model loading occurs.
- Keep storage/key work on one joined or retained non-daemon worker; Tk stays on
its owner thread. Hold the store/owner until worker teardown. Queue only safe
outcomes and booleans; wipe loaded capability buffers, never display/log/copy
keys or raw exception/native path data. Imports use the existing validated
store; no format reimplementation.
- File selection and confirmation precede mutation. Initial import and explicit
replacement are distinct; explain that verified import removes the selected
transfer file when possible. Report saved-but-transfer-remains separately.
Cancellation before commit preserves prior pairing and transfer; cancellation
after commit must report saved state. Postcommit uncertainty requires explicit
status reload and must not claim rollback.
- Forget confirms removal of this desktop copy only, explains that copies remain
authorized until OBS Forget/Replace, and preserves preferences, models and
transcripts. Failed deletion does not claim success. Corrupt/inaccessible
state stays an error; allow explicit refresh/forget, not silent replacement.
- One child dialog per Settings window. Escape, window close and parent close
signal cancellation and wait asynchronously for worker teardown. If close
overlaps a committed/uncertain mutation, keep its outcome visible for Close
acknowledgement. Do not destroy the parent with a live pairing worker.
- Use the existing theme, readable status/actions, responsive wrapping and
keyboard focus. Test normal/compact sizes and enlarged fonts with rendered
fixtures. Physical scaling/assistive-technology and actual OBS UI acceptance
remain separate.

## Verification

From the owning worktree, use the original `.venv/Scripts/python.exe` with
`PYTHONPATH` set to this worktree. Run focused pairing-owner/store/UI and Settings
tests first, then affected privacy/configuration/boundary checks. Use disposable
Windows roots and synthetic capabilities for persistence, second-process owner,
cancel/reset/forget and real-Tk integration; no consumer pairing or microphone.
Capture the dialog in normal/compact/error states and inspect the actual images.
Independent review reads source, diff, contract and test/render evidence.

## Implemented source and evidence

`ObsPairingDialog` now opens lazily from Speech & privacy on Windows and retains
one worker/store owner until close. Normal and compact dark layouts keep status
and cancellation together in the top card; a scrollable body and fixed footer
preserve access with longer messages and enlarged text. Unexpected operation
failures mark the saved state unknown and require explicit Refresh; only a typed
precommit cancellation claims that the old pairing was preserved. Parent close
waits for storage teardown and leaves committed outcomes visible for acknowledgement.

The desktop store now exposes `claim_owner()` using the protected, noninheritable
share-zero `obs-pairing-owner-v1.lock`. Its six real Windows tests include actual
child-process contention/succession and noninheritance, plus unsafe ACL, nonempty
sentinel and reparse refusal. The larger owner/storage/privacy bundle passes 125
tests. The integrated ten-module setup/settings/store/privacy/configuration/backup/
boundary bundle passes **204 tests with no skips in 33.28 seconds**. After the
final unknown-state and status-card adjustment, all **23 UI tests** pass again
in 5.14 seconds. Tests include disposable actual DPAPI import, dialog reopen and
Forget while preserving unrelated files, worker cancellation on both sides of
commit, parent-close waiting, startup failure, lazy/singleton entry and Settings
reset/discard isolation. Test fixtures that render transient dialogs map their
parent before measuring geometry; hidden-window dimensions are not layout evidence.

Exact local commands from the owning worktree:

```powershell
$py = "C:\Users\unknown\Projects\Mindict\.venv\Scripts\python.exe"
$env:PYTHONPATH = (Get-Location).Path
& $py -m pytest tests/test_obs_pairing_ui.py tests/test_obs_pairing_owner.py tests/test_obs_pairing_store.py tests/test_settings_ui.py tests/test_settings.py tests/test_config.py tests/test_privacy.py tests/test_backup.py tests/test_backup_store.py tests/test_repo_boundaries.py -o addopts='' -q
& $py -m pytest tests/test_obs_pairing_ui.py -o addopts='' -q
& $py tests/capture_settings.py
& $py native/obs-plugin/tools/test_native.py --toolchain "C:\Users\unknown\.local\llvm-mingw-20260616-ucrt-x86_64" --output "C:\Users\unknown\Projects\Mindict\.grok\obs-enrollment-flow\setup-native-final" --build "C:\Users\unknown\Projects\Mindict\.grok\obs-enrollment-flow\build-a" --headers "C:\Users\unknown\Projects\Mindict\.grok\obs-native-build\headers" --ui
```

Settings capture passes. The actual desktop dialog captures at
`.grok/obs-enrollment-flow/desktop-ui-captures/` cover unpaired, paired, compact
uncertain state and enlarged text; root inspected normal/compact/enlarged images.
The revised error card keeps its explanation visible instead of placing it below
the setup instructions. Native dialog acceptance passes through real Windows
TaskDialog activation and targeted Escape, with three fixture captures and no
mutations or OBS startup; the native driver passes 29 top-level commands. Three
existing test-only heap-shim link warnings remain. Native and desktop fixture
checks do not establish actual OBS integration, physical scaling/input or assistive
technology. Final independent desktop source/test/render review is clear. The
reviewer reran 29 focused UI/owner tests and a 187-test related bundle against
the final source, and directly checked mapped modality, Escape and an ambiguous
post-mutation close result. The latter probes supplement the committed tests;
they do not establish physical assistive-technology acceptance. Separate native
fixture/driver review is also clear, including all three captures and current
source/artifact/nested-receipt hashes. The final combined local evidence is
`.grok/obs-enrollment-flow/desktop-setup-verification.json`.

CI [34758907354](https://github.com/RioPlay/utterleaf/actions/runs/34758907354) passes
all five desktop jobs at `15e1c776821abf14abfe6bec7786c8c133d867f7`, with release
skipped. That is the earlier linked native checkpoint; it does not validate this
desktop setup follow-up; its own CI remains pending.

## Non-goals and stop

No Android edits, plugin installation/publication, OBS launch, connection/Arm,
audio pipe, PCM, live recognition, new Settings preference or global redesign in
this step. Those remain required subsequent components, not removed scope.
Stop changing this setup increment when the named checks and independent review
pass; preserve its evidence, then continue real frontend/connection/Arm/audio
integration rather than declaring live OBS complete.
Loading
Loading