diff --git a/docs/desktop-roadmap.md b/docs/desktop-roadmap.md index 5b5517a..3109496 100644 --- a/docs/desktop-roadmap.md +++ b/docs/desktop-roadmap.md @@ -247,6 +247,26 @@ and headless smoke pass. Nine synthetic UI renders were inspected; independent source review is clear. Its active plan records exact commands and remaining acceptance limits. +The controller source is preserved in draft PR #40 at `a0ad377`; all five desktop +jobs passed in [CI 34774733389](https://github.com/RioPlay/utterleaf/actions/runs/34774733389). +The next [mix provenance and routing-boundary increment](plans/active/obs-mix-provenance.md) +is active on `feat/obs-mix-provenance`. Its bounded assignment model and explicit +version-2 routing codec pass 224 focused tests, native/Python interoperability, +796 affected desktop tests, all 51 native verification commands, the linked build +and headless refusal. Independent component review is clear and 446 recorded +hash comparisons match for codec checkpoint `5183cc2`. +The integration implements a bounded native observer, private routing +history, explicit-v2 receiver/controller transport and compact Mix details tab. +Final verification passes 873 affected desktop tests, 48 Tk checks, all 55 native +commands, the 25-command linked build and headless refusal. Eleven UI captures +were inspected; 488 recorded hash comparisons match the final inputs and outputs. +Independent review is clear after correcting stop/drain, stale refresh and +metadata retirement races, callback scheduling contention and cancellation cleanup. +Production source requires audio version 2 before Arm. Source CI is tracked on +[draft PR #41](https://github.com/RioPlay/utterleaf/pull/41); the integration +receipts are separate from the earlier codec evidence. Live OBS, +export, distribution and application-entry gates remain open. + Current integrated desktop regression: **1,432 passed, 13 skipped in 39.47 seconds**, including the reviewed OBS control, native identity, pipe and audio handshake. The combined focused OBS/pipe/privacy/configuration/boundary/packaging bundle diff --git a/docs/plans/active/obs-mix-provenance.md b/docs/plans/active/obs-mix-provenance.md new file mode 100644 index 0000000..b2efb11 --- /dev/null +++ b/docs/plans/active/obs-mix-provenance.md @@ -0,0 +1,239 @@ +# OBS mix provenance and routing boundaries + +## Goal and area + +Continue the full [OBS audio design](obs-audio-design.md) on +`feat/obs-mix-provenance`, from reviewed controller source `a0ad377` in draft +PR #40. All five desktop CI jobs passed for that source in run `34774733389`. + +Describe each selected OBS bus using its explicit label and observed input +assignments. Disclose buses with the same assigned inputs as the primary mix. +Preserve historical attribution and a timestamped boundary when routing changes; +keep normal capture continuous when the capture identity and format remain valid. + +Area: bounded immutable desktop metadata, versioned native/Python framing, +native source/routing observation and worker transport, receiver/transcript +ownership, visible mix descriptions and focused cross-language/race tests. + +## Constraints + +- Resolve the primary bus from the live output's actual slot-zero audio encoder. + Additional tracks are selected OBS buses, never stereo channels or inferred + speakers. Equal assignment sets describe configuration, not identical PCM. +- No OBS routing, stream, recording, monitoring, profile or encoder mutation. + Preserve explicit Arm, session identity, cancellation and bounded audio queues. +- Enumerate and copy metadata outside the audio callback. Retain only selected + buses, with fixed source counts and UTF-8 byte limits. Labels/source identifiers + must not leak through logs, exception text, dataclass representations or backups. +- Distinguish observation time from a proven sample boundary. OBS signal timing + and buffered audio require an explicit uncertainty contract; do not invent + precise attribution from a callback timestamp. Preserve uncertainty visibly + when exact effective timing cannot be established. +- Metadata history and pending updates must remain bounded in memory for an + unlimited session, using the existing private transcript ownership model. + Missing, malformed, oversized, stale or inconsistent updates fail visibly. +- Source/capture identity or format changes that cannot preserve the authorized + primary mix terminate safely. Ordinary assignment changes must not silently + rewrite earlier labels or be presented as a new inferred speaker. + +## Acceptance + +1. Immutable models enforce six buses, at most 128 selected input records, + unique source identities and exact selected-bus labels. Tests distinguish + identical names from identical identities and empty from duplicate mixes. +2. Independent native wire vectors decode with Python under strict version and + size checks. Identity, revision, selected buses and monotonic observation/ + boundary rules are validated before receiver state or audio ownership changes. +3. Initial metadata describes the actual selected primary and optional buses. + Native signal/enumeration/teardown fixtures cover changes during snapshot, + attachment, callback publication, stop, cancellation and source destruction. +4. Routing changes preserve earlier transcript attribution and refresh later + descriptions. Bounded update bursts, slow recognition and cancellation do not + create an unbounded metadata list, block audio callbacks or lose cleanup. +5. UI and eventual exports distinguish input assignments, same-input mixes and + uncertain transition intervals. Synthetic evidence does not claim real OBS + stream compatibility or physical-device usability. + +## Verification + +Start with focused model/protocol/receiver/transcription tests and native +metadata/wire/stream fixtures. Then run the affected OBS regression and canonical +native build/test driver when integration changes its linked inputs. Record +exact commands, final source hashes and independent review results here. + +Commands for the current integration, from the owning worktree. The earlier +codec checkpoint retains separate receipts in the parent output directory. + +```powershell +$py = "C:/Users/unknown/Projects/Mindict/.venv/Scripts/python.exe" +$toolchain = "C:/Users/unknown/.local/llvm-mingw-20260616-ucrt-x86_64" +$obsBin = "C:/Program Files/obs-studio/bin/64bit" +$headers = "C:/Users/unknown/Projects/Mindict/.grok/obs-native-build/headers" +$output = "C:/Users/unknown/Projects/Mindict/.grok/obs-mix-provenance/integration" +& $py -m pytest tests/test_obs_mix.py tests/test_obs_routing_protocol.py tests/test_obs_protocol.py -o addopts= -q +& $py native/obs-plugin/tools/build.py --toolchain $toolchain --obs-bin $obsBin --headers $headers --output "$output/build" +& $py native/obs-plugin/tools/smoke.py --build "$output/build" *> "$output/build/smoke-run.log" +& $py native/obs-plugin/tools/test_native.py --toolchain $toolchain --build "$output/build" --headers $headers --output "$output/verification" +& $py -m pytest tests/test_obs_session_ui.py -o addopts= -q +``` + +## Current work and open decisions + +The preceding controller checkpoint remains frozen. Bounded immutable models, +Python framing and the independent C encoder are implemented and independently +reviewed. The following codec evidence belongs to checkpoint `5183cc2`; +it does not verify the in-progress integration described below. + +The focused model/routing/legacy codec bundle passes **224 tests**. The canonical +driver passes **796 affected desktop tests** and all **51 native commands**; +the **24-command linked build** and headless refusal smoke pass. The audit matches +all **446** recorded source, SDK, tool, generated-file, runtime, artifact and log +hashes. The existing native v1 wire vectors remain unchanged. Independent C and +Python review is clear, including the corrected blank/invisible-label policy. + +Local evidence lives under `.grok/obs-mix-provenance/`: build/smoke receipts in +`build/`, canonical `verification/test-receipt.json`, focused `codec/` evidence +and `hash-audit.json`. No OBS application, audio device, consumer profile or +model was used. This proves the component checks, not the remaining package +acceptance or live OBS behavior. + +### Current integration + +The receiver, private routing history, transcript ownership and explicit v2 +pipe/controller path are implemented. Routing is validated against accepted +per-bus sequence positions before publication. Initial metadata must follow Start +and precede PCM; a pre-Start Disarm creates neither audio nor metadata stores. +Each later accepted observation has exactly the next wire revision. Equal +observation timestamps are allowed; these timestamps are not sample boundaries. + +The private journal queues at most eight bounded records and performs file I/O +on its own worker. Disk-backed history remains with the transcript owner after +audio is released. Cancellation clears visible metadata immediately, then waits +off the UI thread for owned files and active readers to close. Cleanup failure +remains visible even after a formerly complete or cancelled result. Review found +and fixed an internal recognition-cancellation path that had omitted history +closure after ownership transfer. + +The view places latest mix details beside the transcript in a tab, preserving +the window dimensions and minimum preview heights. It distinguishes complete, +same-input, different-input and unassigned mixes and states timing uncertainty. +It shows configuration observations without identifying speakers or asserting +audibility. Private names are absent from representations and cleared on discard; +selecting text does not publish it to the platform selection clipboard. + +Final integration verification passes **873 affected desktop tests**, **48 Tk +checks**, all **55 native verification commands**, the **25-command linked +build** and headless refusal smoke. Eleven synthetic captures include normal +and compact mix-detail tabs. All **488** recorded source, SDK, tool, runtime, +generated-file, artifact, log and UI source/render hash comparisons match. +Evidence lives in `.grok/obs-mix-provenance/integration/`, with the separate +`ui-integration-verification.json` also checked by its `hash-audit.json`. +Earlier 120-check integration evidence describes its original source stage only. + +Independent receiver/controller/pipe, private-history, view, native integration +and build-driver reviews are clear. Review corrected cached private metadata +retention on explicit close and external destruction. Keyboard traversal and the +full six-bus, 128-source display bounds are covered. These checks used no OBS +application, audio device, consumer profile or model. Source CI results are +tracked on [draft PR #41](https://github.com/RioPlay/utterleaf/pull/41). + +Integration CI exposed a platform assumption in the keyboard-navigation fixture. +The macOS runner uses Tk 9.0.4, whose +[event generation](https://github.com/tcltk/tk/blob/core-9-0-4/macosx/tkMacOSXKeyboard.c#L650) +remaps Option-letter keysyms through the active layout. The fixture verifies the +installed mnemonic binding and invokes its registered action with the explicit +letter on every host. Ctrl+Tab still uses delivered key events everywhere; +Alt-letter delivery remains checked on Windows/X11. Physical Aqua mnemonic +delivery remains unverified. Runtime and render inputs are unchanged. + +Native observation uses fixed watcher/snapshot limits and an OBS monotonic clock +declared by the newly pinned public `util/platform.h`; its ISC notice is retained. +Independent review found and fixed an old-worker/new-session snapshot race by +serializing lifecycle mutation and snapshot access. Synthetic close/reopen checks +pass. OBS's own signal-disconnect calls can wait for foreign signal callbacks +before the local quiescence timeout begins; this is a public-API teardown limit, +not a proven globally bounded close. Scheduler callbacks use a nonblocking +frontend post and fail visibly if its state lock is contended. + +Final native review corrected three lifecycle races. Normal stop or Disarm can +close observation before initial Routing reaches the pipe, so the exact-generation +immutable snapshot remains readable through the bounded audio drain; worker +retirement wipes it. Stale queued refresh commands cannot stop a newer capture. +Snapshot comparison and publication revalidate lifecycle state under the same +lock as worker retirement, preventing a paused refresh from republishing after +retirement. Deterministic fixtures cover each race and callback scheduling +contention. Independent reruns pass on the same frozen native inputs. + +Production source advertises and requires audio version 2. Protocol/command +versions remain 1. Older audio-version-1 plugins and unknown future versions are +refused before Arm. Actual live OBS/audio/load acceptance, application entry, +export integration and native distribution remain open. + +Copy source names and UUIDs entirely inside the enumeration callback. The pinned +[core implementation](https://github.com/obsproject/obs-studio/blob/ba2f32bdf791005443988a4955e963663e16b1ed/libobs/obs.c) +holds the source mutex across that callback; source renaming and UUID reset use +the same mutex. A retained reference also prevents destruction. This is a +pinned-implementation constraint: the public getter API does not separately +promise borrowed-string lifetime or thread safety outside enumeration. + +### Observation semantics + +The public [source API](https://docs.obsproject.com/reference-sources) describes +assignment and rename signals but supplies no audio timestamp or documented +sample-effective ordering. The [core API](https://docs.obsproject.com/reference-core) +provides input enumeration and source lifecycle signals. These support observed +configuration snapshots, not proof of which input was audible in each sample. + +Keep PCM continuous on ordinary assignment changes. Do not pause/drop capture to +make labels appear more precise. Record the OBS monotonic observation timestamp +and exact per-bus next-sequence positions when the worker publishes an update. +Those positions serialize metadata with the transport; they are not the instant +the routing took effect. Describe historical entries as observed assignments, +with uncertainty explicit. A later observation never rewrites earlier entries. + +### Version 2 routing component + +The 12-byte ULAP header retains its existing layout. Version 2 adds kind 5; +existing Start/Audio/Gap/End body layouts are unchanged. Python encoding/decoding +requires explicit version selection and rejects version mixing. The native +runtime and compatibility reply now select version 2, and the desktop status +check requires it before Arm. Explicit legacy codec wrappers and the legacy +stream fixture retain version 1; the new runtime fixture verifies Start, +initial Routing, continuous Audio, later Routing and End under version 2. + +Routing body, little-endian: + +| Part | Fields | +| --- | --- | +| Prefix, 36 bytes | session ID (16 bytes), revision (u64), observed-at ns (u64), primary bus (u8), selected mask (u8), source count (u16) | +| Each selected bus | bus (u8), next sequence (u64), label UTF-8 length (u16), label bytes | +| Each source | source ID (16 bytes), selected assignment mask (u8), name UTF-8 length (u16), name bytes | + +Buses appear once in ascending selected order; sources appear once in ascending +raw-ID order. All source masks are nonempty subsets of the selected mask. +There are at most six bus labels of 64 UTF-8 bytes and 128 source names of 128 +bytes. The maximum body is 19,302 bytes. Revisions range from 1 to u64-max minus +one; next-sequence positions may include u64-max after sequence exhaustion. + +Whitespace-only and joiner-only text, controls and directional overrides are rejected. +Normal RTL text, meaningful Persian non-joiners and emoji joiners are preserved; +names are not silently trimmed or normalized. Invalid input produces generic +errors and cannot leave partially encoded native output. Source IDs/names and +bus labels are absent from model/frame representations. + +The codec does not validate session authority or update order. Integration must +require initial revision 1, exact next revisions, monotonic observation time, +unchanged selected/primary buses, and positions equal to the receiver's accepted +sequences before committing a record. Its private history must be bounded in +memory and survive slow recognition without silently losing observations. + +## Non-goals and stop + +Recorded-file stream selection and global PTS handling follow as a separate +reviewed increment. Do not add diarization, automatic source selection, microphone +capture, model downloads, plugin installation or publication here. + +Stop editing this package after the complete metadata/boundary contract, +integration checks and independent review pass, with the roadmap accurately +describing remaining live OBS/audio/load and native distribution gates. Keep the +full desktop and original Android completion goal active. diff --git a/native/obs-plugin/README.md b/native/obs-plugin/README.md index 0d9bffd..3ee1f42 100644 --- a/native/obs-plugin/README.md +++ b/native/obs-plugin/README.md @@ -63,20 +63,37 @@ remain open. All 647 focused desktop tests, 41 Tk tests, 51 native verification commands, the linked build and headless smoke pass. Independent source and render review is clear; its plan records exact evidence and remaining gates. +The [mix provenance increment](../../docs/plans/active/obs-mix-provenance.md) +adds an original bounded version-2 Routing encoder and matching Python metadata +model/codec. Independent byte vectors and native/Python interoperability pass; +the final canonical run passes 796 affected desktop tests and all 51 native +commands, plus the linked build and headless refusal at codec checkpoint `5183cc2`. +The current integration adds bounded native input observation, continuous +version-2 routing/audio transport, private desktop history and a compact Mix +details tab. Production source advertises audio version 2; desktop status requires +that version before Arm. Final integration passes 873 affected desktop tests, +48 Tk checks, all 55 native commands, the 25-command linked build and headless +refusal. Independent review is clear, including normal stop/Disarm metadata drain, +stale refresh, retirement/publication races and nonblocking callback scheduling. +The 488-comparison hash audit includes current UI source and eleven captures. +Source CI is tracked on [draft PR #41](https://github.com/RioPlay/utterleaf/pull/41). +OBS frontend/audio acceptance, application entry, +export integration and native distribution remain open. + The native test driver needs the desktop virtualenv for NumPy-backed protocol validation, and `--build` plus `--headers` for conversion fixtures. ## Inputs and legal boundary `dependencies.json` pins the exact OBS source revision -`ba2f32bdf791005443988a4955e963663e16b1ed` for 41 public resources, including the -frontend and audio-resampler headers, configuration template and license. The obs-websocket API header -is pinned separately at `1ef34bf48110c2a18184e50e41cd0b1a855e2147`, for 42 total +`ba2f32bdf791005443988a4955e963663e16b1ed` for 42 public resources, including the +frontend, audio-resampler and platform clock headers, configuration template and license. The obs-websocket API header +is pinned separately at `1ef34bf48110c2a18184e50e41cd0b1a855e2147`, for 43 total resources. The build verifies every resource's URL, byte count and SHA-256 before compiling. It generates `obsconfig.h`, derives local import libraries from installed `obs.dll` and `obs-frontend-api.dll`, embeds the common-controls activation manifest, and -emits source/input/generated-file receipts. The nine ISC header notices in +emits source/input/generated-file receipts. The ten ISC header notices in `OBS-HEADER-NOTICES.txt` and the obs-websocket notice reproduce each header's complete leading comment. diff --git a/native/obs-plugin/dependencies.json b/native/obs-plugin/dependencies.json index ef49833..b759a3e 100644 --- a/native/obs-plugin/dependencies.json +++ b/native/obs-plugin/dependencies.json @@ -2,6 +2,12 @@ "obs_revision": "ba2f32bdf791005443988a4955e963663e16b1ed", "obs_websocket_revision": "1ef34bf48110c2a18184e50e41cd0b1a855e2147", "resources": { + "libobs/util/platform.h": { + "url": "https://raw.githubusercontent.com/obsproject/obs-studio/ba2f32bdf791005443988a4955e963663e16b1ed/libobs/util/platform.h", + "sha256": "af2afe7059d2d911a36b62ce4ff4429d2cb15cc69794cb174bf0568dc3792bba", + "git_blob_sha1": "592d9eca6e5c8c16286fb0fff1a506557ff8af0a", + "bytes": 7266 + }, "libobs/media-io/audio-resampler.h": { "url": "https://raw.githubusercontent.com/obsproject/obs-studio/ba2f32bdf791005443988a4955e963663e16b1ed/libobs/media-io/audio-resampler.h", "sha256": "f10fe20d41dab554661ea590b9274bb58035dd7d0ffbda91ba5e197b407b7e57", diff --git a/native/obs-plugin/src/audio_capture.c b/native/obs-plugin/src/audio_capture.c index 090b3f6..968f30c 100644 --- a/native/obs-plugin/src/audio_capture.c +++ b/native/obs-plugin/src/audio_capture.c @@ -267,6 +267,15 @@ bool ul_audio_capture_inspect_frontend(uint8_t additional_mask, return valid; } +bool ul_audio_capture_matches_frontend(const ul_audio_capture_spec *expected) +{ + ul_audio_capture_spec current; + if (expected == NULL || + !ul_audio_capture_inspect_frontend(expected->mix_mask, ¤t)) + return false; + return same_spec(expected, ¤t); +} + ul_audio_capture *ul_audio_capture_create_worker( const ul_audio_capture_spec *spec) { diff --git a/native/obs-plugin/src/audio_capture.h b/native/obs-plugin/src/audio_capture.h index 11de1b4..1d65fb2 100644 --- a/native/obs-plugin/src/audio_capture.h +++ b/native/obs-plugin/src/audio_capture.h @@ -32,6 +32,8 @@ typedef struct ul_audio_capture_spec { * and releases its own reference to the current streaming output. */ bool ul_audio_capture_inspect_frontend(uint8_t additional_mask, ul_audio_capture_spec *out); +/* Frontend-thread identity/format guard for an active generation. */ +bool ul_audio_capture_matches_frontend(const ul_audio_capture_spec *expected); /* Worker-only. Allocates bounded per-bus queues and independent converters. */ ul_audio_capture *ul_audio_capture_create_worker( diff --git a/native/obs-plugin/src/audio_metadata.c b/native/obs-plugin/src/audio_metadata.c new file mode 100644 index 0000000..daa9d30 --- /dev/null +++ b/native/obs-plugin/src/audio_metadata.c @@ -0,0 +1,661 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#include "audio_metadata.h" + +#include +#ifndef UL_AUDIO_METADATA_TIME_NS +#include +#include +#define UL_AUDIO_METADATA_TIME_NS() os_gettime_ns() +#endif + +#include + +#include +#include + +#ifndef UL_AUDIO_METADATA_CLOSE_TIMEOUT_MS +#define UL_AUDIO_METADATA_CLOSE_TIMEOUT_MS 1000u +#endif + +#ifndef UL_AUDIO_METADATA_STABLE_ATTEMPTS +#define UL_AUDIO_METADATA_STABLE_ATTEMPTS 8u +#endif + +#ifndef UL_AUDIO_METADATA_CALLBACK_ENTERED +#define UL_AUDIO_METADATA_CALLBACK_ENTERED() ((void)0) +#endif + +#ifndef UL_AUDIO_METADATA_TAKE_LOCKED +#define UL_AUDIO_METADATA_TAKE_LOCKED() ((void)0) +#endif +#ifndef UL_AUDIO_METADATA_BEFORE_PUBLISH +#define UL_AUDIO_METADATA_BEFORE_PUBLISH() ((void)0) +#endif + +typedef struct watched_source { + obs_source_t *source; +} watched_source; + +typedef struct metadata_state { + SRWLOCK lock; + atomic_bool active; + atomic_bool failed; + atomic_bool dirty; + atomic_bool queued; + atomic_bool poisoned; + atomic_uint_fast64_t readers; + atomic_uint_fast64_t token; + atomic_uintptr_t generation; + uint8_t primary_bus; + uint8_t bus_mask; + ul_audio_metadata_schedule schedule; + signal_handler_t *core_handler; + watched_source watched[UL_AUDIO_METADATA_MAX_WATCHED_SOURCES]; + size_t watched_count; + ul_audio_metadata_snapshot latest; +} metadata_state; + +typedef struct snapshot_build { + ul_audio_metadata_snapshot snapshot; + watched_source watched[UL_AUDIO_METADATA_MAX_WATCHED_SOURCES]; + bool newly_connected[UL_AUDIO_METADATA_MAX_WATCHED_SOURCES]; + size_t watched_count; + bool failed; +} snapshot_build; + +static metadata_state metadata = { + SRWLOCK_INIT, + ATOMIC_VAR_INIT(false), ATOMIC_VAR_INIT(false), + ATOMIC_VAR_INIT(false), ATOMIC_VAR_INIT(false), + ATOMIC_VAR_INIT(false), ATOMIC_VAR_INIT(0u), + ATOMIC_VAR_INIT(0u), ATOMIC_VAR_INIT(0u), + 0u, 0u, NULL, NULL, {{0}}, 0u, {0}, +}; + +static void metadata_signal(void *data, calldata_t *parameters); +static void core_signal(void *data, calldata_t *parameters); + +static bool valid_mask(uint8_t primary_bus, uint8_t bus_mask) +{ + return primary_bus < UL_AUDIO_METADATA_MIXES && bus_mask != 0u && + (bus_mask & ~0x3fu) == 0u && + (bus_mask & (uint8_t)(1u << primary_bus)) != 0u; +} + +static size_t bounded_c_string(const char *text, size_t maximum) +{ + size_t length = 0u; + if (text == NULL) + return maximum + 1u; + while (length <= maximum && text[length] != '\0') + length++; + return length; +} + +static bool parse_uuid(const char *text, uint8_t out[16]) +{ + static const uint8_t positions[4] = {8u, 13u, 18u, 23u}; + size_t input = 0u, output = 0u, dash = 0u; + if (text == NULL || bounded_c_string(text, 36u) != 36u) + return false; + while (input < 36u) { + unsigned high, low; + if (dash < 4u && input == positions[dash]) { + if (text[input++] != '-') + return false; + dash++; + continue; + } +#define HEX_VALUE(ch, value) \ + (((ch) >= '0' && (ch) <= '9') ? ((value) = (unsigned)((ch) - '0'), true) : \ + ((ch) >= 'a' && (ch) <= 'f') ? ((value) = (unsigned)((ch) - 'a' + 10), true) : \ + ((ch) >= 'A' && (ch) <= 'F') ? ((value) = (unsigned)((ch) - 'A' + 10), true) : false) + if (input + 1u >= 36u || output >= 16u || + !HEX_VALUE(text[input], high) || + !HEX_VALUE(text[input + 1u], low)) + return false; +#undef HEX_VALUE + out[output++] = (uint8_t)((high << 4) | low); + input += 2u; + } + return output == 16u && dash == 4u; +} + +static bool valid_text(const uint8_t *text, size_t length, size_t maximum) +{ + size_t index = 0u; + bool has_visible = false; + if (text == NULL || length == 0u || length > maximum) + return false; + while (index < length) { + uint32_t codepoint; + uint8_t first = text[index++]; + if (first < 0x80u) { + codepoint = first; + } else if (first >= 0xc2u && first <= 0xdfu) { + if (index >= length || (text[index] & 0xc0u) != 0x80u) + return false; + codepoint = ((uint32_t)(first & 0x1fu) << 6) | + (uint32_t)(text[index++] & 0x3fu); + } else if (first >= 0xe0u && first <= 0xefu) { + uint8_t second; + if (length - index < 2u) + return false; + second = text[index]; + if ((second & 0xc0u) != 0x80u || + (text[index + 1u] & 0xc0u) != 0x80u || + (first == 0xe0u && second < 0xa0u) || + (first == 0xedu && second >= 0xa0u)) + return false; + codepoint = ((uint32_t)(first & 0x0fu) << 12) | + ((uint32_t)(second & 0x3fu) << 6) | + (uint32_t)(text[index + 1u] & 0x3fu); + index += 2u; + } else if (first >= 0xf0u && first <= 0xf4u) { + uint8_t second; + if (length - index < 3u) + return false; + second = text[index]; + if ((second & 0xc0u) != 0x80u || + (text[index + 1u] & 0xc0u) != 0x80u || + (text[index + 2u] & 0xc0u) != 0x80u || + (first == 0xf0u && second < 0x90u) || + (first == 0xf4u && second > 0x8fu)) + return false; + codepoint = ((uint32_t)(first & 0x07u) << 18) | + ((uint32_t)(second & 0x3fu) << 12) | + ((uint32_t)(text[index + 1u] & 0x3fu) << 6) | + (uint32_t)(text[index + 2u] & 0x3fu); + index += 3u; + } else { + return false; + } + if (codepoint <= 0x1fu || + (codepoint >= 0x7fu && codepoint <= 0x9fu) || + codepoint == 0x061cu || codepoint == 0x200bu || + codepoint == 0x200eu || codepoint == 0x200fu || + (codepoint >= 0x2028u && codepoint <= 0x202eu) || + (codepoint >= 0x2066u && codepoint <= 0x2069u) || + codepoint == 0xfeffu) + return false; + if (codepoint != 0x20u && codepoint != 0x00a0u && + codepoint != 0x1680u && + !(codepoint >= 0x2000u && codepoint <= 0x200au) && + codepoint != 0x202fu && codepoint != 0x205fu && + codepoint != 0x3000u && codepoint != 0x200cu && + codepoint != 0x200du) + has_visible = true; + } + return has_visible; +} + +static bool was_watched(obs_source_t *source) +{ + size_t index; + for (index = 0u; index < metadata.watched_count; ++index) + if (metadata.watched[index].source == source) + return true; + return false; +} + +static void disconnect_source(obs_source_t *source) +{ + signal_handler_t *handler = obs_source_get_signal_handler(source); + if (handler != NULL) { + signal_handler_disconnect(handler, "audio_mixers", metadata_signal, + &metadata); + signal_handler_disconnect(handler, "rename", metadata_signal, + &metadata); + signal_handler_disconnect(handler, "remove", metadata_signal, + &metadata); + } +} + +static bool connect_source(obs_source_t *source) +{ + signal_handler_t *handler = obs_source_get_signal_handler(source); + if (handler == NULL) + return false; + signal_handler_connect(handler, "audio_mixers", metadata_signal, + &metadata); + signal_handler_connect(handler, "rename", metadata_signal, &metadata); + signal_handler_connect(handler, "remove", metadata_signal, &metadata); + return true; +} + +static void sort_sources(ul_audio_metadata_snapshot *snapshot) +{ + size_t index; + for (index = 1u; index < snapshot->source_count; ++index) { + ul_audio_metadata_source value = snapshot->sources[index]; + size_t position = index; + while (position != 0u && + memcmp(snapshot->sources[position - 1u].source_id, + value.source_id, 16u) > 0) { + snapshot->sources[position] = snapshot->sources[position - 1u]; + position--; + } + snapshot->sources[position] = value; + } +} + +static bool enumerate_source(void *data, obs_source_t *source) +{ + snapshot_build *build = data; + ul_audio_metadata_source *record; + obs_source_t *retained; + const char *uuid, *name; + uint32_t mixers; + size_t length; + bool old; + if (build->failed || source == NULL || obs_source_removed(source) || + obs_source_get_type(source) != OBS_SOURCE_TYPE_INPUT || + (obs_source_get_output_flags(source) & OBS_SOURCE_AUDIO) == 0u) + return !build->failed; + if (build->watched_count == UL_AUDIO_METADATA_MAX_WATCHED_SOURCES) { + build->failed = true; + return false; + } + retained = obs_source_get_ref(source); + if (retained == NULL) { + build->failed = true; + return false; + } + old = was_watched(source); + build->watched[build->watched_count].source = retained; + build->newly_connected[build->watched_count] = !old; + build->watched_count++; + if (!old && !connect_source(retained)) { + build->failed = true; + return false; + } + mixers = obs_source_get_audio_mixers(retained) & build->snapshot.bus_mask; + if (mixers == 0u) + return true; + if (build->snapshot.source_count == UL_AUDIO_METADATA_MAX_SOURCES) { + build->failed = true; + return false; + } + record = &build->snapshot.sources[build->snapshot.source_count]; + /* The pinned obs_enum_sources implementation holds sources_mutex through + * this callback; public rename/UUID reset takes the same mutex. Keep both + * getters and their complete bounded copies inside this callback. */ + uuid = obs_source_get_uuid(retained); + name = obs_source_get_name(retained); + if (!parse_uuid(uuid, record->source_id) || name == NULL) { + build->failed = true; + return false; + } + length = bounded_c_string(name, UL_AUDIO_METADATA_MAX_SOURCE_NAME_BYTES); + if (!valid_text((const uint8_t *)name, length, + UL_AUDIO_METADATA_MAX_SOURCE_NAME_BYTES)) { + build->failed = true; + return false; + } + record->selected_mask = (uint8_t)mixers; + record->name_length = (uint16_t)length; + memcpy(record->name, name, length); + build->snapshot.source_count++; + return true; +} + +static void cleanup_build(snapshot_build *build, bool keep_connections) +{ + size_t index; + for (index = 0u; index < build->watched_count; ++index) { + if (!keep_connections && build->newly_connected[index]) + disconnect_source(build->watched[index].source); + obs_source_release(build->watched[index].source); + } + build->watched_count = 0u; +} + +static void adopt_watchers(snapshot_build *build) +{ + size_t index; + for (index = 0u; index < metadata.watched_count; ++index) { + size_t candidate; + bool retained = false; + for (candidate = 0u; candidate < build->watched_count; ++candidate) + if (metadata.watched[index].source == + build->watched[candidate].source) { + retained = true; + break; + } + if (!retained) + disconnect_source(metadata.watched[index].source); + obs_source_release(metadata.watched[index].source); + } + metadata.watched_count = build->watched_count; + memcpy(metadata.watched, build->watched, + build->watched_count * sizeof(build->watched[0])); + build->watched_count = 0u; +} + +static bool same_payload(const ul_audio_metadata_snapshot *left, + const ul_audio_metadata_snapshot *right) +{ + ul_audio_metadata_snapshot first = *left, second = *right; + first.token = second.token = 0u; + first.observed_at_ns = second.observed_at_ns = 0u; + return memcmp(&first, &second, sizeof(first)) == 0; +} + +static bool build_and_publish(bool initial, uintptr_t expected_generation) +{ + snapshot_build build = {0}; + uint8_t bus; + size_t index; + uint64_t next_token, observed; + build.snapshot.primary_bus = metadata.primary_bus; + build.snapshot.bus_mask = metadata.bus_mask; + for (bus = 0u; bus < UL_AUDIO_METADATA_MIXES; ++bus) { + ul_audio_metadata_bus *label; + if ((metadata.bus_mask & (uint8_t)(1u << bus)) == 0u) + continue; + label = &build.snapshot.buses[build.snapshot.bus_count++]; + label->bus = bus; + label->label_length = 5u; + memcpy(label->label, "Mix 1", 5u); + label->label[4] = (uint8_t)('1' + bus); + } + obs_enum_sources(enumerate_source, &build); + if (build.failed) + goto failure; + sort_sources(&build.snapshot); + for (index = 1u; index < build.snapshot.source_count; ++index) + if (memcmp(build.snapshot.sources[index - 1u].source_id, + build.snapshot.sources[index].source_id, 16u) == 0) + goto failure; + adopt_watchers(&build); + UL_AUDIO_METADATA_BEFORE_PUBLISH(); + AcquireSRWLockExclusive(&metadata.lock); + if (!atomic_load_explicit(&metadata.active, memory_order_acquire) || + atomic_load_explicit(&metadata.failed, memory_order_acquire) || + atomic_load_explicit(&metadata.generation, memory_order_acquire) != + expected_generation) { + ReleaseSRWLockExclusive(&metadata.lock); + goto failure; + } + if (!initial && same_payload(&build.snapshot, &metadata.latest)) { + ReleaseSRWLockExclusive(&metadata.lock); + return true; + } + next_token = atomic_load_explicit(&metadata.token, memory_order_relaxed); + if (next_token == UINT64_MAX) { + ReleaseSRWLockExclusive(&metadata.lock); + goto failure; + } + observed = UL_AUDIO_METADATA_TIME_NS(); + if (!initial && observed <= metadata.latest.observed_at_ns) { + if (metadata.latest.observed_at_ns == UINT64_MAX) { + ReleaseSRWLockExclusive(&metadata.lock); + goto failure; + } + observed = metadata.latest.observed_at_ns + 1u; + } + build.snapshot.token = next_token + 1u; + build.snapshot.observed_at_ns = observed; + metadata.latest = build.snapshot; + atomic_store_explicit(&metadata.token, next_token + 1u, + memory_order_release); + ReleaseSRWLockExclusive(&metadata.lock); + return true; + +failure: + cleanup_build(&build, false); + return false; +} + +static void schedule_if_needed(void) +{ + bool expected = false; + if (!atomic_load_explicit(&metadata.active, memory_order_acquire) || + atomic_load_explicit(&metadata.failed, memory_order_acquire) || + !atomic_load_explicit(&metadata.dirty, memory_order_acquire) || + !atomic_compare_exchange_strong_explicit( + &metadata.queued, &expected, true, + memory_order_acq_rel, memory_order_acquire)) + return; + if (metadata.schedule == NULL || + !metadata.schedule(atomic_load_explicit(&metadata.generation, + memory_order_acquire))) { + atomic_store_explicit(&metadata.failed, true, memory_order_release); + atomic_store_explicit(&metadata.queued, false, memory_order_release); + } +} + +static void mark_dirty(void) +{ + atomic_store_explicit(&metadata.dirty, true, memory_order_release); + schedule_if_needed(); +} + +static void metadata_signal(void *data, calldata_t *parameters) +{ + (void)parameters; + if (data != &metadata) + return; + atomic_fetch_add_explicit(&metadata.readers, 1u, memory_order_seq_cst); + UL_AUDIO_METADATA_CALLBACK_ENTERED(); + if (atomic_load_explicit(&metadata.active, memory_order_acquire)) + mark_dirty(); + atomic_fetch_sub_explicit(&metadata.readers, 1u, memory_order_seq_cst); +} + +static void core_signal(void *data, calldata_t *parameters) +{ + metadata_signal(data, parameters); +} + +static void disconnect_all(void) +{ + size_t index; + if (metadata.core_handler != NULL) { + signal_handler_disconnect(metadata.core_handler, "source_create", + core_signal, &metadata); + signal_handler_disconnect(metadata.core_handler, "source_remove", + core_signal, &metadata); + signal_handler_disconnect(metadata.core_handler, "source_destroy", + core_signal, &metadata); + signal_handler_disconnect(metadata.core_handler, "source_rename", + core_signal, &metadata); + } + for (index = 0u; index < metadata.watched_count; ++index) { + disconnect_source(metadata.watched[index].source); + obs_source_release(metadata.watched[index].source); + } + metadata.watched_count = 0u; + metadata.core_handler = NULL; +} + +bool ul_audio_metadata_open_frontend(uintptr_t generation, + uint8_t primary_bus, uint8_t bus_mask, + ul_audio_metadata_schedule schedule) +{ + unsigned attempt; + bool stable = false; + if (generation == 0u || schedule == NULL || + !valid_mask(primary_bus, bus_mask) || + atomic_load_explicit(&metadata.active, memory_order_acquire) || + atomic_load_explicit(&metadata.poisoned, memory_order_acquire) || + atomic_load_explicit(&metadata.readers, memory_order_acquire) != 0u) + return false; + metadata.primary_bus = primary_bus; + metadata.bus_mask = bus_mask; + metadata.schedule = schedule; + metadata.core_handler = obs_get_signal_handler(); + if (metadata.core_handler == NULL) + return false; + metadata.watched_count = 0u; + atomic_store_explicit(&metadata.failed, false, memory_order_release); + atomic_store_explicit(&metadata.dirty, false, memory_order_release); + atomic_store_explicit(&metadata.queued, false, memory_order_release); + AcquireSRWLockExclusive(&metadata.lock); + memset(&metadata.latest, 0, sizeof(metadata.latest)); + atomic_store_explicit(&metadata.token, 0u, memory_order_release); + atomic_store_explicit(&metadata.generation, generation, + memory_order_release); + atomic_store_explicit(&metadata.active, true, memory_order_release); + ReleaseSRWLockExclusive(&metadata.lock); + signal_handler_connect(metadata.core_handler, "source_create", core_signal, + &metadata); + signal_handler_connect(metadata.core_handler, "source_remove", core_signal, + &metadata); + signal_handler_connect(metadata.core_handler, "source_destroy", core_signal, + &metadata); + signal_handler_connect(metadata.core_handler, "source_rename", core_signal, + &metadata); + for (attempt = 0u; attempt < UL_AUDIO_METADATA_STABLE_ATTEMPTS; ++attempt) { + atomic_store_explicit(&metadata.dirty, false, memory_order_release); + if (!build_and_publish(attempt == 0u, generation)) + break; + if (atomic_load_explicit(&metadata.failed, memory_order_acquire)) + break; + if (!atomic_load_explicit(&metadata.dirty, memory_order_acquire)) { + stable = true; + break; + } + } + if (!stable || + atomic_load_explicit(&metadata.failed, memory_order_acquire)) { + atomic_store_explicit(&metadata.failed, true, memory_order_release); + (void)ul_audio_metadata_close_frontend(generation, false); + return false; + } + atomic_store_explicit(&metadata.queued, false, memory_order_release); + schedule_if_needed(); + return true; +} + +bool ul_audio_metadata_refresh_frontend(uintptr_t generation) +{ + bool success; + if (!atomic_load_explicit(&metadata.active, memory_order_acquire) || + generation == 0u || generation != atomic_load_explicit( + &metadata.generation, memory_order_acquire) || + atomic_load_explicit(&metadata.failed, memory_order_acquire)) + return false; + atomic_store_explicit(&metadata.dirty, false, memory_order_release); + success = build_and_publish(false, generation); + if (atomic_load_explicit(&metadata.failed, memory_order_acquire)) + success = false; + if (!success) + atomic_store_explicit(&metadata.failed, true, memory_order_release); + atomic_store_explicit(&metadata.queued, false, memory_order_release); + if (success) + schedule_if_needed(); + return success; +} + +bool ul_audio_metadata_close_frontend(uintptr_t generation, bool all) +{ + ULONGLONG started; + uintptr_t current; + AcquireSRWLockExclusive(&metadata.lock); + current = atomic_load_explicit(&metadata.generation, memory_order_acquire); + if (!atomic_load_explicit(&metadata.active, memory_order_acquire)) { + ReleaseSRWLockExclusive(&metadata.lock); + return generation == current || (all && current != 0u); + } + if (!all && (generation == 0u || generation != current)) { + ReleaseSRWLockExclusive(&metadata.lock); + return false; + } + atomic_store_explicit(&metadata.active, false, memory_order_release); + ReleaseSRWLockExclusive(&metadata.lock); + disconnect_all(); + started = GetTickCount64(); + while (atomic_load_explicit(&metadata.readers, memory_order_acquire) != 0u) { + if (GetTickCount64() - started >= UL_AUDIO_METADATA_CLOSE_TIMEOUT_MS) { + atomic_store_explicit(&metadata.poisoned, true, + memory_order_release); + return false; + } + SwitchToThread(); + } + atomic_store_explicit(&metadata.queued, false, memory_order_release); + atomic_store_explicit(&metadata.dirty, false, memory_order_release); + return true; +} + +void ul_audio_metadata_fail_frontend(uintptr_t generation) +{ + AcquireSRWLockShared(&metadata.lock); + if (generation != 0u && + atomic_load_explicit(&metadata.active, memory_order_acquire) && + generation == atomic_load_explicit(&metadata.generation, + memory_order_acquire)) + atomic_store_explicit(&metadata.failed, true, memory_order_release); + ReleaseSRWLockShared(&metadata.lock); +} + +void ul_audio_metadata_abandon_after_shutdown(void) +{ + AcquireSRWLockExclusive(&metadata.lock); + atomic_store_explicit(&metadata.active, false, memory_order_release); + atomic_store_explicit(&metadata.failed, true, memory_order_release); + atomic_store_explicit(&metadata.poisoned, true, memory_order_release); + atomic_store_explicit(&metadata.queued, false, memory_order_release); + atomic_store_explicit(&metadata.dirty, false, memory_order_release); + ReleaseSRWLockExclusive(&metadata.lock); +} + +bool ul_audio_metadata_take_worker(uintptr_t generation, uint64_t after_token, + ul_audio_metadata_snapshot *out) +{ + if (generation == 0u || out == NULL) + return false; + AcquireSRWLockShared(&metadata.lock); + UL_AUDIO_METADATA_TAKE_LOCKED(); + if (generation != atomic_load_explicit(&metadata.generation, + memory_order_acquire) || + atomic_load_explicit(&metadata.failed, memory_order_acquire) || + metadata.latest.token <= after_token) { + ReleaseSRWLockShared(&metadata.lock); + return false; + } + *out = metadata.latest; + ReleaseSRWLockShared(&metadata.lock); + return true; +} + +bool ul_audio_metadata_failed(uintptr_t generation) +{ + return generation != 0u && generation == atomic_load_explicit( + &metadata.generation, memory_order_acquire) && + atomic_load_explicit(&metadata.failed, memory_order_acquire); +} + +void ul_audio_metadata_retire_worker(uintptr_t generation) +{ + if (generation == 0u) + return; + AcquireSRWLockExclusive(&metadata.lock); + if (generation == atomic_load_explicit(&metadata.generation, + memory_order_acquire)) { + atomic_store_explicit(&metadata.failed, true, memory_order_release); + atomic_store_explicit(&metadata.queued, false, memory_order_release); + atomic_store_explicit(&metadata.dirty, false, memory_order_release); + atomic_store_explicit(&metadata.token, 0u, memory_order_release); + SecureZeroMemory(&metadata.latest, sizeof(metadata.latest)); + } + ReleaseSRWLockExclusive(&metadata.lock); +} + +bool ul_audio_metadata_request_worker(uintptr_t generation) +{ + bool accepted = false; + AcquireSRWLockShared(&metadata.lock); + if (generation != 0u && + atomic_load_explicit(&metadata.active, memory_order_acquire) && + generation == atomic_load_explicit(&metadata.generation, + memory_order_acquire) && + !atomic_load_explicit(&metadata.failed, memory_order_acquire)) { + mark_dirty(); + accepted = !atomic_load_explicit(&metadata.failed, + memory_order_acquire); + } + ReleaseSRWLockShared(&metadata.lock); + return accepted; +} diff --git a/native/obs-plugin/src/audio_metadata.h b/native/obs-plugin/src/audio_metadata.h new file mode 100644 index 0000000..ccb46e3 --- /dev/null +++ b/native/obs-plugin/src/audio_metadata.h @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#ifndef UTTERLEAF_OBS_AUDIO_METADATA_H +#define UTTERLEAF_OBS_AUDIO_METADATA_H + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define UL_AUDIO_METADATA_MIXES 6u +#define UL_AUDIO_METADATA_MAX_SOURCES 128u +#define UL_AUDIO_METADATA_MAX_WATCHED_SOURCES 512u +#define UL_AUDIO_METADATA_MAX_BUS_LABEL_BYTES 64u +#define UL_AUDIO_METADATA_MAX_SOURCE_NAME_BYTES 128u + +typedef struct ul_audio_metadata_bus { + uint8_t bus; + uint16_t label_length; + uint8_t label[UL_AUDIO_METADATA_MAX_BUS_LABEL_BYTES]; +} ul_audio_metadata_bus; + +typedef struct ul_audio_metadata_source { + uint8_t source_id[16]; + uint8_t selected_mask; + uint16_t name_length; + uint8_t name[UL_AUDIO_METADATA_MAX_SOURCE_NAME_BYTES]; +} ul_audio_metadata_source; + +/* token is a local publication token, not the contiguous wire revision. */ +typedef struct ul_audio_metadata_snapshot { + uint64_t token; + uint64_t observed_at_ns; + uint8_t primary_bus; + uint8_t bus_mask; + uint8_t bus_count; + uint16_t source_count; + ul_audio_metadata_bus buses[UL_AUDIO_METADATA_MIXES]; + ul_audio_metadata_source sources[UL_AUDIO_METADATA_MAX_SOURCES]; +} ul_audio_metadata_snapshot; + +typedef bool (*ul_audio_metadata_schedule)(uintptr_t generation); + +/* One plugin-global observer is supported. These calls are frontend-thread + * only except take_worker/request_worker/retire_worker/failed. The scheduler must only enqueue a numeric + * generation. It must be thread-safe, nonblocking, and must never wait for a + * frontend lock, refresh, close, or queued task; OBS invokes it while holding + * its signal mutex. It must not invoke refresh reentrantly. open publishes the + * complete initial snapshot before returning success. + * + * close must run while OBS is live and without a lock needed by the scheduler. + * OBS signal disconnection may wait for foreign callbacks before the internal + * callback-drain timeout begins, so this API does not bound that OBS wait. */ +bool ul_audio_metadata_open_frontend(uintptr_t generation, + uint8_t primary_bus, uint8_t bus_mask, + ul_audio_metadata_schedule schedule); +bool ul_audio_metadata_refresh_frontend(uintptr_t generation); +bool ul_audio_metadata_close_frontend(uintptr_t generation, bool all); +void ul_audio_metadata_fail_frontend(uintptr_t generation); +/* Terminal unload fallback after OBS signal/global teardown may have begun. + * Makes static callbacks inert and calls no OBS API. */ +void ul_audio_metadata_abandon_after_shutdown(void); + +/* Single transport-worker reader. A true return copies the newest immutable + * observation whose local token is greater than after_token. The same + * generation may take the final snapshot after frontend close disconnects + * watchers, allowing an already-accepted tail to finish. Reopen changes the + * generation before publishing new data, so stale workers cannot cross it. */ +bool ul_audio_metadata_take_worker(uintptr_t generation, uint64_t after_token, + ul_audio_metadata_snapshot *out); +bool ul_audio_metadata_request_worker(uintptr_t generation); +/* Final transport-worker release clears the private immutable snapshot and + * prevents queued refresh from republishing it before frontend disconnect. */ +void ul_audio_metadata_retire_worker(uintptr_t generation); +bool ul_audio_metadata_failed(uintptr_t generation); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/native/obs-plugin/src/audio_protocol.c b/native/obs-plugin/src/audio_protocol.c index 4cad64d..64509a0 100644 --- a/native/obs-plugin/src/audio_protocol.c +++ b/native/obs-plugin/src/audio_protocol.c @@ -13,6 +13,7 @@ _Static_assert(sizeof(float) == 4u && FLT_RADIX == 2 && FLT_MANT_DIG == 24 && #define UL_AUDIO_KIND_AUDIO 2u #define UL_AUDIO_KIND_GAP 3u #define UL_AUDIO_KIND_END 4u +#define UL_AUDIO_KIND_ROUTING 5u static bool valid_sample_rate(uint32_t sample_rate) { @@ -43,6 +44,12 @@ static void put_u32(uint8_t *out, uint32_t value) out[3] = (uint8_t)(value >> 24); } +static void put_u16(uint8_t *out, uint16_t value) +{ + out[0] = (uint8_t)value; + out[1] = (uint8_t)(value >> 8); +} + static void put_u64(uint8_t *out, uint64_t value) { size_t index; @@ -50,29 +57,37 @@ static void put_u64(uint8_t *out, uint64_t value) out[index] = (uint8_t)(value >> (index * 8u)); } -static void put_header(uint8_t *out, uint8_t kind, uint32_t body_size) +static void put_header(uint8_t *out, uint8_t version, uint8_t kind, + uint32_t body_size) { memcpy(out, "ULAP", 4u); - out[4] = 1u; + out[4] = version; out[5] = kind; out[6] = 0u; out[7] = 0u; put_u32(out + 8u, body_size); } -size_t ul_audio_encode_start(uint8_t *out, size_t capacity, - const uint8_t session_id[16], - uint32_t sample_rate, uint8_t primary_bus, - uint8_t bus_mask, uint64_t origin_ns) +static bool valid_version(uint8_t version) +{ + return version == UL_AUDIO_PROTOCOL_LEGACY_VERSION || + version == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION; +} + +size_t ul_audio_encode_start_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint32_t sample_rate, + uint8_t primary_bus, uint8_t bus_mask, uint64_t origin_ns) { const size_t packet_size = UL_AUDIO_HEADER_BYTES + 30u; uint8_t *body; - if (!valid_common(out, capacity, session_id, packet_size) || + if (!valid_version(version) || + !valid_common(out, capacity, session_id, packet_size) || !valid_sample_rate(sample_rate) || primary_bus > 5u || bus_mask == 0u || bus_mask > 0x3fu || (bus_mask & (uint8_t)(1u << primary_bus)) == 0u) return 0u; - put_header(out, UL_AUDIO_KIND_START, 30u); + put_header(out, version, UL_AUDIO_KIND_START, 30u); body = out + UL_AUDIO_HEADER_BYTES; memcpy(body, session_id, 16u); put_u32(body + 16u, sample_rate); @@ -82,14 +97,25 @@ size_t ul_audio_encode_start(uint8_t *out, size_t capacity, return packet_size; } -size_t ul_audio_encode_audio(uint8_t *out, size_t capacity, - const uint8_t session_id[16], uint8_t bus, - uint64_t sequence, uint64_t timestamp_ns, - uint32_t frames, const float *stereo_pcm) +size_t ul_audio_encode_start(uint8_t *out, size_t capacity, + const uint8_t session_id[16], + uint32_t sample_rate, uint8_t primary_bus, + uint8_t bus_mask, uint64_t origin_ns) +{ + return ul_audio_encode_start_version( + UL_AUDIO_PROTOCOL_LEGACY_VERSION, out, capacity, session_id, + sample_rate, primary_bus, bus_mask, origin_ns); +} + +size_t ul_audio_encode_audio_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t bus, uint64_t sequence, + uint64_t timestamp_ns, uint32_t frames, const float *stereo_pcm) { size_t sample_count, pcm_size, packet_size, index; uint8_t *body, *pcm_out; - if (frames == 0u || frames > UL_AUDIO_MAX_FRAMES || bus > 5u || + if (!valid_version(version) || frames == 0u || + frames > UL_AUDIO_MAX_FRAMES || bus > 5u || sequence > UL_AUDIO_MAX_SEQUENCE || stereo_pcm == NULL) return 0u; sample_count = (size_t)frames * 2u; @@ -101,7 +127,8 @@ size_t ul_audio_encode_audio(uint8_t *out, size_t capacity, if (!isfinite(stereo_pcm[index])) return 0u; } - put_header(out, UL_AUDIO_KIND_AUDIO, (uint32_t)(37u + pcm_size)); + put_header(out, version, UL_AUDIO_KIND_AUDIO, + (uint32_t)(37u + pcm_size)); body = out + UL_AUDIO_HEADER_BYTES; memcpy(body, session_id, 16u); body[16] = bus; @@ -117,18 +144,29 @@ size_t ul_audio_encode_audio(uint8_t *out, size_t capacity, return packet_size; } -size_t ul_audio_encode_gap(uint8_t *out, size_t capacity, - const uint8_t session_id[16], uint8_t bus, - uint64_t first_sequence, uint64_t count, - uint64_t timestamp_ns) +size_t ul_audio_encode_audio(uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t bus, + uint64_t sequence, uint64_t timestamp_ns, + uint32_t frames, const float *stereo_pcm) +{ + return ul_audio_encode_audio_version( + UL_AUDIO_PROTOCOL_LEGACY_VERSION, out, capacity, session_id, bus, + sequence, timestamp_ns, frames, stereo_pcm); +} + +size_t ul_audio_encode_gap_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t bus, uint64_t first_sequence, + uint64_t count, uint64_t timestamp_ns) { const size_t packet_size = UL_AUDIO_HEADER_BYTES + 41u; uint8_t *body; - if (!valid_common(out, capacity, session_id, packet_size) || bus > 5u || + if (!valid_version(version) || + !valid_common(out, capacity, session_id, packet_size) || bus > 5u || first_sequence > UL_AUDIO_MAX_SEQUENCE || count == 0u || count > UINT64_MAX - first_sequence) return 0u; - put_header(out, UL_AUDIO_KIND_GAP, 41u); + put_header(out, version, UL_AUDIO_KIND_GAP, 41u); body = out + UL_AUDIO_HEADER_BYTES; memcpy(body, session_id, 16u); body[16] = bus; @@ -138,14 +176,24 @@ size_t ul_audio_encode_gap(uint8_t *out, size_t capacity, return packet_size; } -size_t ul_audio_encode_end(uint8_t *out, size_t capacity, - const uint8_t session_id[16], uint8_t reason, - const ul_audio_end_sequence *last_sequences, - size_t sequence_count) +size_t ul_audio_encode_gap(uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t bus, + uint64_t first_sequence, uint64_t count, + uint64_t timestamp_ns) +{ + return ul_audio_encode_gap_version( + UL_AUDIO_PROTOCOL_LEGACY_VERSION, out, capacity, session_id, bus, + first_sequence, count, timestamp_ns); +} + +size_t ul_audio_encode_end_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t reason, + const ul_audio_end_sequence *last_sequences, size_t sequence_count) { size_t body_size, packet_size, index; uint8_t *body; - if (sequence_count > 6u || + if (!valid_version(version) || sequence_count > 6u || (sequence_count == 0u && reason != UL_AUDIO_END_DISARMED) || (sequence_count != 0u && last_sequences == NULL) || reason < UL_AUDIO_END_STREAM_STOPPED || @@ -161,7 +209,7 @@ size_t ul_audio_encode_end(uint8_t *out, size_t capacity, (entry->has_sequence && entry->sequence > UL_AUDIO_MAX_SEQUENCE)) return 0u; } - put_header(out, UL_AUDIO_KIND_END, (uint32_t)body_size); + put_header(out, version, UL_AUDIO_KIND_END, (uint32_t)body_size); body = out + UL_AUDIO_HEADER_BYTES; memcpy(body, session_id, 16u); body[16] = reason; @@ -174,3 +222,163 @@ size_t ul_audio_encode_end(uint8_t *out, size_t capacity, } return packet_size; } + +size_t ul_audio_encode_end(uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t reason, + const ul_audio_end_sequence *last_sequences, + size_t sequence_count) +{ + return ul_audio_encode_end_version( + UL_AUDIO_PROTOCOL_LEGACY_VERSION, out, capacity, session_id, reason, + last_sequences, sequence_count); +} + +static bool valid_utf8_label(const uint8_t *text, size_t length, + size_t maximum) +{ + size_t index = 0u; + bool has_visible_codepoint = false; + if (text == NULL || length == 0u || length > maximum) + return false; + while (index < length) { + uint32_t codepoint; + uint8_t first = text[index++]; + if (first < 0x80u) { + codepoint = first; + } else if (first >= 0xc2u && first <= 0xdfu) { + if (index >= length || (text[index] & 0xc0u) != 0x80u) + return false; + codepoint = ((uint32_t)(first & 0x1fu) << 6) | + (uint32_t)(text[index++] & 0x3fu); + } else if (first >= 0xe0u && first <= 0xefu) { + uint8_t second; + if (length - index < 2u) + return false; + second = text[index]; + if ((second & 0xc0u) != 0x80u || + (text[index + 1u] & 0xc0u) != 0x80u || + (first == 0xe0u && second < 0xa0u) || + (first == 0xedu && second >= 0xa0u)) + return false; + codepoint = ((uint32_t)(first & 0x0fu) << 12) | + ((uint32_t)(second & 0x3fu) << 6) | + (uint32_t)(text[index + 1u] & 0x3fu); + index += 2u; + } else if (first >= 0xf0u && first <= 0xf4u) { + uint8_t second; + if (length - index < 3u) + return false; + second = text[index]; + if ((second & 0xc0u) != 0x80u || + (text[index + 1u] & 0xc0u) != 0x80u || + (text[index + 2u] & 0xc0u) != 0x80u || + (first == 0xf0u && second < 0x90u) || + (first == 0xf4u && second > 0x8fu)) + return false; + codepoint = ((uint32_t)(first & 0x07u) << 18) | + ((uint32_t)(second & 0x3fu) << 12) | + ((uint32_t)(text[index + 1u] & 0x3fu) << 6) | + (uint32_t)(text[index + 2u] & 0x3fu); + index += 3u; + } else { + return false; + } + if (codepoint <= 0x1fu || + (codepoint >= 0x7fu && codepoint <= 0x9fu) || + codepoint == 0x061cu || codepoint == 0x200bu || + codepoint == 0x200eu || codepoint == 0x200fu || + (codepoint >= 0x2028u && codepoint <= 0x202eu) || + (codepoint >= 0x2066u && codepoint <= 0x2069u) || + codepoint == 0xfeffu) + return false; + if (codepoint != 0x20u && codepoint != 0x00a0u && + codepoint != 0x1680u && + !(codepoint >= 0x2000u && codepoint <= 0x200au) && + codepoint != 0x202fu && codepoint != 0x205fu && + codepoint != 0x3000u && codepoint != 0x200cu && + codepoint != 0x200du) + has_visible_codepoint = true; + } + return has_visible_codepoint; +} + +size_t ul_audio_encode_routing( + uint8_t *out, size_t capacity, const uint8_t session_id[16], + uint64_t revision, uint64_t observed_at_ns, uint8_t primary_bus, + uint8_t bus_mask, const ul_audio_routing_bus *buses, size_t bus_count, + const ul_audio_routing_source *sources, size_t source_count) +{ + size_t body_size = 36u, packet_size, index; + uint8_t expected_bus = 0u; + uint8_t *body, *cursor; + if (out == NULL || session_id == NULL || + capacity < UL_AUDIO_HEADER_BYTES + 36u || + revision == 0u || revision > UL_AUDIO_MAX_SEQUENCE || + primary_bus > 5u || bus_mask == 0u || bus_mask > 0x3fu || + (bus_mask & (uint8_t)(1u << primary_bus)) == 0u || + bus_count == 0u || bus_count > 6u || buses == NULL || + source_count > UL_AUDIO_MAX_ROUTING_SOURCES || + (source_count != 0u && sources == NULL)) + return 0u; + for (index = 0u; index < bus_count; ++index) { + while (expected_bus < 6u && + (bus_mask & (uint8_t)(1u << expected_bus)) == 0u) + ++expected_bus; + if (expected_bus >= 6u || buses[index].bus != expected_bus || + !valid_utf8_label(buses[index].label, buses[index].label_length, + UL_AUDIO_MAX_BUS_LABEL_BYTES)) + return 0u; + body_size += 11u + buses[index].label_length; + ++expected_bus; + } + while (expected_bus < 6u && + (bus_mask & (uint8_t)(1u << expected_bus)) == 0u) + ++expected_bus; + if (expected_bus != 6u) + return 0u; + for (index = 0u; index < source_count; ++index) { + if (sources[index].selected_mask == 0u || + (sources[index].selected_mask & ~bus_mask) != 0u || + !valid_utf8_label(sources[index].name, sources[index].name_length, + UL_AUDIO_MAX_SOURCE_NAME_BYTES) || + (index != 0u && + memcmp(sources[index - 1u].source_id, + sources[index].source_id, 16u) >= 0)) + return 0u; + body_size += 19u + sources[index].name_length; + } + if (body_size > UL_AUDIO_MAX_ROUTING_BODY_BYTES) + return 0u; + packet_size = UL_AUDIO_HEADER_BYTES + body_size; + if (capacity < packet_size) + return 0u; + + put_header(out, 2u, UL_AUDIO_KIND_ROUTING, (uint32_t)body_size); + body = out + UL_AUDIO_HEADER_BYTES; + memcpy(body, session_id, 16u); + put_u64(body + 16u, revision); + put_u64(body + 24u, observed_at_ns); + body[32] = primary_bus; + body[33] = bus_mask; + put_u16(body + 34u, (uint16_t)source_count); + cursor = body + 36u; + for (index = 0u; index < bus_count; ++index) { + *cursor++ = buses[index].bus; + put_u64(cursor, buses[index].next_sequence); + cursor += 8u; + put_u16(cursor, (uint16_t)buses[index].label_length); + cursor += 2u; + memcpy(cursor, buses[index].label, buses[index].label_length); + cursor += buses[index].label_length; + } + for (index = 0u; index < source_count; ++index) { + memcpy(cursor, sources[index].source_id, 16u); + cursor += 16u; + *cursor++ = sources[index].selected_mask; + put_u16(cursor, (uint16_t)sources[index].name_length); + cursor += 2u; + memcpy(cursor, sources[index].name, sources[index].name_length); + cursor += sources[index].name_length; + } + return packet_size; +} diff --git a/native/obs-plugin/src/audio_protocol.h b/native/obs-plugin/src/audio_protocol.h index 5dd44b5..b91a803 100644 --- a/native/obs-plugin/src/audio_protocol.h +++ b/native/obs-plugin/src/audio_protocol.h @@ -10,6 +10,21 @@ #define UL_AUDIO_MAX_FRAMES 8192u #define UL_AUDIO_MAX_PACKET_BYTES 65585u #define UL_AUDIO_MAX_SEQUENCE (UINT64_MAX - UINT64_C(1)) +#define UL_AUDIO_MAX_ROUTING_SOURCES 128u +#define UL_AUDIO_MAX_BUS_LABEL_BYTES 64u +#define UL_AUDIO_MAX_SOURCE_NAME_BYTES 128u +#define UL_AUDIO_MAX_ROUTING_BODY_BYTES 19302u +#define UL_AUDIO_MAX_ROUTING_PACKET_BYTES \ + (UL_AUDIO_HEADER_BYTES + UL_AUDIO_MAX_ROUTING_BODY_BYTES) +#define UL_AUDIO_PROTOCOL_LEGACY_VERSION 1u +#define UL_AUDIO_PROTOCOL_PROVENANCE_VERSION 2u +#ifndef UL_AUDIO_RUNTIME_VERSION +#define UL_AUDIO_RUNTIME_VERSION UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +#endif +#if UL_AUDIO_RUNTIME_VERSION != UL_AUDIO_PROTOCOL_LEGACY_VERSION && \ + UL_AUDIO_RUNTIME_VERSION != UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +#error "UL_AUDIO_RUNTIME_VERSION must be 1 or 2" +#endif enum ul_audio_end_reason { UL_AUDIO_END_STREAM_STOPPED = 1, @@ -25,6 +40,20 @@ typedef struct ul_audio_end_sequence { uint64_t sequence; } ul_audio_end_sequence; +typedef struct ul_audio_routing_bus { + uint8_t bus; + uint64_t next_sequence; + const uint8_t *label; + size_t label_length; +} ul_audio_routing_bus; + +typedef struct ul_audio_routing_source { + uint8_t source_id[16]; + uint8_t selected_mask; + const uint8_t *name; + size_t name_length; +} ul_audio_routing_source; + /* These encoders allocate nothing and return the exact encoded byte count. * Zero means invalid metadata, invalid PCM, or insufficient output capacity. * The caller's output bytes are left untouched on failure. Session IDs are @@ -34,20 +63,44 @@ size_t ul_audio_encode_start(uint8_t *out, size_t capacity, const uint8_t session_id[16], uint32_t sample_rate, uint8_t primary_bus, uint8_t bus_mask, uint64_t origin_ns); +size_t ul_audio_encode_start_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint32_t sample_rate, + uint8_t primary_bus, uint8_t bus_mask, uint64_t origin_ns); size_t ul_audio_encode_audio(uint8_t *out, size_t capacity, const uint8_t session_id[16], uint8_t bus, uint64_t sequence, uint64_t timestamp_ns, uint32_t frames, const float *stereo_pcm); +size_t ul_audio_encode_audio_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t bus, uint64_t sequence, + uint64_t timestamp_ns, uint32_t frames, const float *stereo_pcm); size_t ul_audio_encode_gap(uint8_t *out, size_t capacity, const uint8_t session_id[16], uint8_t bus, uint64_t first_sequence, uint64_t count, uint64_t timestamp_ns); +size_t ul_audio_encode_gap_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t bus, uint64_t first_sequence, + uint64_t count, uint64_t timestamp_ns); size_t ul_audio_encode_end(uint8_t *out, size_t capacity, const uint8_t session_id[16], uint8_t reason, const ul_audio_end_sequence *last_sequences, size_t sequence_count); +size_t ul_audio_encode_end_version( + uint8_t version, uint8_t *out, size_t capacity, + const uint8_t session_id[16], uint8_t reason, + const ul_audio_end_sequence *last_sequences, size_t sequence_count); + +/* Routing is an observation-only version-2 record. Labels and source names + * are already-private UTF-8 bytes and are never logged here. */ +size_t ul_audio_encode_routing( + uint8_t *out, size_t capacity, const uint8_t session_id[16], + uint64_t revision, uint64_t observed_at_ns, uint8_t primary_bus, + uint8_t bus_mask, const ul_audio_routing_bus *buses, size_t bus_count, + const ul_audio_routing_source *sources, size_t source_count); #endif diff --git a/native/obs-plugin/src/audio_stream.c b/native/obs-plugin/src/audio_stream.c index 7266390..f2cc790 100644 --- a/native/obs-plugin/src/audio_stream.c +++ b/native/obs-plugin/src/audio_stream.c @@ -4,6 +4,9 @@ #include "audio_protocol.h" #include "audio_queue.h" #include "session_protocol.h" +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +#include "audio_metadata.h" +#endif #include #include @@ -32,6 +35,9 @@ #ifndef UL_AUDIO_STREAM_COMMAND_TIMEOUT_MS #define UL_AUDIO_STREAM_COMMAND_TIMEOUT_MS 1000u #endif +#ifndef UL_AUDIO_STREAM_METADATA_POLL_MS +#define UL_AUDIO_STREAM_METADATA_POLL_MS 250u +#endif #define UL_AUDIO_STREAM_DISARMED_CONTROL 100 #define UL_AUDIO_STREAM_STOPPING_CONTROL 101 @@ -60,6 +66,12 @@ typedef struct stream_state { bool disarm_consumed; bool disarm_accepted; ULONGLONG drain_started_at; +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + uintptr_t metadata_generation; + uint64_t metadata_token; + uint64_t routing_revision; + ULONGLONG metadata_requested_at; +#endif } stream_state; static DWORD io_timeout(const stream_state *state, DWORD requested); @@ -215,10 +227,10 @@ static int send_converted_audio(stream_state *state, uint8_t bus, const float *stereo) { size_t size; - size = ul_audio_encode_audio(state->packet, sizeof(state->packet), - state->session, bus, block->info.sequence, - block->info.timestamp_ns, block->info.frames, - stereo); + size = ul_audio_encode_audio_version( + UL_AUDIO_RUNTIME_VERSION, state->packet, sizeof(state->packet), + state->session, bus, block->info.sequence, block->info.timestamp_ns, + block->info.frames, stereo); if (!write_packet(state, size)) return UL_AUDIO_STREAM_TRANSPORT_ERROR; state->sent[bus] = true; @@ -286,8 +298,9 @@ static bool send_end(stream_state *state, uint8_t reason, bool allow_disarm) entries[count].sequence = state->last_sequence[bus]; count++; } - size = ul_audio_encode_end(state->packet, sizeof(state->packet), - state->session, reason, entries, count); + size = ul_audio_encode_end_version( + UL_AUDIO_RUNTIME_VERSION, state->packet, sizeof(state->packet), + state->session, reason, entries, count); if (!write_packet(state, size)) return false; return receive_end_ack(state, allow_disarm); @@ -314,16 +327,30 @@ static int stage_first_blocks(stream_state *state, bool all_stopped = true; DWORD wait_result; int control = poll_control(state); - if (control == UL_AUDIO_STREAM_STOPPING_CONTROL) - return UL_AUDIO_STREAM_INCOMPLETE; if (control != UL_AUDIO_STREAM_OK && - control != UL_AUDIO_STREAM_DISARMED_CONTROL) + control != UL_AUDIO_STREAM_DISARMED_CONTROL && + control != UL_AUDIO_STREAM_STOPPING_CONTROL) return UL_AUDIO_STREAM_TRANSPORT_ERROR; if (ul_audio_capture_failed(state->capture)) return UL_AUDIO_STREAM_SOURCE_CHANGED; +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + if (state->metadata_generation == 0u || + ul_audio_metadata_failed(state->metadata_generation)) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + if (GetTickCount64() - state->metadata_requested_at >= + UL_AUDIO_STREAM_METADATA_POLL_MS) { + if (!ul_audio_metadata_request_worker( + state->metadata_generation)) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + state->metadata_requested_at = GetTickCount64(); + } +#endif wait_result = WaitForSingleObject(state->stop_event, 0u); - if (wait_result == WAIT_OBJECT_0) - return UL_AUDIO_STREAM_INCOMPLETE; + if (wait_result == WAIT_OBJECT_0 && !state->draining) { + state->draining = true; + state->drain_started_at = GetTickCount64(); + deactivate(state); + } if (wait_result == WAIT_FAILED) return UL_AUDIO_STREAM_SOURCE_CHANGED; for (bus = 0u; bus < UL_AUDIO_CAPTURE_MIXES; ++bus) { @@ -344,8 +371,7 @@ static int stage_first_blocks(stream_state *state, ul_audio_queue_result status = ul_audio_queue_status(queue); if (status == UL_AUDIO_QUEUE_OK) all_stopped = false; - else if (!state->disarm_accepted || - status != UL_AUDIO_QUEUE_STOPPED) + else if (status != UL_AUDIO_QUEUE_STOPPED) return UL_AUDIO_STREAM_SOURCE_CHANGED; } } @@ -354,14 +380,20 @@ static int stage_first_blocks(stream_state *state, if (state->disarm_accepted && all_stopped) return ready_count == 0u ? UL_AUDIO_STREAM_DISARMED_CONTROL : UL_AUDIO_STREAM_INCOMPLETE; - if (GetTickCount64() - started_at >= UL_AUDIO_STREAM_FIRST_TIMEOUT_MS) - return UL_AUDIO_STREAM_SOURCE_CHANGED; - wait_result = WaitForSingleObject(state->stop_event, - UL_AUDIO_STREAM_POLL_MS); - if (wait_result == WAIT_OBJECT_0) + if (state->draining && all_stopped) return UL_AUDIO_STREAM_INCOMPLETE; - if (wait_result == WAIT_FAILED) + if (state->draining && io_timeout(state, 1u) == 0u) + return UL_AUDIO_STREAM_INCOMPLETE; + if (GetTickCount64() - started_at >= UL_AUDIO_STREAM_FIRST_TIMEOUT_MS) return UL_AUDIO_STREAM_SOURCE_CHANGED; + if (state->draining) { + Sleep(UL_AUDIO_STREAM_POLL_MS); + } else { + wait_result = WaitForSingleObject(state->stop_event, + UL_AUDIO_STREAM_POLL_MS); + if (wait_result == WAIT_FAILED) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + } } if (!ul_audio_capture_origin(state->capture, &state->origin_ns)) return UL_AUDIO_STREAM_SOURCE_CHANGED; @@ -394,8 +426,9 @@ static int finish_empty_disarm(stream_state *state) size_t size; if (!wait_cleanup(state)) return UL_AUDIO_STREAM_INCOMPLETE; - size = ul_audio_encode_end(state->packet, sizeof(state->packet), - state->session, UL_AUDIO_END_DISARMED, NULL, 0u); + size = ul_audio_encode_end_version( + UL_AUDIO_RUNTIME_VERSION, state->packet, sizeof(state->packet), + state->session, UL_AUDIO_END_DISARMED, NULL, 0u); if (!write_packet(state, size) || !receive_end_ack(state, false)) return UL_AUDIO_STREAM_TRANSPORT_ERROR; return UL_AUDIO_STREAM_OK; @@ -407,11 +440,10 @@ static int send_gap_failure(stream_state *state, uint8_t bus, size_t size; if (!valid_block(state, bus, block, true) || block->info.gap.count == 0u) return source_failure(state, UL_AUDIO_END_SOURCE_CHANGED); - size = ul_audio_encode_gap(state->packet, sizeof(state->packet), - state->session, bus, - block->info.gap.first_sequence, - block->info.gap.count, - block->info.gap.timestamp_ns); + size = ul_audio_encode_gap_version( + UL_AUDIO_RUNTIME_VERSION, state->packet, sizeof(state->packet), + state->session, bus, block->info.gap.first_sequence, + block->info.gap.count, block->info.gap.timestamp_ns); if (!write_packet(state, size)) { deactivate(state); return UL_AUDIO_STREAM_TRANSPORT_ERROR; @@ -421,6 +453,94 @@ static int send_gap_failure(stream_state *state, uint8_t bus, return UL_AUDIO_STREAM_SOURCE_CHANGED; } +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +static int send_pending_routing(stream_state *state, bool required) +{ + ul_audio_metadata_snapshot snapshot; + ul_audio_routing_bus buses[UL_AUDIO_CAPTURE_MIXES]; + ul_audio_routing_source sources[UL_AUDIO_METADATA_MAX_SOURCES]; + size_t bus_count = 0u, index, size; + uint8_t bus; + if (state->metadata_generation == 0u || + ul_audio_metadata_failed(state->metadata_generation)) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + if (!ul_audio_metadata_take_worker(state->metadata_generation, + state->metadata_token, &snapshot)) + return required ? UL_AUDIO_STREAM_SOURCE_CHANGED : UL_AUDIO_STREAM_OK; + if (snapshot.primary_bus != state->spec->primary_bus || + snapshot.bus_mask != state->spec->mix_mask || + snapshot.bus_count == 0u || + snapshot.bus_count > UL_AUDIO_CAPTURE_MIXES || + snapshot.source_count > UL_AUDIO_METADATA_MAX_SOURCES || + state->routing_revision >= UL_AUDIO_MAX_SEQUENCE) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + for (bus = 0u; bus < UL_AUDIO_CAPTURE_MIXES; ++bus) { + const ul_audio_metadata_bus *metadata_bus; + if ((snapshot.bus_mask & (uint8_t)(1u << bus)) == 0u) + continue; + if (bus_count >= snapshot.bus_count) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + metadata_bus = &snapshot.buses[bus_count]; + if (metadata_bus->bus != bus || metadata_bus->label_length == 0u || + metadata_bus->label_length > UL_AUDIO_MAX_BUS_LABEL_BYTES) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + buses[bus_count].bus = bus; + buses[bus_count].next_sequence = state->next_sequence[bus]; + buses[bus_count].label = metadata_bus->label; + buses[bus_count].label_length = metadata_bus->label_length; + bus_count++; + } + if (bus_count != snapshot.bus_count) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + for (index = 0u; index < snapshot.source_count; ++index) { + const ul_audio_metadata_source *source = &snapshot.sources[index]; + memcpy(sources[index].source_id, source->source_id, 16u); + sources[index].selected_mask = source->selected_mask; + sources[index].name = source->name; + sources[index].name_length = source->name_length; + } + size = ul_audio_encode_routing( + state->packet, sizeof(state->packet), state->session, + state->routing_revision + 1u, snapshot.observed_at_ns, + snapshot.primary_bus, snapshot.bus_mask, buses, bus_count, sources, + snapshot.source_count); + if (size == 0u) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + if (!write_packet(state, size)) + return UL_AUDIO_STREAM_TRANSPORT_ERROR; + state->routing_revision++; + state->metadata_token = snapshot.token; + return UL_AUDIO_STREAM_OK; +} + +static int poll_metadata(stream_state *state) +{ + ULONGLONG now = GetTickCount64(); + int result = send_pending_routing(state, false); + if (result != UL_AUDIO_STREAM_OK) + return result; + if (now - state->metadata_requested_at >= UL_AUDIO_STREAM_METADATA_POLL_MS) { + if (!ul_audio_metadata_request_worker(state->metadata_generation)) + return UL_AUDIO_STREAM_SOURCE_CHANGED; + state->metadata_requested_at = now; + } + return UL_AUDIO_STREAM_OK; +} +#else +static int send_pending_routing(stream_state *state, bool required) +{ + (void)state; + (void)required; + return UL_AUDIO_STREAM_OK; +} + +static int poll_metadata(stream_state *state) +{ + (void)state; + return UL_AUDIO_STREAM_OK; +} +#endif + static int drain_stream(stream_state *state) { ULONGLONG last_activity = GetTickCount64(); @@ -438,6 +558,15 @@ static int drain_stream(stream_state *state) if (control == UL_AUDIO_STREAM_DISARMED_CONTROL || control == UL_AUDIO_STREAM_STOPPING_CONTROL) stopping = true; + if (!stopping) { + int metadata_result = poll_metadata(state); + if (metadata_result == UL_AUDIO_STREAM_SOURCE_CHANGED) + return source_failure(state, UL_AUDIO_END_SOURCE_CHANGED); + if (metadata_result != UL_AUDIO_STREAM_OK) { + deactivate(state); + return UL_AUDIO_STREAM_TRANSPORT_ERROR; + } + } if (ul_audio_capture_failed(state->capture)) return source_failure(state, UL_AUDIO_END_SOURCE_CHANGED); if (!stopping) { @@ -517,9 +646,10 @@ static int drain_stream(stream_state *state) (void)send_end(state, UL_AUDIO_END_SOURCE_CHANGED, false); return UL_AUDIO_STREAM_SOURCE_CHANGED; } - size_t size = ul_audio_encode_gap( - state->packet, sizeof(state->packet), state->session, - bus, gap.first_sequence, gap.count, gap.timestamp_ns); + size_t size = ul_audio_encode_gap_version( + UL_AUDIO_RUNTIME_VERSION, state->packet, + sizeof(state->packet), state->session, bus, + gap.first_sequence, gap.count, gap.timestamp_ns); if (!write_packet(state, size)) return UL_AUDIO_STREAM_TRANSPORT_ERROR; (void)send_end(state, UL_AUDIO_END_TRANSPORT_ERROR, false); @@ -571,15 +701,22 @@ static int run_initialized(stream_state *state) return UL_AUDIO_STREAM_SOURCE_CHANGED; } } - size = ul_audio_encode_start(state->packet, sizeof(state->packet), - state->session, state->spec->sample_rate, - state->spec->primary_bus, - state->spec->mix_mask, state->origin_ns); + size = ul_audio_encode_start_version( + UL_AUDIO_RUNTIME_VERSION, state->packet, sizeof(state->packet), + state->session, state->spec->sample_rate, state->spec->primary_bus, + state->spec->mix_mask, state->origin_ns); if (!write_packet(state, size)) { deactivate(state); return UL_AUDIO_STREAM_TRANSPORT_ERROR; } state->started = true; + result = send_pending_routing(state, true); + if (result != UL_AUDIO_STREAM_OK) { + deactivate(state); + if (result == UL_AUDIO_STREAM_SOURCE_CHANGED) + (void)send_end(state, UL_AUDIO_END_SOURCE_CHANGED, false); + return result; + } for (bus = 0u; bus < UL_AUDIO_CAPTURE_MIXES; ++bus) { if ((state->spec->mix_mask & (1u << bus)) == 0u) continue; @@ -604,6 +741,20 @@ int ul_audio_stream_run(ul_audio_capture *capture, HANDLE cleanup_complete, ul_audio_disarm_callback disarm, void *disarm_context) +{ + return ul_audio_stream_run_metadata( + capture, spec, admission, session, stop_event, cleanup_complete, + disarm, disarm_context, 0u); +} + +int ul_audio_stream_run_metadata(ul_audio_capture *capture, + const ul_audio_capture_spec *spec, + ul_admission *admission, + const uint8_t session[16], HANDLE stop_event, + HANDLE cleanup_complete, + ul_audio_disarm_callback disarm, + void *disarm_context, + uintptr_t metadata_generation) { stream_state state = {0}; state.capture = capture; @@ -614,6 +765,12 @@ int ul_audio_stream_run(ul_audio_capture *capture, state.cleanup_complete = cleanup_complete; state.disarm = disarm; state.disarm_context = disarm_context; +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + state.metadata_generation = metadata_generation; + state.metadata_requested_at = GetTickCount64(); +#else + (void)metadata_generation; +#endif return run_initialized(&state); } @@ -622,6 +779,15 @@ int ul_audio_stream_run_disarmed(ul_audio_capture *capture, ul_admission *admission, const uint8_t session[16], HANDLE stop_event, HANDLE cleanup_complete) +{ + return ul_audio_stream_run_disarmed_metadata( + capture, spec, admission, session, stop_event, cleanup_complete, 0u); +} + +int ul_audio_stream_run_disarmed_metadata( + ul_audio_capture *capture, const ul_audio_capture_spec *spec, + ul_admission *admission, const uint8_t session[16], HANDLE stop_event, + HANDLE cleanup_complete, uintptr_t metadata_generation) { stream_state state = {0}; state.capture = capture; @@ -634,6 +800,12 @@ int ul_audio_stream_run_disarmed(ul_audio_capture *capture, state.disarm_accepted = true; state.draining = true; state.drain_started_at = GetTickCount64(); +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + state.metadata_generation = metadata_generation; + state.metadata_requested_at = GetTickCount64(); +#else + (void)metadata_generation; +#endif return run_initialized(&state); } diff --git a/native/obs-plugin/src/audio_stream.h b/native/obs-plugin/src/audio_stream.h index 4198ba4..f7e321e 100644 --- a/native/obs-plugin/src/audio_stream.h +++ b/native/obs-plugin/src/audio_stream.h @@ -42,6 +42,19 @@ int ul_audio_stream_run(ul_audio_capture *capture, ul_audio_disarm_callback disarm, void *disarm_context); +/* Version-selectable runtime entry used by the plugin session worker. A + * version-2 run requires the current generation's metadata observation. Its + * final immutable snapshot remains usable after frontend watcher disconnect + * so an already-accepted stop/Disarm tail can finish; version 1 ignores it. */ +int ul_audio_stream_run_metadata(ul_audio_capture *capture, + const ul_audio_capture_spec *spec, + ul_admission *admission, + const uint8_t session[16], HANDLE stop_event, + HANDLE cleanup_complete, + ul_audio_disarm_callback disarm, + void *disarm_context, + uintptr_t metadata_generation); + /* Continues after plugin_state already consumed Disarm and atomically observed * an attached capture. The capture must already be deactivated. */ int ul_audio_stream_run_disarmed(ul_audio_capture *capture, @@ -49,6 +62,10 @@ int ul_audio_stream_run_disarmed(ul_audio_capture *capture, ul_admission *admission, const uint8_t session[16], HANDLE stop_event, HANDLE cleanup_complete); +int ul_audio_stream_run_disarmed_metadata( + ul_audio_capture *capture, const ul_audio_capture_spec *spec, + ul_admission *admission, const uint8_t session[16], HANDLE stop_event, + HANDLE cleanup_complete, uintptr_t metadata_generation); /* Completes a valid Disarm before capture emitted Start. The only wire packet * is End(DISARMED) with zero sequence entries, followed by its exact receipt. diff --git a/native/obs-plugin/src/bridge.c b/native/obs-plugin/src/bridge.c index 6288900..4c52ca8 100644 --- a/native/obs-plugin/src/bridge.c +++ b/native/obs-plugin/src/bridge.c @@ -9,6 +9,7 @@ #include "pairing_ui.h" #include "vendor_dispatch.h" #include "frontend_dispatch.h" +#include "audio_metadata.h" OBS_DECLARE_MODULE() @@ -93,6 +94,14 @@ static bool queue_cleanup(uintptr_t generation) return queued; } +/* OBS can invoke source signals while holding its signal mutex. This callback + * must not acquire frontend_gate, which teardown can hold during disconnect. */ +static bool queue_metadata(uintptr_t generation) +{ + return generation != 0u && + ul_frontend_dispatch_try_post(4u, generation); +} + static void dispatch_frontend(unsigned command, uintptr_t generation) { ul_audio_capture *capture; @@ -106,14 +115,23 @@ static void dispatch_frontend(unsigned command, uintptr_t generation) queued_capture = 0u; capture = ul_plugin_capture_retain(generation); if (capture != NULL) { - bool connected = ul_audio_capture_connect_frontend(capture, generation); + bool metadata_open = + ul_plugin_metadata_open_frontend(generation); + bool connected = metadata_open && + ul_audio_capture_connect_frontend(capture, generation); + if (!connected && metadata_open) + (void)ul_plugin_metadata_close_frontend(generation, + false); ul_plugin_capture_attached(generation, capture, connected); ul_audio_capture_release(capture); } } else if (command == 3u && queued_cleanup == generation) { queued_cleanup = 0u; + (void)ul_plugin_metadata_close_frontend(generation, false); ul_audio_capture_disconnect_frontend(generation, false); ul_plugin_capture_cleanup_complete(generation); + } else if (command == 4u) { + (void)ul_plugin_metadata_refresh_frontend(generation); } } ReleaseSRWLockExclusive(&frontend_gate); @@ -164,12 +182,16 @@ static void frontend_event(enum obs_frontend_event event, void *private_data) stream_busy = true; ul_plugin_capture_stop_frontend(); ul_audio_capture_disconnect_frontend(0u, true); - ul_plugin_stream_event(UL_STREAM_STOPPING); break; + ul_plugin_stream_event(UL_STREAM_STOPPING); + (void)ul_plugin_metadata_close_frontend(0u, true); + break; case OBS_FRONTEND_EVENT_STREAMING_STOPPED: stream_busy = false; ul_plugin_capture_stop_frontend(); ul_audio_capture_disconnect_frontend(0u, true); - ul_plugin_stream_event(UL_STREAM_STOPPED); break; + ul_plugin_stream_event(UL_STREAM_STOPPED); + (void)ul_plugin_metadata_close_frontend(0u, true); + break; default: break; } } @@ -178,15 +200,16 @@ static void frontend_event(enum obs_frontend_event event, void *private_data) } /* Normal EXIT still owns live OBS audio. Disconnect before closing the * frontend gate and before any native worker join. */ + ul_vendor_set_enabled(false); + ul_plugin_stop_accepting(); AcquireSRWLockExclusive(&frontend_gate); if (frontend_open) { ul_plugin_capture_stop_frontend(); + (void)ul_plugin_metadata_close_frontend(0u, true); ul_audio_capture_disconnect_frontend(0u, true); } ReleaseSRWLockExclusive(&frontend_gate); close_frontend(); - ul_vendor_set_enabled(false); - ul_plugin_stop_accepting(); if (prepare_registered) obs_websocket_vendor_unregister_request(vendor, "PrepareSession"); if (issue_registered) @@ -208,7 +231,8 @@ bool obs_module_load(void) return false; if (!ul_frontend_dispatch_open(dispatch_frontend) || !ul_plugin_set_arm_scheduler(queue_arm) || - !ul_plugin_set_capture_schedulers(queue_capture, queue_cleanup)) { + !ul_plugin_set_capture_schedulers(queue_capture, queue_cleanup) || + !ul_plugin_set_metadata_scheduler(queue_metadata)) { ul_frontend_dispatch_close(); ul_plugin_close(); return false; @@ -257,6 +281,7 @@ void obs_module_unload(void) /* Native-only fallback: frontend/websocket teardown order is not assumed. */ close_frontend(); ul_audio_capture_abandon_after_shutdown(); + ul_audio_metadata_abandon_after_shutdown(); ul_vendor_set_enabled(false); ul_plugin_close(); } diff --git a/native/obs-plugin/src/frontend_dispatch.c b/native/obs-plugin/src/frontend_dispatch.c index 7d497ac..b7a019f 100644 --- a/native/obs-plugin/src/frontend_dispatch.c +++ b/native/obs-plugin/src/frontend_dispatch.c @@ -5,7 +5,7 @@ #define UL_FRONTEND_WM_COMMAND (WM_APP + 0x51) #define UL_FRONTEND_COMMAND_MIN 1u -#define UL_FRONTEND_COMMAND_MAX 3u +#define UL_FRONTEND_COMMAND_MAX 4u static SRWLOCK state_lock = SRWLOCK_INIT; static HWND dispatch_window; @@ -99,7 +99,8 @@ bool ul_frontend_dispatch_open(ul_frontend_dispatch_callback callback) return false; } -bool ul_frontend_dispatch_post(unsigned command, uintptr_t generation) +static bool post_command(unsigned command, uintptr_t generation, + bool nonblocking) { HWND window; bool valid; @@ -107,7 +108,12 @@ bool ul_frontend_dispatch_post(unsigned command, uintptr_t generation) if (command < UL_FRONTEND_COMMAND_MIN || command > UL_FRONTEND_COMMAND_MAX || generation == 0u) return false; - AcquireSRWLockShared(&state_lock); + if (nonblocking) { + if (!TryAcquireSRWLockShared(&state_lock)) + return false; + } else { + AcquireSRWLockShared(&state_lock); + } valid = opened && !closing && dispatch_window != NULL; window = dispatch_window; if (!valid) { @@ -120,6 +126,16 @@ bool ul_frontend_dispatch_post(unsigned command, uintptr_t generation) return valid; } +bool ul_frontend_dispatch_post(unsigned command, uintptr_t generation) +{ + return post_command(command, generation, false); +} + +bool ul_frontend_dispatch_try_post(unsigned command, uintptr_t generation) +{ + return post_command(command, generation, true); +} + void ul_frontend_dispatch_close(void) { HWND window; diff --git a/native/obs-plugin/src/frontend_dispatch.h b/native/obs-plugin/src/frontend_dispatch.h index 6ef3c07..e1e346b 100644 --- a/native/obs-plugin/src/frontend_dispatch.h +++ b/native/obs-plugin/src/frontend_dispatch.h @@ -11,13 +11,15 @@ typedef void (*ul_frontend_dispatch_callback)(unsigned command, /* Open and owner cleanup run on the frontend thread, which pumps messages. * The caller pins this module. Open must finish before concurrent post/close; * it is one-shot for this DLL lifetime, including failure. Post is safe from any thread for - * commands 1..3 and nonzero generations; the caller bounds pending work. + * commands 1..4 and nonzero generations; the caller bounds pending work. * Close from another thread makes the dispatcher inert, and the frontend * thread must call close again to destroy the window and unregister the class. * A callback already copied by the owner may finish after non-owner close; * callbacks must recheck their own lifetime gate before accessing owned state. */ bool ul_frontend_dispatch_open(ul_frontend_dispatch_callback callback); bool ul_frontend_dispatch_post(unsigned command, uintptr_t generation); +/* Callback-safe variant: fails immediately if dispatcher state is contended. */ +bool ul_frontend_dispatch_try_post(unsigned command, uintptr_t generation); void ul_frontend_dispatch_close(void); #endif diff --git a/native/obs-plugin/src/plugin_state.c b/native/obs-plugin/src/plugin_state.c index 2da80ad..c038541 100644 --- a/native/obs-plugin/src/plugin_state.c +++ b/native/obs-plugin/src/plugin_state.c @@ -2,6 +2,8 @@ #include "plugin_state.h" #include "session_protocol.h" #include "audio_stream.h" +#include "audio_protocol.h" +#include "audio_metadata.h" #include @@ -57,6 +59,7 @@ typedef struct ul_plugin_runtime { ul_arm_scheduler schedule_arm; ul_arm_scheduler schedule_capture; ul_arm_scheduler schedule_cleanup; + ul_arm_scheduler schedule_metadata; ul_audio_capture_spec capture_spec; ul_audio_capture *capture; bool capture_spec_ready; @@ -301,10 +304,17 @@ static int capture_worker(ul_plugin_runtime *runtime, ul_admission *admission) goto done; } result = was_attached +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + ? ul_audio_stream_run_disarmed_metadata( + capture, &spec, admission, + runtime->session_options.session, runtime->stream_stop, + runtime->cleanup_complete, generation) +#else ? ul_audio_stream_run_disarmed( capture, &spec, admission, runtime->session_options.session, runtime->stream_stop, runtime->cleanup_complete) +#endif : ul_audio_stream_finish_empty_disarm( admission, runtime->session_options.session, runtime->cleanup_complete); @@ -321,9 +331,16 @@ static int capture_worker(ul_plugin_runtime *runtime, ul_admission *admission) InterlockedCompareExchange(&runtime->closing, 0, 0) == 0; ReleaseSRWLockShared(&runtime->session_lock); if (accepted) { - int streamed = ul_audio_stream_run(capture, &spec, admission, + int streamed = +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + ul_audio_stream_run_metadata(capture, &spec, admission, + runtime->session_options.session, runtime->stream_stop, + runtime->cleanup_complete, request_disarm, runtime, generation); +#else + ul_audio_stream_run(capture, &spec, admission, runtime->session_options.session, runtime->stream_stop, runtime->cleanup_complete, request_disarm, runtime); +#endif result = streamed == UL_AUDIO_STREAM_OK ? UL_ADMISSION_AUTH_OK : UL_ADMISSION_REJECTED; } done: @@ -333,6 +350,9 @@ static int capture_worker(ul_plugin_runtime *runtime, ul_admission *admission) runtime->capture = NULL; ReleaseSRWLockExclusive(&runtime->session_lock); ul_audio_capture_release(capture); +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + ul_audio_metadata_retire_worker(generation); +#endif /* Posting never waits for the frontend, including when that thread is * joining us during revocation. A stale generation cannot detach new hooks. */ (void)cleanup(generation); @@ -980,6 +1000,126 @@ bool ul_plugin_set_capture_schedulers(ul_arm_scheduler attach, return accepted; } +bool ul_plugin_set_metadata_scheduler(ul_arm_scheduler refresh) +{ + ul_plugin_runtime *runtime = runtime_acquire(); + bool accepted = false; + if (runtime == NULL || refresh == NULL) { + if (runtime != NULL) + runtime_release(runtime); + return false; + } + AcquireSRWLockExclusive(&runtime->operation_lock); + AcquireSRWLockExclusive(&runtime->session_lock); + if (runtime->schedule_metadata == NULL && runtime->worker == NULL && + InterlockedCompareExchange(&runtime->closing, 0, 0) == 0) { + runtime->schedule_metadata = refresh; + accepted = true; + } + ReleaseSRWLockExclusive(&runtime->session_lock); + ReleaseSRWLockExclusive(&runtime->operation_lock); + runtime_release(runtime); + return accepted; +} + +bool ul_plugin_metadata_open_frontend(uintptr_t generation) +{ +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + ul_plugin_runtime *runtime = runtime_acquire(); + ul_audio_capture_spec spec; + ul_arm_scheduler scheduler = NULL; + bool current = false, opened; + if (runtime == NULL || generation == 0u) + return false; + AcquireSRWLockShared(&runtime->session_lock); + if (runtime->session_generation == generation && + runtime->session_phase == UL_SESSION_STARTED && + runtime->capture_spec_ready && runtime->capture_spec_valid) { + spec = runtime->capture_spec; + scheduler = runtime->schedule_metadata; + current = scheduler != NULL; + } + ReleaseSRWLockShared(&runtime->session_lock); + runtime_release(runtime); + if (!current) + return false; + opened = ul_audio_metadata_open_frontend( + generation, spec.primary_bus, spec.mix_mask, scheduler); + if (!opened) + return false; + runtime = runtime_acquire(); + if (runtime != NULL) { + AcquireSRWLockShared(&runtime->session_lock); + current = runtime->session_generation == generation && + runtime->session_phase == UL_SESSION_STARTED && + runtime->capture_spec_ready && runtime->capture_spec_valid; + ReleaseSRWLockShared(&runtime->session_lock); + runtime_release(runtime); + } else { + current = false; + } + if (!current) + (void)ul_audio_metadata_close_frontend(generation, false); + return current; +#else + (void)generation; + return true; +#endif +} + +bool ul_plugin_metadata_refresh_frontend(uintptr_t generation) +{ +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + ul_plugin_runtime *runtime = runtime_acquire(); + ul_audio_capture_spec spec; + bool current = false; + if (runtime == NULL || generation == 0u) + return false; + AcquireSRWLockShared(&runtime->session_lock); + if (runtime->session_generation == generation && + runtime->session_phase == UL_SESSION_STARTED && + runtime->capture_spec_ready && runtime->capture_spec_valid) { + spec = runtime->capture_spec; + current = true; + } + ReleaseSRWLockShared(&runtime->session_lock); + runtime_release(runtime); + if (!current) + return false; + if (!ul_audio_capture_matches_frontend(&spec) || + !ul_audio_metadata_refresh_frontend(generation)) { + runtime = runtime_acquire(); + if (runtime == NULL) + return false; + AcquireSRWLockExclusive(&runtime->session_lock); + if (runtime->session_generation == generation && + runtime->session_phase == UL_SESSION_STARTED) { + ul_audio_metadata_fail_frontend(generation); + if (runtime->capture != NULL) + ul_audio_capture_deactivate(runtime->capture); + } + ReleaseSRWLockExclusive(&runtime->session_lock); + runtime_release(runtime); + return false; + } + return true; +#else + (void)generation; + return true; +#endif +} + +bool ul_plugin_metadata_close_frontend(uintptr_t generation, bool all) +{ +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + return ul_audio_metadata_close_frontend(generation, all); +#else + (void)generation; + (void)all; + return true; +#endif +} + bool ul_plugin_capture_inspect_request(uintptr_t *generation, uint8_t *mask) { ul_plugin_runtime *runtime = runtime_acquire(); diff --git a/native/obs-plugin/src/plugin_state.h b/native/obs-plugin/src/plugin_state.h index e020f9e..3e4a7f8 100644 --- a/native/obs-plugin/src/plugin_state.h +++ b/native/obs-plugin/src/plugin_state.h @@ -49,6 +49,7 @@ typedef bool (*ul_arm_scheduler)(uintptr_t generation); bool ul_plugin_set_arm_scheduler(ul_arm_scheduler scheduler); bool ul_plugin_set_capture_schedulers(ul_arm_scheduler attach, ul_arm_scheduler cleanup); +bool ul_plugin_set_metadata_scheduler(ul_arm_scheduler refresh); void ul_plugin_arm_checked(uintptr_t generation, bool idle); void ul_plugin_stream_event(ul_stream_event event); ul_session_phase ul_plugin_session_status(void); @@ -68,6 +69,12 @@ void ul_plugin_capture_cleanup_complete(uintptr_t generation); /* Deactivates and drains callback readers without calling OBS. */ void ul_plugin_capture_stop_frontend(void); +/* Frontend-only provenance observer lifecycle for the current capture + * generation. Refresh also revalidates the streaming output/audio format. */ +bool ul_plugin_metadata_open_frontend(uintptr_t generation); +bool ul_plugin_metadata_refresh_frontend(uintptr_t generation); +bool ul_plugin_metadata_close_frontend(uintptr_t generation, bool all); + /* * Permanently pins this DLL generation before callbacks may be registered. * A process may start this component once; close is final and cannot be reset. diff --git a/native/obs-plugin/src/vendor_dispatch.c b/native/obs-plugin/src/vendor_dispatch.c index c2338cd..7b8de92 100644 --- a/native/obs-plugin/src/vendor_dispatch.c +++ b/native/obs-plugin/src/vendor_dispatch.c @@ -93,7 +93,7 @@ void ul_vendor_status(obs_data_t *request, obs_data_t *response, void *private_d return; obs_data_set_int(response, "protocolVersion", 1); obs_data_set_int(response, "commandVersion", 1); - obs_data_set_int(response, "audioVersion", 1); + obs_data_set_int(response, "audioVersion", 2); obs_data_set_int(response, "maxBusMask", 63); obs_data_set_bool(response, "ok", true); } diff --git a/native/obs-plugin/tests/audio_capture_test.c b/native/obs-plugin/tests/audio_capture_test.c index 01cf0a2..8f240cc 100644 --- a/native/obs-plugin/tests/audio_capture_test.c +++ b/native/obs-plugin/tests/audio_capture_test.c @@ -276,6 +276,8 @@ static void test_inspection_and_replacement(void) assert(spec.audio_identity == (uintptr_t)&audio_a); assert(spec.output_identity == (uintptr_t)&output_a); assert(release_calls == 1u); + assert(ul_audio_capture_matches_frontend(&spec)); + assert(release_calls == 2u); assert(!ul_audio_capture_inspect_frontend(0x40u, &zero)); assert(memcmp(&zero, &(ul_audio_capture_spec){0}, sizeof(zero)) == 0); current_output = 0u; @@ -287,6 +289,7 @@ static void test_inspection_and_replacement(void) capture = ul_audio_capture_create_worker(&spec); assert(capture != NULL); current_output = (uintptr_t)&output_b; + assert(!ul_audio_capture_matches_frontend(&spec)); assert(!ul_audio_capture_connect_frontend(capture, 1u)); current_output = (uintptr_t)&output_a; ul_audio_capture_release(capture); diff --git a/native/obs-plugin/tests/audio_metadata_test.c b/native/obs-plugin/tests/audio_metadata_test.c new file mode 100644 index 0000000..f196933 --- /dev/null +++ b/native/obs-plugin/tests/audio_metadata_test.c @@ -0,0 +1,648 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#include + +#include +#include +#include +#include +#include + +static HANDLE callback_entered; +static HANDLE callback_continue; +static HANDLE take_locked; +static HANDLE take_continue; +static HANDLE scheduler_entered; +static HANDLE scheduler_continue; +static HANDLE before_publish; +static HANDLE publish_continue; +static volatile LONG pause_callback; +static volatile LONG pause_take; +static volatile LONG pause_scheduler; +static volatile LONG pause_publish; +static uint64_t test_time_ns(void); +static void test_callback_entered(void); +static void test_take_locked(void); +static void test_before_publish(void); +#define UL_AUDIO_METADATA_CALLBACK_ENTERED() test_callback_entered() +#define UL_AUDIO_METADATA_TAKE_LOCKED() test_take_locked() +#define UL_AUDIO_METADATA_TIME_NS() test_time_ns() +#define UL_AUDIO_METADATA_BEFORE_PUBLISH() test_before_publish() +#define UL_AUDIO_METADATA_CLOSE_TIMEOUT_MS 5000u +#include "../src/audio_metadata.c" + +typedef struct signal_slot { + const char *name; + signal_callback_t callback; + void *data; +} signal_slot; + +struct signal_handler { + signal_slot slots[8]; + size_t count; +}; + +struct obs_source { + struct signal_handler handler; + enum obs_source_type type; + uint32_t flags; + uint32_t mixers; + bool removed; + bool enumerable; + bool no_handler; + LONG refs; + char uuid[37]; + char name[160]; +}; + +static struct signal_handler core_handler; +static struct obs_source sources[UL_AUDIO_METADATA_MAX_WATCHED_SOURCES + 1u]; +static size_t source_count; +static uint64_t fake_time; +static uint32_t schedule_calls; +static uintptr_t scheduled_generation; +static bool schedule_success; +static bool mutate_during_enumeration; + +static void test_callback_entered(void) +{ + if (InterlockedCompareExchange(&pause_callback, 0, 0) != 0) { + SetEvent(callback_entered); + assert(WaitForSingleObject(callback_continue, 5000u) == WAIT_OBJECT_0); + } +} + +static void test_take_locked(void) +{ + if (InterlockedCompareExchange(&pause_take, 0, 0) != 0) { + SetEvent(take_locked); + assert(WaitForSingleObject(take_continue, 5000u) == WAIT_OBJECT_0); + } +} + +static void test_before_publish(void) +{ + if (InterlockedCompareExchange(&pause_publish, 0, 0) != 0) { + SetEvent(before_publish); + assert(WaitForSingleObject(publish_continue, 5000u) == WAIT_OBJECT_0); + } +} + +static bool schedule_refresh(uintptr_t generation) +{ + schedule_calls++; + scheduled_generation = generation; + if (InterlockedCompareExchange(&pause_scheduler, 0, 0) != 0) { + SetEvent(scheduler_entered); + assert(WaitForSingleObject(scheduler_continue, 5000u) == WAIT_OBJECT_0); + } + return schedule_success; +} + +signal_handler_t *obs_get_signal_handler(void) +{ + return &core_handler; +} + +void signal_handler_connect(signal_handler_t *handler, const char *name, + signal_callback_t callback, void *data) +{ + size_t index; + assert(handler != NULL && name != NULL && callback != NULL); + for (index = 0u; index < handler->count; ++index) + if (strcmp(handler->slots[index].name, name) == 0 && + handler->slots[index].callback == callback && + handler->slots[index].data == data) + return; + assert(handler->count < sizeof(handler->slots) / sizeof(handler->slots[0])); + handler->slots[handler->count++] = (signal_slot){name, callback, data}; +} + +void signal_handler_disconnect(signal_handler_t *handler, const char *name, + signal_callback_t callback, void *data) +{ + size_t index; + assert(handler != NULL); + for (index = 0u; index < handler->count; ++index) { + signal_slot *slot = &handler->slots[index]; + if (strcmp(slot->name, name) == 0 && slot->callback == callback && + slot->data == data) { + handler->slots[index] = handler->slots[--handler->count]; + return; + } + } +} + +static void fire_signal(struct signal_handler *handler, const char *name) +{ + signal_slot copy[8]; + size_t count = handler->count, index; + memcpy(copy, handler->slots, count * sizeof(copy[0])); + for (index = 0u; index < count; ++index) + if (strcmp(copy[index].name, name) == 0) + copy[index].callback(copy[index].data, NULL); +} + +void obs_enum_sources(bool (*callback)(void *, obs_source_t *), void *data) +{ + size_t index; + for (index = 0u; index < source_count; ++index) + if (sources[index].enumerable && !sources[index].removed) { + if (!callback(data, &sources[index])) + break; + if (mutate_during_enumeration) { + mutate_during_enumeration = false; + strcpy(sources[index].name, "After snapshot signal"); + fire_signal(&sources[index].handler, "rename"); + } + } +} + +enum obs_source_type obs_source_get_type(const obs_source_t *source) +{ + return source->type; +} + +uint32_t obs_source_get_output_flags(const obs_source_t *source) +{ + return source->flags; +} + +bool obs_source_removed(const obs_source_t *source) +{ + return source->removed; +} + +obs_source_t *obs_source_get_ref(obs_source_t *source) +{ + if (source->removed) + return NULL; + InterlockedIncrement(&source->refs); + return source; +} + +void obs_source_release(obs_source_t *source) +{ + assert(InterlockedDecrement(&source->refs) >= 0); +} + +signal_handler_t *obs_source_get_signal_handler(const obs_source_t *source) +{ + return source->no_handler ? NULL : (signal_handler_t *)&source->handler; +} + +uint32_t obs_source_get_audio_mixers(const obs_source_t *source) +{ + return source->mixers; +} + +const char *obs_source_get_uuid(const obs_source_t *source) +{ + return source->uuid; +} + +const char *obs_source_get_name(const obs_source_t *source) +{ + return source->name; +} + +static uint64_t test_time_ns(void) +{ + fake_time += 10u; + return fake_time; +} + +static void make_source(size_t index, uint32_t mixers) +{ + struct obs_source *source = &sources[index]; + memset(source, 0, sizeof(*source)); + source->type = OBS_SOURCE_TYPE_INPUT; + source->flags = OBS_SOURCE_AUDIO; + source->mixers = mixers; + source->enumerable = true; + assert(snprintf(source->uuid, sizeof(source->uuid), + "00000000-0000-0000-0000-%012llx", + (unsigned long long)(index + 1u)) == 36); + assert(snprintf(source->name, sizeof(source->name), + "Input %u", (unsigned)index) > 0); +} + +static void reset_fixture(void) +{ + size_t index; + assert(!atomic_load_explicit(&metadata.active, memory_order_acquire)); + memset(&core_handler, 0, sizeof(core_handler)); + memset(sources, 0, sizeof(sources)); + source_count = 0u; + fake_time = 1000u; + schedule_calls = 0u; + scheduled_generation = 0u; + schedule_success = true; + mutate_during_enumeration = false; + InterlockedExchange(&pause_callback, 0); + InterlockedExchange(&pause_take, 0); + InterlockedExchange(&pause_scheduler, 0); + InterlockedExchange(&pause_publish, 0); + ResetEvent(callback_entered); + ResetEvent(callback_continue); + ResetEvent(take_locked); + ResetEvent(take_continue); + ResetEvent(scheduler_entered); + ResetEvent(scheduler_continue); + ResetEvent(before_publish); + ResetEvent(publish_continue); + atomic_store_explicit(&metadata.poisoned, false, memory_order_release); + for (index = 0u; index < UL_AUDIO_METADATA_MAX_WATCHED_SOURCES + 1u; + ++index) + assert(sources[index].refs == 0); +} + +static void test_initial_snapshot_stabilizes(void) +{ + ul_audio_metadata_snapshot snapshot; + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + mutate_during_enumeration = true; + assert(ul_audio_metadata_open_frontend(6u, 0u, 1u, schedule_refresh)); + assert(ul_audio_metadata_take_worker(6u, 0u, &snapshot)); + assert(snapshot.token == 2u && snapshot.source_count == 1u); + assert(snapshot.sources[0].name_length == strlen("After snapshot signal")); + assert(memcmp(snapshot.sources[0].name, "After snapshot signal", + snapshot.sources[0].name_length) == 0); + assert(ul_audio_metadata_close_frontend(6u, false)); +} + +static void test_initial_change_remove_and_close(void) +{ + ul_audio_metadata_snapshot snapshot; + uint64_t token; + reset_fixture(); + source_count = 4u; + make_source(0u, 1u); + make_source(1u, 3u); + make_source(2u, 3u); + make_source(3u, 0u); + sources[0].uuid[35] = '2'; + sources[1].uuid[35] = '1'; + sources[2].flags = OBS_SOURCE_VIDEO; + assert(ul_audio_metadata_open_frontend(7u, 0u, 3u, schedule_refresh)); + assert(ul_audio_metadata_take_worker(7u, 0u, &snapshot)); + assert(snapshot.token == 1u && snapshot.observed_at_ns == 1010u); + assert(snapshot.primary_bus == 0u && snapshot.bus_mask == 3u); + assert(snapshot.bus_count == 2u && snapshot.source_count == 2u); + assert(snapshot.buses[0].bus == 0u && snapshot.buses[1].bus == 1u); + assert(memcmp(snapshot.buses[0].label, "Mix 1", 5u) == 0); + assert(memcmp(snapshot.buses[1].label, "Mix 2", 5u) == 0); + assert(snapshot.sources[0].source_id[15] == 1u); + assert(snapshot.sources[1].source_id[15] == 2u); + assert(snapshot.sources[0].selected_mask == 3u); + assert(snapshot.sources[1].selected_mask == 1u); + assert(sources[0].refs == 1 && sources[1].refs == 1 && + sources[3].refs == 1); + token = snapshot.token; + + sources[0].mixers = 2u; + strcpy(sources[0].name, "Renamed"); + fire_signal(&sources[0].handler, "audio_mixers"); + fire_signal(&sources[0].handler, "rename"); + assert(schedule_calls == 1u && scheduled_generation == 7u); + assert(ul_audio_metadata_refresh_frontend(7u)); + assert(ul_audio_metadata_take_worker(7u, token, &snapshot)); + assert(snapshot.token == 2u && snapshot.sources[1].selected_mask == 2u); + token = snapshot.token; + + fire_signal(&sources[0].handler, "rename"); + assert(ul_audio_metadata_refresh_frontend(7u)); + assert(!ul_audio_metadata_take_worker(7u, token, &snapshot)); + + sources[3].mixers = 1u; + fire_signal(&sources[3].handler, "audio_mixers"); + assert(ul_audio_metadata_refresh_frontend(7u)); + assert(ul_audio_metadata_take_worker(7u, token, &snapshot)); + assert(snapshot.token == 3u && snapshot.source_count == 3u); + token = snapshot.token; + sources[3].removed = true; + sources[3].enumerable = false; + fire_signal(&sources[3].handler, "remove"); + assert(ul_audio_metadata_refresh_frontend(7u)); + assert(ul_audio_metadata_take_worker(7u, token, &snapshot)); + assert(snapshot.token == 4u && snapshot.source_count == 2u); + assert(sources[3].refs == 0 && sources[3].handler.count == 0u); + token = snapshot.token; + + sources[0].removed = true; + sources[0].enumerable = false; + fire_signal(&sources[0].handler, "remove"); + assert(ul_audio_metadata_refresh_frontend(7u)); + assert(ul_audio_metadata_take_worker(7u, token, &snapshot)); + assert(snapshot.token == 5u && snapshot.source_count == 1u); + token = snapshot.token; + assert(sources[0].refs == 0 && sources[0].handler.count == 0u); + assert(ul_audio_metadata_close_frontend(7u, false)); + assert(sources[1].refs == 0 && sources[1].handler.count == 0u); + assert(core_handler.count == 0u); + assert(ul_audio_metadata_take_worker(7u, 0u, &snapshot)); + assert(snapshot.token == token); + ul_audio_metadata_retire_worker(7u); + assert(!ul_audio_metadata_take_worker(7u, 0u, &snapshot)); +} + +static void test_visible_bounds_and_text_failures(void) +{ + size_t index; + reset_fixture(); + source_count = UL_AUDIO_METADATA_MAX_SOURCES + 1u; + for (index = 0u; index < source_count; ++index) + make_source(index, 1u); + assert(!ul_audio_metadata_open_frontend(8u, 0u, 1u, schedule_refresh)); + for (index = 0u; index < source_count; ++index) + assert(sources[index].refs == 0 && sources[index].handler.count == 0u); + + reset_fixture(); + source_count = UL_AUDIO_METADATA_MAX_WATCHED_SOURCES + 1u; + for (index = 0u; index < source_count; ++index) + make_source(index, 0u); + assert(!ul_audio_metadata_open_frontend(9u, 0u, 1u, schedule_refresh)); + for (index = 0u; index < source_count; ++index) + assert(sources[index].refs == 0 && sources[index].handler.count == 0u); + + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + strcpy(sources[0].name, " "); + assert(!ul_audio_metadata_open_frontend(10u, 0u, 1u, + schedule_refresh)); + assert(sources[0].refs == 0 && sources[0].handler.count == 0u); + + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + memcpy(sources[0].name, "\xe2\x80\x8d\xe2\x80\x8c", 7u); + assert(!ul_audio_metadata_open_frontend(11u, 0u, 1u, + schedule_refresh)); + + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + memset(sources[0].name, 'x', UL_AUDIO_METADATA_MAX_SOURCE_NAME_BYTES + 1u); + sources[0].name[UL_AUDIO_METADATA_MAX_SOURCE_NAME_BYTES + 1u] = '\0'; + assert(!ul_audio_metadata_open_frontend(11u, 0u, 1u, + schedule_refresh)); + + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + sources[0].no_handler = true; + assert(!ul_audio_metadata_open_frontend(11u, 0u, 1u, + schedule_refresh)); + assert(sources[0].refs == 0); +} + +static void test_scheduler_failure_is_terminal(void) +{ + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + assert(ul_audio_metadata_open_frontend(12u, 0u, 1u, schedule_refresh)); + schedule_success = false; + fire_signal(&sources[0].handler, "audio_mixers"); + assert(schedule_calls == 1u && ul_audio_metadata_failed(12u)); + assert(!ul_audio_metadata_refresh_frontend(12u)); + assert(ul_audio_metadata_close_frontend(12u, false)); +} + +static void test_worker_request_and_frontend_failure(void) +{ + ul_audio_metadata_snapshot snapshot; + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + assert(ul_audio_metadata_open_frontend(13u, 0u, 1u, schedule_refresh)); + assert(ul_audio_metadata_take_worker(13u, 0u, &snapshot)); + schedule_calls = 0u; + assert(!ul_audio_metadata_request_worker(12u)); + assert(ul_audio_metadata_request_worker(13u)); + assert(schedule_calls == 1u); + assert(ul_audio_metadata_refresh_frontend(13u)); + assert(!ul_audio_metadata_failed(13u)); + ul_audio_metadata_fail_frontend(12u); + assert(!ul_audio_metadata_failed(13u)); + ul_audio_metadata_fail_frontend(13u); + assert(ul_audio_metadata_failed(13u)); + assert(!ul_audio_metadata_take_worker(13u, 0u, &snapshot)); + assert(!ul_audio_metadata_request_worker(13u)); + assert(ul_audio_metadata_close_frontend(13u, false)); +} + +static void test_abandon_makes_callbacks_inert(void) +{ + ul_audio_metadata_snapshot snapshot; + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + assert(ul_audio_metadata_open_frontend(16u, 0u, 1u, schedule_refresh)); + schedule_calls = 0u; + ul_audio_metadata_abandon_after_shutdown(); + fire_signal(&sources[0].handler, "audio_mixers"); + assert(schedule_calls == 0u && ul_audio_metadata_failed(16u)); + assert(!ul_audio_metadata_take_worker(16u, 0u, &snapshot)); + assert(!ul_audio_metadata_request_worker(16u)); +} + +typedef struct refresh_context { + uintptr_t generation; + bool result; +} refresh_context; + +static DWORD WINAPI refresh_thread(void *parameter) +{ + refresh_context *context = parameter; + context->result = ul_audio_metadata_refresh_frontend(context->generation); + return 0u; +} + +static void test_retire_blocks_inflight_republish(void) +{ + ul_audio_metadata_snapshot snapshot; + refresh_context context = {17u, true}; + HANDLE worker; + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + assert(ul_audio_metadata_open_frontend(17u, 0u, 1u, schedule_refresh)); + assert(ul_audio_metadata_take_worker(17u, 0u, &snapshot)); + strcpy(sources[0].name, "Changed while retiring"); + InterlockedExchange(&pause_publish, 1); + worker = CreateThread(NULL, 0u, refresh_thread, &context, 0u, NULL); + assert(worker != NULL); + assert(WaitForSingleObject(before_publish, 5000u) == WAIT_OBJECT_0); + ul_audio_metadata_retire_worker(17u); + SetEvent(publish_continue); + assert(WaitForSingleObject(worker, 5000u) == WAIT_OBJECT_0); + assert(!context.result && ul_audio_metadata_failed(17u)); + assert(!ul_audio_metadata_take_worker(17u, 0u, &snapshot)); + assert(ul_audio_metadata_close_frontend(17u, false)); + CloseHandle(worker); + InterlockedExchange(&pause_publish, 0); +} + +static DWORD WINAPI signal_thread(void *unused) +{ + (void)unused; + fire_signal(&sources[0].handler, "audio_mixers"); + return 0u; +} + +typedef struct close_context { + HANDLE done; + uintptr_t generation; + bool result; +} close_context; + +static DWORD WINAPI close_thread(void *parameter) +{ + close_context *context = parameter; + context->result = ul_audio_metadata_close_frontend(context->generation, + false); + SetEvent(context->done); + return 0u; +} + +static void test_close_waits_for_inflight_callback(void) +{ + close_context context = {0}; + HANDLE signal_worker, close_worker; + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + assert(ul_audio_metadata_open_frontend(13u, 0u, 1u, schedule_refresh)); + InterlockedExchange(&pause_callback, 1); + signal_worker = CreateThread(NULL, 0u, signal_thread, NULL, 0u, NULL); + assert(signal_worker != NULL); + assert(WaitForSingleObject(callback_entered, 5000u) == WAIT_OBJECT_0); + context.done = CreateEventW(NULL, TRUE, FALSE, NULL); + context.generation = 13u; + assert(context.done != NULL); + close_worker = CreateThread(NULL, 0u, close_thread, &context, 0u, NULL); + assert(close_worker != NULL); + assert(WaitForSingleObject(context.done, 20u) == WAIT_TIMEOUT); + SetEvent(callback_continue); + assert(WaitForSingleObject(signal_worker, 5000u) == WAIT_OBJECT_0); + assert(WaitForSingleObject(close_worker, 5000u) == WAIT_OBJECT_0); + assert(context.result); + CloseHandle(signal_worker); + CloseHandle(close_worker); + CloseHandle(context.done); +} + +static void test_close_waits_for_nonblocking_scheduler(void) +{ + close_context context = {0}; + HANDLE signal_worker, close_worker; + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + assert(ul_audio_metadata_open_frontend(16u, 0u, 1u, schedule_refresh)); + InterlockedExchange(&pause_scheduler, 1); + signal_worker = CreateThread(NULL, 0u, signal_thread, NULL, 0u, NULL); + assert(signal_worker != NULL); + assert(WaitForSingleObject(scheduler_entered, 5000u) == WAIT_OBJECT_0); + context.done = CreateEventW(NULL, TRUE, FALSE, NULL); + context.generation = 16u; + assert(context.done != NULL); + close_worker = CreateThread(NULL, 0u, close_thread, &context, 0u, NULL); + assert(close_worker != NULL); + assert(WaitForSingleObject(context.done, 20u) == WAIT_TIMEOUT); + SetEvent(scheduler_continue); + assert(WaitForSingleObject(signal_worker, 5000u) == WAIT_OBJECT_0); + assert(WaitForSingleObject(close_worker, 5000u) == WAIT_OBJECT_0); + assert(context.result); + CloseHandle(signal_worker); + CloseHandle(close_worker); + CloseHandle(context.done); +} + +typedef struct take_context { + uintptr_t generation; + bool result; + ul_audio_metadata_snapshot snapshot; +} take_context; + +static DWORD WINAPI take_thread(void *parameter) +{ + take_context *context = parameter; + context->result = ul_audio_metadata_take_worker( + context->generation, 0u, &context->snapshot); + return 0u; +} + +static void test_take_serializes_close_and_reopen(void) +{ + take_context taking = {14u, false, {0}}; + close_context closing = {0}; + ul_audio_metadata_snapshot snapshot; + HANDLE take_worker, close_worker; + reset_fixture(); + source_count = 1u; + make_source(0u, 1u); + assert(ul_audio_metadata_open_frontend(14u, 0u, 1u, schedule_refresh)); + InterlockedExchange(&pause_take, 1); + take_worker = CreateThread(NULL, 0u, take_thread, &taking, 0u, NULL); + assert(take_worker != NULL); + assert(WaitForSingleObject(take_locked, 5000u) == WAIT_OBJECT_0); + closing.done = CreateEventW(NULL, TRUE, FALSE, NULL); + closing.generation = 14u; + assert(closing.done != NULL); + close_worker = CreateThread(NULL, 0u, close_thread, &closing, 0u, NULL); + assert(close_worker != NULL); + assert(WaitForSingleObject(closing.done, 20u) == WAIT_TIMEOUT); + SetEvent(take_continue); + assert(WaitForSingleObject(take_worker, 5000u) == WAIT_OBJECT_0); + assert(WaitForSingleObject(close_worker, 5000u) == WAIT_OBJECT_0); + assert(taking.result && taking.snapshot.token == 1u && closing.result); + InterlockedExchange(&pause_take, 0); + assert(ul_audio_metadata_open_frontend(15u, 0u, 1u, schedule_refresh)); + assert(!ul_audio_metadata_take_worker(14u, 0u, &snapshot)); + assert(ul_audio_metadata_take_worker(15u, 0u, &snapshot)); + assert(snapshot.token == 1u); + assert(ul_audio_metadata_close_frontend(15u, false)); + CloseHandle(take_worker); + CloseHandle(close_worker); + CloseHandle(closing.done); +} + +int main(void) +{ + callback_entered = CreateEventW(NULL, TRUE, FALSE, NULL); + callback_continue = CreateEventW(NULL, TRUE, FALSE, NULL); + take_locked = CreateEventW(NULL, TRUE, FALSE, NULL); + take_continue = CreateEventW(NULL, TRUE, FALSE, NULL); + scheduler_entered = CreateEventW(NULL, TRUE, FALSE, NULL); + scheduler_continue = CreateEventW(NULL, TRUE, FALSE, NULL); + before_publish = CreateEventW(NULL, TRUE, FALSE, NULL); + publish_continue = CreateEventW(NULL, TRUE, FALSE, NULL); + assert(callback_entered != NULL && callback_continue != NULL && + take_locked != NULL && take_continue != NULL && + scheduler_entered != NULL && scheduler_continue != NULL && + before_publish != NULL && publish_continue != NULL); + test_initial_snapshot_stabilizes(); + test_initial_change_remove_and_close(); + test_visible_bounds_and_text_failures(); + test_scheduler_failure_is_terminal(); + test_worker_request_and_frontend_failure(); + test_close_waits_for_inflight_callback(); + test_close_waits_for_nonblocking_scheduler(); + test_take_serializes_close_and_reopen(); + test_retire_blocks_inflight_republish(); + test_abandon_makes_callbacks_inert(); + CloseHandle(callback_entered); + CloseHandle(callback_continue); + CloseHandle(take_locked); + CloseHandle(take_continue); + CloseHandle(scheduler_entered); + CloseHandle(scheduler_continue); + CloseHandle(before_publish); + CloseHandle(publish_continue); + puts("native OBS metadata observation and lifecycle passed"); + return 0; +} diff --git a/native/obs-plugin/tests/audio_protocol_test.c b/native/obs-plugin/tests/audio_protocol_test.c index 14e58a9..ceaddee 100644 --- a/native/obs-plugin/tests/audio_protocol_test.c +++ b/native/obs-plugin/tests/audio_protocol_test.c @@ -71,6 +71,37 @@ static void test_fixed_vectors(void) assert(memcmp(out, expected_end, size) == 0); } +static void test_explicit_versions(void) +{ + const float pcm[2] = {0.25f, -0.25f}; + const ul_audio_end_sequence entry = {0u, true, 0u}; + uint8_t out[64]; + assert(ul_audio_encode_start_version(2u, out, sizeof(out), session, + 48000u, 0u, 1u, 0u) == 42u); + assert(out[4] == 2u && out[5] == 1u); + assert(ul_audio_encode_audio_version(2u, out, sizeof(out), session, + 0u, 0u, 0u, 1u, pcm) == 57u); + assert(out[4] == 2u && out[5] == 2u); + assert(ul_audio_encode_gap_version(2u, out, sizeof(out), session, + 0u, 0u, 1u, 0u) == 53u); + assert(out[4] == 2u && out[5] == 3u); + assert(ul_audio_encode_end_version(2u, out, sizeof(out), session, + UL_AUDIO_END_STREAM_STOPPED, + &entry, 1u) == 39u); + assert(out[4] == 2u && out[5] == 4u); + reset(out, sizeof(out)); + assert(ul_audio_encode_start_version(0u, out, sizeof(out), session, + 48000u, 0u, 1u, 0u) == 0u); + assert(ul_audio_encode_audio_version(3u, out, sizeof(out), session, + 0u, 0u, 0u, 1u, pcm) == 0u); + assert(ul_audio_encode_gap_version(0u, out, sizeof(out), session, + 0u, 0u, 1u, 0u) == 0u); + assert(ul_audio_encode_end_version(3u, out, sizeof(out), session, + UL_AUDIO_END_STREAM_STOPPED, + &entry, 1u) == 0u); + assert_untouched(out, sizeof(out)); +} + static void test_start_validation(void) { static const uint32_t rates[] = {16000, 32000, 44100, 48000, 88200, 96000}; @@ -171,6 +202,199 @@ static void test_end_validation(void) assert_untouched(out, sizeof(out)); } +static size_t encode_routing_fixture(uint8_t *out, size_t capacity) +{ + static const uint8_t main_label[] = "Main"; + static const uint8_t aux_label[] = "Aux"; + static const uint8_t alpha_name[] = "A"; + static const uint8_t beta_name[] = "B"; + static const ul_audio_routing_bus buses[] = { + {1u, 7u, main_label, sizeof(main_label) - 1u}, + {2u, UINT64_MAX, aux_label, sizeof(aux_label) - 1u}, + }; + static const ul_audio_routing_source sources[] = { + {{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}, 2u, + alpha_name, sizeof(alpha_name) - 1u}, + {{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}, 4u, + beta_name, sizeof(beta_name) - 1u}, + }; + return ul_audio_encode_routing( + out, capacity, session, 1u, UINT64_C(0x0102030405060708), + 1u, 6u, buses, 2u, sources, 2u); +} + +static void test_routing_fixed_vector(void) +{ + static const uint8_t expected[] = { + 'U','L','A','P',2,5,0,0,105,0,0,0, + '0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f', + 1,0,0,0,0,0,0,0, 8,7,6,5,4,3,2,1, 1,6,2,0, + 1, 7,0,0,0,0,0,0,0, 4,0, 'M','a','i','n', + 2, 0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff, 3,0, 'A','u','x', + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1, 2, 1,0, 'A', + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2, 4, 1,0, 'B', + }; + uint8_t out[sizeof(expected)]; + size_t size = encode_routing_fixture(out, sizeof(out)); + assert(size == sizeof(expected)); + assert(memcmp(out, expected, sizeof(expected)) == 0); +} + +static void test_routing_validation(void) +{ + static const uint8_t label[] = "Main"; + static const uint8_t name[] = "Input"; + static const uint8_t emoji[] = { + 0xf0,0x9f,0x91,0xa9, 0xe2,0x80,0x8d, 0xf0,0x9f,0x92,0xbb + }; + static const uint8_t persian_zwnj[] = { + 0xd9,0x85, 0xdb,0x8c, 0xe2,0x80,0x8c, + 0xd8,0xb1, 0xd9,0x88, 0xd9,0x85 + }; + static const uint8_t blank_whitespace[] = {' ', 0xc2,0xa0}; + static const uint8_t blank_joiners[] = { + 0xe2,0x80,0x8c, 0xe2,0x80,0x8d + }; + static const uint8_t blank_ideographic[] = {0xe3,0x80,0x80}; + static const uint8_t invalid_text[][3] = { + {'x','\n','y'}, {0xc0,0xaf,'x'}, {0xed,0xa0,0x80}, + {0xf4,0x90,0x80}, {0xc2,0x85,'x'}, {0xe2,0x80,0xa8}, + {0xe2,0x81,0xa6}, {0xe2,'x','y'}, {0xd8,0x9c,'x'}, + {0xe2,0x80,0x8b}, {0xe2,0x80,0x8e}, {0xe2,0x80,0x8f}, + {0xef,0xbb,0xbf}, + }; + ul_audio_routing_bus buses[2] = { + {0u, 0u, label, sizeof(label) - 1u}, + {1u, UINT64_MAX, emoji, sizeof(emoji)}, + }; + ul_audio_routing_source sources[2] = { + {{0}, 1u, name, sizeof(name) - 1u}, + {{0}, 2u, name, sizeof(name) - 1u}, + }; + uint8_t out[256]; + size_t index; + sources[1].source_id[15] = 1u; + assert(ul_audio_encode_routing(out, sizeof(out), session, 1u, 0u, + 0u, 3u, buses, 2u, sources, 2u) != 0u); + reset(out, sizeof(out)); +#define BAD_ROUTE(...) do { \ + assert(ul_audio_encode_routing(__VA_ARGS__) == 0u); \ + assert_untouched(out, sizeof(out)); \ +} while (0) + BAD_ROUTE(NULL, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + BAD_ROUTE(out, sizeof(out), NULL, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + BAD_ROUTE(out, sizeof(out), session, 0u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + BAD_ROUTE(out, sizeof(out), session, UINT64_MAX, 0u, 0u, 3u, + buses, 2u, sources, 2u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 2u, 1u, + buses, 1u, sources, 1u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 0u, + buses, 0u, sources, 0u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 64u, + buses, 1u, sources, 1u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + NULL, 2u, sources, 2u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, NULL, 1u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, UL_AUDIO_MAX_ROUTING_SOURCES + 1u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 1u, sources, 2u); + buses[0].bus = 1u; + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + buses[0].bus = 0u; + buses[0].label_length = 0u; + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + buses[0].label_length = UL_AUDIO_MAX_BUS_LABEL_BYTES + 1u; + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + buses[0].label_length = sizeof(label) - 1u; + for (index = 0u; index < sizeof(invalid_text) / sizeof(invalid_text[0]); ++index) { + buses[0].label = invalid_text[index]; + buses[0].label_length = sizeof(invalid_text[index]); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + } + buses[0].label = blank_whitespace; + buses[0].label_length = sizeof(blank_whitespace); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + buses[0].label = blank_joiners; + buses[0].label_length = sizeof(blank_joiners); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + buses[0].label = blank_ideographic; + buses[0].label_length = sizeof(blank_ideographic); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + buses[0].label = persian_zwnj; + buses[0].label_length = sizeof(persian_zwnj); + assert(ul_audio_encode_routing(out, sizeof(out), session, 1u, 0u, + 0u, 3u, buses, 2u, sources, 2u) != 0u); + reset(out, sizeof(out)); + buses[0].label = label; + buses[0].label_length = sizeof(label) - 1u; + sources[0].selected_mask = 0u; + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + sources[0].selected_mask = 4u; + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + sources[0].selected_mask = 1u; + sources[0].name_length = UL_AUDIO_MAX_SOURCE_NAME_BYTES + 1u; + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + sources[0].name_length = sizeof(name) - 1u; + memcpy(sources[1].source_id, sources[0].source_id, 16u); + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); + sources[1].source_id[15] = 1u; + sources[0].source_id[15] = 2u; + BAD_ROUTE(out, sizeof(out), session, 1u, 0u, 0u, 3u, + buses, 2u, sources, 2u); +#undef BAD_ROUTE +} + +static void test_routing_maximum_and_capacity(void) +{ + uint8_t *out = malloc(UL_AUDIO_MAX_ROUTING_PACKET_BYTES); + uint8_t label[UL_AUDIO_MAX_BUS_LABEL_BYTES]; + uint8_t name[UL_AUDIO_MAX_SOURCE_NAME_BYTES]; + ul_audio_routing_bus buses[6]; + ul_audio_routing_source sources[UL_AUDIO_MAX_ROUTING_SOURCES]; + size_t index; + assert(out != NULL); + memset(label, 'L', sizeof(label)); + memset(name, 'N', sizeof(name)); + memset(sources, 0, sizeof(sources)); + for (index = 0u; index < 6u; ++index) + buses[index] = (ul_audio_routing_bus){ + (uint8_t)index, (uint64_t)index, label, sizeof(label)}; + for (index = 0u; index < UL_AUDIO_MAX_ROUTING_SOURCES; ++index) { + sources[index].source_id[15] = (uint8_t)index; + sources[index].selected_mask = 1u; + sources[index].name = name; + sources[index].name_length = sizeof(name); + } + assert(ul_audio_encode_routing( + out, UL_AUDIO_MAX_ROUTING_PACKET_BYTES, session, + UL_AUDIO_MAX_SEQUENCE, UINT64_MAX, 0u, 0x3fu, + buses, 6u, sources, UL_AUDIO_MAX_ROUTING_SOURCES) == + UL_AUDIO_MAX_ROUTING_PACKET_BYTES); + reset(out, UL_AUDIO_MAX_ROUTING_PACKET_BYTES); + assert(ul_audio_encode_routing( + out, UL_AUDIO_MAX_ROUTING_PACKET_BYTES - 1u, session, + 1u, 0u, 0u, 0x3fu, buses, 6u, + sources, UL_AUDIO_MAX_ROUTING_SOURCES) == 0u); + assert_untouched(out, UL_AUDIO_MAX_ROUTING_PACKET_BYTES); + free(out); +} + static int emit_frames(void) { const float pcm[2] = {0.25f, -0.25f}; @@ -193,16 +417,59 @@ static int emit_frames(void) return 0; } +static int emit_frames_v2(void) +{ + const float pcm[2] = {0.25f, -0.25f}; + const ul_audio_end_sequence entries[2] = {{1, true, 9}, {2, false, 0}}; + uint8_t out[64]; + size_t size; + if (_setmode(_fileno(stdout), _O_BINARY) == -1) + return 1; +#define EMIT(expression) do { \ + size = (expression); \ + if (size == 0u || fwrite(out, 1u, size, stdout) != size) return 1; \ +} while (0) + EMIT(ul_audio_encode_start_version(2u, out, sizeof(out), session, + 48000u, 1u, 6u, 10u)); + EMIT(ul_audio_encode_audio_version(2u, out, sizeof(out), session, + 1u, 7u, 456u, 1u, pcm)); + EMIT(ul_audio_encode_gap_version(2u, out, sizeof(out), session, + 1u, 8u, 2u, 789u)); + EMIT(ul_audio_encode_end_version(2u, out, sizeof(out), session, + UL_AUDIO_END_STREAM_STOPPED, + entries, 2u)); +#undef EMIT + return 0; +} + +static int emit_routing(void) +{ + uint8_t out[128]; + size_t size; + if (_setmode(_fileno(stdout), _O_BINARY) == -1) + return 1; + size = encode_routing_fixture(out, sizeof(out)); + return size == 0u || fwrite(out, 1u, size, stdout) != size; +} + int main(int argc, char **argv) { if (argc == 2 && strcmp(argv[1], "--emit") == 0) return emit_frames(); + if (argc == 2 && strcmp(argv[1], "--emit-v2") == 0) + return emit_frames_v2(); + if (argc == 2 && strcmp(argv[1], "--emit-routing") == 0) + return emit_routing(); assert(argc == 1); test_fixed_vectors(); + test_explicit_versions(); test_start_validation(); test_audio_validation_and_maximum(); test_gap_validation(); test_end_validation(); - puts("native ULAP encoder vectors and bounds passed"); + test_routing_fixed_vector(); + test_routing_validation(); + test_routing_maximum_and_capacity(); + puts("native ULAP encoder and routing vectors and bounds passed"); return 0; } diff --git a/native/obs-plugin/tests/audio_stream_routing_test.c b/native/obs-plugin/tests/audio_stream_routing_test.c new file mode 100644 index 0000000..05d3a18 --- /dev/null +++ b/native/obs-plugin/tests/audio_stream_routing_test.c @@ -0,0 +1,345 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* Reuse the SDK-free capture/admission plane without rerunning its legacy main. */ +#define main ul_audio_stream_legacy_fixture_main +#include "audio_stream_test.c" +#undef main + +#include "../src/audio_metadata.h" + +static struct ul_admission *routing_admission; +static bool routing_initial = true; +static bool routing_changed = true; +static bool routing_failed; +static bool routing_fail_after_audio; +static bool routing_invalid_update; +static uint8_t routing_mask = 1u; +static unsigned routing_change_after_audio = 1u; +static bool routing_closed; +static unsigned routing_requests; + +static void fill_snapshot(ul_audio_metadata_snapshot *out, uint64_t token, + const char *source_name) +{ + static const uint8_t source_id[16] = { + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 + }; + memset(out, 0, sizeof(*out)); + out->token = token; + out->observed_at_ns = token == 1u ? 100u : 200u; + out->primary_bus = 0u; + out->bus_mask = routing_mask; + for (uint8_t bus = 0u; bus < UL_AUDIO_CAPTURE_MIXES; ++bus) { + ul_audio_metadata_bus *label; + if ((routing_mask & (uint8_t)(1u << bus)) == 0u) + continue; + label = &out->buses[out->bus_count++]; + label->bus = bus; + label->label_length = 5u; + memcpy(label->label, "Mix 1", 5u); + label->label[4] = (uint8_t)('1' + bus); + } + out->source_count = 1u; + memcpy(out->sources[0].source_id, source_id, sizeof(source_id)); + out->sources[0].selected_mask = routing_mask; + out->sources[0].name_length = routing_invalid_update && token == 2u + ? 1u : (uint16_t)strlen(source_name); + if (routing_invalid_update && token == 2u) + out->sources[0].name[0] = ' '; + else + memcpy(out->sources[0].name, source_name, + out->sources[0].name_length); +} + +bool ul_audio_metadata_take_worker(uintptr_t generation, uint64_t after_token, + ul_audio_metadata_snapshot *out) +{ + assert(generation == 77u && out != NULL); + if (routing_failed) + return false; + if (routing_initial && after_token < 1u) { + fill_snapshot(out, 1u, "Desktop"); + return true; + } + if (routing_changed && after_token < 2u && routing_admission != NULL && + routing_admission->audio_count >= routing_change_after_audio) { + fill_snapshot(out, 2u, "Desktop renamed"); + return true; + } + return false; +} + +bool ul_audio_metadata_request_worker(uintptr_t generation) +{ + assert(generation == 77u); + routing_requests++; + return !routing_failed && !routing_closed; +} + +bool ul_audio_metadata_failed(uintptr_t generation) +{ + assert(generation == 77u); + return routing_failed || + (routing_fail_after_audio && routing_admission != NULL && + routing_admission->audio_count >= 1u); +} + +static uint64_t routing_revision(const struct ul_admission *admission, + unsigned write_index) +{ + assert(admission->writes[write_index][4] == 2u && + admission->writes[write_index][5] == 5u); + return get_u64(admission->writes[write_index] + 28u); +} + +static uint64_t routing_position(const struct ul_admission *admission, + unsigned write_index, unsigned bus_index) +{ + /* header12 + session16 + revision8 + observed8 + primary/mask/count2 */ + return get_u64(admission->writes[write_index] + 49u + bus_index * 16u); +} + +static void test_initial_and_changed_routing_order(void) +{ + struct ul_audio_capture capture = {0}; + struct ul_admission admission = {0}; + ul_audio_capture_spec spec = spec_for(5u); + HANDLE stop = CreateEventW(NULL, TRUE, FALSE, NULL); + HANDLE cleanup = CreateEventW(NULL, TRUE, FALSE, NULL); + assert(stop != NULL && cleanup != NULL); + capture.origin_ready = true; + enqueue(&capture, 0u, 0u, 0u, (ul_audio_gap){0}, 1.0f); + enqueue(&capture, 0u, 1u, 20833u, (ul_audio_gap){0}, 2.0f); + enqueue(&capture, 0u, 2u, 41666u, (ul_audio_gap){0}, 5.0f); + enqueue(&capture, 2u, 0u, 0u, (ul_audio_gap){0}, 3.0f); + enqueue(&capture, 2u, 1u, 20833u, (ul_audio_gap){0}, 4.0f); + admission.stop_event = stop; + admission.stop_after_audio = 4u; + admission.slow_audio_ms = 2u; + routing_admission = &admission; + routing_initial = routing_changed = true; + routing_failed = false; + routing_fail_after_audio = false; + routing_invalid_update = false; + routing_closed = false; + routing_mask = 5u; + routing_change_after_audio = 3u; + routing_requests = 0u; + assert(ul_audio_stream_run_metadata( + &capture, &spec, &admission, session, stop, cleanup, + fake_disarm, NULL, 77u) == UL_AUDIO_STREAM_OK); + assert(admission.write_count == 9u); + assert(admission.writes[0][4] == 2u && admission.writes[0][5] == 1u); + assert(routing_revision(&admission, 1u) == 1u && + routing_position(&admission, 1u, 0u) == 0u && + routing_position(&admission, 1u, 1u) == 0u); + assert(admission.writes[2][5] == 2u && admission.writes[2][28] == 0u && + get_u64(admission.writes[2] + 29u) == 0u); + assert(admission.writes[3][5] == 2u && admission.writes[3][28] == 2u); + assert(admission.writes[4][5] == 2u && admission.writes[4][28] == 0u && + get_u64(admission.writes[4] + 29u) == 1u); + assert(routing_revision(&admission, 5u) == 2u && + routing_position(&admission, 5u, 0u) == 2u && + routing_position(&admission, 5u, 1u) == 1u); + assert(admission.writes[6][5] == 2u && admission.writes[6][28] == 2u && + get_u64(admission.writes[6] + 29u) == 1u); + assert(admission.writes[7][5] == 2u && admission.writes[7][28] == 0u && + get_u64(admission.writes[7] + 29u) == 2u); + assert(admission.writes[8][5] == 4u && admission.writes[8][4] == 2u); + assert(routing_requests != 0u); + clear_admission(&admission); + CloseHandle(cleanup); + CloseHandle(stop); + routing_admission = NULL; + routing_mask = 1u; + routing_change_after_audio = 1u; +} + +static void test_missing_initial_snapshot_is_terminal_before_pcm(void) +{ + struct ul_audio_capture capture = {0}; + struct ul_admission admission = {0}; + ul_audio_capture_spec spec = spec_for(1u); + HANDLE stop = CreateEventW(NULL, TRUE, FALSE, NULL); + HANDLE cleanup = CreateEventW(NULL, TRUE, FALSE, NULL); + assert(stop != NULL && cleanup != NULL); + capture.origin_ready = true; + enqueue(&capture, 0u, 0u, 0u, (ul_audio_gap){0}, 1.0f); + admission.stop_event = stop; + routing_admission = &admission; + routing_initial = false; + routing_changed = false; + routing_failed = false; + routing_fail_after_audio = false; + routing_invalid_update = false; + routing_mask = 1u; + routing_closed = false; + assert(ul_audio_stream_run_metadata( + &capture, &spec, &admission, session, stop, cleanup, + fake_disarm, NULL, 77u) == UL_AUDIO_STREAM_SOURCE_CHANGED); + assert(admission.audio_count == 0u && admission.write_count == 2u); + assert(admission.writes[0][5] == 1u && admission.writes[0][4] == 2u); + assert(admission.writes[1][5] == 4u && admission.writes[1][28] == + UL_AUDIO_END_SOURCE_CHANGED); + CloseHandle(cleanup); + CloseHandle(stop); + routing_admission = NULL; +} + +static void test_refresh_failure_stops_after_accepted_prefix(void) +{ + struct ul_audio_capture capture = {0}; + struct ul_admission admission = {0}; + ul_audio_capture_spec spec = spec_for(1u); + HANDLE stop = CreateEventW(NULL, TRUE, FALSE, NULL); + HANDLE cleanup = CreateEventW(NULL, TRUE, FALSE, NULL); + assert(stop != NULL && cleanup != NULL); + capture.origin_ready = true; + enqueue(&capture, 0u, 0u, 0u, (ul_audio_gap){0}, 1.0f); + enqueue(&capture, 0u, 1u, 20833u, (ul_audio_gap){0}, 2.0f); + admission.stop_event = stop; + routing_admission = &admission; + routing_initial = true; + routing_changed = false; + routing_failed = false; + routing_fail_after_audio = true; + routing_invalid_update = false; + routing_mask = 1u; + routing_closed = false; + assert(ul_audio_stream_run_metadata( + &capture, &spec, &admission, session, stop, cleanup, + fake_disarm, NULL, 77u) == UL_AUDIO_STREAM_SOURCE_CHANGED); + assert(admission.audio_count == 1u && admission.write_count == 4u); + assert(admission.writes[0][5] == 1u && + admission.writes[1][5] == 5u && + admission.writes[2][5] == 2u && + admission.writes[3][5] == 4u && + admission.writes[3][28] == UL_AUDIO_END_SOURCE_CHANGED); + CloseHandle(cleanup); + CloseHandle(stop); + routing_admission = NULL; + routing_fail_after_audio = false; +} + +static void test_invalid_changed_snapshot_is_source_failure(void) +{ + struct ul_audio_capture capture = {0}; + struct ul_admission admission = {0}; + ul_audio_capture_spec spec = spec_for(1u); + HANDLE stop = CreateEventW(NULL, TRUE, FALSE, NULL); + HANDLE cleanup = CreateEventW(NULL, TRUE, FALSE, NULL); + assert(stop != NULL && cleanup != NULL); + capture.origin_ready = true; + enqueue(&capture, 0u, 0u, 0u, (ul_audio_gap){0}, 1.0f); + enqueue(&capture, 0u, 1u, 20833u, (ul_audio_gap){0}, 2.0f); + admission.stop_event = stop; + routing_admission = &admission; + routing_initial = routing_changed = true; + routing_failed = routing_fail_after_audio = false; + routing_invalid_update = true; + routing_mask = 1u; + routing_closed = false; + assert(ul_audio_stream_run_metadata( + &capture, &spec, &admission, session, stop, cleanup, + fake_disarm, NULL, 77u) == UL_AUDIO_STREAM_SOURCE_CHANGED); + assert(admission.audio_count == 1u && admission.write_count == 4u); + assert(admission.writes[3][5] == 4u && + admission.writes[3][28] == UL_AUDIO_END_SOURCE_CHANGED); + CloseHandle(cleanup); + CloseHandle(stop); + routing_admission = NULL; + routing_invalid_update = false; +} + +static void test_v2_gap_and_empty_disarm_terminal_packets(void) +{ + struct ul_audio_capture capture = {0}; + struct ul_admission admission = {0}; + ul_audio_capture_spec spec = spec_for(1u); + HANDLE stop = CreateEventW(NULL, TRUE, FALSE, NULL); + HANDLE cleanup = CreateEventW(NULL, TRUE, TRUE, NULL); + assert(stop != NULL && cleanup != NULL); + capture.origin_ready = true; + enqueue(&capture, 0u, 0u, 0u, (ul_audio_gap){0}, 1.0f); + enqueue(&capture, 0u, 2u, 41666u, + (ul_audio_gap){1u, 1u, 20833u}, 2.0f); + admission.stop_event = stop; + routing_admission = &admission; + routing_initial = true; + routing_changed = routing_failed = routing_fail_after_audio = false; + routing_invalid_update = false; + routing_mask = 1u; + routing_closed = false; + assert(ul_audio_stream_run_metadata( + &capture, &spec, &admission, session, stop, cleanup, + fake_disarm, NULL, 77u) == UL_AUDIO_STREAM_SOURCE_CHANGED); + assert(admission.write_count == 5u && + admission.writes[3][5] == 3u && + admission.writes[4][5] == 4u); + for (unsigned index = 0u; index < admission.write_count; ++index) + assert(admission.writes[index][4] == 2u); + clear_admission(&admission); + memset(&admission, 0, sizeof(admission)); + assert(ul_audio_stream_finish_empty_disarm(&admission, session, cleanup) == + UL_AUDIO_STREAM_OK); + assert(admission.write_count == 1u && admission.writes[0][4] == 2u && + admission.writes[0][5] == 4u && + admission.writes[0][28] == UL_AUDIO_END_DISARMED && + admission.writes[0][29] == 0u); + CloseHandle(cleanup); + CloseHandle(stop); + routing_admission = NULL; +} + +static void test_closed_snapshot_finishes_precommitted_tails(void) +{ + for (unsigned disarmed = 0u; disarmed < 2u; ++disarmed) { + struct ul_audio_capture capture = {0}; + struct ul_admission admission = {0}; + ul_audio_capture_spec spec = spec_for(1u); + HANDLE stop = CreateEventW(NULL, TRUE, disarmed == 0u, NULL); + HANDLE cleanup = CreateEventW(NULL, TRUE, TRUE, NULL); + int result; + assert(stop != NULL && cleanup != NULL); + capture.origin_ready = true; + capture.queues[0].status = UL_AUDIO_QUEUE_STOPPED; + enqueue(&capture, 0u, 0u, 0u, (ul_audio_gap){0}, 1.0f); + admission.stop_event = stop; + routing_admission = &admission; + routing_initial = true; + routing_changed = routing_failed = routing_fail_after_audio = false; + routing_invalid_update = false; + routing_mask = 1u; + routing_closed = true; + if (disarmed != 0u) + result = ul_audio_stream_run_disarmed_metadata( + &capture, &spec, &admission, session, stop, cleanup, 77u); + else + result = ul_audio_stream_run_metadata( + &capture, &spec, &admission, session, stop, cleanup, + fake_disarm, NULL, 77u); + assert(result == UL_AUDIO_STREAM_OK && admission.write_count == 4u); + assert(admission.writes[0][5] == 1u && + admission.writes[1][5] == 5u && + admission.writes[2][5] == 2u && + admission.writes[3][5] == 4u && + admission.writes[3][28] == + (disarmed != 0u ? UL_AUDIO_END_DISARMED + : UL_AUDIO_END_STREAM_STOPPED)); + CloseHandle(cleanup); + CloseHandle(stop); + } + routing_admission = NULL; + routing_closed = false; +} + +int main(void) +{ + test_initial_and_changed_routing_order(); + test_missing_initial_snapshot_is_terminal_before_pcm(); + test_refresh_failure_stops_after_accepted_prefix(); + test_invalid_changed_snapshot_is_source_failure(); + test_v2_gap_and_empty_disarm_terminal_packets(); + test_closed_snapshot_finishes_precommitted_tails(); + puts("audio stream v2 routing ordering and failure passed"); + return 0; +} diff --git a/native/obs-plugin/tests/audio_stream_test.c b/native/obs-plugin/tests/audio_stream_test.c index 7aaed01..9d2695a 100644 --- a/native/obs-plugin/tests/audio_stream_test.c +++ b/native/obs-plugin/tests/audio_stream_test.c @@ -406,8 +406,11 @@ static void test_staging_refusals_emit_nothing(void) enqueue(&capture, 0, 0, 10, (ul_audio_gap){0}, 1.0f); spec = spec_for(1u); assert(run_stream(&capture, &spec, &admission, stop) == - UL_AUDIO_STREAM_INCOMPLETE); - assert(admission.write_count == 0u && capture.deactivate_calls == 1u); + UL_AUDIO_STREAM_OK); + assert(admission.write_count == 3u && capture.deactivate_calls == 1u); + assert(admission.writes[0][5] == 1u && admission.writes[1][5] == 2u && + admission.writes[2][5] == 4u && + admission.writes[2][28] == UL_AUDIO_END_STREAM_STOPPED); CloseHandle(stop); memset(&capture, 0, sizeof(capture)); diff --git a/native/obs-plugin/tests/bridge_test.c b/native/obs-plugin/tests/bridge_test.c index e0c2c50..ad64b0e 100644 --- a/native/obs-plugin/tests/bridge_test.c +++ b/native/obs-plugin/tests/bridge_test.c @@ -22,12 +22,16 @@ static ul_arm_scheduler arm_scheduler; static ul_frontend_dispatch_callback queued_task; static uintptr_t queued_parameter; static unsigned queued_command; -static ul_arm_scheduler capture_scheduler, cleanup_scheduler; -static bool allow_dispatch, allow_post, allow_capture_schedulers; +static ul_arm_scheduler capture_scheduler, cleanup_scheduler, metadata_scheduler; +static bool allow_dispatch, allow_post, allow_capture_schedulers, + allow_metadata_scheduler; static bool inspect_requested, allow_inspect, allow_connect, capture_available; static unsigned inspect_calls, inspected_calls, connect_calls, attached_calls; static unsigned capture_releases, stop_calls, disconnect_calls, abandon_calls; static unsigned cleanup_completed_calls; +static unsigned metadata_open_calls, metadata_refresh_calls, + metadata_close_calls, metadata_abandon_calls; +static bool allow_metadata_open; static uintptr_t capture_generation, disconnected_generation; static bool inspected_ok, attached_ok, disconnected_all; static ul_audio_capture *test_capture = (ul_audio_capture *)(uintptr_t)55; @@ -37,6 +41,10 @@ static unsigned queue_calls, idle_checks, output_releases, stream_events; static ul_stream_event last_stream_event; static obs_output_t *current_output = (obs_output_t *)(uintptr_t)3; static bool after_exit; +static SRWLOCK frontend_gate; +static unsigned lifecycle_order, metadata_open_order, connect_order, + metadata_close_order, disconnect_order; +static bool require_metadata_post_unlocked; static void record(char value) { assert(order_size + 1 < sizeof(order)); order[order_size++] = value; order[order_size] = 0; } static uint32_t test_version(void) { return LIBOBS_API_VER; } @@ -87,13 +95,27 @@ bool ul_frontend_dispatch_open(ul_frontend_dispatch_callback callback) { queued_task = callback; return allow_dispatch; } bool ul_frontend_dispatch_post(unsigned command, uintptr_t generation) { - assert(!exiting && command >= 1 && command <= 3 && generation != 0); + assert(!exiting && command >= 1 && command <= 4 && generation != 0); + if (command == 4 && require_metadata_post_unlocked) { + assert(TryAcquireSRWLockExclusive(&frontend_gate)); + ReleaseSRWLockExclusive(&frontend_gate); + } if (!allow_post) return false; queued_command = command; queued_parameter = generation; ++queue_calls; return true; } +bool ul_frontend_dispatch_try_post(unsigned command, uintptr_t generation) +{ return ul_frontend_dispatch_post(command, generation); } void ul_frontend_dispatch_close(void) { /* Retain copied callback for late delivery. */ } bool ul_plugin_set_capture_schedulers(ul_arm_scheduler attach, ul_arm_scheduler cleanup) { capture_scheduler = attach; cleanup_scheduler = cleanup; return allow_capture_schedulers; } +bool ul_plugin_set_metadata_scheduler(ul_arm_scheduler refresh) +{ metadata_scheduler = refresh; return allow_metadata_scheduler; } +bool ul_plugin_metadata_open_frontend(uintptr_t generation) +{ assert(!after_exit && generation == capture_generation); ++metadata_open_calls; metadata_open_order = ++lifecycle_order; return allow_metadata_open; } +bool ul_plugin_metadata_refresh_frontend(uintptr_t generation) +{ assert(!after_exit && generation == capture_generation); ++metadata_refresh_calls; return true; } +bool ul_plugin_metadata_close_frontend(uintptr_t generation, bool all) +{ assert(!after_exit); assert(all || generation == capture_generation); if (all && !exiting) assert(last_stream_event == UL_STREAM_STOPPING || last_stream_event == UL_STREAM_STOPPED); ++metadata_close_calls; metadata_close_order = ++lifecycle_order; return true; } bool ul_plugin_capture_inspect_request(uintptr_t *generation, uint8_t *mask) { assert(!after_exit); *generation = capture_generation; *mask = 4; return inspect_requested; } bool ul_audio_capture_inspect_frontend(uint8_t mask, ul_audio_capture_spec *spec) @@ -111,14 +133,14 @@ void ul_plugin_capture_inspected(uintptr_t generation, const ul_audio_capture_sp ul_audio_capture *ul_plugin_capture_retain(uintptr_t generation) { assert(!after_exit); return capture_available && generation == capture_generation ? test_capture : NULL; } bool ul_audio_capture_connect_frontend(ul_audio_capture *capture, uintptr_t generation) -{ assert(!after_exit && capture == test_capture && generation == capture_generation); ++connect_calls; return allow_connect; } +{ assert(!after_exit && capture == test_capture && generation == capture_generation); ++connect_calls; connect_order = ++lifecycle_order; return allow_connect; } void ul_plugin_capture_attached(uintptr_t generation, ul_audio_capture *capture, bool success) { assert(!after_exit && generation == capture_generation && capture == test_capture); ++attached_calls; attached_ok = success; } void ul_audio_capture_release(ul_audio_capture *capture) { assert(capture == test_capture); ++capture_releases; } void ul_plugin_capture_stop_frontend(void) { assert(!after_exit); ++stop_calls; } void ul_audio_capture_disconnect_frontend(uintptr_t generation, bool all) -{ assert(!after_exit); ++disconnect_calls; disconnected_generation = generation; disconnected_all = all; } +{ assert(!after_exit); ++disconnect_calls; disconnect_order = ++lifecycle_order; disconnected_generation = generation; disconnected_all = all; } void ul_plugin_capture_cleanup_complete(uintptr_t generation) { assert(!after_exit && disconnect_calls != 0 && !disconnected_all); @@ -126,6 +148,7 @@ void ul_plugin_capture_cleanup_complete(uintptr_t generation) ++cleanup_completed_calls; } void ul_audio_capture_abandon_after_shutdown(void) { ++abandon_calls; } +void ul_audio_metadata_abandon_after_shutdown(void) { ++metadata_abandon_calls; } ul_plugin_snapshot ul_plugin_get_status(void) { return snapshot; } void ul_plugin_stop_accepting(void) { assert(!enabled); snapshot.status = UL_PLUGIN_CLOSED; record('S'); } void ul_plugin_close(void) { assert(!enabled); snapshot.status = UL_PLUGIN_CLOSED; record('C'); } @@ -169,16 +192,22 @@ static void reset(void) order_size = 0; order[0] = 0; frontend_open = false; queued_arm = 0; stream_busy = false; - arm_scheduler = capture_scheduler = cleanup_scheduler = NULL; + arm_scheduler = capture_scheduler = cleanup_scheduler = metadata_scheduler = NULL; queued_task = NULL; queued_parameter = 0; queued_command = 0; queued_capture = queued_cleanup = 0; allow_dispatch = allow_post = allow_capture_schedulers = true; + allow_metadata_scheduler = allow_metadata_open = true; allow_inspect = allow_connect = true; inspect_requested = capture_available = inspected_ok = attached_ok = disconnected_all = false; capture_generation = 40; disconnected_generation = 0; inspect_calls = inspected_calls = connect_calls = attached_calls = 0; capture_releases = stop_calls = disconnect_calls = abandon_calls = 0; cleanup_completed_calls = 0; + metadata_open_calls = metadata_refresh_calls = metadata_close_calls = 0; + metadata_abandon_calls = 0; + lifecycle_order = metadata_open_order = connect_order = 0; + metadata_close_order = disconnect_order = 0; + require_metadata_post_unlocked = false; checked_generation = 0; allow_scheduler = has_output = true; current_active = current_output_active = checked_idle = false; @@ -322,6 +351,8 @@ int main(void) assert(!obs_module_load() && tools_calls == 0 && event_calls == 0); reset(); allow_capture_schedulers = false; assert(!obs_module_load() && tools_calls == 0 && event_calls == 0); + reset(); allow_metadata_scheduler = false; + assert(!obs_module_load() && tools_calls == 0 && event_calls == 0); /* Read-only inspection belongs to STARTED and reports unsupported formats. */ reset(); assert(obs_module_load()); inspect_requested = true; @@ -344,12 +375,19 @@ int main(void) dispatch_frontend(2, 39); assert(queued_capture == 40 && connect_calls == 0); queued_task(queued_command, queued_parameter); - assert(connect_calls == 1 && attached_calls == 1 && attached_ok && capture_releases == 1); + assert(connect_calls == 1 && metadata_open_calls == 1 && + metadata_open_order < connect_order && attached_calls == 1 && + attached_ok && capture_releases == 1); queued_task(queued_command, queued_parameter); assert(connect_calls == 1 && capture_releases == 1); allow_connect = false; assert(capture_scheduler(40)); queued_task(queued_command, queued_parameter); - assert(connect_calls == 2 && attached_calls == 2 && !attached_ok && capture_releases == 2); + assert(connect_calls == 2 && metadata_open_calls == 2 && metadata_close_calls == 1 && attached_calls == 2 && !attached_ok && capture_releases == 2); + allow_metadata_open = false; + assert(capture_scheduler(40)); queued_task(queued_command, queued_parameter); + assert(connect_calls == 2 && metadata_open_calls == 3 && + attached_calls == 3 && !attached_ok && capture_releases == 3); + allow_metadata_open = true; capture_available = false; assert(capture_scheduler(40)); queued_task(queued_command, queued_parameter); assert(connect_calls == 2 && queued_capture == 0); @@ -358,10 +396,16 @@ int main(void) dispatch_frontend(3, 39); assert(queued_cleanup == 40 && disconnect_calls == 0 && cleanup_completed_calls == 0); queued_task(queued_command, queued_parameter); - assert(disconnect_calls == 1 && disconnected_generation == 40 && !disconnected_all - && cleanup_completed_calls == 1); + assert(disconnect_calls == 1 && metadata_close_calls == 2 && disconnected_generation == 40 && !disconnected_all + && metadata_close_order < disconnect_order && + cleanup_completed_calls == 1); + queued_task(queued_command, queued_parameter); + assert(disconnect_calls == 1 && metadata_close_calls == 2 && cleanup_completed_calls == 1); + require_metadata_post_unlocked = true; + assert(metadata_scheduler(40)); + require_metadata_post_unlocked = false; queued_task(queued_command, queued_parameter); - assert(disconnect_calls == 1 && cleanup_completed_calls == 1); + assert(metadata_refresh_calls == 1); frontend_event(OBS_FRONTEND_EVENT_STREAMING_STOPPING, NULL); assert(stop_calls == 1 && disconnect_calls == 2 && disconnected_all); frontend_event(OBS_FRONTEND_EVENT_STREAMING_STOPPED, NULL); @@ -373,13 +417,13 @@ int main(void) && cleanup_completed_calls == 1); assert(!capture_scheduler(40) && !cleanup_scheduler(40)); obs_module_unload(); - assert(abandon_calls == 1 && disconnect_calls == 4); + assert(abandon_calls == 1 && metadata_abandon_calls == 1 && disconnect_calls == 4); /* Missed EXIT must use only native abandonment, even with queued capture. */ reset(); assert(obs_module_load()); capture_available = true; assert(capture_scheduler(40)); after_exit = true; obs_module_unload(); queued_task(queued_command, queued_parameter); - assert(abandon_calls == 1 && disconnect_calls == 0 && connect_calls == 0); + assert(abandon_calls == 1 && metadata_abandon_calls == 1 && disconnect_calls == 0 && connect_calls == 0); puts("bridge lifecycle, frontend handoff and stream-busy gate tests passed"); return 0; diff --git a/native/obs-plugin/tests/frontend_dispatch_fault_test.c b/native/obs-plugin/tests/frontend_dispatch_fault_test.c index 8c14719..e0e38fa 100644 --- a/native/obs-plugin/tests/frontend_dispatch_fault_test.c +++ b/native/obs-plugin/tests/frontend_dispatch_fault_test.c @@ -125,13 +125,20 @@ static void scenario(int mode) CloseHandle(worker); CloseHandle(close_started); CloseHandle(close_finished); pump(); assert(callbacks == 0 && closing); - } else { - assert(mode == 6); + } else if (mode == 6) { + assert(!ul_frontend_dispatch_post(5, 42)); fail_post = true; assert(!ul_frontend_dispatch_post(2, 42)); fail_post = false; assert(ul_frontend_dispatch_post(2, 42)); pump(); assert(callbacks == 1); + } else { + assert(mode == 7); + AcquireSRWLockExclusive(&state_lock); + assert(!ul_frontend_dispatch_try_post(2, 42)); + ReleaseSRWLockExclusive(&state_lock); + assert(ul_frontend_dispatch_try_post(2, 42)); + pump(); assert(callbacks == 1); } ul_frontend_dispatch_close(); assert(dispatch_window == NULL && dispatch_class == 0); @@ -147,7 +154,7 @@ int main(int argc, char **argv) if (argc == 2) { scenario(atoi(argv[1])); return 0; } WCHAR executable[MAX_PATH], command[MAX_PATH + 16]; assert(GetModuleFileNameW(NULL, executable, MAX_PATH)); - for (unsigned mode = 1; mode <= 6; ++mode) { + for (unsigned mode = 1; mode <= 7; ++mode) { STARTUPINFOW start = {0}; PROCESS_INFORMATION process = {0}; DWORD code; diff --git a/native/obs-plugin/tests/frontend_dispatch_test.c b/native/obs-plugin/tests/frontend_dispatch_test.c index fc28df0..4eb5370 100644 --- a/native/obs-plugin/tests/frontend_dispatch_test.c +++ b/native/obs-plugin/tests/frontend_dispatch_test.c @@ -26,8 +26,9 @@ static void callback(unsigned command, uintptr_t generation) static DWORD WINAPI worker(void *unused) { (void)unused; - assert(ul_frontend_dispatch_post(2u, (uintptr_t)42u)); + assert(ul_frontend_dispatch_post(4u, (uintptr_t)42u)); assert(!ul_frontend_dispatch_post(0u, 42u)); + assert(!ul_frontend_dispatch_post(5u, 42u)); assert(!ul_frontend_dispatch_post(1u, 0u)); while (InterlockedCompareExchange(&callback_complete, 0, 0) == 0) Sleep(1u); @@ -51,7 +52,7 @@ int main(void) } assert(WaitForSingleObject(thread, 2000u) == WAIT_OBJECT_0); CloseHandle(thread); - assert(callbacks == 1u && seen_command == 2u && seen_generation == 42u); + assert(callbacks == 1u && seen_command == 4u && seen_generation == 42u); assert(!ul_frontend_dispatch_post(1u, (uintptr_t)43u)); close_in_callback = 1; /* The worker's cross-thread close made the dispatcher inert; owner cleanup diff --git a/native/obs-plugin/tests/plugin_state_test.c b/native/obs-plugin/tests/plugin_state_test.c index ce1d071..bc98ac9 100644 --- a/native/obs-plugin/tests/plugin_state_test.c +++ b/native/obs-plugin/tests/plugin_state_test.c @@ -2,6 +2,7 @@ #include #include +#include #include #include #include @@ -9,6 +10,8 @@ #include "../src/plugin_state.h" #include "../src/session_protocol.h" #include "../src/audio_stream.h" +#include "../src/audio_protocol.h" +#include "../src/audio_metadata.h" static BOOL WINAPI shim_GetModuleHandleExW(DWORD flags, LPCWSTR address, HMODULE *module); @@ -71,6 +74,27 @@ static int shim_ul_audio_stream_run_disarmed( ul_audio_capture *capture, const ul_audio_capture_spec *spec, ul_admission *admission, const uint8_t session[16], HANDLE stop_event, HANDLE cleanup_complete); +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +static int shim_ul_audio_stream_run_metadata( + ul_audio_capture *capture, const ul_audio_capture_spec *spec, + ul_admission *admission, const uint8_t session[16], HANDLE stop_event, + HANDLE cleanup_complete, ul_audio_disarm_callback disarm, + void *disarm_context, uintptr_t metadata_generation); +static int shim_ul_audio_stream_run_disarmed_metadata( + ul_audio_capture *capture, const ul_audio_capture_spec *spec, + ul_admission *admission, const uint8_t session[16], HANDLE stop_event, + HANDLE cleanup_complete, uintptr_t metadata_generation); +static bool shim_ul_audio_capture_matches_frontend( + const ul_audio_capture_spec *expected); +static bool shim_ul_audio_metadata_open_frontend( + uintptr_t generation, uint8_t primary_bus, uint8_t bus_mask, + ul_audio_metadata_schedule schedule); +static bool shim_ul_audio_metadata_refresh_frontend(uintptr_t generation); +static bool shim_ul_audio_metadata_close_frontend(uintptr_t generation, + bool all); +static void shim_ul_audio_metadata_fail_frontend(uintptr_t generation); +static void shim_ul_audio_metadata_retire_worker(uintptr_t generation); +#endif #define GetModuleHandleExW shim_GetModuleHandleExW #define ul_pairing_store_open shim_ul_pairing_store_open @@ -104,6 +128,16 @@ static int shim_ul_audio_stream_run_disarmed( #define ul_audio_stream_run shim_ul_audio_stream_run #define ul_audio_stream_finish_empty_disarm shim_ul_audio_stream_finish_empty_disarm #define ul_audio_stream_run_disarmed shim_ul_audio_stream_run_disarmed +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +#define ul_audio_stream_run_metadata shim_ul_audio_stream_run_metadata +#define ul_audio_stream_run_disarmed_metadata shim_ul_audio_stream_run_disarmed_metadata +#define ul_audio_capture_matches_frontend shim_ul_audio_capture_matches_frontend +#define ul_audio_metadata_open_frontend shim_ul_audio_metadata_open_frontend +#define ul_audio_metadata_refresh_frontend shim_ul_audio_metadata_refresh_frontend +#define ul_audio_metadata_close_frontend shim_ul_audio_metadata_close_frontend +#define ul_audio_metadata_fail_frontend shim_ul_audio_metadata_fail_frontend +#define ul_audio_metadata_retire_worker shim_ul_audio_metadata_retire_worker +#endif #define UL_PLUGIN_AUTH_TIMEOUT_MS 40u #define UL_PLUGIN_READY_TIMEOUT_MS 40u #define UL_PLUGIN_START_TIMEOUT_MS 80u @@ -141,6 +175,16 @@ static int shim_ul_audio_stream_run_disarmed( #undef ul_audio_stream_run #undef ul_audio_stream_finish_empty_disarm #undef ul_audio_stream_run_disarmed +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +#undef ul_audio_stream_run_metadata +#undef ul_audio_stream_run_disarmed_metadata +#undef ul_audio_capture_matches_frontend +#undef ul_audio_metadata_open_frontend +#undef ul_audio_metadata_refresh_frontend +#undef ul_audio_metadata_close_frontend +#undef ul_audio_metadata_fail_frontend +#undef ul_audio_metadata_retire_worker +#endif struct ul_pairing_store { int unused; @@ -641,6 +685,85 @@ static int shim_ul_audio_stream_run_disarmed( ? UL_AUDIO_STREAM_OK : UL_AUDIO_STREAM_INCOMPLETE; } +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +static uintptr_t fake_metadata_generation; +static unsigned fake_metadata_open_count, fake_metadata_refresh_count, + fake_metadata_close_count, fake_metadata_fail_count, + fake_metadata_retire_count; +static bool fake_metadata_open_result = true; +static bool fake_metadata_match_result = true; + +static int shim_ul_audio_stream_run_metadata( + ul_audio_capture *capture, const ul_audio_capture_spec *spec, + ul_admission *admission, const uint8_t exact_session[16], + HANDLE stop_event, HANDLE cleanup_complete, + ul_audio_disarm_callback disarm, void *disarm_context, + uintptr_t metadata_generation) +{ + fake_metadata_generation = metadata_generation; + return shim_ul_audio_stream_run(capture, spec, admission, exact_session, + stop_event, cleanup_complete, disarm, + disarm_context); +} + +static int shim_ul_audio_stream_run_disarmed_metadata( + ul_audio_capture *capture, const ul_audio_capture_spec *spec, + ul_admission *admission, const uint8_t exact_session[16], + HANDLE stop_event, HANDLE cleanup_complete, + uintptr_t metadata_generation) +{ + fake_metadata_generation = metadata_generation; + return shim_ul_audio_stream_run_disarmed(capture, spec, admission, + exact_session, stop_event, + cleanup_complete); +} + +static bool shim_ul_audio_capture_matches_frontend( + const ul_audio_capture_spec *expected) +{ + return expected != NULL && fake_metadata_match_result; +} + +static bool shim_ul_audio_metadata_open_frontend( + uintptr_t generation, uint8_t primary_bus, uint8_t bus_mask, + ul_audio_metadata_schedule schedule) +{ + assert(generation != 0u && primary_bus < 6u && bus_mask != 0u && + schedule != NULL); + fake_metadata_generation = generation; + fake_metadata_open_count++; + return fake_metadata_open_result; +} + +static bool shim_ul_audio_metadata_refresh_frontend(uintptr_t generation) +{ + assert(generation == fake_metadata_generation); + fake_metadata_refresh_count++; + return true; +} + +static bool shim_ul_audio_metadata_close_frontend(uintptr_t generation, + bool all) +{ + (void)all; + assert(generation == 0u || generation == fake_metadata_generation); + fake_metadata_close_count++; + return true; +} + +static void shim_ul_audio_metadata_fail_frontend(uintptr_t generation) +{ + assert(generation == fake_metadata_generation); + fake_metadata_fail_count++; +} + +static void shim_ul_audio_metadata_retire_worker(uintptr_t generation) +{ + if (generation == fake_metadata_generation) + fake_metadata_retire_count++; +} +#endif + static bool queued_arm_scheduler(uintptr_t generation) { fake_scheduled_generation = generation; @@ -666,6 +789,14 @@ static bool queued_cleanup_scheduler(uintptr_t generation) return fake_cleanup_schedule_result; } +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +static bool queued_metadata_scheduler(uintptr_t generation) +{ + fake_metadata_generation = generation; + return true; +} +#endif + static DWORD WINAPI export_thread(void *unused) { (void)unused; @@ -844,6 +975,11 @@ static int start_capture_runtime(void) failures += check(ul_plugin_set_capture_schedulers( queued_capture_scheduler, queued_cleanup_scheduler), "install capture schedulers before worker"); +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + failures += check(ul_plugin_set_metadata_scheduler( + queued_metadata_scheduler), + "install metadata scheduler before worker"); +#endif return failures; } @@ -1176,6 +1312,55 @@ static int scenario_capture_clean(void) return failures; } +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION +static int scenario_metadata_lifecycle(void) +{ + uint8_t session[16] = {0x6d, 2, 3, 4}; + uintptr_t generation = 0u; + ul_audio_capture_spec spec = capture_spec(0u, 5u); + int failures = start_capture_runtime(); + LONG deactivations; + failures += reach_capture_started(session, 4u, &generation); + ul_plugin_capture_inspected(generation, &spec); + failures += check(WaitForSingleObject(fake_capture_schedule_entered, + 2000u) == WAIT_OBJECT_0, + "capture exists before metadata lifecycle checks"); + failures += check(!ul_plugin_metadata_open_frontend(generation + 1u), + "stale generation cannot open metadata"); + failures += check(ul_plugin_metadata_open_frontend(generation) && + fake_metadata_open_count == 1u && + fake_metadata_generation == generation, + "current generation opens bounded metadata observer"); + deactivations = InterlockedCompareExchange( + &fake_capture_deactivate_count, 0, 0); + failures += check(!ul_plugin_metadata_refresh_frontend(generation + 1u) && + fake_metadata_fail_count == 0u && + InterlockedCompareExchange( + &fake_capture_deactivate_count, 0, 0) == + deactivations, + "stale metadata task cannot stop current capture"); + failures += check(ul_plugin_metadata_refresh_frontend(generation) && + fake_metadata_refresh_count == 1u, + "matching output refreshes metadata"); + fake_metadata_match_result = false; + failures += check(!ul_plugin_metadata_refresh_frontend(generation) && + fake_metadata_fail_count == 1u && + InterlockedCompareExchange( + &fake_capture_deactivate_count, 0, 0) > + deactivations, + "identity change terminally fails metadata"); + failures += check(ul_plugin_metadata_close_frontend(generation, false) && + fake_metadata_close_count == 1u, + "generation closes metadata observer"); + ul_plugin_stop_accepting(); + ul_plugin_close(); + failures += check(fake_metadata_retire_count == 1u, + "worker retirement clears private metadata snapshot"); + close_capture_events(); + return failures; +} +#endif + static int scenario_capture_failures(void) { uint8_t session[16] = {0x91}; @@ -1905,6 +2090,10 @@ static int run_child(const char *scenario) return scenario_disarm_lifecycle(); if (strcmp(scenario, "disarm-close") == 0) return scenario_disarm_close_pending(); +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + if (strcmp(scenario, "metadata") == 0) + return scenario_metadata_lifecycle(); +#endif return 1; } @@ -1941,7 +2130,11 @@ int main(int argc, char **argv) L"arm-start-timeout", L"arm-stale", L"arm-bound", L"arm-teardown", L"arm-close", L"capture-clean", L"capture-failures", L"capture-cancel", L"capture-event-fail", L"cleanup-event-fail", - L"stream-event-fail", L"disarm-lifecycle", L"disarm-close"}; + L"stream-event-fail", L"disarm-lifecycle", L"disarm-close" +#if UL_AUDIO_RUNTIME_VERSION == UL_AUDIO_PROTOCOL_PROVENANCE_VERSION + , L"metadata" +#endif + }; wchar_t executable[32768]; size_t index; int failures = 0; diff --git a/native/obs-plugin/tests/test_audio_protocol.py b/native/obs-plugin/tests/test_audio_protocol.py index f03be74..3aac7e7 100644 --- a/native/obs-plugin/tests/test_audio_protocol.py +++ b/native/obs-plugin/tests/test_audio_protocol.py @@ -11,6 +11,7 @@ import numpy as np from utterleaf import obs_protocol as protocol +from utterleaf.obs_mix import BusLabel, MixSnapshot, SourceAssignment ROOT = Path(__file__).resolve().parents[1] @@ -50,14 +51,15 @@ def main() -> None: output.mkdir(parents=True, exist_ok=True) executable = output / "audio_protocol_test.exe" compile_result = run([ - args.compiler.resolve(), "-std=c11", "-Wall", "-Wextra", "-Werror", + args.compiler.resolve(), "-std=c11", + "-Wall", "-Wextra", "-Werror", ROOT / "src/audio_protocol.c", ROOT / "tests/audio_protocol_test.c", "-o", executable, ]) if compile_result.stdout or compile_result.stderr: raise AssertionError("strict native compilation produced output") native = run([executable]) - if native.stdout.strip() != "native ULAP encoder vectors and bounds passed" or native.stderr: + if native.stdout.strip() != "native ULAP encoder and routing vectors and bounds passed" or native.stderr: raise AssertionError("unexpected native fixture output") emitted = run([executable, "--emit"], binary=True) @@ -79,6 +81,39 @@ def main() -> None: ] if frames != expected: raise AssertionError("Python decoder disagrees with native wire bytes") + + emitted_v2 = run([executable, "--emit-v2"], binary=True) + if emitted_v2.stderr: + raise AssertionError("native version-2 emitter wrote diagnostics") + decoder_v2 = protocol.FrameDecoder(version=protocol.PROVENANCE_VERSION) + frames_v2 = decoder_v2.feed(emitted_v2.stdout) + decoder_v2.finish() + if frames_v2 != expected: + raise AssertionError("Python version-2 decoder disagrees with native wire bytes") + + emitted_routing = run([executable, "--emit-routing"], binary=True) + if emitted_routing.stderr: + raise AssertionError("native routing emitter wrote diagnostics") + routing = protocol.RoutingFrame( + SESSION, 1, 0x0102030405060708, ((1, 7), (2, protocol.UINT64_MAX)), + MixSnapshot( + 1, 6, + ( + SourceAssignment(bytes(15) + b"\x01", "A", 2), + SourceAssignment(bytes(15) + b"\x02", "B", 4), + ), + (BusLabel(1, "Main"), BusLabel(2, "Aux")), + ), + ) + routing_decoder = protocol.FrameDecoder(version=protocol.PROVENANCE_VERSION) + routing_frames = routing_decoder.feed(emitted_routing.stdout) + routing_decoder.finish() + if routing_frames != [routing]: + raise AssertionError("Python decoder disagrees with native routing bytes") + if emitted_routing.stdout != protocol.encode_frame( + routing, version=protocol.PROVENANCE_VERSION + ): + raise AssertionError("Python encoder disagrees with native routing bytes") print("native/Python ULAP interoperability passed") diff --git a/native/obs-plugin/tests/test_vendor_dispatch.py b/native/obs-plugin/tests/test_vendor_dispatch.py index 028e5e9..ec3330e 100644 --- a/native/obs-plugin/tests/test_vendor_dispatch.py +++ b/native/obs-plugin/tests/test_vendor_dispatch.py @@ -106,7 +106,7 @@ def test_defaults_do_not_authorize_a_request(self): def test_status_is_exact_read_only_compatibility_metadata(self): expected = {"ok": True, "protocolVersion": 1, "commandVersion": 1, - "audioVersion": 1, "maxBusMask": 63} + "audioVersion": 2, "maxBusMask": 63} self.assertEqual(self.dispatch({}, operation="status"), expected) self.assertEqual(self.dispatch({}, operation="status"), expected) self.assertEqual([VENDOR.ul_vendor_test_count(i) for i in range(3)], [0, 0, 0]) diff --git a/native/obs-plugin/tools/build.py b/native/obs-plugin/tools/build.py index 63bb7cb..f2a6014 100644 --- a/native/obs-plugin/tools/build.py +++ b/native/obs-plugin/tools/build.py @@ -17,12 +17,12 @@ "bridge", "plugin_state", "pairing_ui", "vendor_dispatch", "pairing_store", "authorization", "admission", "handshake", "crypto", "session_protocol", "audio_protocol", "audio_queue", "audio_convert", "audio_capture", - "audio_stream", "frontend_dispatch", + "audio_stream", "audio_metadata", "frontend_dispatch", ) ISC_HEADERS = ( "callback/calldata.h", "callback/proc.h", "callback/signal.h", "util/base.h", "util/bmem.h", "util/c99defs.h", "util/darray.h", - "util/text-lookup.h", "util/util_uint64.h", + "util/text-lookup.h", "util/util_uint64.h", "util/platform.h", ) diff --git a/native/obs-plugin/tools/test_native.py b/native/obs-plugin/tools/test_native.py index 666e4e4..65a5284 100644 --- a/native/obs-plugin/tools/test_native.py +++ b/native/obs-plugin/tools/test_native.py @@ -51,6 +51,8 @@ def main() -> None: "tests/audio_convert_test.c", "tests/audio_convert.def", "tests/test_audio_convert.py", "src/audio_capture.c", "src/audio_capture.h", "tests/audio_capture_test.c", "src/audio_stream.c", "src/audio_stream.h", "tests/audio_stream_test.c", + "tests/audio_stream_routing_test.c", + "src/audio_metadata.c", "src/audio_metadata.h", "tests/audio_metadata_test.c", "src/frontend_dispatch.c", "src/frontend_dispatch.h", "tests/frontend_dispatch_test.c", "tests/frontend_dispatch_fault_test.c", "tests/admission_io_test.c", "tests/admission_io_fault_test.c", @@ -74,6 +76,10 @@ def main() -> None: desktop_tests = ( "tests/test_obs_audio_pipe.py", "tests/test_obs_audio_arm.py", "tests/test_obs_audio_disarm.py", "tests/test_obs_protocol.py", "tests/test_obs_session.py", + "tests/test_obs_mix.py", "tests/test_obs_routing_protocol.py", + "tests/test_obs_routing_session.py", "tests/test_obs_routing_store.py", + "tests/test_obs_transcription_routing.py", "tests/test_obs_routing_pipe.py", + "tests/test_obs_controller_routing.py", "tests/test_windows_pipe.py", "tests/test_obs_control_status.py", "tests/test_obs_control.py", "tests/test_obs_control_enrollment.py", "tests/test_obs_websocket.py", "tests/test_obs_websocket_disconnect.py", @@ -148,6 +154,7 @@ def run(name: str, arguments: list[str | Path], timeout: int = 60) -> None: audio_protocol = output / "audio_protocol_test.exe" audio_queue = output / "audio_queue_test.exe" audio_stream = output / "audio_stream_test.exe" + audio_stream_routing = output / "audio_stream_routing_test.exe" frontend_dispatch = output / "frontend_dispatch_test.exe" frontend_fault = output / "frontend_dispatch_fault_test.exe" admission_io = output / "admission_io_test.exe" @@ -164,6 +171,7 @@ def run(name: str, arguments: list[str | Path], timeout: int = 60) -> None: "-o", audio_queue]) run("audio-queue-test", [audio_queue]) run("audio-stream-build", [*flags, "-D_M_X64=100", + "-DUL_AUDIO_RUNTIME_VERSION=1", "-DUL_AUDIO_STREAM_FIRST_TIMEOUT_MS=60", "-DUL_AUDIO_STREAM_IDLE_TIMEOUT_MS=60", "-DUL_AUDIO_STREAM_WRITE_TIMEOUT_MS=40", @@ -174,6 +182,19 @@ def run(name: str, arguments: list[str | Path], timeout: int = 60) -> None: ROOT / "src/session_protocol.c", ROOT / "tests/audio_stream_test.c", "-o", audio_stream]) run("audio-stream-test", [audio_stream]) + run("audio-stream-routing-build", [*flags, "-D_M_X64=100", "-DUL_AUDIO_RUNTIME_VERSION=2", + "-DUL_AUDIO_STREAM_FIRST_TIMEOUT_MS=60", + "-DUL_AUDIO_STREAM_IDLE_TIMEOUT_MS=60", + "-DUL_AUDIO_STREAM_WRITE_TIMEOUT_MS=40", + "-DUL_AUDIO_STREAM_ACK_TIMEOUT_MS=40", + "-DUL_AUDIO_STREAM_POLL_MS=1", + "-DUL_AUDIO_STREAM_DRAIN_TIMEOUT_MS=100", + "-DUL_AUDIO_STREAM_METADATA_POLL_MS=1", + ROOT / "src/audio_stream.c", ROOT / "src/audio_protocol.c", + ROOT / "src/session_protocol.c", + ROOT / "tests/audio_stream_routing_test.c", + "-o", audio_stream_routing]) + run("audio-stream-routing-test", [audio_stream_routing]) run("frontend-dispatch-build", [*flags, "-D_M_X64=100", ROOT / "src/frontend_dispatch.c", ROOT / "tests/frontend_dispatch_test.c", "-luser32", @@ -253,7 +274,7 @@ def run(name: str, arguments: list[str | Path], timeout: int = 60) -> None: pairing_dll, authorization_dll]) artifacts = [fixed, fault, identity, dll, crypto, authorization_state, authorization_dll, pairing_state, pairing_dll, plugin_state, session_protocol, admission_io, - admission_io_fault, audio_protocol, audio_queue, audio_stream, + admission_io_fault, audio_protocol, audio_queue, audio_stream, audio_stream_routing, frontend_dispatch, frontend_fault] if args.build is not None: convert_stub = output / "audio_convert_stub_test.exe" @@ -278,6 +299,11 @@ def run(name: str, arguments: list[str | Path], timeout: int = 60) -> None: "-o", capture_test]) run("audio-capture-test", [capture_test]) artifacts.append(capture_test) + metadata_test = output / "audio_metadata_test.exe" + run("audio-metadata-build", [*capture_flags, ROOT / "tests/audio_metadata_test.c", + "-o", metadata_test]) + run("audio-metadata-test", [metadata_test]) + artifacts.append(metadata_test) bridge_test = output / "bridge_test.exe" run("bridge-wrapper-build", [*flags, "-D_M_X64=100", f"-I{headers / 'libobs'}", f"-I{headers / 'frontend/api'}", f"-I{headers / 'obs-websocket'}", @@ -307,6 +333,7 @@ def run(name: str, arguments: list[str | Path], timeout: int = 60) -> None: "schema": 2, "scope": "pairing/admission/Arm/Disarm, read-only compatibility, desktop controller and live local recognition fixtures; optional libobs dispatch and synthetic capture/conversion; no OBS application, model loading or audio devices", "desktop_audio_pipe": "passed: Windows pipe, controller, transport, receiver, committed-window recognition, bounded model output and cancellation fixtures", "audio_stream": "passed: synthetic bounded transport fixture", + "audio_metadata": "passed: bounded observation and lifecycle fixture" if args.build is not None else "not run: supply --build and --headers", "frontend_dispatch": "passed: real Windows message-only window fixture", "audio_capture": "passed: pinned public OBS SDK synthetic fixture" if args.build is not None else "not run: supply --build and --headers", "vendor_dispatch": "passed" if args.build is not None else "not run: supply --build and --headers", diff --git a/tests/capture_obs_session.py b/tests/capture_obs_session.py index cecc2af..0f5a78b 100644 --- a/tests/capture_obs_session.py +++ b/tests/capture_obs_session.py @@ -14,6 +14,8 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1])) from utterleaf.obs_session_ui import ObsSessionWindow +from utterleaf.obs_mix import BusLabel, MixSnapshot, SourceAssignment +from utterleaf.obs_protocol import RoutingFrame class SnapshotSource: @@ -108,6 +110,14 @@ def recognition(state, message, *, preview="", tracks=0, completed=0, incomplete ), } +STATES["active"][0].routing = RoutingFrame( + b"s" * 16, 2, 80_000_000_000, ((0, 20), (2, 20)), + MixSnapshot(0, 5, + (SourceAssignment(b"a" * 16, "Host microphone", 5), + SourceAssignment(b"b" * 16, "Guest call and desktop audio", 5)), + (BusLabel(0, "Mix 1"), BusLabel(2, "Mix 3"))), +) + CAPTURES = ( ("disabled", "disabled", "840x720+80+60"), ("ready", "ready", "840x720+80+60"), @@ -118,6 +128,8 @@ def recognition(state, message, *, preview="", tracks=0, completed=0, incomplete ("compact-disabled", "disabled", "560x520+80+60"), ("compact-active", "active", "560x520+80+60"), ("compact-degraded", "degraded", "560x520+80+60"), + ("active-mixes", "active", "840x720+80+60"), + ("compact-mixes", "active", "560x520+80+60"), ) @@ -141,6 +153,7 @@ def capture(output: Path | None = None) -> Path: controller.value, coordinator.value = STATES[state_name] window.root.geometry(geometry) window.refresh() + window.preview_tabs.select(1 if name.endswith("-mixes") else 0) for _ in range(5): root.update() time.sleep(0.04) diff --git a/tests/test_obs_control_status.py b/tests/test_obs_control_status.py index 4956d76..f976f1b 100644 --- a/tests/test_obs_control_status.py +++ b/tests/test_obs_control_status.py @@ -16,7 +16,7 @@ "ok": True, "protocolVersion": 1, "commandVersion": 1, - "audioVersion": 1, + "audioVersion": 2, "maxBusMask": 63, } @@ -51,7 +51,7 @@ def _connected(monkeypatch, responses, *, vendor=True): def test_plugin_status_uses_exact_empty_request_and_typed_result(monkeypatch): control, transport = _connected(monkeypatch, [_status_response]) with control: - assert control.plugin_status() == ObsPluginStatus(1, 1, 1, 63) + assert control.plugin_status() == ObsPluginStatus(1, 1, 2, 63) request = json.loads(transport.sent[-1])["d"] assert request["requestType"] == "CallVendorRequest" assert request["requestData"] == { @@ -63,11 +63,11 @@ def test_repeated_status_is_inert_before_and_after_preparation(monkeypatch): responses = [_status_response, _status_response, _vendor_response, _prepare_response, _status_response] control, transport = _connected(monkeypatch, responses) with control: - assert control.plugin_status() == ObsPluginStatus(1, 1, 1, 63) - assert control.plugin_status() == ObsPluginStatus(1, 1, 1, 63) + assert control.plugin_status() == ObsPluginStatus(1, 1, 2, 63) + assert control.plugin_status() == ObsPluginStatus(1, 1, 2, 63) session = control.prepare_session(KEY) assert len(session) == 16 - assert control.plugin_status() == ObsPluginStatus(1, 1, 1, 63) + assert control.plugin_status() == ObsPluginStatus(1, 1, 2, 63) operations = [ json.loads(item)["d"]["requestData"]["requestType"] for item in transport.sent[2:] @@ -78,7 +78,8 @@ def test_repeated_status_is_inert_before_and_after_preparation(monkeypatch): @pytest.mark.parametrize("change", [ lambda data: data.update(protocolVersion=2), lambda data: data.update(commandVersion=2), - lambda data: data.update(audioVersion=2), + lambda data: data.update(audioVersion=1), + lambda data: data.update(audioVersion=3), lambda data: data.update(maxBusMask=62), lambda data: data.update(commandVersion=True), lambda data: data.update(extra=0), diff --git a/tests/test_obs_controller.py b/tests/test_obs_controller.py index 613d4ea..e75dfc0 100644 --- a/tests/test_obs_controller.py +++ b/tests/test_obs_controller.py @@ -11,7 +11,7 @@ from utterleaf import obs_controller as module from utterleaf.obs_control import ObsControlCancelled, ObsControlDisconnected, ObsControlError -from utterleaf.obs_control import StreamEvent, StreamSnapshot +from utterleaf.obs_control import ObsPluginStatus, StreamEvent, StreamSnapshot from utterleaf.obs_audio_pipe import ObsAudioPipeCancelled from utterleaf.obs_protocol import AudioFrame, EndFrame, EndReason, StartFrame from utterleaf.obs_session import ObsCaptureSession @@ -120,6 +120,7 @@ def connect(self, *args, **kwargs): def plugin_status(self): self.calls.append("status") + return ObsPluginStatus(1, 1, 1, 63) def stream_status(self): self.calls.append("idle") diff --git a/tests/test_obs_controller_routing.py b/tests/test_obs_controller_routing.py new file mode 100644 index 0000000..6e7eeb1 --- /dev/null +++ b/tests/test_obs_controller_routing.py @@ -0,0 +1,189 @@ +"""Versioned controller integration with real private routing ownership.""" + +import io +import threading +from dataclasses import replace + +import pytest + +from test_obs_controller import Control, Pipe, SESSION, ORIGIN, arm, audio, eventually, start_frame +from utterleaf import capture_store +from utterleaf.config import Config +from utterleaf.obs_control import ObsPluginStatus +from utterleaf.obs_controller import ObsSessionController +from utterleaf.obs_mix import BusLabel, MixSnapshot, SourceAssignment +from utterleaf.obs_protocol import EndFrame, EndReason, RoutingFrame +from utterleaf.obs_routing_store import ObsRoutingStore +from utterleaf.obs_session import ObsCaptureSession +from utterleaf.obs_transcription import ObsTranscriptionCoordinator, ObsTranscriptionError +from utterleaf.transcript import Segment, Transcript + + +def observation(revision=1): + return RoutingFrame( + SESSION, revision, ORIGIN + revision, ((0, revision - 1),), + MixSnapshot(0, 1, (SourceAssignment(b"s" * 16, f"Private input {revision}", 1),), + (BusLabel(0, "Stream"),)), + ) + + +@pytest.fixture +def session(monkeypatch): + monkeypatch.setattr(capture_store, "WINDOW_SECONDS", 0.001) + control, pipe = Control(), Pipe() + control.plugin_status = lambda: ObsPluginStatus(1, 1, 2, 63) + histories = [] + journals = [] + recognized = threading.Event() + + def recognize(samples, config, *, cancel): + assert config.allow_network is False + recognized.set() + return Transcript((Segment(0, len(samples) / 16000, "local text"),), "en") + + owner = ObsTranscriptionCoordinator(Config(), recognizer_factory=lambda cfg: recognize, + journal_factory=io.BytesIO) + journal_factory = io.BytesIO + + def history_factory(): + journal = journal_factory() + history = ObsRoutingStore(journal_factory=lambda: journal) + journals.append(journal) + histories.append(history) + return history + + def receiver(*args, **kwargs): + assert kwargs["protocol_version"] == 2 + return ObsCaptureSession(*args, **kwargs, routing_factory=history_factory) + + def connect_pipe(*args, **kwargs): + assert kwargs["protocol_version"] == 2 + return pipe.connect(*args, **kwargs) + + controller = ObsSessionController(owner, control_factory=control.connect, + pipe_factory=connect_pipe, receiver_factory=receiver) + + def set_journal_factory(factory): + nonlocal journal_factory + journal_factory = factory + + yield controller, control, pipe, owner, histories, journals, recognized, set_journal_factory + controller.cancel() + assert controller.wait(3), "controller or routing owner did not finish cleanup" + if any(history.cleanup_failed for history in histories): + with pytest.raises(ObsTranscriptionError): + owner.close(3) + else: + assert owner.close(3) + for journal in journals: + io.BytesIO.close(journal) + + +def test_routing_updates_flow_through_live_controller_without_audio_gaps(session): + controller, control, pipe, owner, histories, journals, recognized, _ = session + arm((controller, control, pipe, owner)) + first, second = observation(), observation(2) + pipe.batches.put([start_frame(), first, audio()]) + assert recognized.wait(2), "recognition must start before capture ends" + assert controller.snapshot().routing == first + assert "Private input" not in repr(controller.snapshot()) + pipe.batches.put([second, audio(1), EndFrame(SESSION, EndReason.STREAM_STOPPED, ((0, 1),))]) + assert controller.wait(3) + assert controller.snapshot().state == "complete" + assert controller.snapshot().routing == second + assert tuple(owner.iter_routing_observations()) == (first, second) + assert [segment.start for segment in owner.iter_segments()] == pytest.approx([0, 0.001]) + assert len(histories) == 1 and not journals[0].closed + controller.cancel() + assert controller.snapshot().routing is None + assert controller.wait(3) and journals[0].closed + assert tuple(owner.iter_routing_observations()) == () + + +@pytest.mark.parametrize("bad", [None, "revision", "position", "identity"]) +def test_unverified_routing_is_never_published_or_transcribed(session, bad): + controller, control, pipe, owner, histories, _, recognized, _ = session + arm((controller, control, pipe, owner)) + frame = observation() + if bad == "revision": + frame = replace(frame, revision=2) + elif bad == "position": + frame = replace(frame, positions=((0, 1),)) + elif bad == "identity": + frame = replace(frame, session_id=b"x" * 16) + pipe.batches.put([start_frame(), *([] if bad is None else [frame]), audio()]) + assert controller.wait(3) + assert controller.snapshot().state == "incomplete" + assert controller.snapshot().routing is None + assert not histories and not recognized.is_set() + assert not tuple(owner.iter_routing_observations()) + + +@pytest.mark.parametrize("after_end", [False, True]) +@pytest.mark.parametrize("close_error", [False, True]) +def test_cancel_waits_for_history_close_and_reports_late_failure(session, after_end, close_error): + controller, control, pipe, owner, histories, _, recognized, set_factory = session + closing, release = threading.Event(), threading.Event() + + class SlowClose(io.BytesIO): + def close(self): + closing.set() + assert release.wait(3), "test did not release history cleanup" + if close_error: + raise OSError("private routing path must not escape") + super().close() + + set_factory(SlowClose) + try: + arm((controller, control, pipe, owner)) + pipe.batches.put([start_frame(), observation(), audio()]) + assert recognized.wait(2) + if after_end: + pipe.batches.put([EndFrame(SESSION, EndReason.STREAM_STOPPED, ((0, 0),))]) + assert controller.wait(3) + assert controller.snapshot().state == "complete" + controller.cancel() + assert closing.wait(2) + assert not controller.wait(0) + assert controller.snapshot().state == "cancelling" + assert controller.snapshot().routing is None + finally: + release.set() + assert controller.wait(3) + assert histories[0].wait_closed(0) + assert controller.snapshot().state == ("error" if close_error else "cancelled") + assert "private routing path" not in repr(controller.snapshot()) + + +def test_history_write_failure_interrupts_waiting_pipe(session): + controller, control, pipe, owner, histories, _, _, set_factory = session + release = threading.Event() + + class FailedWrite(io.BytesIO): + def write(self, value): + assert release.wait(3) + raise OSError("private routing path") + + set_factory(FailedWrite) + try: + arm((controller, control, pipe, owner)) + pipe.batches.put([start_frame(), observation()]) + eventually(lambda: controller.snapshot().routing is not None) + release.set() + assert controller.wait(3), "borrowed history failure must interrupt an idle pipe read" + assert owner.failed and controller.snapshot().state == "error" + assert "private routing path" not in repr(controller.snapshot()) + assert histories[0].failed + finally: + release.set() + + +@pytest.mark.parametrize("version", [None, True, 0, 3, "2"]) +def test_unknown_audio_version_never_becomes_armable(session, version): + controller, control, pipe, owner, histories, *_ = session + control.plugin_status = lambda: ObsPluginStatus(1, 1, version, 63) + assert controller.connect("127.0.0.1", 4455, "test", expected_executable="fixture.exe") + assert controller.wait(3) + assert controller.snapshot().state == "error" + assert not controller.arm(b"k" * 32) + assert not pipe.read_started.is_set() and not histories diff --git a/tests/test_obs_mix.py b/tests/test_obs_mix.py new file mode 100644 index 0000000..f679350 --- /dev/null +++ b/tests/test_obs_mix.py @@ -0,0 +1,249 @@ +"""Invariant and privacy tests for bounded OBS mix provenance.""" + +from dataclasses import FrozenInstanceError + +import pytest + +from utterleaf.obs_mix import ( + BusLabel, + MAX_LABEL_BYTES, + MAX_SOURCE_NAME_BYTES, + MAX_SOURCES, + MixMetadataError, + MixRelationship, + MixSnapshot, + SourceAssignment, +) + + +PRIMARY = b"primary-source!!" +GUEST = b"guest-source-000" + + +class IntSubclass(int): + pass + + +class StrSubclass(str): + pass + + +class BytesSubclass(bytes): + pass + + +def source(source_id=PRIMARY, name="Complete mix", bus_mask=0b001): + return SourceAssignment(source_id, name, bus_mask) + + +def snapshot(*, primary=0, mask=0b111, sources=(), names=("Stream", "Guest", "Archive")): + labels = tuple(BusLabel(bus, name) for bus, name in zip(range(6), names) if mask & (1 << bus)) + return MixSnapshot(primary, mask, tuple(sources), labels) + + +def test_snapshot_is_frozen_and_queries_selected_bus_assignments(): + complete = source(bus_mask=0b011) + guest = source(GUEST, "Guest 🎧", 0b110) + mixes = snapshot(sources=(guest, complete)) + + assert mixes.inputs_for_bus(0) == (complete,) + assert mixes.inputs_for_bus(1) == (guest, complete) + assert mixes.inputs_for_bus(2) == (guest,) + with pytest.raises(FrozenInstanceError): + mixes.primary_bus = 1 + with pytest.raises(FrozenInstanceError): + complete.name = "changed" + + +def test_invalid_metadata_uses_one_generic_public_error(): + with pytest.raises(MixMetadataError, match="^Invalid OBS mix metadata$") as caught: + SourceAssignment(PRIMARY, "private\nname", 1) + assert "private" not in str(caught.value) + + +def test_relationship_uses_only_exact_source_id_sets(): + sources = ( + source(GUEST, "Same display name", 0b100), + source(PRIMARY, "Same display name", 0b011), + ) + mixes = snapshot(mask=0b1111, sources=sources, names=("P", "Copy", "Other", "Empty")) + + assert mixes.relationship(0) is MixRelationship.PRIMARY + assert mixes.relationship(1) is MixRelationship.SAME_INPUTS + assert mixes.relationship(2) is MixRelationship.DIFFERENT_INPUTS + assert mixes.relationship(3) is MixRelationship.UNASSIGNED + + +def test_two_unassigned_selected_buses_are_unassigned(): + mixes = snapshot(primary=0, mask=0b011, names=("Main", "Spare")) + + assert mixes.inputs_for_bus(0) == () + assert mixes.inputs_for_bus(1) == () + assert mixes.relationship(0) is MixRelationship.PRIMARY + assert mixes.relationship(1) is MixRelationship.UNASSIGNED + + +@pytest.mark.parametrize( + "source_id,name,bus_mask", + [ + (b"short", "name", 1), + (bytearray(16), "name", 1), + (PRIMARY, "", 1), + (PRIMARY, "x" * (MAX_SOURCE_NAME_BYTES + 1), 1), + (PRIMARY, "name", 0), + (PRIMARY, "name", 64), + (PRIMARY, "name", True), + ], +) +def test_source_assignment_rejects_inexact_or_unbounded_data(source_id, name, bus_mask): + with pytest.raises(ValueError, match="^Invalid OBS mix metadata$"): + SourceAssignment(source_id, name, bus_mask) + + +def test_primitive_subclasses_are_rejected(): + with pytest.raises(ValueError): + SourceAssignment(BytesSubclass(PRIMARY), "name", 1) + with pytest.raises(ValueError): + SourceAssignment(PRIMARY, StrSubclass("name"), 1) + with pytest.raises(ValueError): + SourceAssignment(PRIMARY, "name", IntSubclass(1)) + with pytest.raises(ValueError): + BusLabel(IntSubclass(0), "Main") + with pytest.raises(ValueError): + MixSnapshot(IntSubclass(0), 1, (), (BusLabel(0, "Main"),)) + + +@pytest.mark.parametrize( + "text", + [ + "line\nbreak", + "delete\x7f", + "control\x85", + "arabic mark\u061c", + "zero width space\u200b", + "left-to-right mark\u200e", + "right-to-left mark\u200f", + "separator\u2028", + "paragraph\u2029", + "embed\u202a", + "override\u202e", + "isolate\u2066", + "pop isolate\u2069", + "byte-order mark\ufeff", + "\ud800", + ], +) +def test_display_text_rejects_control_and_bidirectional_formatting(text): + with pytest.raises(ValueError, match="^Invalid OBS mix metadata$"): + SourceAssignment(PRIMARY, text, 1) + with pytest.raises(ValueError, match="^Invalid OBS mix metadata$"): + BusLabel(0, text) + + +@pytest.mark.parametrize("text", ["", " ", "\u00a0", "\u200c", "\u200d", " \u200c\u200d "]) +def test_display_text_rejects_whitespace_or_joiners_without_visible_text(text): + with pytest.raises(MixMetadataError, match="^Invalid OBS mix metadata$"): + SourceAssignment(PRIMARY, text, 1) + with pytest.raises(MixMetadataError, match="^Invalid OBS mix metadata$"): + BusLabel(0, text) + + +def test_display_text_bounds_and_preserves_unicode_joiners_and_spacing(): + label = BusLabel(0, "👩\u200d💻 Mix") + assignment = SourceAssignment(PRIMARY, " می\u200cکروفن ", 1) + assert label.label == "👩\u200d💻 Mix" + assert assignment.name == " می\u200cکروفن " + + with pytest.raises(ValueError): + BusLabel(0, "🌿" * (MAX_LABEL_BYTES // 4 + 1)) + with pytest.raises(ValueError): + SourceAssignment(PRIMARY, "🌿" * (MAX_SOURCE_NAME_BYTES // 4 + 1), 1) + + +@pytest.mark.parametrize("bus,label", [(True, "Main"), (6, "Main"), (0, ""), (0, b"Main")]) +def test_bus_label_rejects_inexact_bus_or_text(bus, label): + with pytest.raises(ValueError, match="^Invalid OBS mix metadata$"): + BusLabel(bus, label) + + +@pytest.mark.parametrize( + "primary,mask,sources,labels", + [ + (True, 1, (), (BusLabel(0, "Main"),)), + (0, True, (), (BusLabel(0, "Main"),)), + (0, 0, (), ()), + (1, 1, (), (BusLabel(0, "Main"),)), + (0, 1, [], (BusLabel(0, "Main"),)), + (0, 1, (), [BusLabel(0, "Main")]), + (0, 0b11, (), (BusLabel(0, "Main"),)), + (0, 0b11, (), (BusLabel(1, "Other"), BusLabel(0, "Main"))), + ], +) +def test_snapshot_rejects_inexact_empty_or_incomplete_selection(primary, mask, sources, labels): + with pytest.raises(ValueError, match="^Invalid OBS mix metadata$"): + MixSnapshot(primary, mask, sources, labels) + + +def test_snapshot_rejects_more_than_six_labels(): + labels = tuple(BusLabel(0, f"Label {index}") for index in range(7)) + + with pytest.raises(MixMetadataError, match="^Invalid OBS mix metadata$"): + MixSnapshot(0, 1, (), labels) + + +def test_snapshot_requires_sorted_unique_ids_and_selected_source_masks(): + first = source(b"0000000000000001", "First", 0b01) + second = source(b"0000000000000002", "Second", 0b10) + labels = (BusLabel(0, "Main"), BusLabel(1, "Other")) + + with pytest.raises(ValueError): + MixSnapshot(0, 0b11, (second, first), labels) + with pytest.raises(ValueError): + MixSnapshot(0, 0b11, (first, first), labels) + with pytest.raises(ValueError): + MixSnapshot(0, 0b01, (second,), (labels[0],)) + + +def test_snapshot_caps_sources_at_128(): + sources = tuple( + source(index.to_bytes(16, "big"), f"Input {index}", 1) + for index in range(MAX_SOURCES + 1) + ) + with pytest.raises(ValueError): + snapshot(mask=1, sources=sources, names=("Main",)) + + +def test_exact_text_and_source_count_bounds_are_accepted(): + sources = tuple( + source(index.to_bytes(16, "big"), "n" * MAX_SOURCE_NAME_BYTES, 1) + for index in range(MAX_SOURCES) + ) + mixes = MixSnapshot( + 0, + 1, + sources, + (BusLabel(0, "l" * MAX_LABEL_BYTES),), + ) + + assert len(mixes.sources) == MAX_SOURCES + assert len(mixes.labels[0].label.encode("utf-8")) == MAX_LABEL_BYTES + + +@pytest.mark.parametrize("bus", [1, -1, 6, True, 0.0]) +def test_nonselected_or_inexact_bus_queries_are_rejected(bus): + mixes = snapshot(mask=1, names=("Main",)) + with pytest.raises(ValueError, match="^Invalid OBS mix metadata$"): + mixes.inputs_for_bus(bus) + with pytest.raises(ValueError, match="^Invalid OBS mix metadata$"): + mixes.relationship(bus) + + +def test_representations_hide_source_ids_and_all_display_text(): + assignment = source(PRIMARY, "Private input", 0b11) + label = BusLabel(0, "Private bus") + mixes = MixSnapshot(0, 0b11, (assignment,), (label, BusLabel(1, "Also private"))) + + rendered = " ".join((repr(assignment), repr(label), repr(mixes))) + assert "primary-source" not in rendered + assert "Private" not in rendered + assert "Also private" not in rendered diff --git a/tests/test_obs_routing_pipe.py b/tests/test_obs_routing_pipe.py new file mode 100644 index 0000000..b48030f --- /dev/null +++ b/tests/test_obs_routing_pipe.py @@ -0,0 +1,196 @@ +"""Explicit OBS protocol-v2 selection at the authenticated pipe boundary.""" + +from __future__ import annotations + +import hashlib +import hmac +import struct +import time + +import numpy as np +import pytest + +from utterleaf import obs_audio_pipe +from utterleaf import obs_protocol +from utterleaf.obs_mix import BusLabel, MixSnapshot, SourceAssignment + + +SESSION = bytes(range(16)) +ACK_DOMAIN = b"Utterleaf OBS audio server ack v1\0" + + +class ProcessLease: + pid = 314 + + def __init__(self) -> None: + self.closed = False + self.verified: list[int] = [] + + def verify_pid(self, pid, *, cancelled, deadline) -> None: + assert not cancelled() + assert deadline > time.monotonic() + assert pid == self.pid + self.verified.append(pid) + + def close(self) -> None: + self.closed = True + + +class AuthenticatedPipe: + def __init__(self) -> None: + self.pending = bytearray() + self.writes: list[bytes] = [] + self.closed = False + self.max_chunk = 23 + + def server_pid(self) -> int: + return ProcessLease.pid + + def write_all(self, data, *, deadline) -> None: + assert deadline > time.monotonic() + self.writes.append(data) + if len(data) == 56: + assert data[:8] == b"ULAH\x01\x01\x00\x00" + mac = hmac.new(data[24:], ACK_DOMAIN + data[:24], hashlib.sha256).digest() + self.pending.extend(b"ULAH\x01\x02\x00\x00" + data[8:24] + mac) + return + magic, version, kind, reserved, session, mask, status, trailing = struct.unpack( + "<4sBBH16sBBH", data + ) + assert (magic, version, reserved, session, trailing) == ( + b"ULAC", 1, 0, SESSION, 0 + ) + if kind == 1: + assert 0 <= mask <= 63 and status == 0 + self.pending.extend( + struct.pack("<4sBBH16sBBH", b"ULAC", 1, 2, 0, session, mask, 1, 0) + ) + else: + assert (kind, mask, status) == (3, 0, 0) + + def read(self, maximum, *, deadline) -> bytes: + assert deadline > time.monotonic() + count = min(maximum, self.max_chunk, len(self.pending)) + result = bytes(self.pending[:count]) + del self.pending[:count] + return result + + def available_bytes(self, *, deadline) -> int: + assert deadline > time.monotonic() + return len(self.pending) + + def wait_for_disconnect(self, *, deadline) -> None: + assert deadline > time.monotonic() + assert not self.pending + assert self.writes[-1] == b"ULAC\x01\x03\x00\x00" + SESSION + b"\x00" * 4 + self.closed = True + + def close(self) -> None: + self.closed = True + + +def connect_v2(monkeypatch): + pipe, lease = AuthenticatedPipe(), ProcessLease() + + def open_pipe(name, **controls): + assert name == "\\\\.\\pipe\\Utterleaf.OBS." + SESSION.hex() + assert controls["cancelled"]() is False + return pipe + + monkeypatch.setattr(obs_audio_pipe.windows_pipe, "connect", open_pipe) + connection = obs_audio_pipe.connect( + SESSION, + lease, + cancelled=lambda: False, + deadline=time.monotonic() + 2, + protocol_version=obs_protocol.PROVENANCE_VERSION, + ) + connection.arm(deadline=time.monotonic() + 2) + return connection, pipe, lease + + +def routing_stream() -> tuple[obs_protocol.Frame, ...]: + snapshot = MixSnapshot( + 0, + 3, + (SourceAssignment(b"s" * 16, "Desktop", 3),), + (BusLabel(0, "Program"), BusLabel(1, "Guest")), + ) + pcm = np.array(((0.25, -0.25), (0.5, -0.5)), dtype="= len(first_bytes) + 4: + raise OSError("Second private raw detail") + return super().write(bytes(data[:4])) + + journal = FailedWriter() + store = storage.ObsRoutingStore(journal_factory=lambda: journal) + assert store.append(first) + assert store.append(second) + store.finish() + with pytest.raises(storage.ObsRoutingStoreError, match="storage failed"): + store.wait_ready() + assert journal.getvalue() == first_bytes + assert "Second private raw detail" not in str(store.error) + close(store) + + +def test_creation_failure_is_sanitized_and_default_requires_local_filesystem(monkeypatch): + monkeypatch.setattr( + storage, + "require_local_filesystem", + lambda _path: (_ for _ in ()).throw(storage.LocalFilesystemError("private path")), + ) + monkeypatch.setattr( + storage.tempfile, + "TemporaryFile", + lambda **_kwargs: pytest.fail("created history on unverified filesystem"), + ) + store = storage.ObsRoutingStore() + with pytest.raises(storage.ObsRoutingStoreError, match="storage failed") as caught: + store.wait_ready() + assert "private path" not in str(caught.value) + close(store) + + +def test_invalid_journal_factory_result_is_a_sanitized_storage_failure(): + store = storage.ObsRoutingStore(journal_factory=lambda: object()) + with pytest.raises(storage.ObsRoutingStoreError, match="storage failed"): + store.wait_ready() + assert "journal" not in str(store.error).lower() + close(store) + + +def test_cancel_is_nonblocking_while_file_close_is_slow(): + close_started = threading.Event() + release_close = threading.Event() + + class SlowClose(io.BytesIO): + def close(self): + close_started.set() + assert release_close.wait(2) + super().close() + + journal = SlowClose() + store = finish(storage.ObsRoutingStore(journal_factory=lambda: journal)) + returned = threading.Event() + caller = threading.Thread(target=lambda: (store.close(), returned.set())) + caller.start() + try: + assert close_started.wait(2) + assert returned.wait(0.5) + assert not store.wait_closed(0.05) + finally: + release_close.set() + caller.join(2) + assert store.wait_closed(2) + + +def test_cancel_waits_for_active_streaming_reader_before_claiming_cleanup(): + journal = io.BytesIO() + store = storage.ObsRoutingStore(journal_factory=lambda: journal) + assert store.append(routing(1)) + assert store.append(routing(2)) + finish(store) + + observations = store.iter_observations() + assert next(observations).revision == 1 + store.close() + assert not store.wait_closed(0.05) + observations.close() + assert store.wait_closed(2) + assert journal.closed + + +def test_close_before_drain_suppresses_reading_and_wait_ready_success(): + factory_started = threading.Event() + release_factory = threading.Event() + + def factory(): + factory_started.set() + assert release_factory.wait(2) + return io.BytesIO() + + store = storage.ObsRoutingStore(journal_factory=factory) + assert factory_started.wait(2) + assert store.append(routing()) + store.close() + release_factory.set() + assert store.wait_closed(2) + with pytest.raises(TranscriptionCancelled): + store.wait_ready() + assert tuple(store.iter_observations()) == () + + +def test_wait_ready_honors_external_cancellation_without_taking_ownership(): + factory_started = threading.Event() + release_factory = threading.Event() + cancelled = threading.Event() + + def factory(): + factory_started.set() + assert release_factory.wait(2) + return io.BytesIO() + + store = storage.ObsRoutingStore(journal_factory=factory) + assert factory_started.wait(2) + cancelled.set() + with pytest.raises(TranscriptionCancelled): + store.wait_ready(cancelled.is_set) + assert not store.wait_closed(0), "wait_ready took cleanup ownership" + + store.close() + release_factory.set() + assert store.wait_closed(2) + + +def test_close_failure_is_terminal_sanitized_and_reported_after_wait(): + class FailedClose(io.BytesIO): + def close(self): + raise OSError("private cleanup detail") + + journal = FailedClose() + store = finish(storage.ObsRoutingStore(journal_factory=lambda: journal)) + store.close() + assert store.wait_closed(2) + assert store.failed + assert store.cleanup_failed + assert str(store.error) == "Private OBS routing history cleanup failed." + assert "private cleanup detail" not in str(store.error) + io.BytesIO.close(journal) + + +def test_iterator_rejects_nonrouting_or_corrupt_records_without_private_bytes(): + journal = io.BytesIO() + store = storage.ObsRoutingStore(journal_factory=lambda: journal) + assert store.append(routing(name="Hidden input")) + finish(store) + data = bytearray(journal.getvalue()) + data[5] = 1 + journal.seek(0) + journal.write(data) + + with pytest.raises(storage.ObsRoutingStoreError, match="^Invalid OBS routing history$"): + tuple(store.iter_observations()) + assert store.failed + assert "Hidden" not in str(store.error) + close(store) + + +def test_iterator_sanitizes_arbitrary_storage_read_failures(): + class FailedRead(io.BytesIO): + def read(self, _size=-1): + raise RuntimeError("private source read detail") + + journal = FailedRead() + store = storage.ObsRoutingStore(journal_factory=lambda: journal) + assert store.append(routing(name="Hidden input")) + finish(store) + + with pytest.raises(storage.ObsRoutingStoreError, match="^Invalid OBS routing history$"): + tuple(store.iter_observations()) + assert "private source read detail" not in str(store.error) + close(store) + + +def test_maximum_record_is_bounded_and_streamed_without_materializing_history(): + labels = tuple(BusLabel(bus, "l" * 64) for bus in range(6)) + sources = tuple( + SourceAssignment(index.to_bytes(16, "big"), "n" * 128, 0x3F) + for index in range(128) + ) + frame = RoutingFrame( + SESSION, + 1, + 2**64 - 1, + tuple((bus, 2**64 - 1) for bus in range(6)), + MixSnapshot(5, 0x3F, sources, labels), + ) + encoded = encode_frame(frame, version=PROVENANCE_VERSION) + assert len(encoded) == storage.MAX_ROUTING_RECORD_BYTES + journal = io.BytesIO() + store = storage.ObsRoutingStore(journal_factory=lambda: journal) + assert store.append(frame) + finish(store) + iterator = store.iter_observations() + assert next(iterator) == frame + with pytest.raises(StopIteration): + next(iterator) + close(store) + + +def test_finish_and_nonexact_frames_are_rejected_without_publication(): + store = storage.ObsRoutingStore(journal_factory=io.BytesIO) + with pytest.raises(TypeError, match="exact RoutingFrame"): + store.append(object()) + store.finish() + assert not store.append(routing()) + store.wait_ready() + assert tuple(store.iter_observations()) == () + close(store) diff --git a/tests/test_obs_session_ui.py b/tests/test_obs_session_ui.py index 9123c64..fc46323 100644 --- a/tests/test_obs_session_ui.py +++ b/tests/test_obs_session_ui.py @@ -9,6 +9,8 @@ import pytest from utterleaf import obs_session_ui as ui +from utterleaf.obs_mix import BusLabel, MixSnapshot, SourceAssignment +from utterleaf.obs_protocol import RoutingFrame class RecognitionState(Enum): @@ -478,3 +480,138 @@ def test_browse_is_explicit_and_render_never_opens_picker(opened, monkeypatch): assert calls == [] window.browse_button.invoke() assert len(calls) == 1 and window.executable_var.get() == r"C:\OBS\obs64.exe" + + +def routing_observation(revision=1, *, name="Guest and desktop"): + return RoutingFrame( + b"s" * 16, revision, 100 + revision, ((0, 0), (1, 0), (2, 0), (3, 0)), + MixSnapshot(0, 15, + (SourceAssignment(b"a" * 16, name, 3), + SourceAssignment(b"b" * 16, "Music", 4)), + tuple(BusLabel(bus, f"Mix {bus + 1}") for bus in range(4))), + ) + + +def test_mix_details_are_local_read_only_and_distinguish_assignments(opened, tk_root): + snapshot = session("active", primary=0, buses=(0, 1, 2, 3), seconds=25) + snapshot.routing = routing_observation() + window, controller, _coordinator, actions = opened( + snapshot, recognition(RecognitionState.RUNNING, preview="Original transcript", tracks=4), + ) + window.preview_tabs.select(1) + tk_root.update() + details = window.mix_details.get("1.0", "end-1c") + assert "Complete streaming mix" in details + assert "Same assigned inputs as the primary mix; audio may differ" in details + assert "Different assigned inputs" in details and "No observed assigned inputs" in details + assert "before or after this observation" in details + assert "Guest and desktop" in details and "Music" in details + assert "update 1" in details + assert state(window.mix_details) == "disabled" + assert not int(window.mix_details.cget("exportselection")) + assert actions.calls == [] + + controller.value.routing = routing_observation(2, name="Renamed guest") + window.refresh() + assert window.preview.get("1.0", "end-1c") == "Original transcript" + assert "Renamed guest" in window.mix_details.get("1.0", "end-1c") + assert "Guest and desktop" not in window.mix_details.get("1.0", "end-1c") + assert window.preview_tabs.index("current") == 1 + assert "Renamed guest" not in repr(window) + + controller.value.state = "cancelling" + window.refresh() + assert window.mix_details.get("1.0", "end-1c") == "" + assert window.preview_tabs.index("current") == 0 + assert str(window.preview_tabs.tab(1, "state")) == "disabled" + + +@pytest.mark.parametrize("geometry", ["560x520", "840x720"]) +def test_mix_details_share_preview_space_and_keep_stop_accessible(opened, tk_root, geometry): + snapshot = session("active", primary=0, buses=(0, 1, 2, 3)) + snapshot.routing = routing_observation(name="Long private input name " * 5) + window, _controller, _coordinator, actions = opened(snapshot) + window.root.geometry(geometry + "+40+40") + window.preview_tabs.select(1) + tk_root.update() + bottom = window.root.winfo_rooty() + window.root.winfo_height() + assert window.mix_details.winfo_ismapped() + assert window.mix_details.winfo_height() >= 100 + for widget in (window.stop_button, window.cancel_button, window.close_button): + assert widget.winfo_ismapped() + assert widget.winfo_rooty() + widget.winfo_height() <= bottom + assert actions.calls == [] + + +@pytest.mark.parametrize("external_destroy", [False, True]) +def test_closing_view_releases_cached_private_routing(opened, tk_root, external_destroy): + snapshot = session("active", primary=0, buses=(0, 1, 2, 3)) + snapshot.routing = routing_observation() + window, _controller, _coordinator, actions = opened(snapshot) + assert window._last_routing is snapshot.routing + if external_destroy: + window.root.destroy() + tk_root.update() + else: + window.close() + assert window.closed and window._last_routing is None + assert actions.calls == [("close",)] + + +def test_mix_tabs_support_keyboard_navigation_without_actions(opened, tk_root): + snapshot = session("active", primary=0, buses=(0, 1, 2, 3)) + snapshot.routing = routing_observation() + window, _controller, _coordinator, actions = opened(snapshot) + window.preview.focus_force() + tk_root.update() + window.root.event_generate("") + tk_root.update() + assert window.preview_tabs.index("current") == 1 + window.root.event_generate("") + tk_root.update() + assert window.preview_tabs.index("current") == 0 + # ttk binds tab mnemonics to Option on Aqua and Alt on other backends. + aqua = tk_root.tk.call("tk", "windowingsystem") == "aqua" + modifier = "Option" if aqua else "Alt" + binding = tk_root.tk.splitlist(window.root.bind(f"<{modifier}-Key>")) + assert binding == ("ttk::notebook::MnemonicActivation", str(window.root), "%K") + # Aqua remaps generated Option-letter keysyms through the keyboard layout. + # Exercise the registered mnemonic with its letter explicitly on every host; + # Ctrl+Tab above still covers delivered key events on Aqua. + callback = (*binding[:-1], "m") + assert tk_root.tk.call("catch", callback) == 3 # Tcl's handled-event break. + tk_root.update() + assert window.preview_tabs.index("current") == 1 + if not aqua: + window.preview_tabs.select(0) + window.root.event_generate(f"<{modifier}-m>") + tk_root.update() + assert window.preview_tabs.index("current") == 1 + assert actions.calls == [] + + +def test_maximum_mix_details_remain_bounded_scrollable_and_local(opened, tk_root): + snapshot = session("active", primary=0, buses=tuple(range(6))) + snapshot.routing = RoutingFrame( + b"s" * 16, 1, 0, tuple((bus, 0) for bus in range(6)), + MixSnapshot(0, 63, + tuple(SourceAssignment(index.to_bytes(16, "big"), + f"Input {index:03d} ".ljust(128, "x"), 63) + for index in range(128)), + tuple(BusLabel(bus, f"Label {bus} ".ljust(64, "x")) for bus in range(6))), + ) + window, _controller, _coordinator, actions = opened(snapshot) + window.root.geometry("560x520+40+40") + window.preview_tabs.select(1) + tk_root.update() + details = window.mix_details.get("1.0", "end-1c") + assert details.count("Input 127 ") == 6 + assert details.count("Same assigned inputs") == 5 + assert len(details) < 110_000 + assert window.mix_details.yview()[1] < 1 + assert window.mix_details.winfo_height() >= 100 + assert window.stop_button.winfo_ismapped() + window.mix_details.yview_moveto(1) + tk_root.update() + assert window.mix_details.yview()[1] == 1 + assert actions.calls == [] diff --git a/tests/test_obs_transcription_routing.py b/tests/test_obs_transcription_routing.py new file mode 100644 index 0000000..e8fe3d3 --- /dev/null +++ b/tests/test_obs_transcription_routing.py @@ -0,0 +1,268 @@ +"""Routing-history ownership tests for live OBS transcription.""" + +from __future__ import annotations + +import io +import threading + +import numpy as np +import pytest + +from utterleaf.capture_store import CaptureStore +from utterleaf.config import Config +from utterleaf.obs_mix import BusLabel, MixSnapshot, SourceAssignment +from utterleaf.obs_protocol import RoutingFrame +from utterleaf.obs_routing_store import ObsRoutingStore, ObsRoutingStoreError +from utterleaf.obs_session import CapturedTrack, ObsCaptureResult, ObsSessionError +from utterleaf.obs_transcription import ( + ObsTranscriptionCoordinator, + ObsTranscriptionError, + ObsTranscriptionState, +) +from utterleaf.transcript import Segment, Transcript, TranscriptionCancelled + + +ORIGIN = 4_000_000_000 +RATE = 16_000 +SESSION = b"routing-session!" + + +def routing(revision=1): + return RoutingFrame( + SESSION, + revision, + ORIGIN + revision, + ((0, revision - 1),), + MixSnapshot( + 0, + 1, + (SourceAssignment(revision.to_bytes(16, "big"), "Input", 1),), + (BusLabel(0, "Stream"),), + ), + ) + + +def history(*frames, journal=None): + journal = journal or io.BytesIO() + store = ObsRoutingStore(journal_factory=lambda: journal) + for frame in frames: + assert store.append(frame) + store.finish() + return store, journal + + +def coordinator(): + def factory(_config): + def recognize(audio, _snapshot, *, cancel): + return Transcript((Segment(0, len(audio) / RATE, "recognized"),), "en") + + return recognize + + return ObsTranscriptionCoordinator( + Config(), recognizer_factory=factory, journal_factory=io.BytesIO + ) + + +def capture_result(*, routing_history=None): + audio = CaptureStore(RATE) + assert audio.append(np.ones(160, dtype=np.float32)) + audio.finish() + track = CapturedTrack(0, ORIGIN, 160, 0, audio) + result = ObsCaptureResult( + (track,), + primary_bus=0, + origin_ns=ORIGIN, + clean_end=True, + reason="OBS capture finished.", + routing_history=routing_history, + ) + return track, result + + +def complete_with_history(*frames, journal=None): + routing_store, journal = history(*frames, journal=journal) + owner = coordinator() + owner.attach_routing_history(routing_store) + track, result = capture_result(routing_history=routing_store) + owner.add_track(track, origin_ns=ORIGIN, sample_rate=RATE, primary_bus=0) + owner.finish_capture(result) + assert owner.wait(2) + return owner, result, routing_store, journal + + +def test_history_transfers_once_and_survives_audio_result_close_for_export(): + frame = routing() + owner, result, routing_store, journal = complete_with_history(frame) + + assert result.closed + assert owner.snapshot().state is ObsTranscriptionState.COMPLETE + assert tuple(owner.iter_routing_observations()) == (frame,) + assert not journal.closed + with pytest.raises(ObsSessionError, match="not transferable"): + result.take_routing_history() + + assert owner.close(2) + assert routing_store.wait_closed(0) + assert journal.closed + + +def test_version1_result_has_no_routing_history_or_fabricated_observations(): + owner = coordinator() + track, result = capture_result() + owner.add_track(track, origin_ns=ORIGIN, sample_rate=RATE, primary_bus=0) + owner.finish_capture(result) + + assert owner.wait(2) + assert owner.snapshot().state is ObsTranscriptionState.COMPLETE + assert tuple(owner.iter_routing_observations()) == () + assert owner.close(2) + + +def test_result_history_must_match_the_borrowed_live_store_before_transfer(): + borrowed, _ = history(routing()) + different, _ = history(routing()) + owner = coordinator() + owner.attach_routing_history(borrowed) + track, result = capture_result(routing_history=different) + owner.add_track(track, origin_ns=ORIGIN, sample_rate=RATE, primary_bus=0) + + with pytest.raises(ObsTranscriptionError, match="changed its routing history"): + owner.finish_capture(result) + assert result.take_routing_history() is different + + result.close() + different.close() + assert different.wait_closed(2) + owner.cancel() + assert not owner.wait(0.05) + borrowed.close() + assert owner.wait(2) + assert owner.close(2) + + +def test_cancel_does_not_close_borrowed_history_but_wait_includes_owner_cleanup(): + borrowed = ObsRoutingStore(journal_factory=io.BytesIO) + owner = coordinator() + owner.attach_routing_history(borrowed) + + owner.cancel() + assert borrowed.append(routing()), "transcription closed a receiver-owned history" + borrowed.finish() + borrowed.wait_ready() + assert not owner.wait(0.05) + + borrowed.close() + assert owner.wait(2) + assert owner.snapshot().state is ObsTranscriptionState.CANCELLED + assert owner.close(2) + + +def test_cancelled_owned_history_waits_for_active_reader_release(): + owner, _, _, journal = complete_with_history(routing(1), routing(2)) + observations = owner.iter_routing_observations() + assert next(observations).revision == 1 + + owner.cancel() + assert not owner.wait(0.05) + assert not journal.closed + observations.close() + + assert owner.wait(2) + assert owner.snapshot().state is ObsTranscriptionState.CANCELLED + assert owner.close(2) + assert journal.closed + + +def test_internal_recognizer_cancellation_discards_transferred_history(): + entered = threading.Event() + release = threading.Event() + + def factory(_config): + def recognize(_audio, _snapshot, *, cancel): + entered.set() + assert release.wait(2) + raise TranscriptionCancelled("private recognizer cancellation detail") + + return recognize + + journal = io.BytesIO() + routing_store, _ = history(routing(), journal=journal) + owner = ObsTranscriptionCoordinator( + Config(), recognizer_factory=factory, journal_factory=io.BytesIO + ) + owner.attach_routing_history(routing_store) + track, result = capture_result(routing_history=routing_store) + owner.add_track(track, origin_ns=ORIGIN, sample_rate=RATE, primary_bus=0) + assert entered.wait(2) + + owner.finish_capture(result) + release.set() + + assert owner.wait(2) + assert routing_store.wait_closed(0) + assert journal.closed + snapshot = owner.snapshot() + assert snapshot.state is ObsTranscriptionState.CANCELLED + assert "private recognizer" not in snapshot.message + assert owner.close(2) + + +def test_borrowed_storage_failure_is_a_cheap_sanitized_failure_signal(): + def failed_factory(): + raise OSError("private source name and path") + + routing_store = ObsRoutingStore(journal_factory=failed_factory) + with pytest.raises(ObsRoutingStoreError, match="storage failed"): + routing_store.wait_ready() + owner = coordinator() + owner.attach_routing_history(routing_store) + + assert owner.failed + snapshot = owner.snapshot() + assert snapshot.state is ObsTranscriptionState.FAILED + assert snapshot.message == "Private OBS routing history storage failed." + assert "private source name" not in snapshot.message + + owner.cancel() + routing_store.close() + assert owner.wait(2) + assert owner.close(2) + + +def test_late_owned_history_cleanup_failure_overrides_complete_and_cancelled(): + class FailedClose(io.BytesIO): + def close(self): + raise OSError("private source cleanup detail") + + journal = FailedClose() + owner, _, _, _ = complete_with_history(routing(), journal=journal) + assert owner.snapshot().state is ObsTranscriptionState.COMPLETE + + owner.cancel() + assert owner.wait(2) + snapshot = owner.snapshot() + assert snapshot.state is ObsTranscriptionState.FAILED + assert snapshot.message == "Private OBS routing history cleanup failed." + assert "private source cleanup detail" not in snapshot.message + assert snapshot.preview == "" + assert owner.failed + assert tuple(owner.iter_routing_observations()) == () + owner.cancel() + assert owner.snapshot() == snapshot + with pytest.raises(ObsTranscriptionError, match="could not be discarded") as caught: + owner.close(2) + assert "private source cleanup detail" not in str(caught.value) + io.BytesIO.close(journal) + + +def test_routing_history_cannot_be_read_before_transcription_finishes(): + routing_store, _ = history(routing()) + owner = coordinator() + owner.attach_routing_history(routing_store) + with pytest.raises(ObsTranscriptionError, match="Wait for OBS transcription"): + tuple(owner.iter_routing_observations()) + + owner.cancel() + routing_store.close() + assert owner.wait(2) + assert owner.close(2) diff --git a/utterleaf/obs_audio_pipe.py b/utterleaf/obs_audio_pipe.py index feece89..a0f81b7 100644 --- a/utterleaf/obs_audio_pipe.py +++ b/utterleaf/obs_audio_pipe.py @@ -91,10 +91,11 @@ class ObsAudioPipe: No control socket is consulted after the independently owned lease is issued. """ - def __init__(self, pipe, peer, session_id: bytes, cancelled: Callable[[], bool]): + def __init__(self, pipe, peer, session_id: bytes, cancelled: Callable[[], bool], + *, protocol_version: int = obs_protocol.VERSION): + self._decoder = obs_protocol.FrameDecoder(version=protocol_version) self._pipe, self._peer = pipe, peer self._session_id, self._cancelled = session_id, cancelled - self._decoder = obs_protocol.FrameDecoder() self._io_lock = threading.Lock() self._closed = threading.Event() self._armed = False @@ -278,7 +279,8 @@ def close(self) -> None: def connect(session_id: bytes, peer: windows_peer_identity.VerifiedProcessLease, - *, cancelled: Callable[[], bool], deadline: float) -> ObsAudioPipe: + *, cancelled: Callable[[], bool], deadline: float, + protocol_version: int = obs_protocol.VERSION) -> ObsAudioPipe: """Consume an independent authenticated-control process lease on every exit. The pipe name is constructed from the exact 16-byte nonsecret session ID. @@ -291,6 +293,9 @@ def connect(session_id: bytes, peer: windows_peer_identity.VerifiedProcessLease, hello = bytearray() try: _check(cancelled, deadline) + if (type(protocol_version) is not int + or protocol_version not in (obs_protocol.VERSION, obs_protocol.PROVENANCE_VERSION)): + raise ObsAudioPipeError("Unsupported OBS audio protocol version") if type(session_id) is not bytes or len(session_id) != 16: raise ObsAudioPipeError("Invalid OBS audio session") peer.verify_pid(peer.pid, cancelled=cancelled, deadline=deadline) @@ -313,7 +318,7 @@ def connect(session_id: bytes, peer: windows_peer_identity.VerifiedProcessLease, or returned_session != session_id or not hmac.compare_digest(mac, expected)): raise ObsAudioPipeError("Invalid OBS audio handshake") _check(cancelled, deadline) - return ObsAudioPipe(pipe, peer, session_id, cancelled) + return ObsAudioPipe(pipe, peer, session_id, cancelled, protocol_version=protocol_version) except BaseException as exc: try: if pipe is not None: diff --git a/utterleaf/obs_control.py b/utterleaf/obs_control.py index 0f64743..3e9c5ed 100644 --- a/utterleaf/obs_control.py +++ b/utterleaf/obs_control.py @@ -33,7 +33,7 @@ _VENDOR_NAME = "Utterleaf" _PLUGIN_PROTOCOL_VERSION = 1 _PLUGIN_COMMAND_VERSION = 1 -_PLUGIN_AUDIO_VERSION = 1 +_PLUGIN_AUDIO_VERSION = 2 _PLUGIN_MAX_BUS_MASK = 63 _OUTPUT_STATES = frozenset({ "OBS_WEBSOCKET_OUTPUT_STARTING", "OBS_WEBSOCKET_OUTPUT_STARTED", diff --git a/utterleaf/obs_controller.py b/utterleaf/obs_controller.py index 4c59eab..c85fecf 100644 --- a/utterleaf/obs_controller.py +++ b/utterleaf/obs_controller.py @@ -13,7 +13,7 @@ from typing import Protocol from utterleaf import obs_audio_pipe, obs_control -from utterleaf.obs_protocol import AudioFrame, EndFrame, StartFrame +from utterleaf.obs_protocol import AudioFrame, EndFrame, RoutingFrame, StartFrame, VERSION, PROVENANCE_VERSION from utterleaf.obs_session import ObsCaptureSession, ObsSessionError @@ -26,6 +26,7 @@ class TranscriptionSink(Protocol): def failed(self) -> bool: ... def add_track(self, track, *, origin_ns: int, sample_rate: int, primary_bus: int) -> None: ... + def attach_routing_history(self, history) -> None: ... def finish_capture(self, result) -> None: ... def cancel(self) -> None: ... def wait(self, timeout: float | None = None) -> bool: ... @@ -40,6 +41,7 @@ class ObsControllerSnapshot: primary_bus: int | None = None buses: tuple[int, ...] = () captured_seconds: float = 0.0 + routing: RoutingFrame | None = field(default=None, repr=False) @dataclass @@ -97,6 +99,9 @@ def __init__(self, transcription: TranscriptionSink, *, self._primary: int | None = None self._buses: tuple[int, ...] = () self._seconds = 0.0 + self._routing: RoutingFrame | None = None + self._audio_version = VERSION + self._cleanup_failed = False self._pipe = None def snapshot(self) -> ObsControllerSnapshot: @@ -104,6 +109,7 @@ def snapshot(self) -> ObsControllerSnapshot: state, message = self._state, self._message degraded, primary, buses, seconds = ( self._degraded, self._primary, self._buses, self._seconds) + routing, cleanup_failed = self._routing, self._cleanup_failed if state == "finalizing": recognition = self._sink.snapshot() terminal = recognition.state.value @@ -112,11 +118,13 @@ def snapshot(self) -> ObsControllerSnapshot: message = recognition.message if state == "cancelling" and self.wait(0): recognition = self._sink.snapshot() - if recognition.state.value == "failed": + if cleanup_failed: + state, message = "error", "Private OBS capture storage could not close cleanly." + elif recognition.state.value == "failed": state, message = "error", recognition.message else: state, message = "cancelled", "OBS transcription discarded." - return ObsControllerSnapshot(state, message, degraded, primary, buses, seconds) + return ObsControllerSnapshot(state, message, degraded, primary, buses, seconds, routing) def connect(self, host: str, port: int, password: str, *, expected_executable: str) -> bool: @@ -171,6 +179,7 @@ def cancel(self) -> None: self._abort.set() with self._condition: self._state, self._message = "cancelling", "Discarding OBS transcription…" + self._routing = None self._condition.notify_all() self._sink.cancel() @@ -232,7 +241,11 @@ def _control_worker(self, request: _ConnectionRequest) -> None: # credential there immediately after the handshake, not on exit. request.password = "" self._check() - control.plugin_status() + compatibility = control.plugin_status() + audio_version = compatibility.audio_version + if type(audio_version) is not int or audio_version not in (VERSION, PROVENANCE_VERSION): + raise _SessionFailure() + self._audio_version = audio_version self._check() refresh_idle = not self._idle_snapshot(control, publish=True) while not self._audio_ended.is_set(): @@ -311,7 +324,8 @@ def _perform_arm(self, control, key: bytearray) -> None: self._check() owned_lease, lease = lease, None pipe = self._pipe_factory(session_id, owned_lease, cancelled=self._audio_cancelled, - deadline=time.monotonic() + obs_control.REQUEST_TIMEOUT) + deadline=time.monotonic() + obs_control.REQUEST_TIMEOUT, + protocol_version=self._audio_version) pipe.arm(additional_mix_mask=self._mask, deadline=time.monotonic() + obs_control.REQUEST_TIMEOUT) self._check() @@ -320,7 +334,8 @@ def _perform_arm(self, control, key: bytearray) -> None: self._pipe = pipe self._armed = True self._state, self._message = "armed", "Waiting for the next OBS stream to start…" - worker = threading.Thread(target=self._audio_worker, args=(session_id, pipe, self._mask), + worker = threading.Thread(target=self._audio_worker, + args=(session_id, pipe, self._mask, self._audio_version), name="utterleaf-obs-audio", daemon=True) worker.start() self._audio_started = True @@ -331,14 +346,15 @@ def _perform_arm(self, control, key: bytearray) -> None: if lease is not None: lease.close() - def _audio_worker(self, session_id: bytes, pipe, mask: int) -> None: + def _audio_worker(self, session_id: bytes, pipe, mask: int, protocol_version: int) -> None: receiver = result = None registered: set[int] = set() origin_ns = sample_rate = 0 + routing_attached = False try: receiver = self._receiver_factory( session_id, buses=tuple(bus for bus in range(6) if mask & (1 << bus)), - stream_active=False) + stream_active=False, protocol_version=protocol_version) while receiver.state in {"armed", "active"}: self._check() frames = pipe.read_frames() @@ -352,6 +368,13 @@ def _audio_worker(self, session_id: bytes, pipe, mask: int) -> None: receiver.notify_stream_started(frame.session_id) origin_ns, sample_rate = frame.origin_ns, frame.sample_rate receiver.accept(frame) + if isinstance(frame, RoutingFrame): + if not routing_attached: + self._sink.attach_routing_history(receiver.routing_history) + routing_attached = True + with self._condition: + self._check() + self._routing = receiver.current_routing if isinstance(frame, AudioFrame): tracks = receiver.live_tracks() for track in tracks: @@ -410,12 +433,31 @@ def _audio_worker(self, session_id: bytes, pipe, mask: int) -> None: self._sink.cancel() self._publish("error", "OBS transcription could not finish. No complete transcript was produced.") finally: + cleanup_failed = False try: if result is not None: - result.close() + try: + result.close() + finally: + result.wait_closed() + if result.routing_history is not None: + cleanup_failed |= result.routing_history.cleanup_failed + except BaseException: + cleanup_failed = True finally: try: if receiver is not None: - receiver.close() + try: + receiver.close() + finally: + receiver.wait_closed() + if receiver.routing_history is not None: + cleanup_failed |= receiver.routing_history.cleanup_failed + except BaseException: + cleanup_failed = True finally: + if cleanup_failed: + with self._condition: + self._cleanup_failed = True + self._publish("error", "Private OBS capture storage could not close cleanly.") self._audio_done.set() diff --git a/utterleaf/obs_mix.py b/utterleaf/obs_mix.py new file mode 100644 index 0000000..e96d9b8 --- /dev/null +++ b/utterleaf/obs_mix.py @@ -0,0 +1,164 @@ +"""Bounded immutable provenance for selected OBS audio mixes. + +The model records OBS-provided assignments only. It does not infer speakers or +inspect PCM, and identifiers and display text stay out of representations. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from enum import Enum + + +MAX_SOURCES = 128 +MAX_LABEL_BYTES = 64 +MAX_SOURCE_NAME_BYTES = 128 + +_MAX_BUS = 5 +_MAX_BUS_MASK = 0x3F +_INVALID = "Invalid OBS mix metadata" + + +class MixMetadataError(ValueError): + """OBS mix provenance violates the bounded domain contract.""" + + +class MixRelationship(str, Enum): + PRIMARY = "primary" + SAME_INPUTS = "same_inputs" + DIFFERENT_INPUTS = "different_inputs" + UNASSIGNED = "unassigned" + + +def _integer(value: object, minimum: int, maximum: int) -> bool: + return type(value) is int and minimum <= value <= maximum + + +def _bounded_text(value: object, maximum_bytes: int) -> bool: + if type(value) is not str or not 0 < len(value) <= maximum_bytes: + return False + has_visible_text = False + for character in value: + codepoint = ord(character) + if ( + codepoint <= 0x1F + or codepoint == 0x061C + or 0x7F <= codepoint <= 0x9F + or codepoint == 0x200B + or 0x200E <= codepoint <= 0x200F + or 0x2028 <= codepoint <= 0x202E + or 0x2066 <= codepoint <= 0x2069 + or codepoint == 0xFEFF + ): + return False + if not character.isspace() and codepoint not in (0x200C, 0x200D): + has_visible_text = True + if not has_visible_text: + return False + try: + return len(value.encode("utf-8")) <= maximum_bytes + except UnicodeEncodeError: + return False + + +@dataclass(frozen=True, slots=True) +class SourceAssignment: + source_id: bytes = field(repr=False) + name: str = field(repr=False) + bus_mask: int + + def __post_init__(self) -> None: + if ( + type(self.source_id) is not bytes + or len(self.source_id) != 16 + or not _bounded_text(self.name, MAX_SOURCE_NAME_BYTES) + or not _integer(self.bus_mask, 1, _MAX_BUS_MASK) + ): + raise MixMetadataError(_INVALID) + + +@dataclass(frozen=True, slots=True) +class BusLabel: + bus: int + label: str = field(repr=False) + + def __post_init__(self) -> None: + if not _integer(self.bus, 0, _MAX_BUS) or not _bounded_text( + self.label, MAX_LABEL_BYTES + ): + raise MixMetadataError(_INVALID) + + +@dataclass(frozen=True, slots=True) +class MixSnapshot: + primary_bus: int + bus_mask: int + sources: tuple[SourceAssignment, ...] + labels: tuple[BusLabel, ...] + + def __post_init__(self) -> None: + if ( + not _integer(self.primary_bus, 0, _MAX_BUS) + or not _integer(self.bus_mask, 1, _MAX_BUS_MASK) + or not self.bus_mask & (1 << self.primary_bus) + or type(self.sources) is not tuple + or not len(self.sources) <= MAX_SOURCES + or any(type(source) is not SourceAssignment for source in self.sources) + or type(self.labels) is not tuple + or len(self.labels) > _MAX_BUS + 1 + or any(type(label) is not BusLabel for label in self.labels) + ): + raise MixMetadataError(_INVALID) + + source_ids = tuple(source.source_id for source in self.sources) + selected_buses = tuple( + bus for bus in range(_MAX_BUS + 1) if self.bus_mask & (1 << bus) + ) + if ( + source_ids != tuple(sorted(source_ids)) + or len(source_ids) != len(set(source_ids)) + or any(source.bus_mask & ~self.bus_mask for source in self.sources) + or tuple(label.bus for label in self.labels) != selected_buses + ): + raise MixMetadataError(_INVALID) + + def inputs_for_bus(self, bus: int) -> tuple[SourceAssignment, ...]: + self._check_selected_bus(bus) + bit = 1 << bus + return tuple(source for source in self.sources if source.bus_mask & bit) + + def relationship(self, bus: int) -> MixRelationship: + self._check_selected_bus(bus) + if bus == self.primary_bus: + return MixRelationship.PRIMARY + primary_ids = { + source.source_id + for source in self.sources + if source.bus_mask & (1 << self.primary_bus) + } + bus_ids = { + source.source_id + for source in self.sources + if source.bus_mask & (1 << bus) + } + if not bus_ids: + return MixRelationship.UNASSIGNED + if primary_ids == bus_ids: + return MixRelationship.SAME_INPUTS + return MixRelationship.DIFFERENT_INPUTS + + def _check_selected_bus(self, bus: int) -> None: + if not _integer(bus, 0, _MAX_BUS) or not self.bus_mask & (1 << bus): + raise MixMetadataError(_INVALID) + + +__all__ = [ + "BusLabel", + "MAX_LABEL_BYTES", + "MAX_SOURCE_NAME_BYTES", + "MAX_SOURCES", + "MixMetadataError", + "MixRelationship", + "MixSnapshot", + "SourceAssignment", +] diff --git a/utterleaf/obs_protocol.py b/utterleaf/obs_protocol.py index b128815..3f8939f 100644 --- a/utterleaf/obs_protocol.py +++ b/utterleaf/obs_protocol.py @@ -13,13 +13,17 @@ import numpy as np +from utterleaf.obs_mix import BusLabel, MixSnapshot, SourceAssignment + MAGIC = b"ULAP" VERSION = 1 +PROVENANCE_VERSION = 2 KIND_START = 1 KIND_AUDIO = 2 KIND_GAP = 3 KIND_END = 4 +KIND_ROUTING = 5 MAX_FRAMES = 8192 CHANNELS = 2 @@ -38,6 +42,13 @@ _GAP = struct.Struct("<16sBQQQ") _END_PREFIX = struct.Struct("<16sBB") _END_SEQUENCE = struct.Struct(" None: previous_bus = bus -Frame: TypeAlias = StartFrame | AudioFrame | GapFrame | EndFrame +@dataclass(frozen=True) +class RoutingFrame: + """An observed assignment snapshot at explicit transport publication positions. + + Neither observation time nor next-sequence positions identify the instant + OBS changed the samples. PCM remains continuous; these are configuration + observations, not speaker attribution or sample-effective routing claims. + A receiver must validate identity, revision order and positions against its + own state. The decoder cannot establish that session-level authority. + """ + + session_id: bytes = field(repr=False) + revision: int + observed_at_ns: int + positions: tuple[tuple[int, int], ...] + snapshot: MixSnapshot = field(repr=False) + + def __post_init__(self) -> None: + if (not _session(self.session_id) + or not _integer(self.revision, 1, MAX_SEQUENCE) + or not _integer(self.observed_at_ns, 0, UINT64_MAX) + or type(self.snapshot) is not MixSnapshot + or type(self.positions) is not tuple + or len(self.positions) != len(self.snapshot.labels)): + raise ProtocolError(_INVALID_FRAME) + for entry, label in zip(self.positions, self.snapshot.labels): + if (type(entry) is not tuple or len(entry) != 2 + or not _bus(entry[0]) or entry[0] != label.bus + or not _integer(entry[1], 0, UINT64_MAX)): + raise ProtocolError(_INVALID_FRAME) + + +Frame: TypeAlias = StartFrame | AudioFrame | GapFrame | EndFrame | RoutingFrame -def encode_frame(frame: Frame) -> bytes: +def encode_frame(frame: Frame, *, version: int = VERSION) -> bytes: """Encode one already-validated frame without transport or authentication.""" + if type(version) is not int or version not in (VERSION, PROVENANCE_VERSION): + raise ProtocolError(_INVALID_FRAME) if type(frame) is StartFrame: kind = KIND_START body = _START.pack( @@ -192,14 +237,29 @@ def encode_frame(frame: Frame) -> bytes: _END_SEQUENCE.pack(bus, UINT64_MAX if sequence is None else sequence) for bus, sequence in frame.last_sequences ) + elif type(frame) is RoutingFrame and version == PROVENANCE_VERSION: + kind = KIND_ROUTING + snapshot = frame.snapshot + parts = [_ROUTING_PREFIX.pack( + frame.session_id, frame.revision, frame.observed_at_ns, + snapshot.primary_bus, snapshot.bus_mask, len(snapshot.sources), + )] + for (bus, sequence), label in zip(frame.positions, snapshot.labels): + encoded = label.label.encode("utf-8") + parts.extend((_ROUTING_BUS.pack(bus, sequence, len(encoded)), encoded)) + for source in snapshot.sources: + encoded = source.name.encode("utf-8") + parts.extend((_ROUTING_SOURCE.pack( + source.source_id, source.bus_mask, len(encoded)), encoded)) + body = b"".join(parts) else: raise ProtocolError(_INVALID_FRAME) if len(body) > MAX_BODY_BYTES: raise ProtocolError(_INVALID_FRAME) - return _HEADER.pack(MAGIC, VERSION, kind, 0, len(body)) + body + return _HEADER.pack(MAGIC, version, kind, 0, len(body)) + body -def _valid_body_length(kind: int, length: int) -> bool: +def _valid_body_length(kind: int, length: int, version: int) -> bool: if length > MAX_BODY_BYTES: return False if kind == KIND_START: @@ -212,9 +272,46 @@ def _valid_body_length(kind: int, length: int) -> bool: if kind == KIND_END: sequence_bytes = length - _END_PREFIX.size return 0 <= sequence_bytes <= 6 * _END_SEQUENCE.size and sequence_bytes % 9 == 0 + if kind == KIND_ROUTING and version == PROVENANCE_VERSION: + return MIN_ROUTING_BODY_BYTES <= length <= MAX_ROUTING_BODY_BYTES return False +def _decode_routing(body: bytes) -> RoutingFrame: + session_id, revision, observed_at_ns, primary, mask, source_count = ( + _ROUTING_PREFIX.unpack_from(body) + ) + if not 1 <= mask <= 0x3F or primary > 5 or not mask & (1 << primary) or source_count > 128: + raise ProtocolError(_INVALID_FRAME) + offset = _ROUTING_PREFIX.size + labels: list[BusLabel] = [] + positions: list[tuple[int, int]] = [] + for selected_bus in range(6): + if not mask & (1 << selected_bus): + continue + bus, sequence, byte_count = _ROUTING_BUS.unpack_from(body, offset) + offset += _ROUTING_BUS.size + if bus != selected_bus or not 1 <= byte_count <= 64 or offset + byte_count > len(body): + raise ProtocolError(_INVALID_FRAME) + label = body[offset:offset + byte_count].decode("utf-8", errors="strict") + offset += byte_count + labels.append(BusLabel(bus, label)) + positions.append((bus, sequence)) + sources: list[SourceAssignment] = [] + for _ in range(source_count): + identity, source_mask, byte_count = _ROUTING_SOURCE.unpack_from(body, offset) + offset += _ROUTING_SOURCE.size + if not 1 <= byte_count <= 128 or offset + byte_count > len(body): + raise ProtocolError(_INVALID_FRAME) + name = body[offset:offset + byte_count].decode("utf-8", errors="strict") + offset += byte_count + sources.append(SourceAssignment(identity, name, source_mask)) + if offset != len(body): + raise ProtocolError(_INVALID_FRAME) + return RoutingFrame(session_id, revision, observed_at_ns, tuple(positions), + MixSnapshot(primary, mask, tuple(sources), tuple(labels))) + + def _decode_body(kind: int, body: bytes) -> Frame: if kind == KIND_START: return StartFrame(*_START.unpack(body)) @@ -238,13 +335,21 @@ def _decode_body(kind: int, body: bytes) -> Frame: except ValueError as exc: raise ProtocolError(_INVALID_FRAME) from exc return EndFrame(session_id, reason, tuple(entries)) + if kind == KIND_ROUTING: + return _decode_routing(body) raise ProtocolError(_INVALID_FRAME) class FrameDecoder: """Incrementally parse bounded packets; any error permanently closes it.""" - def __init__(self) -> None: + def __init__(self, *, version: int = VERSION) -> None: + # Version 2 is explicitly selected during component development. The + # active transport continues to require version 1 until its receiver + # and native runtime both implement mandatory routing observations. + if type(version) is not int or version not in (VERSION, PROVENANCE_VERSION): + raise ProtocolError(_INVALID_STREAM) + self._version = version self._buffer = bytearray() self._closed = False @@ -270,9 +375,9 @@ def feed(self, data: bytes) -> list[Frame]: try: while len(self._buffer) >= HEADER_BYTES: magic, version, kind, reserved, length = _HEADER.unpack_from(self._buffer) - if magic != MAGIC or version != VERSION or reserved != 0: + if magic != MAGIC or version != self._version or reserved != 0: self._fail() - if not _valid_body_length(kind, length): + if not _valid_body_length(kind, length, version): self._fail() packet_bytes = HEADER_BYTES + length if len(self._buffer) < packet_bytes: @@ -285,10 +390,11 @@ def feed(self, data: bytes) -> list[Frame]: self._buffer.clear() self._closed = True raise - except (struct.error, ValueError, OverflowError) as exc: + except (struct.error, ValueError, OverflowError): self._buffer.clear() self._closed = True - raise ProtocolError(_INVALID_STREAM) from exc + # Unicode/metadata exceptions can retain private names or wire bytes. + raise ProtocolError(_INVALID_STREAM) from None return frames def finish(self) -> None: @@ -312,6 +418,7 @@ def finish(self) -> None: "KIND_END", "KIND_GAP", "KIND_START", + "KIND_ROUTING", "MAGIC", "MAX_BODY_BYTES", "MAX_FEED_BYTES", @@ -319,7 +426,10 @@ def finish(self) -> None: "MAX_PACKET_BYTES", "MAX_PCM_BYTES", "MAX_SEQUENCE", + "MAX_ROUTING_BODY_BYTES", "ProtocolError", + "PROVENANCE_VERSION", + "RoutingFrame", "StartFrame", "SUPPORTED_SAMPLE_RATES", "UINT64_MAX", diff --git a/utterleaf/obs_routing_store.py b/utterleaf/obs_routing_store.py new file mode 100644 index 0000000..0a247e3 --- /dev/null +++ b/utterleaf/obs_routing_store.py @@ -0,0 +1,303 @@ +"""Bounded-memory private storage for OBS routing observations.""" + +from __future__ import annotations + +from collections import deque +import tempfile +import threading +from typing import BinaryIO, Callable, Iterator + +from utterleaf.local_filesystem import LocalFilesystemError, require_local_filesystem +from utterleaf.obs_protocol import ( + HEADER_BYTES, + MAX_ROUTING_BODY_BYTES, + PROVENANCE_VERSION, + FrameDecoder, + ProtocolError, + RoutingFrame, + encode_frame, +) +from utterleaf.transcript import TranscriptionCancelled + + +MAX_PENDING_RECORDS = 8 +MAX_ROUTING_RECORD_BYTES = HEADER_BYTES + MAX_ROUTING_BODY_BYTES + +_INVALID = "Invalid OBS routing history" +_STORAGE_FAILED = "Private OBS routing history storage failed." +_QUEUE_FAILED = "Private OBS routing history could not keep up." +_CLEANUP_FAILED = "Private OBS routing history cleanup failed." + +JournalFactory = Callable[[], BinaryIO] + + +class ObsRoutingStoreError(RuntimeError): + """Private routing history cannot continue or be read safely.""" + + +def _default_journal() -> BinaryIO: + try: + directory = require_local_filesystem(tempfile.gettempdir()) + except LocalFilesystemError: + raise ObsRoutingStoreError( + "Private OBS routing history needs a verified local temporary folder." + ) from None + try: + return tempfile.TemporaryFile( + mode="w+b", + prefix="utterleaf-obs-routing-", + buffering=0, + dir=directory, + ) + except OSError: + raise ObsRoutingStoreError( + "Private OBS routing history storage could not be created." + ) from None + + +class ObsRoutingStore: + """Write routing frames asynchronously and retain a disk-backed history.""" + + def __init__(self, *, journal_factory: JournalFactory | None = None) -> None: + if journal_factory is not None and not callable(journal_factory): + raise TypeError("OBS routing history requires a journal factory") + self._journal_factory = journal_factory or _default_journal + self._condition = threading.Condition() + self._file_lock = threading.Lock() + self._pending: deque[bytes] = deque() + self._finishing = False + self._discard = threading.Event() + self._failed = threading.Event() + self._cleanup_failed = threading.Event() + self._ready = threading.Event() + self._closed = threading.Event() + self._error: ObsRoutingStoreError | None = None + self._journal: BinaryIO | None = None + self._written_bytes = 0 + self._worker = threading.Thread( + target=self._run, + name="utterleaf-obs-routing-store", + daemon=True, + ) + try: + self._worker.start() + except BaseException: + raise ObsRoutingStoreError( + "Private OBS routing history storage could not start." + ) from None + + def append(self, frame: RoutingFrame) -> bool: + """Queue one exact frame without waiting or performing file I/O.""" + if type(frame) is not RoutingFrame: + raise TypeError("OBS routing history requires an exact RoutingFrame") + with self._condition: + if self._rejecting_locked(): + return False + try: + record = encode_frame(frame, version=PROVENANCE_VERSION) + except Exception: + raise ObsRoutingStoreError(_INVALID) from None + if len(record) > MAX_ROUTING_RECORD_BYTES: + raise ObsRoutingStoreError(_INVALID) + with self._condition: + if self._rejecting_locked(): + return False + if len(self._pending) >= MAX_PENDING_RECORDS: + self._set_error_locked(_QUEUE_FAILED) + self._finishing = True + self._condition.notify_all() + return False + self._pending.append(record) + self._condition.notify_all() + return True + + def finish(self) -> None: + """Request an asynchronous drain of every accepted record.""" + with self._condition: + if not self._finishing and not self._discard.is_set(): + self._finishing = True + self._condition.notify_all() + + @property + def error(self) -> ObsRoutingStoreError | None: + with self._condition: + return self._error + + @property + def failed(self) -> bool: + return self._failed.is_set() + + @property + def cleanup_failed(self) -> bool: + return self._cleanup_failed.is_set() + + def wait_ready(self, cancelled: Callable[[], bool] = lambda: False) -> None: + """Wait for a successful drain, failing on cancellation or storage loss.""" + while not self._ready.wait(0.05): + if cancelled(): + raise TranscriptionCancelled("OBS routing history cancelled") + if cancelled() or self._discard.is_set(): + raise TranscriptionCancelled("OBS routing history cancelled") + error = self.error + if error is not None: + raise error + + def iter_observations(self) -> Iterator[RoutingFrame]: + """Stream verified routing frames after a successful drain.""" + if not self._ready.is_set(): + raise ObsRoutingStoreError("Wait for OBS routing history before reading it") + self._file_lock.acquire() + try: + with self._condition: + if self._discard.is_set(): + return + if self._error is not None: + raise self._error + journal = self._journal + remaining = self._written_bytes + if journal is None: + raise ProtocolError(_INVALID) + journal.seek(0) + decoder = FrameDecoder(version=PROVENANCE_VERSION) + while remaining: + requested = min(remaining, MAX_ROUTING_RECORD_BYTES) + data = journal.read(requested) + if type(data) is not bytes or not data or len(data) > requested: + raise ProtocolError(_INVALID) + remaining -= len(data) + for frame in decoder.feed(data): + if type(frame) is not RoutingFrame: + raise ProtocolError(_INVALID) + if self._discard.is_set(): + return + yield frame + decoder.finish() + except ObsRoutingStoreError: + raise + except Exception: + self._record_error(_INVALID) + raise ObsRoutingStoreError(_INVALID) from None + finally: + self._file_lock.release() + + def close(self) -> None: + """Set nonblocking discard intent; the writer owns eventual file close.""" + self._discard.set() + with self._condition: + self._pending.clear() + self._condition.notify_all() + + def wait_closed(self, timeout: float | None = None) -> bool: + """Wait until the sole writer has completed its file-close attempt.""" + return self._closed.wait(timeout) + + def _rejecting_locked(self) -> bool: + return ( + self._finishing + or self._discard.is_set() + or self._error is not None + or self._ready.is_set() + ) + + def _set_error_locked(self, message: str) -> None: + if self._error is None: + self._error = ObsRoutingStoreError(message) + self._failed.set() + + def _record_error(self, message: str) -> None: + with self._condition: + self._set_error_locked(message) + self._condition.notify_all() + + def _run(self) -> None: + journal: BinaryIO | None = None + try: + try: + journal = self._journal_factory() + if not all( + callable(getattr(journal, operation, None)) + for operation in ("close", "read", "seek", "truncate", "write") + ): + raise TypeError("Invalid routing history journal") + with self._condition: + self._journal = journal + self._drain(journal) + except BaseException: + self._record_error(_STORAGE_FAILED) + with self._condition: + self._pending.clear() + finally: + self._ready.set() + with self._condition: + while not self._discard.is_set(): + self._condition.wait() + if journal is not None: + try: + with self._file_lock: + journal.close() + except BaseException: + with self._condition: + self._error = ObsRoutingStoreError(_CLEANUP_FAILED) + self._failed.set() + self._cleanup_failed.set() + self._condition.notify_all() + self._closed.set() + + def _drain(self, journal: BinaryIO) -> None: + while True: + with self._condition: + while ( + not self._pending + and not self._finishing + and not self._discard.is_set() + ): + self._condition.wait() + if self._discard.is_set(): + self._pending.clear() + return + if self._pending: + record = self._pending.popleft() + else: + return + try: + self._write_record(journal, record) + except BaseException: + self._record_error(_STORAGE_FAILED) + with self._condition: + self._pending.clear() + return + + def _write_record(self, journal: BinaryIO, record: bytes) -> None: + with self._condition: + start = self._written_bytes + with self._file_lock: + try: + journal.seek(start) + view = memoryview(record) + written = 0 + while written < len(view): + count = journal.write(view[written:]) + if ( + type(count) is not int + or count <= 0 + or count > len(view) - written + ): + raise OSError("OBS routing history write made no progress") + written += count + except BaseException: + try: + journal.seek(start) + journal.truncate(start) + except Exception: + pass + raise + with self._condition: + self._written_bytes = start + len(record) + + +__all__ = [ + "MAX_PENDING_RECORDS", + "MAX_ROUTING_RECORD_BYTES", + "ObsRoutingStore", + "ObsRoutingStoreError", +] diff --git a/utterleaf/obs_session.py b/utterleaf/obs_session.py index d7543c6..5130e27 100644 --- a/utterleaf/obs_session.py +++ b/utterleaf/obs_session.py @@ -18,7 +18,9 @@ import numpy as np from utterleaf.capture_store import CaptureStore -from utterleaf.obs_protocol import AudioFrame, EndFrame, EndReason, GapFrame, StartFrame +from utterleaf.obs_protocol import (AudioFrame, EndFrame, EndReason, GapFrame, + RoutingFrame, StartFrame, VERSION, PROVENANCE_VERSION) +from utterleaf.obs_routing_store import ObsRoutingStore class ObsSessionError(RuntimeError): @@ -43,7 +45,8 @@ class ObsCaptureResult: """ def __init__(self, tracks: tuple[CapturedTrack, ...], *, primary_bus: int | None, - origin_ns: int | None, clean_end: bool, reason: str): + origin_ns: int | None, clean_end: bool, reason: str, + routing_history: ObsRoutingStore | None = None): self.tracks = tracks self.primary_bus = primary_bus self.origin_ns = origin_ns @@ -51,10 +54,14 @@ def __init__(self, tracks: tuple[CapturedTrack, ...], *, primary_bus: int | None self._reason = reason self._ready = False self.closed = False + self.routing_history = routing_history + self._owns_routing_history = routing_history is not None + self._routing_taken = False @property def complete(self) -> bool: return (not self.closed and self._ready and self._clean_end and bool(self.tracks) + and (self.routing_history is None or self.routing_history.error is None) and all(track.received_frames > 0 and track.store.error is None for track in self.tracks)) @@ -62,10 +69,13 @@ def complete(self) -> bool: def empty(self) -> bool: """A verified clean Disarm before audio, distinct from a failed empty capture.""" return (not self.closed and self._ready and self._clean_end and not self.tracks - and self.primary_bus is None and self.origin_ns is None) + and self.primary_bus is None and self.origin_ns is None + and self.routing_history is None) @property def reason(self) -> str: + if self.routing_history is not None and self.routing_history.error is not None: + return "OBS mix history could not be preserved. The capture is incomplete." if any(track.store.error for track in self.tracks): return "OBS audio storage failed. Only the stored portion can be recovered." if self._clean_end and any(track.received_frames == 0 for track in self.tracks): @@ -78,17 +88,40 @@ def wait_ready(self, cancelled: Callable[[], bool] = lambda: False) -> None: try: for track in self.tracks: track.store.wait_ready(cancelled) + if self.routing_history is not None: + self.routing_history.wait_ready(cancelled) except BaseException: self.close() raise self._ready = True + def take_routing_history(self) -> ObsRoutingStore | None: + """Transfer history once; keep borrowed completion/error facts on this result. + + The transcript owner retains it after decoded audio is released. The new + owner must close and wait for it on discard, including failed completion. + """ + if self.closed or self._routing_taken: + raise ObsSessionError("OBS routing history is not transferable") + self._routing_taken = True + self._owns_routing_history = False + return self.routing_history + def close(self) -> None: if self.closed: return self.closed = True - for track in self.tracks: - track.store.close() + try: + for track in self.tracks: + track.store.close() + finally: + if self._owns_routing_history: + self.routing_history.close() + + def wait_closed(self, timeout: float | None = None) -> bool: + """Wait for owned routing cleanup after close, off the UI thread.""" + return (self.routing_history.wait_closed(timeout) + if self._owns_routing_history else True) @dataclass @@ -112,7 +145,10 @@ class ObsCaptureSession: def __init__(self, session_id: bytes, primary_bus: int | None = None, buses: tuple[int, ...] = (), *, - stream_active: bool, store_factory=CaptureStore): + stream_active: bool, store_factory=CaptureStore, + protocol_version: int = VERSION, routing_factory=ObsRoutingStore): + if type(protocol_version) is not int or protocol_version not in (VERSION, PROVENANCE_VERSION): + raise ValueError("Invalid OBS audio protocol version") if type(stream_active) is not bool or stream_active: raise ObsSessionError("Arm OBS transcription while the stream is stopped") if type(session_id) is not bytes or len(session_id) != 16: @@ -128,6 +164,10 @@ def __init__(self, session_id: bytes, primary_bus: int | None = None, self.buses = buses self._mask = sum(1 << bus for bus in buses) self._factory = store_factory + self._routing_factory = routing_factory + self._requires_routing = protocol_version == PROVENANCE_VERSION + self.routing_history: ObsRoutingStore | None = None + self.current_routing: RoutingFrame | None = None self._tracks: dict[int, _Track] = {} self._started = False self._origin_ns: int | None = None @@ -149,8 +189,8 @@ def notify_stream_started(self, session_id: bytes) -> None: self._fail("OBS stream start was stale or repeated") self._started = True - def accept(self, frame: StartFrame | AudioFrame | GapFrame | EndFrame) -> None: - if type(frame) not in (StartFrame, AudioFrame, GapFrame, EndFrame): + def accept(self, frame: StartFrame | AudioFrame | GapFrame | EndFrame | RoutingFrame) -> None: + if type(frame) not in (StartFrame, AudioFrame, GapFrame, EndFrame, RoutingFrame): self._fail("Unexpected OBS audio message") self._check_identity(frame.session_id) if self.state not in {"armed", "active"}: @@ -168,6 +208,13 @@ def accept(self, frame: StartFrame | AudioFrame | GapFrame | EndFrame) -> None: return if self.state != "active": self._fail("OBS audio arrived before the armed stream started") + if isinstance(frame, RoutingFrame): + self._routing(frame) + return + if self._requires_routing and self.current_routing is None: + self._fail("OBS supplied no initial mix observation. The capture is incomplete.") + if self.routing_history is not None and self.routing_history.error is not None: + self._fail("OBS mix history could not be preserved. The capture is incomplete.") if isinstance(frame, AudioFrame): self._audio(frame) elif isinstance(frame, GapFrame): @@ -203,6 +250,30 @@ def _start(self, frame: StartFrame) -> None: self.state = "active" self.reason = "Receiving the armed OBS stream." + def _routing(self, frame: RoutingFrame) -> None: + previous = self.current_routing + expected_revision = 1 if previous is None else previous.revision + 1 + expected_positions = tuple((bus, track.next_sequence) + for bus, track in self._tracks.items()) + if (not self._requires_routing or frame.revision != expected_revision + or (previous is not None and frame.observed_at_ns < previous.observed_at_ns) + or frame.snapshot.primary_bus != self.primary_bus + or frame.snapshot.bus_mask != sum(1 << bus for bus in self.buses) + or frame.positions != expected_positions): + self._fail("OBS mix observation was stale or inconsistent. The capture is incomplete.") + # Observation time can precede buffered PCM and the Start origin. It is + # not an effective sample timestamp. The sequence cut is publication + # order, and metadata does not alter the audio continuity calculation. + try: + if self.routing_history is None: + self.routing_history = self._routing_factory() + accepted = self.routing_history.append(frame) + except Exception: + self._fail("OBS mix history could not start or continue. Check local storage.") + if not accepted: + self._fail("OBS mix history could not keep up. The capture is incomplete.") + self.current_routing = frame + def _audio(self, frame: AudioFrame) -> None: if frame.bus not in self._tracks: self._fail("OBS supplied an audio bus that was not selected") @@ -246,6 +317,8 @@ def _end(self, frame: EndFrame) -> None: else "OBS capture was interrupted. Only the prefix can be recovered.") for track in self._tracks.values(): track.store.finish() + if self.routing_history is not None: + self.routing_history.finish() def _fail(self, reason: str) -> None: # A late callback cannot reopen or mutate an already transferred result. @@ -255,7 +328,9 @@ def _fail(self, reason: str) -> None: self._clean_end = False for track in self._tracks.values(): track.store.finish() - raise ObsSessionError(reason) + if self.routing_history is not None: + self.routing_history.finish() + raise ObsSessionError(reason) from None def connection_lost(self) -> None: """Signal loss of the authenticated audio channel, not control-only lag.""" @@ -268,8 +343,10 @@ def take_result(self) -> ObsCaptureResult: item.next_sequence - 1 if item.next_sequence else None, item.store) for bus, item in self._tracks.items()) result = ObsCaptureResult(tracks, primary_bus=self.primary_bus, origin_ns=self._origin_ns, - clean_end=self._clean_end, reason=self.reason) + clean_end=self._clean_end, reason=self.reason, + routing_history=self.routing_history) self._tracks.clear() + self.routing_history = None self._taken = True return result @@ -288,12 +365,22 @@ def live_tracks(self) -> tuple[CapturedTrack, ...]: def cancel(self) -> None: """Discard this receiver's audio; a transferred result has its own owner.""" - for track in self._tracks.values(): - track.store.close() - self._tracks.clear() - self._session_id = b"" - self._clean_end = False - self.state = "cancelled" - self.reason = "OBS capture cancelled." + try: + for track in self._tracks.values(): + track.store.close() + finally: + self._tracks.clear() + if self.routing_history is not None: + self.routing_history.close() + self.current_routing = None + self._session_id = b"" + self._clean_end = False + self.state = "cancelled" + self.reason = "OBS capture cancelled." + + def wait_closed(self, timeout: float | None = None) -> bool: + """Wait for untransferred routing cleanup after cancel/close.""" + return (self.routing_history.wait_closed(timeout) + if self.routing_history is not None else True) close = cancel diff --git a/utterleaf/obs_session_ui.py b/utterleaf/obs_session_ui.py index 0233833..4d2fc51 100644 --- a/utterleaf/obs_session_ui.py +++ b/utterleaf/obs_session_ui.py @@ -13,6 +13,8 @@ from utterleaf import theme from utterleaf.host import ui_font +from utterleaf.obs_mix import MixRelationship +from utterleaf.obs_protocol import RoutingFrame POLL_MS = 100 @@ -143,6 +145,11 @@ def _styles(self) -> None: padding=(9, 4)) style.configure("Obs.Error.TLabel", background=theme.SURFACE_CONTAINER, foreground=theme.ERROR, font=(family, 9)) + style.configure("Obs.TNotebook", background=theme.SURFACE_CONTAINER, borderwidth=0) + style.configure("Obs.TNotebook.Tab", background=theme.SURFACE_CONTAINER, + foreground=theme.ON_VARIANT, font=(family, 10), padding=(12, 6)) + style.map("Obs.TNotebook.Tab", background=[("selected", theme.SURFACE_LOW)], + foreground=[("selected", theme.PRIMARY), ("disabled", theme.OUTLINE)]) style.configure("Obs.Danger.TButton", background=theme.PRIMARY_CONTAINER, foreground=theme.ON_PRIMARY_CONTAINER, padding=(16, 8)) style.map("Obs.Danger.TButton", background=[("active", theme.OUTLINE_VARIANT)], @@ -289,15 +296,15 @@ def _build_transcript(self, page) -> None: card.grid(row=6, column=0, sticky="nsew", pady=(0, 11)) card.columnconfigure(0, weight=1) card.rowconfigure(2, weight=1) - ttk.Label(card, text="Transcript preview", style="Obs.CardTitle.TLabel").grid( - row=0, column=0, sticky="w" - ) self.progress_text = tk.StringVar(self.root, "Waiting for selected OBS audio.") ttk.Label(card, textvariable=self.progress_text, style="Obs.CardHint.TLabel").grid( - row=1, column=0, sticky="ew", pady=(3, 8) + row=0, column=0, sticky="ew", pady=(0, 8) ) - text_frame = tk.Frame(card, bg=theme.OUTLINE_VARIANT, padx=1, pady=1) - text_frame.grid(row=2, column=0, sticky="nsew") + self.preview_tabs = ttk.Notebook(card, style="Obs.TNotebook") + self.preview_tabs.grid(row=2, column=0, sticky="nsew") + self.preview_tabs.enable_traversal() + text_frame = tk.Frame(self.preview_tabs, bg=theme.OUTLINE_VARIANT, padx=1, pady=1) + self.preview_tabs.add(text_frame, text="Transcript", underline=0) text_frame.columnconfigure(0, weight=1) text_frame.rowconfigure(0, weight=1) self.preview = tk.Text( @@ -311,6 +318,24 @@ def _build_transcript(self, page) -> None: scroll.grid(row=0, column=1, sticky="ns") self.preview.configure(yscrollcommand=scroll.set, state="disabled") + mixes_frame = tk.Frame(self.preview_tabs, bg=theme.OUTLINE_VARIANT, padx=1, pady=1) + mixes_frame.columnconfigure(0, weight=1) + mixes_frame.rowconfigure(0, weight=1) + self.preview_tabs.add(mixes_frame, text="Mix details", underline=0, state="disabled") + self.mix_details = tk.Text( + mixes_frame, wrap="word", undo=False, height=8, borderwidth=0, + padx=12, pady=10, bg=theme.SURFACE_LOW, fg=theme.ON_SURFACE, + selectbackground=theme.PRIMARY_CONTAINER, selectforeground=theme.ON_SURFACE, + exportselection=False, font=(ui_font(), 10), + ) + self.mix_details.grid(row=0, column=0, sticky="nsew") + mix_scroll = ttk.Scrollbar(mixes_frame, orient="vertical", command=self.mix_details.yview) + mix_scroll.grid(row=0, column=1, sticky="ns") + self.mix_details.configure(yscrollcommand=mix_scroll.set, state="disabled") + self.mix_details.tag_configure("heading", foreground=theme.PRIMARY, + font=(ui_font(), 10, "bold")) + self._last_routing = None + export_row = ttk.Frame(card, style="Card.TFrame") self.export_row = export_row export_row.grid(row=3, column=0, sticky="ew", pady=(10, 0)) @@ -586,6 +611,8 @@ def refresh(self) -> None: self.capture_meta.set( f"{primary_text} · {selected_text or 'complete stream mix'} · {self._duration(seconds)}" ) + routing = getattr(session, "routing", None) + self._update_mix_details(routing if state not in {"cancelling", "cancelled"} else None) preview = preview_value if isinstance(preview_value, str) else "" self.preview.configure(state="normal") @@ -622,6 +649,47 @@ def refresh(self) -> None: self._update_export_row() self._update_compact_sections() + def _update_mix_details(self, routing) -> None: + """Render only the latest accepted observation; never rewrite transcript text.""" + if type(routing) is not RoutingFrame: + routing = None + if routing is self._last_routing: + return + self._last_routing = routing + self.mix_details.configure(state="normal") + self.mix_details.delete("1.0", "end") + if routing is None: + self.preview_tabs.select(0) + self.preview_tabs.tab(1, state="disabled") + else: + self.preview_tabs.tab(1, state="normal") + self.mix_details.insert("end", f"Observed assignments · update {routing.revision}\n", "heading") + self.mix_details.insert( + "end", "Configured inputs · change timing is approximate\n\n", + ) + descriptions = { + MixRelationship.PRIMARY: "Complete streaming mix", + MixRelationship.SAME_INPUTS: "Same assigned inputs as the primary mix; audio may differ", + MixRelationship.DIFFERENT_INPUTS: "Different assigned inputs; may contain several speakers", + MixRelationship.UNASSIGNED: "No observed assigned inputs", + } + for label in routing.snapshot.labels: + title = f"Mix {label.bus + 1}" + if label.label != title: + title += f" · {label.label}" + self.mix_details.insert("end", title + "\n", "heading") + self.mix_details.insert("end", descriptions[routing.snapshot.relationship(label.bus)] + "\n") + for source in routing.snapshot.inputs_for_bus(label.bus): + self.mix_details.insert("end", " • " + source.name + "\n") + self.mix_details.insert("end", "\n") + self.mix_details.insert( + "end", "These are configured inputs, not identified speakers or proof of audible sound. " + "Changes may reach the audio before or after this observation. Earlier observations " + "remain in this session's private history.", + ) + self.mix_details.yview_moveto(0) + self.mix_details.configure(state="disabled") + def _update_export_row(self) -> None: if not hasattr(self, "export_row"): return @@ -704,6 +772,7 @@ def close(self) -> None: if self.closed: return self.closed = True + self._last_routing = None self.password_var.set("") if self._poll_id is not None: try: @@ -722,6 +791,7 @@ def _destroyed(self, event) -> None: if event.widget is not self.root or self.closed: return self.closed = True + self._last_routing = None self.password_var.set("") self._poll_id = None try: diff --git a/utterleaf/obs_transcription.py b/utterleaf/obs_transcription.py index 9641d7a..3089d4f 100644 --- a/utterleaf/obs_transcription.py +++ b/utterleaf/obs_transcription.py @@ -23,6 +23,8 @@ from utterleaf.capture_store import CaptureReadState from utterleaf.config import Config from utterleaf.local_filesystem import LocalFilesystemError, require_local_filesystem +from utterleaf.obs_protocol import RoutingFrame +from utterleaf.obs_routing_store import ObsRoutingStore, ObsRoutingStoreError from utterleaf.obs_session import CapturedTrack, ObsCaptureResult from utterleaf.transcript import Transcript, TranscriptionCancelled @@ -141,6 +143,10 @@ def __init__(self, config: Config, *, recognizer_factory: RecognizerFactory | No self._cleanup_done = threading.Event() self._tracks: dict[int, _TrackCursor] = {} self._result: ObsCaptureResult | None = None + self._borrowed_routing_history: ObsRoutingStore | None = None + self._routing_history: ObsRoutingStore | None = None + self._routing_error: ObsRoutingStoreError | None = None + self._routing_cleanup_error: ObsRoutingStoreError | None = None self._origin_ns: int | None = None self._sample_rate: float | None = None self._primary_bus: int | None = None @@ -194,6 +200,20 @@ def add_track(self, track: CapturedTrack, *, origin_ns: int, sample_rate: int, self._message = "Transcribing OBS audio locally." self._condition.notify_all() + def attach_routing_history(self, history: ObsRoutingStore) -> None: + """Borrow live failure state until the final capture transfers ownership.""" + if type(history) is not ObsRoutingStore: + raise TypeError("OBS transcription requires an exact routing history") + with self._condition: + if self._cancel.is_set() or self._closed or self._result is not None: + raise ObsTranscriptionError("OBS transcription cannot attach routing history") + if self._borrowed_routing_history is not None: + if self._borrowed_routing_history is history: + return + raise ObsTranscriptionError("OBS transcription already has routing history") + self._borrowed_routing_history = history + self._condition.notify_all() + def finish_capture(self, result: ObsCaptureResult) -> None: """Accept ownership of the final capture after validating live registrations.""" if not isinstance(result, ObsCaptureResult): @@ -202,6 +222,14 @@ def finish_capture(self, result: ObsCaptureResult) -> None: if self._cancel.is_set() or self._closed or self._result is not None: raise ObsTranscriptionError("OBS transcription cannot accept this result") self._validate_result_locked(result) + try: + history = result.take_routing_history() + except Exception: + raise ObsTranscriptionError( + "OBS capture routing history could not be transferred" + ) from None + self._routing_history = history + self._borrowed_routing_history = None self._result = result self._track_count = len(result.tracks) if not self._fatal: @@ -210,15 +238,18 @@ def finish_capture(self, result: ObsCaptureResult) -> None: self._condition.notify_all() def cancel(self) -> None: - """Set nonblocking stop/discard intent; borrowed stores stay with their owner.""" + """Set discard intent; borrowed audio and routing stay with their owner.""" with self._condition: self._cancel.set() self._discard_requested.set() self._preview = "" - if self._cleanup_error is None: + history = self._routing_history + if self._cleanup_error is None and self._routing_error is None: self._state = ObsTranscriptionState.CANCELLED self._message = "OBS transcription cancelled." self._condition.notify_all() + if history is not None: + history.close() self._request_cleanup() def wait(self, timeout: float | None = None) -> bool: @@ -228,14 +259,28 @@ def wait(self, timeout: float | None = None) -> bool: if not self._discard_requested.is_set(): return True remaining = None if deadline is None else max(0.0, deadline - time.monotonic()) - return self._cleanup_done.wait(remaining) + if not self._cleanup_done.wait(remaining): + return False + with self._condition: + history = self._routing_history or self._borrowed_routing_history + if history is not None: + remaining = None if deadline is None else max(0.0, deadline - time.monotonic()) + if not history.wait_closed(remaining): + return False + self._record_routing_failure(history, cleanup=history.cleanup_failed) + return True @property def failed(self) -> bool: """Cheap failure signal for control and pipe cancellation predicates.""" - return self._failed.is_set() + if self._failed.is_set(): + return True + with self._condition: + history = self._routing_history or self._borrowed_routing_history + return history is not None and history.failed def snapshot(self) -> ObsTranscriptionSnapshot: + self._refresh_routing_failure() with self._condition: return ObsTranscriptionSnapshot( state=self._state, @@ -246,6 +291,24 @@ def snapshot(self) -> ObsTranscriptionSnapshot: incomplete=self._incomplete, ) + def iter_routing_observations(self) -> Iterator[RoutingFrame]: + """Stream the owned routing history after recognition completes.""" + if not self._done.is_set(): + raise ObsTranscriptionError( + "Wait for OBS transcription before reading routing history" + ) + if self._discard_requested.is_set(): + return + with self._condition: + history = self._routing_history + if history is None: + return + try: + yield from history.iter_observations() + except ObsRoutingStoreError: + self._record_routing_failure(history) + raise ObsTranscriptionError("OBS routing history could not be read") from None + def iter_segments(self) -> Iterator[ObsTranscriptSegment]: """Stream the private journal after recognition has stopped.""" if not self._done.is_set(): @@ -299,10 +362,15 @@ def close(self, timeout: float | None = None) -> bool: with self._condition: self._closed = True cleanup_error = self._cleanup_error + routing_cleanup_error = self._routing_cleanup_error if cleanup_error is not None: raise ObsTranscriptionError( "Private OBS transcript storage could not be deleted" ) from cleanup_error + if routing_cleanup_error is not None: + raise ObsTranscriptionError( + "Private OBS routing history could not be discarded" + ) from None return True @staticmethod @@ -346,6 +414,8 @@ def _validate_result_locked(self, result: ObsCaptureResult) -> None: raise ObsTranscriptionError("OBS capture result changed its timeline") if not registered and result.tracks and any(track.received_frames > 0 for track in result.tracks): raise ObsTranscriptionError("OBS capture result skipped live track registration") + if result.routing_history is not self._borrowed_routing_history: + raise ObsTranscriptionError("OBS capture result changed its routing history") def _run(self) -> None: try: @@ -387,6 +457,10 @@ def _run(self) -> None: self._finish_after_failure() finally: if self._discard_requested.is_set(): + with self._condition: + history = self._routing_history + if history is not None: + history.close() self._request_cleanup() def _poll_round(self, cursors: tuple[_TrackCursor, ...]) -> bool: @@ -546,6 +620,27 @@ def _record_cleanup_failure(self, exc: BaseException) -> None: self._message = "Private OBS transcript cleanup failed." self._condition.notify_all() + def _refresh_routing_failure(self) -> None: + with self._condition: + history = self._routing_history or self._borrowed_routing_history + if history is not None and history.error is not None: + self._record_routing_failure(history) + + def _record_routing_failure(self, history: ObsRoutingStore, *, cleanup: bool = False) -> None: + error = history.error + if error is None: + return + with self._condition: + if cleanup or self._routing_error is None: + self._routing_error = error + if cleanup: + self._routing_cleanup_error = error + self._failed.set() + self._preview = "" + self._state = ObsTranscriptionState.FAILED + self._message = str(self._routing_error) + self._condition.notify_all() + def _write_all(self, data: bytes) -> None: view = memoryview(data) written = 0 @@ -624,7 +719,7 @@ def _finish_cancelled_locked(self) -> None: result = self._result if result is not None: result.close() - if self._cleanup_error is None: + if self._cleanup_error is None and self._routing_error is None: self._state = ObsTranscriptionState.CANCELLED self._message = "OBS transcription cancelled." self._done.set()