Skip to content

feat(audio): loopback feeder mode — all app audio under exclusive/ASIO #865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
byrongamatos merged 2 commits into from
Jul 11, 2026
Merged

feat(audio): loopback feeder mode — all app audio under exclusive/ASIO #865

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3d34469
feat(audio): loopback feeder mode — all app audio under exclusive/ASIO
OmikronApex Jul 11, 2026
8f53ea1
fix(audio): close loopback capture context on teardown (release tap w…
byrongamatos Jul 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 120 additions & 10 deletions static/app.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1498,7 +1498,7 @@
// Normalize any stale 'Combo' tokens left from legacy-mode sessions.
if (_getArrangementNamingMode() === 'smart') {
filters.arrHas = _toSmartArrs(filters.arrHas);
filters.arrLacks = _toSmartArrs(filters.arrLacks);

Check warning on line 1501 in static/app.js

View workflow job for this annotation

GitHub Actions / ci / lint 

File has too many lines (12275). Maximum allowed is 1500

Check warning on line 1501 in static/app.js

View workflow job for this annotation

GitHub Actions / ci / lint 

File has too many lines (12326). Maximum allowed is 1500
}
return filters;
} catch {
Expand Down Expand Up @@ -5375,15 +5375,94 @@
if (_elCtx) return;
const el = document.getElementById('audio');
if (!el) throw new Error('no core audio element');
_elCtx = new AudioContext();
_elSource = _elCtx.createMediaElementSource(el);
_elSource.connect(_elCtx.destination);
_elTap = _makeTap(_elCtx);
await _elTap.attach(_elSource);
// Assign the module state ONLY after the whole chain succeeded.
// createMediaElementSource throws InvalidStateError when another
// consumer (highway_3d's analyser tap) already owns the element's
// one-shot source — assigning _elCtx before that throw poisoned every
// later tick into `_elTap.active` TypeErrors (tester log 2026-07-11)
// while the song kept playing on the default device.
const ctx = new AudioContext();
let source, tap;
try {
source = ctx.createMediaElementSource(el);
source.connect(ctx.destination);
tap = _makeTap(ctx);
await tap.attach(source);
} catch (e) {
try { await ctx.close(); } catch (_) { /* already closed */ }
throw e;
}
_elCtx = ctx; _elSource = source; _elTap = tap;
}

// ── Whole-app loopback capture ───────────────────────────────────────────
// Preferred mode: one getDisplayMedia frame-audio capture covers EVERY
// sound the app makes (song, previews, UI) — no per-surface taps, so
// plugin-private AudioContexts (song-preview, future plugins) survive
// exclusive/ASIO output too. The desktop main process answers the request
// with this window's own frame (frame-scoped — no other apps' audio).
// Local playback is silenced via the suppressLocalAudioPlayback track
// constraint, with a page-mute IPC fallback (capture taps frame audio
// before the output mute, so a muted page still feeds the stream).
let _lbStream = null, _lbCtx = null, _lbTap = null, _lbPageMuted = false;
let _loopbackUnavailable = false; // sticky: probe once, then fall back
async function _engageLoopback() {
const stream = await navigator.mediaDevices.getDisplayMedia({
video: true,
audio: { suppressLocalAudioPlayback: true },
});
for (const t of stream.getVideoTracks()) t.stop(); // required, unused
const track = stream.getAudioTracks()[0];
if (!track) {
for (const t of stream.getTracks()) t.stop();
throw new Error('no loopback audio track');
}
try {
// Fresh context per session (not reused) so teardown's close()
// fully releases the tap worklet node — see _teardownLoopback.
_lbCtx = new AudioContext();
if (_lbCtx.state !== 'running') await _lbCtx.resume().catch(() => {});
const source = _lbCtx.createMediaStreamSource(stream);
const tap = _makeTap(_lbCtx);
await tap.attach(source);
const suppressed = track.getSettings?.().suppressLocalAudioPlayback === true;
if (!suppressed && typeof api.setPageMuted === 'function') {
_lbPageMuted = (await api.setPageMuted(true)) === true;
}
if (window._asioDiagEnabled?.()) {
console.log('[asio-diag] loopback: suppressed=', suppressed,
'pageMuted=', _lbPageMuted, 'rate=', _lbCtx.sampleRate);
}
await api.setRendererBus(true, 1.0);
tap.active = true;
_lbStream = stream; _lbTap = tap;
_mode = 'loopback';
console.log('[renderer-bus] engaged: app loopback → engine bus');
} catch (e) {
for (const t of stream.getTracks()) t.stop();
throw e;
}
}
async function _teardownLoopback() {
if (_lbTap) _lbTap.active = false;
if (_lbStream) for (const t of _lbStream.getTracks()) t.stop();
_lbStream = null; _lbTap = null;
// Close the capture context so its tap worklet node is released. The
// context is per-session (not reused): without this, each exclusive⇄
// shared switch orphaned a live worklet on a long-lived context.
if (_lbCtx) {
try { await _lbCtx.close(); } catch (_) { /* already closed */ }
_lbCtx = null;
}
if (_lbPageMuted && typeof api.setPageMuted === 'function') {
try { await api.setPageMuted(false); } catch (_) { /* engine gone */ }
}
_lbPageMuted = false;
}

// ── Engagement state machine ─────────────────────────────────────────────
// 'off' | 'element' | 'stems'
// 'off' | 'loopback' | 'element' | 'stems' (element/stems = fallback when
// loopback capture is unavailable: old desktop main, denied capture)
let _mode = 'off';
let _stemsGraph = null; // { context, masterNode } snapshot while engaged
let _stemsTap = null;
Expand All @@ -5408,7 +5487,9 @@
const prev = _mode;
_mode = 'off';
try { await api.setRendererBus(false, 0); } catch (_) { /* engine gone */ }
if (prev === 'element' && _elCtx) {
if (prev === 'loopback') {
await _teardownLoopback();
} else if (prev === 'element' && _elCtx) {
_elTap.active = false;
await _setSink(_elCtx, false).catch(() => {});
} else if (prev === 'stems' && _stemsGraph) {
Expand Down Expand Up @@ -5467,9 +5548,20 @@

let want = 'off';
if (running && exclusive) {
if (stems) want = 'stems';
// Loopback covers ALL app audio (song, previews, UI), so it
// engages for the whole exclusive session — not just while a
// song is loaded. Per-surface modes remain as fallback when
// loopback capture is unavailable (old desktop main without
// the display-media handler, capture denied).
if (!_loopbackUnavailable) want = 'loopback';
else if (stems) want = 'stems';
else if (elementSong) want = 'element';
}
// Song audio riding the native transport must not ALSO ride the
// loopback (double-carry into the same engine output). The native
// transport plays from the engine, not the page, so page loopback
// never hears it — no conflict; loopback stays engaged for
// previews/UI while the transport owns the song.

// [asio-diag] full decision vector, change-gated (500ms poll —
// steady state must not flood the buffer). This is the feeder-side
Expand All @@ -5481,6 +5573,7 @@
+ ' stems=' + !!stems + ' songAudio=' + !!songAudio
+ ' juceMode=' + !!window._juceMode
+ ' elementSong=' + elementSong
+ ' loopbackUnavailable=' + _loopbackUnavailable
+ ' want=' + want + ' mode=' + _mode;
if (d !== window._lastRendererBusDecision) {
window._lastRendererBusDecision = d;
Expand All @@ -5492,12 +5585,29 @@
const stemsGraphChanged = _mode === 'stems' && stems !== _stemsGraph;
if (want !== _mode || stemsGraphChanged) {
await _disengage();
if (want === 'stems') await _engageStems(stems);
else if (want === 'element') await _engageElement();
try {
if (want === 'loopback') await _engageLoopback();
else if (want === 'stems') await _engageStems(stems);
else if (want === 'element') await _engageElement();
} catch (e) {
if (want === 'loopback') {
// Capture unavailable (no handler in an old desktop
// main, permission denied) — remember and fall back to
// the per-surface modes on the next tick.
_loopbackUnavailable = true;
console.warn('[renderer-bus] loopback capture unavailable — falling back to surface taps:', e);
}
throw e;
}
}
} catch (e) {
console.warn('[renderer-bus] reevaluate failed (will retry):', e);
_mode = 'off';
// A partial engage may have left the bus enabled with no producer
// and the page muted — undo both so a failed tick can't strand
// audio in silence until the next successful engage.
try { await api.setRendererBus(false, 0); } catch (_) { /* engine gone */ }
await _teardownLoopback().catch(() => {});
Comment on lines 5603 to +5610

@coderabbitai coderabbitai Bot Jul 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect surface-mode engage functions for sink flips that precede possible throws
fd -t f 'app.js' static | xargs -I{} ast-grep outline {} --items all --match '_engage(Stems|Element)|_setSink' 2>/dev/null
rg -nP -C6 'async function _engage(Stems|Element)\s*\(' static/app.js

Repository: got-feedBack/feedBack

Length of output: 1301

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Read the engage/disengage and failure-handling sections with line numbers.
sed -n '5465,5535p' static/app.js | cat -n
printf '\n-----\n'
sed -n '5580,5625p' static/app.js | cat -n

printf '\n-----\n'
# Inspect sink helper and any self-cleanup around element capture.
rg -n -C4 '_setSink|_ensureElementCapture|_teardownLoopback|_disengage' static/app.js

Repository: got-feedBack/feedBack

Length of output: 11077

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the exact failure path and the surrounding state transitions.
sed -n '5478,5528p' static/app.js | cat -n
printf '\n-----\n'
sed -n '5590,5615p' static/app.js | cat -n

Repository: got-feedBack/feedBack

Length of output: 3947

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
p = Path('static/app.js')
text = p.read_text()
for needle in ['async function _disengage', 'async function _engageStems', 'async function _engageElement', 'async function _ensureElementCapture', '_mode = \'off\'', '_teardownLoopback']:
    idx = text.find(needle)
    if idx != -1:
        start = max(0, idx - 300)
        end = min(len(text), idx + 1200)
        print(f"\n=== {needle} ===\n")
        print(text[start:end])
PY

Repository: got-feedBack/feedBack

Length of output: 9378

Restore the sink on failed surface-mode engage
_engageStems() and _engageElement() flip their context to the null sink before later awaits can throw. If that happens, the outer catch only sets _mode = 'off' and tears down loopback, so the surface context can stay stranded on the null sink until a later successful engage. Clean up the surface-mode sink in the failure path, or make the engage helpers self-revert on error.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@static/app.js` around lines 5603 - 5610, Update the failure handling around
the surface-mode engage flow, including _engageStems() and _engageElement(), so
any error after switching to the null sink restores the original surface sink
before leaving the outer catch. Ensure failed engages cannot leave the context
muted, while preserving the existing _mode reset, renderer-bus cleanup, and
loopback teardown behavior.

} finally {
_busy = false;
}
Expand Down
Loading
Loading