Repository navigation
Conversation
added 30 commits
October 5, 2026 20:19
Handle the WebView media permission requests in the Tauri shell, and add the macOS Info.plist usage strings and entitlements that recording needs.
Add the takes API: chunked upload of audio and camera parts, recording tracks that own takes, alignment from the capture sidecar, and takes as lanes in the mix export. Exports now pass a limiter at -1 dBFS so summed lanes cannot clip. A take with a camera recording can be rendered to MP4 over the backing mix.
Add the Record button and recording panel, recording tracks with a voice or instrument input each, take folders with one lane per take, a live waveform while recording, and a Recording tab in Settings for the input, camera and latency test. Stems that were not extracted are hidden from the mixer. Includes the e2e and node tests for these flows.
Add the camera to the record flow, upload the camera recording with the take, and a "Make video" action that renders the take over the backing mix and downloads the MP4. Includes the camera delay nudge, the video strings in every language, and the e2e and node tests.
The desktop shell grants the mic without saving the grant, so each page load started with nameless devices and a new track's picker read No input found until a track was armed. In the desktop app only, open the mic once per page to read the names when a track picker is shown or a track is added.
A Playwright spec that attaches to a running desktop app over CDP and is skipped unless STEMDECK_DESKTOP_CDP is set, so CI never runs it. Also more origin rule cases and the verified WebView2 behaviour in mediaperm.rs.
WAV and FLAC keep the -1 dBFS sample-peak limiter, bit-identical below it. MP3, OGG and both MP4 renders now limit at -1.5 dBTP with 4x oversampling, because the encoder adds peak of its own: measured on real songs, the AAC in video.mp4 still decoded at +0.4 dBFS with clipped samples. AAC is written without intensity stereo, which sent single blocks up to 2.7 dB over the encoder's input. Render cache version 4.
The video block now counts the camera parts sent. When the server holds fewer, the end of the recording is missing, so the take is kept without its video instead of with a short one. After one part fails for good the recorder stops sending the rest: a long take queued hundreds of parts that were each refused, and Stop waited minutes for them.
The camera file starts with the first frame captured after MediaRecorder.start(), but its time on the take's clock came from the recorder's start event, which fires 11 to 61 ms after that frame depending on the codec and the main thread's load. The recorder now reads the capture times of the preview's frames and uses the first one after start(), moved onto the audio stamp clock (perfMsAtFrame0 runs a measured 30 ms behind the capture of the sound). Measured against Chromium's own A/V recorder with fake devices: placement error -18 to +32 ms before (21 takes), -4.5 to +2.4 ms after (9 takes, MP4, VP9 and VP8). Engines without capture times keep the start event, and the sidecar says which.
A 3 s take made a video of the whole song. The video now covers the take by default, from its count-in to where it stopped, with the backing playing throughout; "Whole song" stays a choice next to Make video (remembered per browser, translated). The server takes span "take" or "song", validated, "take" by default. When the camera started before the video's start, the render cut the front and restarted the timestamps at 0, which moved the whole camera up to one frame early (25 ms measured on a take). Frames are now shifted first and cut after, so each lands on the nearest frame of the 30 fps grid. Every lane is cut at the same sample, so the take keeps its place. A running render can be stopped from the take's row.
Chromium's file-based fake camera and microphone play a flash and a burst; the take goes through the real recorder, upload and render, and the flash-to-burst distance in the MP4 must match Chromium's own A/V recorder over the same tracks. Also checks the offset's sign and size and that the take's sound is in the video to the sample. Every spec that opens fake devices now mutes the browser.
The card read a mono 22 kHz buffer cut at 180 s. It now uses ffmpeg ebur128 on the full stereo file at its own rate, with the true peak. Songs analysed before are measured again once, when first opened, tracked by a loudness_version field.
Jobs separated before peaks.json existed, or made by hand, opened on the streaming engine and swapped to the full decode once the empty peaks came back. The first /peaks request now builds the file (one streamed pass per stem, about 1 s for a three minute song), and a first take saved on such a job brings the stems into peaks.json instead of writing the take alone.
A job without peaks started the chunked engine and destroyed it moments later for the full decode. A take started in that moment recorded into the closed context. The engine is now picked once the peaks answer is known; the peaks fetch begins with the job fetch, so this rarely waits.
Record waits for the song's engine to be ready and binds to it. A take whose engine goes away mid-take stops there and keeps what it captured; Stop no longer waits forever on a closed context. A take that never read the song's clock is judged by the audio it holds: over half a second it is kept, placed where the playhead stood, and the panel says so in every language. Before, it was deleted as too short however long it was.
The chunked engine kept every source node it scheduled until pause, seek or destroy, and each node held its five second buffer: memory grew 2.1 MB/s on a six stem song and was freed only at Stop. A node now removes itself when it ends. Measured with CDP on two real songs over three minutes: 2.09 MB/s before, flat after (48 to 61 MB).
Native BiquadFilterNodes in an effects slot between each lane's gain and its analyser, and a master slot before the destination, shared by both engines. Smoothed parameters, crossfaded structure changes, bypass that is bit-exact once its fade has run. EQ saved per song in the mix store, applied when a song opens, and sent to the export as fx=.
fx_filters computes the Web Audio biquad coefficients in Python and hands them to ffmpeg's biquad filter in double precision. The mixdown, video.mp4 and Make video renders take a strictly validated fx= value: lane EQ first in each lane, master EQ before the export limiter. A flat EQ leaves the graph untouched. Render cache version 5. Golden vectors from Chromium's own filters hold the browser, the Python maths and an ffmpeg render to each other.
previewLaneEq plays another EQ on a lane (an A/B compare, a soloed band) and null goes back to the saved one; nothing reaches the store or the export. Engine graph comments name the effects slots.
A small EQ tab at the bottom-left of each waveform lane opens the lane's EQ panel as an extra row right under the lane, in the mixer column and the waveform column at the same fixed height, so the rows stay aligned. Open state is kept per song and lane; the tab shows when the EQ is on or bypassed; the panel row has a saved on/bypass switch. Panel contents load on first open. The waves column clips without being a scroll container, so the tab and the panel stick to the visible area at any zoom.
added 10 commits
October 10, 2026 14:50
The recording view scrolled at many common window sizes (1366x768, 1280x720, 1440x900 and smaller: 16 to 28 controls cut off). The scene preview is now the elastic part, sized from the room the strip and the timeline leave, never under 360 px wide. Short-window tiers tighten the header, the controls strip and the timeline; the Recording & Output panel is compacted (resolution and frame rate side by side, smaller switches) with Export MP4 pinned at the bottom; the drawer is 400 px wide. The layout with Scene Layers under the preview now starts below 680 px of workspace width instead of 1000. Measured: no scrolling at nine of ten window sizes in eight states; only a window under 720 px high scrolls.
Performance view fits the window without scrolling.
A wide dialog (1060 px) with a left navigation column (icon and label per section, a search box, an out-of-sync count on Library and a dot on Network), a title and one-line description per section, settings in cards with the text on the left and the control on the right, and a fixed footer with Done. A new Library section holds Storage, Automatic cleanup, Out of sync tracks and a red Danger zone for Reset app data; Logs and Registry sit under Troubleshooting. Search lists matching settings in place; arrow keys move through the sections; below 820 px the navigation becomes a row of chips and below 560 px the dialog fills the screen. Nothing a setting does, stores or calls has changed. Also fixes the Registry Refresh button, which was wired to Test encoders, and help text that showed literal code tags.
Settings dialog redesign.
…k switch Live Controls belonged to the postponed live-streaming work and only showed "Not available yet". The tabs are now Scene Builder and Templates; an unknown or remembered tab value opens the Scene Builder. The "Include metronome click" switch is gone from Recording & Output and from the saved recording settings; an export never asks for a click lane. Old saved settings that still hold the value load without an error and drop it.
Live Controls tab and the metronome click switch removed.
…m into one layer The scene had three separate layers for the song information: Song Title & Artist, Album Art and Waveform Overlay. They are now one Track Info layer with one frame, one visibility switch and four part switches (Album art, Title and artist, Show artist, Waveform) in Edit properties. The default card draws pixel-identical to before at 16:9, 9:16 and 1:1; other part combinations adapt (art only becomes a square, a title with no waveform is centred, a waveform alone is a centred band). Saved scenes are migrated when loaded: the three old layers merge into one Track Info with the title layer's frame, the earliest z-order and the parts the old layers showed; viewing a scene writes nothing and migrating twice gives the same result. Add Layer offers Track Info; the Guide has entries for the row, its controls and the part switches.
Track Info groups the title, album art and waveform layers.
… Performance leave the rail A second window next to the live preview plays the take's recorded video, muted and in step with the playhead (within about 15 ms when settled), while the engine plays the backing and the take. It holds the first or last frame outside the take and shows a note when stopped, with no take, while a take loads and while a new take records. Layout tiers: live, review and layers side by side on very wide windows; two equal windows when each stays at least 55% of the single-window size; otherwise one window with a Live | Take switch. Scene Layers stays flush right and the view still fits every common window size without scrolling. The Karaoke and Performance buttons are gone from the left rail; the Mode menu is the way in, and the Guide explains both on the menu items. Also fixes a missing audioEngine import that made saving a take throw.
Review window plays the recorded take; rail buttons removed.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| </div> | ||
|
|
||
| <div style="display:flex;align-items:center;gap:20px"> | ||
| <button onClick="{{togglePlay}}" aria-label="{{playAria}}" style="width:44px;height:44px;border-radius:50%;border:1px solid rgba(255,255,255,.18);background:rgba(8,10,14,.45);color:#F4F5F7;display:inline-flex;align-items:center;justify-content:center"> |
|
|
||
| <!-- Progress --> | ||
| <div style="padding:0 120px"> | ||
| <div onClick="{{seekBar}}" role="slider" aria-label="Playback position" aria-valuemin="0" aria-valuemax="186" aria-valuenow="{{tInt}}" style="position:relative;height:18px;cursor:pointer"> |
| <!-- Sections panel --> | ||
| <aside style="width:330px;flex:none;display:flex;flex-direction:column;gap:12px;border-left:1px solid #161C24;padding-left:12px"> | ||
| <div role="tablist" aria-label="Side panel" style="display:grid;grid-template-columns:repeat(2, minmax(0, 1fr));gap:8px"> | ||
| <button role="tab" aria-selected="{{queueSel}}" onClick="{{showQueue}}" style="{{queueTabStyle}}">Queue</button> |
| <aside style="width:330px;flex:none;display:flex;flex-direction:column;gap:12px;border-left:1px solid #161C24;padding-left:12px"> | ||
| <div role="tablist" aria-label="Side panel" style="display:grid;grid-template-columns:repeat(2, minmax(0, 1fr));gap:8px"> | ||
| <button role="tab" aria-selected="{{queueSel}}" onClick="{{showQueue}}" style="{{queueTabStyle}}">Queue</button> | ||
| <button role="tab" aria-selected="{{sectionsSel}}" onClick="{{showSections}}" style="{{sectionsTabStyle}}">Sections</button> |
| <div style="flex:1;min-height:0;overflow-y:auto;display:flex;flex-direction:column;gap:8px;padding-right:2px"> | ||
| <sc-for list="{{sections}}" as="sec" hint-placeholder-count="7"> | ||
| <div class="secc" style="{{sec.cardStyle}}"> | ||
| <button onClick="{{sec.seek}}" aria-label="{{sec.aria}}" style="flex:1;min-width:0;display:flex;align-items:center;gap:14px;background:none;border:0;padding:0;text-align:left;color:inherit"> |
| <span class="lbl">Key</span> | ||
| <div style="display:flex;align-items:center;gap:10px"><span style="font-size:24px;font-weight:600">{{keyName}}</span><span title="Transpose (semitones)" class="mono" style="min-width:28px;height:28px;padding:0 4px;box-sizing:border-box;border-radius:14px;border:1.5px solid #F5B72A;color:#F5B72A;font-size:11px;font-weight:700;display:inline-flex;align-items:center;justify-content:center">{{semisLabel}}</span></div> | ||
| <div style="display:flex;gap:8px"> | ||
| <button class="ib" aria-label="Transpose down a semitone" onClick="{{keyDown}}" style="width:44px;height:36px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14"></path></svg></button> |
| <div style="display:flex;align-items:center;gap:10px"><span style="font-size:24px;font-weight:600">{{keyName}}</span><span title="Transpose (semitones)" class="mono" style="min-width:28px;height:28px;padding:0 4px;box-sizing:border-box;border-radius:14px;border:1.5px solid #F5B72A;color:#F5B72A;font-size:11px;font-weight:700;display:inline-flex;align-items:center;justify-content:center">{{semisLabel}}</span></div> | ||
| <div style="display:flex;gap:8px"> | ||
| <button class="ib" aria-label="Transpose down a semitone" onClick="{{keyDown}}" style="width:44px;height:36px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14"></path></svg></button> | ||
| <button class="ib" aria-label="Transpose up a semitone" onClick="{{keyUp}}" style="width:44px;height:36px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14M12 5v14"></path></svg></button> |
| <div style="display:flex;gap:8px"> | ||
| <button class="ib" aria-label="Transpose down a semitone" onClick="{{keyDown}}" style="width:44px;height:36px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14"></path></svg></button> | ||
| <button class="ib" aria-label="Transpose up a semitone" onClick="{{keyUp}}" style="width:44px;height:36px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14M12 5v14"></path></svg></button> | ||
| <button class="ib" onClick="{{keyReset}}" style="flex:1;height:36px;font-size:13px;font-weight:500">Reset</button> |
| <div style="display:flex;align-items:center;gap:14px"> | ||
| <span style="font-size:24px;font-weight:600;width:84px">{{speedPct}}</span> | ||
| <div style="display:flex"> | ||
| <button class="ib" aria-label="Slower" onClick="{{speedDown}}" style="width:44px;height:32px;border-radius:8px 0 0 8px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14"></path></svg></button> |
| <span style="font-size:24px;font-weight:600;width:84px">{{speedPct}}</span> | ||
| <div style="display:flex"> | ||
| <button class="ib" aria-label="Slower" onClick="{{speedDown}}" style="width:44px;height:32px;border-radius:8px 0 0 8px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14"></path></svg></button> | ||
| <button class="ib" aria-label="Faster" onClick="{{speedUp}}" style="width:44px;height:32px;border-radius:0 8px 8px 0;margin-left:-1px"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.2" stroke-linecap="round"><path d="M5 12h14M12 5v14"></path></svg></button> |
| "-maxrate", f"{int(kbps * 1.5)}k", "-bufsize", f"{kbps * 2}k", | ||
| "-pix_fmt", "yuv420p"], width, height, seconds, ref) | ||
| if ok2: | ||
| row["x264_same_kbps"], _peak = bitrates(ffprobe, ref, seconds) |
| if _cache_loaded_for == ident: | ||
| return | ||
| _cache.clear() | ||
| _cache_loaded_for = ident |
| if not drainer.done(): | ||
| drainer.cancel() | ||
| with contextlib.suppress(asyncio.CancelledError): | ||
| await drainer |
| if not task.done() and await request.is_disconnected(): | ||
| task.cancel() | ||
| with contextlib.suppress(asyncio.CancelledError): | ||
| await task |
| if not task.done() and await request.is_disconnected(): | ||
| task.cancel() | ||
| with contextlib.suppress(asyncio.CancelledError): | ||
| await task |
| if fqdn: | ||
| names.add(fqdn) | ||
| names.add(fqdn.split(".", 1)[0]) | ||
| except OSError: |
This was referenced Oct 11, 2026
added 10 commits
October 11, 2026 18:27
…ce and a track-row overflow The full suite had never run against the 0.21.0 work: it loaded zero tests because streamer.spec.mjs set launchOptions inside a describe, which Playwright refuses for the whole run. Those groups moved to streamer-media.spec.mjs. Out-of-date expectations (the Performance rename, the Mode menu as the only way into Karaoke and Performance, the lanes, Track Info, the layout tiers) were updated without dropping any test, specs that leaked state or relied on timing were made robust, and track-header.spec.mjs now measures the side-rail header. Two real bugs found by the run are fixed, each with a spec that fails without the fix: when the song reloaded during a take the recorder sometimes stopped as if it were a pause and lost the "song reloaded" message (capture/takeRecorder.js), and a recording track's card was taller than a short row, so its menu and input picker stuck out (daw.css). Result: 625 passed, 7 skipped (hardware or host gated), 0 failed.
E2E suite up to date and green; take-recorder race and track-row overflow fixed.
…ries ci.yml gets a top-level permissions block set to none (it was the only workflow without one) and checkout, setup-node and upload-artifact are pinned to commit SHAs like the other workflows. .trivyignore records four HIGH advisories in uv.lock with the reason each is not reachable in StemDeck: fsspec CVE-2026-104851 (no reference:// filesystem; models load through hf_hub_download) and hydra-core CVE-2026-106439, CVE-2026-106441 and CVE-2026-106442 (no instantiate or hydra.main on any runtime path). The bump is deferred on purpose: any uv.lock change sends every Windows and Linux desktop install to the full download instead of an in-app update, so it waits for a release that already needs a reinstall.
CI workflow hardening and four dated trivy ignores.
…g in the e2e job The ebur128 filter in ffmpeg 5.1 (Debian 12, Ubuntu 22.04) has no framelog=quiet, so every loudness measurement failed there. Use framelog=verbose, which old and new ffmpeg accept, and guard it with a test. Read a WebM take's length from its last packet instead of the decoder's time=, which ffmpeg 6.1 reports as the start of the last frame. Install ffmpeg in the frontend-e2e job; recording specs need it.
…ngs list Add 74 Guide entries and 76 help sentences in all ten languages for the top bar, Mode menu, library, lyrics, sections bar, Stems popover, grid editor, click track, Export menu, notifications and record panel. With the Guide on, menu buttons now open their menus so the items inside can be explained. Follow the dialog's real Settings sections and drop the stale Input row.
…ve; fix non-object settings bodies Add tests for collect, the four separation workers (with subprocess coverage via scripts/coverage_run.py), stems and backing error paths, startup, restore and the stems-location move, plus four browser specs for Settings storage, footer time, queue progress and library tags. A JSON body that is not an object no longer makes the settings endpoints answer 500. Closes #797
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
This is the whole of 0.21.0 on one branch: 124 commits since v0.20.0. The usual
next-releasebranch no longer exists, so this pull request goes straight tomain. It stays a draft until I have built and tried it on every desktop platform.Merging it publishes the Docker
:edgeimage, so I will merge it at release time only. Please merge with a merge commit, not a squash, so the per-feature history and theCloseslines in the commit messages survive.What is in it
Modes
Karaoke Mode
Performance Mode
Stems and mixes per mode
GPU video encoder
Settings
Imports and playback
Hardening
How it was checked
On the final commit, on Windows, 2026-10-11:
pytest tests -q: 3178 passed, 5 skipped, 0 failed (5 minutes 55 seconds).tests/js/: none failing.ruff checkandruff format --check: clean. The translation coverage check: all ten tables complete.uv.lockhas no change against v0.20.0.pyproject.tomlgains one line, the AGPL licence.From earlier in the work, measured on my machine and not repeated on the final commit:
What is not tested yet
Known problems
Each known problem has its own issue. The release checklist is #795.
Fix or decide before the release:
Platform tracking: macOS #786, Linux #787.
More issues for the problems that can wait will be linked here as they open. Older problems found on the way are listed as separate issues and are not part of this branch.
Notes for review
main.rs, a newmediaperm.rs, the macOSInfo.plistandEntitlements.plist,Cargo.tomlandCargo.lock). The in-app updater replaces the executable andbackend/, so those changes travel with an update.uv.lockis unchanged, so the runtime id is unchanged and Windows and Linux users should still be offered the in-app update; I will confirm that on the built release.uv.lockhas no change against v0.20.0.