Repository navigation
Give PATH and yt-dlp the verified FFmpeg, and split the e2e job - #685
Merged
Merged
Conversation
added 2 commits
September 23, 2026 17:54
Setup can decide the FFmpeg in the data directory is unusable and settle on another: an Intel pair on Apple Silicon (#637), a build missing an encoder. It tells the backend through STEMDECK_FFMPEG and STEMDECK_FFPROBE, and separation honours that. PATH and yt-dlp were given FFMPEG_DIR instead, which in a desktop install is always data/ffmpeg, whatever setup concluded about it. So local files separated fine while every YouTube import ran the binary setup had thrown out, and the error named a binary the app had supposedly stopped using (#651). ffmpeg_dir() now answers for both: the directory holding the ffmpeg this process actually runs, provided ffprobe is beside it. yt-dlp takes one directory and looks for both there, so a split pair gets no directory at all and is left to PATH, where the desktop shell has already put the verified FFmpeg first. So do Docker and source runs, which have no FFmpeg of their own and were already right. A hand-set STEMDECK_FFMPEG_DIR is still honoured, because the binary defaults into it. The same fault had a second victim, not in the report. Setup also accepts FFmpeg nested under data/ffmpeg/bin/ (#248). yt-dlp was handed data/ffmpeg there, which holds no binary, and it runs dir/ffmpeg literally, with no fallback to PATH, so those installs had no FFmpeg for any import at all. The nested directory is what gets handed over now. The directory in use is logged at startup, and so is a data-directory FFmpeg that is being passed over, since the failure this prevents was silent. download.py also stops importing FFMPEG_DIR as a value. It was bound when the module first loaded, which is right in production and was the reason one of the new tests could not show main's behaviour on its own. A cold start with the #651 environment does: main gives yt-dlp data\ffmpeg, the rejected copy; this gives it the verified directory. Nine tests over real files: the rejected-copy case for ffmpeg_dir, PATH, yt-dlp and the log line, the nested layout, and the four setups that have to keep working.
The e2e job had become the slowest check on every pull request, about six minutes, growing with every spec (#686). It runs serially on purpose: the suite shares one backend and one jobs directory, and several specs change server state, so more workers against that one backend would interfere and fail intermittently. Raising workers was never the fix. Each shard is its own runner with its own backend, so this adds parallelism without adding sharing. Files stay whole within a shard, fullyParallel being off, so tests that rely on order inside a file still get it. One failing shard does not cancel the others, and each uploads its report under its own name, since a second upload under the same name fails the job. Run locally, all three against a fresh server each: 87, 90 and 81 tests, the whole 258, all passing, so no spec leaned on another file's leftovers. Free on a public repository; each shard pays its own setup, about a minute and a half. No required status check names frontend-e2e, so the per-shard names block nothing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #651
Closes #684
Closes #686
What was wrong
The backend had two separate answers to "which FFmpeg".
STEMDECK_FFMPEGandSTEMDECK_FFPROBE.FFMPEG_DIR. In a desktop install that is alwaysdata/ffmpeg, whatever setup concluded about it.That broke YouTube imports in two situations.
data/ffmpeg/bin/, a layout setup accepts. yt-dlp was handeddata/ffmpeg, which holds no binary. yt-dlp runsdir/ffmpegliterally and never falls back to PATH, so it had no FFmpeg at all.The fix
ffmpeg_dir()inapp/core/config.pygives the directory of the ffmpeg this process actually runs, provided ffprobe sits beside it. Bothconfigure_portable_environment()(PATH) and_base_ydl_opts()(yt-dlp) now use it.yt-dlp takes one directory and looks for both binaries there. So when ffmpeg and ffprobe are in different places, no directory is handed over and PATH decides. The shell has already put the verified FFmpeg first on PATH.
data/ffmpegdata/ffmpegbin/layout (#684)data/ffmpeg, no binarydata/ffmpeg/binSTEMDECK_FFMPEG_DIRset by handThe FFmpeg directory in use is logged at startup. So is a data-directory FFmpeg that is being skipped, since the failure this prevents was silent. Nothing is deleted: the rejected copy stays where it is, as #643 intends. No Rust change and no new environment variable.
Also: split the e2e job across three runners (#686)
The
frontend-e2ejob took about six minutes, running one test at a time. It has to stay serial within a run, because the suite shares one backend and one jobs directory and several specs change server state, so raisingworkerswould cause intermittent failures. It now runs as three shards. Each shard is its own runner with its own backend, so nothing is shared. Files stay whole within a shard, one failing shard does not cancel the others, and each uploads its report under its own name. No required status check namesfrontend-e2e, so the per-shard names block nothing.Run locally, each shard against a fresh server: 87, 90 and 81 tests, the full 258, all passing. No spec depends on another file's leftovers.
Verification
data\ffmpeg, the rejected copy; this branch gives it the verified directory.allin1_inferpackage, which is missing locally and present in CI.uv.lockare clean.