Skip to content

Give PATH and yt-dlp the verified FFmpeg, and split the e2e job - #685

Merged
thcp merged 2 commits into
mainfrom
fix/ffmpeg-dir-651
Sep 23, 2026
Merged

thcp merged 2 commits into
mainfrom
fix/ffmpeg-dir-651

Conversation

@thcp

@thcp thcp commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #651
Closes #684
Closes #686

What was wrong

The backend had two separate answers to "which FFmpeg".

  • Separation used the pair setup verified. The desktop shell passes it in STEMDECK_FFMPEG and STEMDECK_FFPROBE.
  • PATH and yt-dlp were given FFMPEG_DIR. In a desktop install that is always data/ffmpeg, whatever setup concluded about it.

That broke YouTube imports in two situations.

The fix

ffmpeg_dir() in app/core/config.py gives the directory of the ffmpeg this process actually runs, provided ffprobe sits beside it. Both configure_portable_environment() (PATH) and _base_ydl_opts() (yt-dlp) now use it.

yt-dlp takes one directory and looks for both binaries there. So when ffmpeg and ffprobe are in different places, no directory is handed over and PATH decides. The shell has already put the verified FFmpeg first on PATH.

Setup yt-dlp location, before After
Data-directory FFmpeg verified data/ffmpeg data/ffmpeg
Data-directory copy rejected (#651) the rejected copy the verified FFmpeg's directory
Nested bin/ layout (#684) data/ffmpeg, no binary data/ffmpeg/bin
Docker, source with FFmpeg on PATH none none
STEMDECK_FFMPEG_DIR set by hand that directory that directory

The FFmpeg directory in use is logged at startup. So is a data-directory FFmpeg that is being skipped, since the failure this prevents was silent. Nothing is deleted: the rejected copy stays where it is, as #643 intends. No Rust change and no new environment variable.

Also: split the e2e job across three runners (#686)

The frontend-e2e job took about six minutes, running one test at a time. It has to stay serial within a run, because the suite shares one backend and one jobs directory and several specs change server state, so raising workers would cause intermittent failures. It now runs as three shards. Each shard is its own runner with its own backend, so nothing is shared. Files stay whole within a shard, one failing shard does not cancel the others, and each uploads its report under its own name. No required status check names frontend-e2e, so the per-shard names block nothing.

Run locally, each shard against a fresh server: 87, 90 and 81 tests, the full 258, all passing. No spec depends on another file's leftovers.

Verification

  • Cold start with the An FFmpeg that setup rejected is still what yt-dlp gets, and still first on PATH #651 environment (a rejected pair in the data directory, the verified one elsewhere): main gives yt-dlp data\ffmpeg, the rejected copy; this branch gives it the verified directory.
  • 9 new tests over real files, all failing against main. They cover the rejected-copy case for the directory, PATH, yt-dlp and the log line; the nested layout; and the four setups that have to keep working.
  • Full pytest suite: 1,089 pass. The one failure needs the optional allin1_infer package, which is missing locally and present in CI.
  • ruff, bandit and uv.lock are clean.

Thales added 2 commits September 23, 2026 17:54
Setup can decide the FFmpeg in the data directory is unusable and settle on
another: an Intel pair on Apple Silicon (#637), a build missing an encoder.
It tells the backend through STEMDECK_FFMPEG and STEMDECK_FFPROBE, and
separation honours that. PATH and yt-dlp were given FFMPEG_DIR instead, which
in a desktop install is always data/ffmpeg, whatever setup concluded about
it. So local files separated fine while every YouTube import ran the binary
setup had thrown out, and the error named a binary the app had supposedly
stopped using (#651).

ffmpeg_dir() now answers for both: the directory holding the ffmpeg this
process actually runs, provided ffprobe is beside it. yt-dlp takes one
directory and looks for both there, so a split pair gets no directory at all
and is left to PATH, where the desktop shell has already put the verified
FFmpeg first. So do Docker and source runs, which have no FFmpeg of their own
and were already right. A hand-set STEMDECK_FFMPEG_DIR is still honoured,
because the binary defaults into it.

The same fault had a second victim, not in the report. Setup also accepts
FFmpeg nested under data/ffmpeg/bin/ (#248). yt-dlp was handed data/ffmpeg
there, which holds no binary, and it runs dir/ffmpeg literally, with no
fallback to PATH, so those installs had no FFmpeg for any import at all. The
nested directory is what gets handed over now.

The directory in use is logged at startup, and so is a data-directory FFmpeg
that is being passed over, since the failure this prevents was silent.

download.py also stops importing FFMPEG_DIR as a value. It was bound when the
module first loaded, which is right in production and was the reason one of
the new tests could not show main's behaviour on its own. A cold start with
the #651 environment does: main gives yt-dlp data\ffmpeg, the rejected copy;
this gives it the verified directory.

Nine tests over real files: the rejected-copy case for ffmpeg_dir, PATH,
yt-dlp and the log line, the nested layout, and the four setups that have to
keep working.
The e2e job had become the slowest check on every pull request, about six
minutes, growing with every spec (#686). It runs serially on purpose: the
suite shares one backend and one jobs directory, and several specs change
server state, so more workers against that one backend would interfere and
fail intermittently. Raising workers was never the fix.

Each shard is its own runner with its own backend, so this adds parallelism
without adding sharing. Files stay whole within a shard, fullyParallel being
off, so tests that rely on order inside a file still get it. One failing
shard does not cancel the others, and each uploads its report under its own
name, since a second upload under the same name fails the job.

Run locally, all three against a fresh server each: 87, 90 and 81 tests, the
whole 258, all passing, so no spec leaned on another file's leftovers. Free on
a public repository; each shard pays its own setup, about a minute and a half.
No required status check names frontend-e2e, so the per-shard names block
nothing.
@thcp thcp changed the title Give PATH and yt-dlp the FFmpeg setup verified, not the data directory Give PATH and yt-dlp the verified FFmpeg, and split the e2e job Sep 23, 2026
@thcp
thcp merged commit d59adbd into main Sep 23, 2026
12 checks passed
@thcp
thcp deleted the fix/ffmpeg-dir-651 branch September 23, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant