[thcp]

Closes #838

What changed

• Runner pinned to ubuntu-24.04 for frontend-e2e only, with a comment to revisit after the Ubuntu 26 label migration ([Ubuntu] ubuntu-latest label will use Ubuntu 26.04 in November 2026 actions/runner-images#14748).
• scripts/ci/retry.sh: up to 3 attempts, a per-attempt timeout that kills the whole process group, pauses of 10 s then 20 s. Every attempt is logged with exit code and duration; a retried success leaves a warning annotation; the last exit code is returned unchanged. tests/ci/test_retry_sh.sh (28 checks) runs in the lint job.
• Browsers: npx playwright install chromium on its own, cached (~/.cache/ms-playwright, keyed on the lockfile's Playwright version), 10 minute step limit.
• System packages: playwright install-deps chromium on its own, run as root through the retry wrapper so the timeout can kill apt. apt gets Acquire::Retries=3, http/https timeouts of 20 s and DPkg::Lock::Timeout=60 via /etc/apt/apt.conf.d, so Playwright's own apt-get uses them. The downloaded .debs are cached (keyed on Playwright and image version). Attempts after the first drop azure.archive.ubuntu.com from /etc/apt/apt-mirrors.txt, so they start on archive.ubuntu.com: apt falls back only on an error, never on a trickle. 10 minute step limit.
• ffmpeg: no longer apt. A static BtbN build (autobuild-2026-09-30-13-08, ffmpeg 8.1.3, gpl), pinned by SHA256, cached, and checked against the pin on every run before it is unpacked. The step prints ffmpeg -encoders | grep -E "libvorbis|libmp3lame" and fails by name if libvorbis, libmp3lame, aac or flac is missing.
• Job summary: a table of each setup step's seconds and cache hit, miss or partial, written with if: always(), so a step killed by its limit shows as "did not finish".

Why ffmpeg from BtbN

• Static download: GitHub's release CDN, not the Ubuntu mirror that stalled. BtbN keeps month-end builds for about two years and is already the desktop app's Windows source. Has libvorbis, libmp3lame and chromaprint.
• Third-party setup action: rejected. It downloads the binary itself, so the checksum pin would live in someone else's code and need a review on every bump. The workflow can pin it directly in a few lines.
• apt with cached .debs: still on the slow mirror whenever any of its 87 packages gets an update, which would be a cache miss.
• shaka-project static builds (the macOS source): no libvorbis.

Step limits

10 minutes each. A healthy run takes about 20 s for the system packages and seconds for the rest. The 12-minute installs in #838 were at 29 kB/s; with the .debs cached they should not recur. If they do, failing at 10 minutes and retrying (apt resumes partial downloads) beats using up the whole 30 minute job, which stays as the outer limit.

What did not change

• Job names frontend-e2e (1) to (3), the selector, the scoped and full cadence, e2e-full.yml.
• Specs, app code, uv.lock, package files.
• Other workflows. The last 200 runs show no slow or failed apt or download step outside frontend-e2e (Linux Rust Check's apt step peaks at 72 s, the test and linux-installer apt steps stay under 60 s), so they are not pinned or changed.

Evidence

Run 39827053220 (first e2e-full, 6c3cf58), attempt 1: shard 1 spent 12m23s in playwright install --with-deps, then 16m28s in apt's ffmpeg until the job limit. Shard 3: 11m38s, then 17m41s. Shard 2: 22 s and 11 s. The log has "Fetched 20.9 MB in 11min 56s (29.2 kB/s)" from azure.archive.ubuntu.com; the Chromium download itself took 9 s. The image was ubuntu-24.04 20261012.334.1. Same pattern in runs 39823430200, 39792892907 and 39827062156.

Trade-offs

• A retry hides a flaky step for up to 3 attempts. The warning annotation and the log show it.
• The pin must be revisited after the migration.
• e2e now runs ffmpeg 8.1.3 instead of Ubuntu's 6.1. That matches the desktop builds, not the Docker image.

Results on this PR

• First push (3bb1874): the slow mirror came back. One shard in each of runs 39843695691 and 39843702780 spent three 170 s attempts fetching fonts from Azure at about 40 kB/s, and failed at 9 minutes instead of 30. That is why the second commit drops Azure on a retry.
• Second push (ca231fb), cold .deb cache, run 39854523178: shard 2's first attempt timed out at 170 s on Azure, the second dropped Azure and finished in 13 s. All 3 shards green.
• Warm caches, manual e2e-full run 39854515365: browsers 1 s (hit), system packages 10 to 39 s (hit), ffmpeg 7 to 10 s (hit, checksum and unpack). All 3 shards green.
• Encoders printed in every shard: libvorbis and libmp3lame present; export-ogg specs pass.

Not verified

• A step reaching its 10 minute limit. The per-attempt timeout fired for real; the step limit did not.
• Caches from this branch are not visible to other PRs. They start helping once the change is on big-release.