Skip to content

Bump certifi and pillow, require Python 3.11, clear the trivy ignores - #828

Merged
thcp merged 2 commits into
big-releasefrom
chore/uv-lock-security-bump
Oct 21, 2026
Merged

Bump certifi and pillow, require Python 3.11, clear the trivy ignores#828
thcp merged 2 commits into
big-releasefrom
chore/uv-lock-security-bump

Conversation

@thcp

@thcp thcp commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

Closes #827

What changed

All 8 original trivy findings in uv.lock (certifi and pillow) are gone, and a local trivy fs --scanners vuln --severity HIGH,CRITICAL --ignore-unfixed uv.lock reports 0 findings.

Commit 1: uv lock --upgrade-package for each flagged package only. No blanket upgrade.

Package Old New Advisory
certifi 2026.4.22 2026.10.19 GHSA-6q48-6c7w-9gw2
pillow 12.3.0 13.0.0 CVE-2026-112403 and 6 more
urllib3 2.7.0 2.8.0 CVE-2026-97687, CVE-2026-97689
anyio 4.13.0 4.14.2 CVE-2026-63374
fsspec 2026.3.0 2026.9.0 CVE-2026-104851
hydra-core 1.3.5 1.3.7 CVE-2026-106439, -106441, -106442

Commit 2: requires-python goes from >=3.10,<3.14 to >=3.11,<3.14. Pillow 13.0.0 requires Python 3.11, so with 3.10 allowed the lock kept a pillow 12.3.0 entry for 3.10, and trivy reads every entry. Every shipped build already runs 3.12 (desktop, Docker, CI). A plain uv lock then only drops the 3.10 branches:

  • Removed: backports-asyncio-runner, exceptiongroup, tomli.
  • Dropped their 3.10-only versions: contourpy, matplotlib, networkx, numpy, onnxruntime, pillow, scikit-learn, scipy.
  • No package gets a new version.

scripts/macos/make-runtime-pack.sh no longer picks or accepts a 3.10 interpreter, because installing the project on 3.10 would now fail.

.trivyignore: removed the anyio, urllib3, fsspec and hydra-core entries, since the lock now takes each fix. Kept CVE-2025-32434 (the Intel macOS torch pin) and DS-0002 (the Dockerfile gosu drop). Both comments are still true.

What did not change

  • torch, torchaudio and torchvision are unchanged. The Rust drift test uv_lock_resolves_the_cpu_torch_pair passes.
  • No app, desktop or Rust code.
  • ruff target-version stays py310. Moving it to py311 would flag 16 UP041/UP017 rewrites in app/ and tests/, which belong in a separate change.

Where the lock is used

  • Uses it: CI (uv sync --frozen), ./run.sh setup (uv sync), the trivy scan, and the desktop runtimeId hash.
  • Does not use it: the desktop packages (pip install "$REPO_ROOT" in make-portable.sh, make-portable.ps1 and make-runtime-pack.sh) and Docker (pip install .). These resolve from PyPI against pyproject at build time. The deps-audit job compiles from pyproject too.

So the bump makes CI test what actually ships.

Desktop update effect

The runtimeId changes from py3.12-9b2b87d8ac2ddbc3 to py3.12-585d1341908f343b. The in-app updater will not offer this release, and every Windows and Linux desktop install does one full download. This is expected for 0.21.0.

Verified locally (macOS, Python 3.12, fresh env from the final lock)

  • pytest: 3315 passed, 9 skipped, 5 failed. The 5 are the known ogg/libvorbis failures on this host.
  • ruff check, ruff format --check and bandit -ll are clean. All 61 npm run test:js files pass. cargo test: 172 passed.
  • trivy with the CI flags (vuln, secret, misconfig, .trivyignore): exit 0.
  • Playwright:
    • After commit 2: the 5 specs select-e2e picks, plus export-menu and import-background. 49 passed.
    • After commit 1, the full suite: 688 passed. The failures that reproduce on a rerun (trash-restore x3, streamer-media, video-sync) also fail on plain big-release.
  • Real separation through the HTTP API on a 20 s file finished on MPS. Stems, peaks, beats, the wav and mp3 mixdowns and the zip all returned 200.
  • The certifi CA bundle loads into ssl. Pillow 13 decodes PNG, JPEG, GIF, TIFF and WebP. torchvision transforms and matplotlib PNG output work.

Release notes

  • Python 3.10 is no longer supported; every shipped build already used 3.12.

thcp added 2 commits October 21, 2026 11:50
@thcp
chore(deps): bump certifi, pillow and the packages the .trivyignore w…
1d2df0a 
…as waiting on

Targeted uv lock --upgrade-package, no blanket upgrade:

- certifi 2026.4.22 -> 2026.10.19 (GHSA-6q48-6c7w-9gw2)
- pillow 12.3.0 -> 13.0.0 on Python 3.11+ (seven CVEs fixed in 13.0.0)
- urllib3 2.7.0 -> 2.8.0 (CVE-2026-97687, CVE-2026-97689)
- anyio 4.13.0 -> 4.14.2 (CVE-2026-63374)
- fsspec 2026.3.0 -> 2026.9.0 (CVE-2026-104851)
- hydra-core 1.3.5 -> 1.3.7 (CVE-2026-106439, -106441, -106442)

No other package moved; torch, torchaudio and torchvision are unchanged.
The deferred ignores for urllib3, anyio, fsspec and hydra-core are removed
because the lock now takes each fix. The torch CVE-2025-32434 and DS-0002
entries stay; both comments are still true.

Pillow 13.0.0 requires Python 3.11, and requires-python is still >=3.10, so
the lock keeps a pillow 12.3.0 entry for Python 3.10 only. Trivy still
reports it. Every shipped build (desktop, Docker, CI) runs Python 3.12.

This changes the desktop runtimeId, so the in-app updater does not offer
this release and every Windows and Linux desktop install does one full
download. Accepted for 0.21.0.

Closes #827
@thcp
chore(deps): require Python 3.11 so Pillow 13 can resolve everywhere
d2e0c01 
Pillow 13.0.0, which fixes the seven Pillow advisories trivy reports,
requires Python 3.11. With requires-python at >=3.10 the lock kept a
pillow 12.3.0 entry for Python 3.10, and trivy reads every entry in the
lock. Every shipped build (desktop, Docker, CI) already runs Python 3.12.

Re-locked with plain uv lock. The only lock changes are the Python 3.10
branches going away: backports-asyncio-runner, exceptiongroup and tomli
are removed, and the 3.10-only versions of contourpy, matplotlib,
networkx, numpy, onnxruntime, pillow, scikit-learn and scipy are dropped.
No package gains a new version; torch, torchaudio and torchvision are
unchanged.

make-runtime-pack.sh no longer picks or accepts a Python 3.10
interpreter, since installing the project on it would now fail.
@thcp
thcp marked this pull request as ready for review October 21, 2026 11:59
@thcp
thcp merged commit 040ee8b into big-release Oct 21, 2026
11 checks passed
This was referenced Oct 21, 2026
Closed
Merged
thcp added a commit that referenced this pull request Oct 22, 2026
@thcp
ci(e2e): scoped e2e on staging PRs, full suite after each merge to bi…
6c3cf58 
…g-release (#836)

Closes #835

## What changed

- **PR to `main`, push to `main`, release:** full suite, as before.
- **PR to any other base:** smoke + changed specs + mapped specs. A hub
module (former `full` source entries: `main.js`, `state.js`, `mixer.js`,
`catalog.js`, `index.html`, `app/api/jobs.py`, `stems.py`,
`app/core/**`, ...) or an unmapped file under `static/` or `app/` now
adds a fixed **core set** (17 specs) instead of everything. `uv.lock`
and `pyproject.toml` add core too.
- **Full suite on a non-main PR** only for a change to how the suite
runs: `playwright.config.mjs`, `tests/e2e/helpers*`, `serve.sh`,
`seed.py`, `package*.json`, the three e2e workflows, the selector,
`spec-map.json`, or unmapped code under `tests/e2e/`.
- **New `e2e-full.yml`:** full suite on every push to `big-release`,
nightly (04:17 UTC) on `big-release`, and `workflow_dispatch` with a
`ref` input.
- **New reusable `e2e.yml`** (`workflow_call`, inputs `mode`,
`base-ref`, `checkout-ref`) holds the steps once. Same pinned SHAs,
`permissions: {}`, no secrets.
- **Concurrency:** a new push to a PR cancels that PR's older e2e run.
Pushes, releases, nightly and manual runs are never cancelled.
`timeout-minutes: 30` per shard.

## What did not change

- Application code, specs, `uv.lock`, package files. Three shards,
`--pass-with-no-tests`.
- Check names gain a prefix: `frontend-e2e / frontend-e2e (1)` to `(3)`.
A reusable workflow always prefixes the caller job. No branch protection
or ruleset requires these names today (checked).

## Real diffs, before and after

| Diff | Files | Before | After |
|---|---|---|---|
| #806 (queue.js, catalog.js) | 5 | FULL 105 | 30 (core), includes
queue-progress |
| #820 | 7 | FULL 105 | 59 (core) |
| #821 | 2 | 47 | 47 |
| #823 | 8 | 5 | 5 |
| #828 (uv.lock) | 4 | 5 | 22 (core) |
| #819, #830, #831 | 7 to 19 | FULL 105 | FULL 105 (edit
`spec-map.json`) |
| #829 | 19 | FULL 105 | FULL 105 (edits `helpers.mjs`) |
| #822 | 33 | FULL 105 | FULL 105 (edits `seed.py`) |
| `main...big-release` | 331 | FULL | FULL (PR to main) |

`npx playwright test --list <selected>` matches the selector exactly for
#806 (30 files, 200 tests) and #820 (59 files, 465 tests). Full suite:
724 tests in 105 files. Core + smoke alone: 159 tests in 22 files.

## Trade-off

A non-main PR can pass while a spec outside its core and mapped sets
breaks. The push to `big-release` run catches it on the merge commit,
minutes later.

## Limits

- **The nightly schedule only starts once `e2e-full.yml` reaches
`main`.** GitHub runs scheduled workflows from the default branch, and
the job checks out `big-release` explicitly for that reason.
- Neither concurrency nor the timeout helps a hosted runner GitHub never
assigns (queue time does not count toward the timeout). Fix by hand:
re-run the job, or `gh run cancel <id>` then `gh run rerun <id>
--failed`.
- This PR changes `ci.yml`, so its own CI runs the full suite. Expected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@thcp