Phase 3 — Performance: memory & throughput #274

Activity

1. thcp

   added

   performanceSpeed, memory, and resource efficiencySpeed, memory, and resource efficiency

   on Jul 16, 2026
2. thcp commented on Jul 16, 2026

   

   thcp

   on Jul 16, 2026

   CollaboratorAuthor

   More actions

   Implementation plan — Phase 3: Performance

   Findings covered: #286, #287, #288, #289 (+#285 from Phase 2), #290, plus the --shifts quality setting from the roadmap. 5 PRs (one explicitly a stretch). Depends on Phase 1's timing telemetry (#293) for the before/after numbers — measure first, then optimize.

   ---

   PR-3A — SSE dirty-flag + tear-proof job serialization (#289, #285)

   app/core/models.py: add version: int = 0 to Job (exclude from to_state() and from _JOB_FIELDS — internal like cancel_requested). _set() increments job.version after all field writes.

   app/api/events.py stream loop: replace serialize-and-compare with:

   last_v = -1
   while loop.time() < deadline:
       v = job.version
       if v != last_v:
           snapshot = job.to_state()
           if job.version != v:      # torn read: writer ran mid-serialize
               continue               # loop immediately, re-serialize
           yield f"data: {json.dumps(snapshot)}\n\n"
           last_v = v
           ...

   • Fixes Non-atomic multi-field job updates visible to SSE #285 for real: a snapshot taken while _set was mid-flight is detected (version moved) and discarded, so a torn progress/stage pair never reaches the client.
   • Perf: idle connections now do one int compare per 0.2 s tick instead of a full to_state() + json.dumps — at the 200-connection cap that eliminates ~1,000 serializations/s.
   • Keepalive cadence unchanged (count ticks, emit : keepalive every ~75).
   • Terminal check moves to after a successful yield (status is in the snapshot), plus a cheap job.status in TERMINAL check on the no-change path so already-terminal jobs still close promptly.

   Tests: extend the SSE tests: no duplicate frames when state is static; frame emitted on _set; a simulated concurrent _set during serialize (monkeypatch to_state to bump version mid-call) yields only consistent snapshots; terminal close still works.

   ---

   PR-3B — Single-pass streamed peaks + presence (#286, #287)

   New module app/pipeline/audio_stats.py:

   def scan_stem(path: Path, buckets: int = 1500) -> tuple[list[list[float]], float]:
       """One streamed pass: ([min,max] per bucket, rms). Constant memory."""

   • info = sf.info(path); blocksize = max(1, ceil(info.frames / buckets)); iterate sf.blocks(path, blocksize=blocksize, dtype="float32", always_2d=True); per block: [float(b[:,0].min()), float(b[:,0].max())] + accumulate np.sum(b[:,0]**2) and count → rms = sqrt(sumsq/n).
   • Memory: one block ≈ frames/1500 × channels × 4 B ≈ 2–3 MB for a 20-min stereo stem (vs ~420 MB today).

   app/pipeline/collect.py — compute_stem_peaks: keep the signature and peaks.json format identical; body delegates to scan_stem and now also returns {name: rms} collected during the same pass.

   app/pipeline/runner.py — _run_common: drop the separate compute_stem_presence call (analyze.py version and its 6 ffmpeg spawns are deleted); after compute_stem_peaks returns the rms map, derive presence inline: normalize the 6 demucs-stem rms values to 0–100 relative to max (exact logic from analyze.compute_stem_presence lines 236–246). original/mix are scanned for peaks but excluded from presence, matching today.

   • Ordering note: presence is currently computed before the mix step and peaks after; move the combined scan to the current peaks position (after mix creation) — presence lands a few seconds later within the same job, invisible to the UI since both are only consumed at done.
   • Known, accepted delta: presence RMS is now full-rate channel-0 instead of 22.05 kHz ffmpeg-downmixed mono. Presence is normalized-relative (0–100 vs the loudest stem), so displayed values shift by ≲1–2 points. Document in the PR body; update any test fixtures.

   Tests: golden test — generate multi-tone WAVs, compare new peaks output to the old implementation's output (copy old impl into the test as reference) within float tolerance; rms correctness vs np.sqrt(np.mean(x**2)) full-load reference; presence normalization unchanged for equal inputs; peaks.json schema untouched (frontend contract).

   Measured claim for the PR body: run a 20-min track before/after with stage_timings (Phase 1) and report the post= delta + peak RSS if measurable.

   ---

   PR-3C — Demucs startup cost: measure, publish, decide (#288)

   Explicitly a measurement task — subprocess isolation (kill-on-cancel, crash containment) is a design feature we keep.

   • separate.py: record time.monotonic() at Popen and at the first stderr byte that parses as a progress percentage → job.stage_timings["separate_startup"]. One-line addition to the existing read loop.
   • After it ships, run 3 jobs on a representative machine (CPU and GPU), post the numbers to Model weights reloaded per job #288, and close it with a decision comment: keep subprocess (expected: startup is 5–15 s of a 1–15 min stage = not worth a persistent-worker's complexity), or open a follow-up with a concrete design if it's >20% of separate time on GPU.

   ---

   PR-3D — Separation quality setting: --shifts (#274 roadmap item)

   Follows the exact demucs_device settings pattern (the newest, most complete example of the runtime-settings plumbing):

   • app/core/settings.py: get_separation_quality() / set_separation_quality(v) with choices ("standard", "best"), default standard, env seed STEMDECK_SEPARATION_QUALITY.
   • app/main.py: add to _settings_payload() + POST handler (422 on bad value).
   • app/pipeline/separate.py: in _demucs_cmd, when quality == "best" append ["--shifts", "2"]. Applies on any device (CPU users accept the 2× wait knowingly).
   • Settings UI (static/js/catalog.js): General tab, next to Compute device: select "Separation quality" — Standard / Best (2× slower), wired identically to the video-height select. Desc: "Best runs the separator twice with randomized shifts and averages the result — cleaner stems, twice the time."
   • Tests: settings round-trip + rejection; _demucs_cmd includes/omits --shifts 2.

   ---

   PR-3E (stretch, do last, OK to defer) — Export render cache (#290)

   Server-mode nicety; skip if time-boxed effort exceeds a day.

   • Key: sha1(job_id, ext, stems, gains, start, end, export_sample_rate); store at CACHE_DIR/mixdown/<key>.<ext>.
   • Tee-while-streaming: _stream_ffmpeg writes chunks to <key>.tmp as it yields; on clean returncode==0 finish, os.replace tmp→final; on any failure/disconnect, unlink tmp. Cache hit → FileResponse (first-byte latency for hits drops to ~0, misses unchanged).
   • Bound: after each finalize, prune dir to 20 files / 500 MB by mtime.
   • Invalidation: key includes every render input; stems are immutable per job id; export_sample_rate is in the key so a settings change misses cleanly.
   • Tests: hit/miss, failed render leaves no cache entry, prune bound.

   ---

   Sequencing & estimates

   3A (0.5d) → 3B (1d, the big memory win) → 3C (0.25d) → 3D (0.5d) → 3E (stretch 1d).
   Manual verification: 3B on the longest track in the library — confirm identical waveforms render in the studio (peaks contract) and presence bars look sane; 3A via devtools EventSource frames during a job.
3. thcp commented on Jul 17, 2026

   

   thcp

   on Jul 17, 2026

   CollaboratorAuthor

   More actions

   Phase 3 status: 4 of 5 shipped, closing out for now

   PR	Closes	Commit
   #305	#289, #285 (SSE dirty-flag + tear-proof serialization)	c896b9c
   #306	#286, #287 (single-pass streamed peaks + presence)	6eb1741
   #307	#288 (startup-cost instrumentation)	5e8caeb
   #308	--shifts separation quality setting (roadmap item)	68c449d

   #288 stays open: PR-3C only instruments stage_timings["separate_startup"] -- closing it needs real timing numbers from a few jobs run on representative CPU/GPU hardware, which isn't something producible from a dev sandbox. Whoever has real hardware handy: run a few jobs, check separate_startup in the job's metadata.json or the completion log line, and post the numbers here with a keep-or-redesign decision.

   #290 (export render cache) deferred: explicitly marked a stretch item in the implementation plan ("OK to defer" / "skip if time-boxed effort exceeds a day"). More substantial than the other four (tee-while-streaming cache, bounded prune, cache-key invalidation across stems/gains/region/sample-rate) -- left as a standalone backlog item rather than rushed into this phase.

   Leaving this issue open until #288 and #290 land.
4. thcp commented on Jul 17, 2026

   

   thcp

   on Jul 17, 2026

   CollaboratorAuthor

   More actions

   Update: #290 shipped too

   PR #311 (`679eb78`) added the mixdown render cache -- all 5 originally-planned Phase 3 items are now done except #288, which stays open by design (it needs real hardware timing data to close with a decision, not more code -- see the comment on #288 itself and the #309 follow-up it produced).

   Phase 3 final tally: #305, #306, #307, #308, #310 (bonus follow-up), #311. All merged.
5. thcp commented on Jul 17, 2026

   

   thcp

   on Jul 17, 2026

   CollaboratorAuthor

   More actions

   All 5 findings + the #309 follow-up are closed. This tracking issue is done — closing.
6. thcp

   closed this as completedon Jul 17, 2026
7. thcp

   self-assigned this

   on Oct 1, 2026