You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
[Sad Catapella B5][Performance] Repeated linear manifest lookups make unchanged runs quadratic in file count #91
run_experiment() lines 524–543 calls previous.file(rel) for every file, making the lookup portion O(N²) for N unchanged images.
check_sources() lines 518–534 repeats this for each supplied path, leading to O(M×N) lookup work for M caller images. The code also creates a redundant sorted list for manifest leaves; see RunManifest.leaves().
Tests cover only three entries. Reuse tests cover small experiments here.
Fix / benchmarks
Create an immutable path→FileEntry lookup once (or explicitly build one per batch from the manifest) and use it consistently in run and source checks, with duplicate-path validation. Preserve existing RunManifest JSON schema/order. Benchmark 100, 1k, 10k, 100k images in a no-change rerun and check_sources; compare slope/latency and memory overhead. The iterator/tuple public interface can remain unchanged. Numba is not suitable: Python dictionary indexing removes the algorithmic problem.
Activity