[Sad Catapella B9][Performance infrastructure] Add reproducible end-to-end and kernel benchmarks before Numba acceleration #106

Basis

Sad Catapella Batch 9, source d7a538e56d41ac1093ba79adcad397c21b3dfd32. Confirmed benchmarking infrastructure gap, not a measured performance regression; no benchmarks were run or code modified.

Evidence

• Existing experiments/binning_study/README.md lines 15–46 documents histogram accuracy/error studies, with limited CSV read/write timing in histogram_experiment.main lines 292–317. These scripts predate the production pixel/block/orchestration and multi-channel histogram paths; they do not benchmark candidate mask kernels, metadata indexing, QC, chunk reads, or whole-pipeline RSS.
• pyproject.toml lines 37–48 has no benchmark extra or Numba, and CI workflow runs lint/pytest but no reproducible performance job. No acceleration claims should be made from those correctness tests.
• Existing identified candidate issues: [Audit/Performance] Mask component and pair scans repeat expensive full-plane work #69 (piece_counts, pair_counts, _scan_plane); [Sad Catapella B3][Performance] Histogram run materializes every plane's bins and rewrites the entire CSV #77 (full histogram table RAM), [Sad Catapella B3][Performance] Misaligned Zarr block reads may decode the same chunk repeatedly #78 (Zarr chunk I/O), [Sad Catapella B4][Performance] Mask-looking filename ownership checks scale quadratically with image count #84/[Sad Catapella B4][Performance] Design reconciliation and duplicate checks repeatedly scan all rows #85/[Sad Catapella B5][Performance] Repeated linear manifest lookups make unchanged runs quadratic in file count #91/[Sad Catapella B6][Performance] Pixel-size single-linkage grouping can perform quadratic unique-calibration comparisons #95 (quadratic Python/indexing issues), [Sad Catapella B7][Performance] IntensityPanel recomputes image pooling and distribution curves for every setting or channel change #97/[Sad Catapella B8][Performance] Notebook plan rebuild repeatedly stats every TIFF and reparses the manifest on settings edits #102 (notebook repeated work); normalization Folders named single are refused, but existing lab datasets use them #24 (leave-one-out QC). The 64-bit downsample candidate [Sad Catapella B3][Scientific correctness] OME-Zarr pyramid mean corrupts uint64/int64 precision #75 is proposed for deletion by open PR PP-2: remove convert_tiffs, zarr_export, the moved-name shims and the old axis functions #66, so do not benchmark it unless feature retained.

Proposed benchmark framework

A versioned benchmarks/ directory with pinned input generator, seed, baseline commit, Python/NumPy/SciPy/Zarr/Numba versions, hardware/core/thread count, dtype, image and chunk geometry, compression, compiler warmup, and tabular results. Run metrics: wall time, CPU time, peak RSS, allocation volume, storage reads/decompressed bytes, and throughput (pixels/s or images/s). Profile first with production code (e.g. py-spy/scalene/cProfile and memory profiling), do not assume Numba improves C-backed NumPy/SciPy.

Tier 1 — mask kernels: 512²/4096²/16384² (the largest only if resources allow), 1–10^5 objects, contiguous/sparse int32 label IDs, fragmented objects, 1/2/4/8/12 mask types, T/Z >1, realistic label adjacency; independent oracle and bit-exact evidence JSON. Compare existing SciPy, algorithmically improved NumPy and optional njit union-find/streaming overlap. Include cold JIT and warm repeats, single and multi-thread controls. Require deterministic tie-breaking and no uint32/uint64 coercion.

Tier 2 — data pipeline: whole TIFF, tiled TIFF, compressed TIFF, NGFF 2/3, block vs whole, chunk boundary mismatch, 256/65536 bins, 1k–100k FOVs, 1–8 channels, no-change and 1% changed re-runs, filesystem metadata count and CSV memory, simulated fault + concurrent writers for correctness.

Tier 3 — QC/UX: 5/50/500/5000 FOV per biological repeat, 2–20 biological repeats, unequal FOV counts, 1–8 channels, threshold edits and channel switching. Normalize performance comparisons to identical scientific decision policy; normalization code/benchmarks belong in its repo.

Acceptance

• Produce scripts/reports that compare specified existing and candidate implementations on the same persisted fixture and pin commit hashes; avoid speed claims without confidence intervals and repeats.
• Verify exact count/identity and numerical tolerances independently before performance acceptance; assert invalid data remains rejected, tests pass and peak RSS meets limits.
• Define a documented threshold for accepting JIT complexity only after baseline measurements (e.g. worthwhile warm-runtime improvement with no unbounded compile/wheel burden), maintain a pure NumPy/SciPy fallback only as an explicitly selected identical-semantics backend, not a silent scientific fallback.
• Benchmark PR PP-2: remove convert_tiffs, zarr_export, the moved-name shims and the old axis functions #66 post-merge architecture rather than optimizing deleted conversion code. This issue covers shared benchmark infra and should not replace each algorithm's specific issue.

No tests, benchmarks or speedups have been executed or measured.