fix(data_plane): bound warm SketchStore memory with retention horizon

[zzylol]

Root cause (the leak)

asap-data-plane RSS grew ~0.74 GiB/min unbounded under steady OTLP sketch ingest, freezing a 251 GiB node. The [MEMORY_DIAG] showed a flat ~20218 sid count yet climbing RSS and 0.00 KB sealed — so the growth is per-instance, in current_epoch.

• SketchStore::append_sample/append_precompute always build the per-sid store via SidStoreData::new(), which sets epoch_capacity: None (index/epoch_columnar.rs). maybe_rotate_epoch() therefore returns early on the very first match arm — nothing ever seals (hence 0.00 KB sealed).
• The persistence flusher (persistence/flusher.rs) and approx_memory_bytes only ever operate on sealed_epochs (list_sealed_epochs -> evict_sealed_epoch). With nothing sealed, the eviction/retention path never sees current_epoch at all.
• lifecycle/eviction.rs (SchemaEvictionService) only drops whole Expired sids (schema retirement). In steady state sids stay Active forever, so it never fires.

Net: current_epoch's columnar Vecs accumulate one ~19 KB pane per 30s tumbling window per sid forever — memory was O(active_series × total_elapsed_time). Two unbounded sources: (1) nothing seals, (2) even sealed epochs only evict by count (max_epochs), never by age for current_epoch.

Fix

A time-based retention horizon on SidStoreData:

• On each insert, evict windows whose END is older than newest_end − horizon from current_epoch (and drop fully-behind sealed_epochs). Memory becomes O(active_series × horizon).
• Configurable via env ASAP_SKETCH_RETENTION_MS (0 disables), resolved once via OnceLock off the hot path. Default 2h (DEFAULT_SKETCH_RETENTION_MS).
• This is retention/eviction, not force-seal-and-read-only-sealed: recent unsealed windows stay queryable via the overlap scan.

Why it can't regress #323–#326

The 2h horizon comfortably exceeds the ~30m max range-query window plus the delta-stitching carry-in's Full-base reach. Eviction keys on window-END, so a pane straddling the cutoff survives until fully behind the horizon — the overlap scan (range_query_overlap_into) and carry-in (collect_ending_at_or_before) still see everything within the horizon. Older data lives in the cold/raw tier.

Tests

cargo test -p data_plane — 739 lib tests pass (0 fail). New regression tests:

• evict_window_ends_before_* — drops old / keeps straddling+recent, recomputes bounds, no-op fast path.
• retention_bounds_window_count_over_long_elapsed_time — 4h of 30s panes stays bounded to ~horizon/window (not 480).
• retention_disabled_keeps_full_history — None horizon = legacy behavior.
• retention_keeps_windows_within_horizon_queryable + e2e retention_bounds_memory_yet_keeps_recent_windows_queryable — 30m range query within horizon still resolves with its Full carry-in base intact.

cargo clippy -p data_plane — no new errors/warnings in the changed files.

🤖 Generated with Claude Code