feat(e2e): all-five-sketch runtime path + harness — controller, processor, fake-exporter, P1–P9

[zzylol]

Summary

Lands the real e2e: PromQL → controller → agent (sketchcol) → backend (precompute_engine) → PromQL response, through the modified-OTLP wire format (typed Metric.data = {DDSketch | KLLSketch | HLLSketch | CountSketch | CountMinSketch}). Each sketch type now has a runtime-verified path with a soak against its accuracy envelope; the P1–P9 harness drives the full sweep matrix.

Sketch	Query	Result	Envelope
DDSketch	histogram_quantile(0.5/0.9/0.99,…)	21.12 / 47.95 / 104.60	relative_quantile, ε=0.01
KLLSketch	histogram_quantile(0.5,…)	18.26	rank_quantile, ε=0.16
HLLSketch	count(http_requests_total)	149.68 distinct	relative_cardinality, ε=0.008
CountSketch	sum_over_time(http_requests_total[1m])	24266	additive_frequency, ε=0.03
CountMinSketch	sum_over_time(http_requests_total[1m])	145735	additive_frequency, ε≈0.0027, δ=0.03125

Cross-cutting fixes

• Controller data_sink (types.rs, config/agent.rs): AgentDataSink enum picks between Otlp{endpoint,…} and PrometheusScrape{…} instead of always emitting prometheus.
• OpAMP framing (opamp/mod.rs): strip varint header from incoming WS payloads before proto decode; outbound ServerToAgent carries ReportFullState + Accept/Offer capability bitmask so agents accept and apply config.
• ddsketchprocessor: replaced DataDog sketches-go with sketchlib-go/DDSketch so the proto envelope is decodable by asap_sketchlib's DDSketchState.
• Sketchcollector builder: builder-config-sketches.yaml (renamed from builder-config-ddonly.yaml) compiles all five sketch processors plus opampextension under OCB v0.141.0.

Harness (P1–P9)

• P2 deploy/fake-exporter/swappable_filter.go — atomic.Pointer-backed Stream.AttributeFilter, swap via POST /control/projection. No SDK patch needed.
• P4 deploy/fake-exporter/raw_tee.go — hour-bucketed JSONL ground-truth writer, byte-for-byte compatible with the Rust RawSample wire format. Wired into runSynthetic + runTraceReplay.
• P5 deploy/scripts/promql_replay.py — fixed-QPS PromQL fan-out, plan-id-tagged JSONL.
• P6 deploy/scripts/plan_transition.py — t_query_in / t_plan_ready / t_first_hit / t_steady against the controller plan-id stream + 1 Hz docker stats.
• P7 deploy/scripts/run_e2e_sweep.sh — {DDSketch,KLL,CS,CMS,HLL} × {N=1,10} × {scrape=100ms,1s} × {card=1e3,1e4,1e5} (60 cells).
• P8 deploy/scripts/accuracy_reduce.py — cold-truth ⋈ replay → relative-error CSV.
• P9 deploy/scripts/e2e_plots.py — pareto / bandwidth / transition / latency-CDF figures.

Known limitation

Stock OTel gateway 0.108 can't translate DDSketch / HLLSketch through prometheusremotewrite, so warm-tier sketch ingest is dropped at the gateway — the cold-tier path (P1+P4) carries the e2e flow today. Tracked in PROGRESS.md follow-ups: enable OTLP ingest on precompute_engine (or swap the backend image to query_engine_rust, which already has it).

Companion PR in ASAPQuery-backend lands the matching query_statistic impls + --cold-store-root wiring.

Test plan

• All five sketches: live PromQL → expected accuracy envelope (table above)
• docker compose -f base.yml -f agents-N1.yml -f baseline-b3-delta.yml -f e2e-overlay.yml up -d brings the stack up cleanly
• swappable_filter HTTP swap returns {"applied":"zone"}
• raw_tee writes ground-truth JSONL with the right path layout (8 unit tests incl. concurrent-writer race + format anchor)
• P5 smoke: 17 queries / 6 s, p50 2.4 ms, p99 1.3 s
• P8 smoke: 4.1 M ground-truth samples, 17 query rows, accuracy CSV produced
• P9 smoke: pareto_acc_vs_thru.png + query_latency_cdf.png from real data
• Full P7 sweep over 60 cells (hours wall) — separate run
• OTel submodule bumps deferred (excluded from this PR)

🤖 Generated with Claude Code