docs(design): asap edge precompute framework — host adapters + phased extraction

[zzylol]

Summary

Drafts the framework refactor that decouples ASAP's edge runtime from being a fork of the OTel Collector. Forward-looking design doc; no code changes.

The doc lives at `docs/design-asap-edge-framework.md` and addresses the question that came up in conversation: can ASAPCollector be applied to other edge data planes (Telegraf, Vector, OTAP Dataflow), and if so, can we make the project a framework instead of a collector mod?

What's in it

• §3 Current state — what's already host-neutral (sketchlib-go, sketch_core / asap_sketchlib, the wire envelope, controller plan logic) vs what's coupled to OTel today (windowing, delta apply, snapshot cache, batching all live inside Go OTel `processor.Processor` impls).
• §4 Five-layer architecture — sketch algorithms, wire format, operator runtime, host adapters, control plane.
• §5 Key abstractions — `Sample`, `Sketch`, `Operator`, `OperatorConfig`, `SketchFrame`, `Host` trait. Each with a Rust signature and a one-paragraph rationale.
• §6 Concrete host adapters — OTel Collector (refactored to ~50 LoC shim post-Phase-1), Telegraf (~300 LoC, cleanest non-OTel fit), Vector (~500 LoC Rust transform), OTAP Dataflow (~500 LoC Rust + Arrow batched-add path).
• §7 Control plane — OpAMP for OTel; HTTP-poll for everything else, leveraging the existing `GET /api/v1/plan` endpoint.
• §8 Phased migration — six phases that each keep the b3-delta e2e green. Phase 1 (Go runtime extract) ~2 weeks; Phase 3 (Telegraf) ~1 week.
• §9 Risks — sketch-algo divergence between Go and Rust runtimes; perf regression from runtime extraction; controller-side OTel-yaml coupling; OpAMP non-portability. Each with mitigation + gate.
• §10 Open questions — five concrete decisions the project needs to make before Phase 1 starts (repo layout, sketch trait at Layer 1, host field on the controller plan, operator-side cold-tier policy, wire-version policy).
• Appendix A — line-by-line mapping from today's code onto the proposed layers.
• Appendix B — why this is the right time (the runtime just got correct in fix(ddsketch): make delta_transmission actually work end-to-end #210+fix(processors): forward inputs through windowed sketch processors #211+rename "datacollector" in self-monitoring metric names when having a project name #71; OTAP Dataflow is shipping; external interest exists).

Decision required to proceed

The doc is gating the host-adapter work. The five "decision-required" checkboxes at the end need answers before Phase 1 starts.

Test plan

• Doc renders cleanly.
• Cross-references to existing code paths verified (PRs cited, file paths accurate as of `main` post-fix(processors): forward inputs through windowed sketch processors #211 / rename "datacollector" in self-monitoring metric names when having a project name #71).
• Review by stakeholders.

🤖 Generated with Claude Code

[zzylol]

Update — sketch-core retirement (per review)

Pushed 140c737. The original draft listed Layer 1 as "sketchlib-go + asap_sketchlib / sketch_core". Looking at the actual code, that's a misleading framing:

• asap_sketchlib already ships apply_delta in-crate for countmin, count, hll (mirroring sketchlib-go's per-sketch delta.go layout):

  asap_sketchlib/src/sketches/countmin.rs:397   pub fn apply_delta(&mut self, delta: CmDelta)
  asap_sketchlib/src/sketches/count.rs:799      pub fn apply_delta(&mut self, delta: CountDelta)
  asap_sketchlib/src/sketches/hll.rs:430        pub fn apply_delta(&mut self, delta: HllDelta)
  

• sketch-core has apply_delta for DDSketch + CountSketch — the two sketches asap_sketchlib doesn't have it for, plus a bunch of *_sketchlib.rs adapter wrappers that re-export asap_sketchlib's types onto backend traits.
• sketch-core lives in three on-disk copies (ASAPQuery/asap-common/, ASAPQuery-backend/asap-common/, sketchlib-bench/sketch-core/) that drift independently. Already a real source of bugs.

The split is accidental. The framework should treat asap_sketchlib as the single Rust algorithm crate, mirroring sketchlib-go on the Go side, and retire sketch-core entirely.

Doc updates:

• New §3.3 enumerates what's in sketch-core and where each file goes (algorithm-level → asap_sketchlib; backend-glue → asap-query-engine/src/precompute_operators/).
• §4 Layer 1 simplified: "sketchlib-go (Go), asap_sketchlib (Rust); sketch-core RETIRED".
• §8 gains a Phase 0.5 — Retire sketch-core, ordered before Phase 2 (Rust runtime extraction). Strict ordering: (1) land asap_sketchlib PRs adding DDSketch + CountSketch apply_delta, (2) switch ASAPQuery-backend / sketchlib-bench / ASAPQuery imports, (3) delete the three copies.
• §9 R1 risk: sketch-core retirement is itself a divergence-vector reduction.
• §12 repo layout adds the out-of-tree changes.
• §13 decision-required gets an explicit checkbox for Phase 0.5.
• Appendix A maps every sketch-core file to its destination.

Phase 0.5 is small in scope (~3 small PRs, mostly mechanical) but unblocks the bigger Phase 2 work.