Skip to content

docs(design): rename Operator/Host/Sample/SketchFrame; trait composition; phase renumber - #216

Merged
zzylol merged 1 commit into
mainfrom
docs/design-edge-framework-naming-and-design-revisions
May 2, 2026
Merged

zzylol merged 1 commit into
mainfrom
docs/design-edge-framework-naming-and-design-revisions

Conversation

@zzylol

@zzylol zzylol commented May 2, 2026

Copy link
Copy Markdown
Contributor

Summary

In-place revisions to `docs/design-asap-edge-framework.md` per naming + design audit.

Renames (collision resolutions)

Old New Reason
`Operator`, `asap-operator-{go,rs}` `Precompute`, `asap-precompute-{go,rs}` Avoids k8s-operator / DataFusion-operator collision; matches existing `precompute_engine` naming on the backend.
`Host`, `asap-host-*` `Adapter`, `asap-{otelcol,telegraf,vector,otap}` "Host" is overloaded (network / container / OTel-SDK). Drop the `host-` prefix.
`Sample` `Observation` Avoids `prometheus::Sample` / `pmetric::NumberDataPoint` collision; matches the `observe()` verb.
`SketchFrame` (proposed rename) keep `SketchEnvelope` (existing proto name) "Frame" is overloaded (link layer / video / stack / dataframe); "envelope" is the established term for tagged-union wire payloads.
`WindowMode { Batch, Window, Sliding }` `AggregationMode { Tumbling, Sliding, Batch }` Drop the `Window` variant inside a `Window`-prefixed enum.

Design changes

  • `Sketch` trait split into `Sketch` + `QuantileSketch` + `CardinalitySketch` + `FrequencySketch`. Each impl only exposes queries it actually answers — matches `asap_sketchlib`'s current shape (KllSketch::quantile, HllSketch::estimate, CountMinSketch::estimate).
  • Adapters emit raw `Observation`s. `agg_id` resolution moves into the runtime (was duplicated across host adapters); single source of truth.
  • One `Precompute` instance per sketch type, generic over `SketchT`. Matches today's per-processor shape.
  • Crash recovery deferred to a future `PersistentPrecompute: Precompute` trait; not in Phase 2 scope.
  • New `ControlChannel` trait abstracts OpAMP / HTTP-poll / FileWatch (§5.7, §7).
  • Backpressure config: `max_series` + `OnOverflow` enum. Needed once runtime is extracted from OTel pipeline backpressure.

Phasing

  • Phase 0.5 (sketch-core retirement, already shipped) moved out of the design doc into `adr-0001-retire-sketch-core.md` (to be authored). Doc stays forward-looking.
  • Phases renumbered to 1–7 (no decimals): lock design → extract Go → extract Rust → Telegraf → OTAP → Vector → optional rename to `asap-edge`.

R1 mitigation rewrite

Old: cross-language byte-corpus comparison.
New: cross-language statistical-output comparison (P50/P90/P99/count/sum/cardinality/top-K within each sketch's published error bound). Byte corpus is invalid because hash seeds and storage growth strategies legitimately differ across runtimes (we discovered this during the wire-format alignment audit). Harness to live in `sketchlib-bench`.

Stats

  • 822 → 945 lines (+123).
  • All cross-references updated (§5.* / §6.* / §8 phase numbers / §9 R1+R2+R4 / §10 / §12 / §13 / Appendix A).

Out of scope

  • Adding the actual ADR files (`adr-0001`, `adr-0002`, `adr-0003`) — separate PRs.
  • Code changes — this is forward-looking design only.

@zzylol
zzylol force-pushed the docs/design-edge-framework-naming-and-design-revisions branch 6 times, most recently from 6532ed6 to 20bf4f6 Compare May 2, 2026 03:49
…ion; phase renumber

In-place revisions to the edge precompute framework design doc per
naming and design audit:

NAMING — collisions resolved:
- Operator → Precompute (avoids k8s-operator / DataFusion-operator
  confusion; matches existing `precompute_engine` naming on backend).
- asap-operator-{go,rs} → asap-precompute-{go,rs}.
- Host → Adapter; asap-host-* → asap-{otelcol,telegraf,vector,otap}
  (drop host- prefix; "Host" is overloaded).
- Sample → Observation (avoids prometheus::Sample / pmetric collision;
  matches observe() verb).
- SketchFrame → keep SketchEnvelope (existing proto name; "Frame" is
  overloaded; "envelope" is the standard term for tagged-union wire
  payloads).
- WindowMode { Batch, Window, Sliding } → AggregationMode { Tumbling,
  Sliding, Batch } (drop Window variant inside Window-prefixed enum).

DESIGN — abstractions sharpened:
- Sketch trait split into Sketch + QuantileSketch + CardinalitySketch
  + FrequencySketch (each sketch only exposes queries it can answer;
  matches asap_sketchlib current shape).
- Adapters emit raw Observations; runtime resolves agg_id via matchers
  internally (single source of truth, simpler adapter contract).
- One Precompute instance per sketch type (matches today's per-processor
  shape); generic over SketchT instead of dyn dispatch.
- Crash recovery (snapshot/restore) deferred to future
  PersistentPrecompute trait; not in Phase 2 scope.
- New ControlChannel trait abstracts OpAMP / HTTP-poll / FileWatch
  (§5.7 + §7).
- New backpressure config: max_series + OnOverflow (Drop | Block |
  EvictOldest); needed once runtime is extracted from OTel pipeline.

PHASING — renumbered, ADR landing:
- Phase 0.5 (sketch-core retirement, already shipped) moved to
  adr-0001-retire-sketch-core.md; design doc stays forward-looking.
- Phases 1-7 (no decimals): lock design → extract Go → extract Rust
  → Telegraf → OTAP → Vector → optional rename to asap-edge.
- R1 mitigation rewritten: cross-language *statistical-output*
  comparison (P50/P90/cardinality within sketch error bound), not
  byte-corpus (byte corpus is invalid because hash seeds and storage
  growth strategies legitimately differ).

Length: 822 → 945 lines (+123). All cross-references updated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol force-pushed the docs/design-edge-framework-naming-and-design-revisions branch from 20bf4f6 to e444ba2 Compare May 2, 2026 03:51
@zzylol
zzylol merged commit e96f0f9 into main May 2, 2026
@zzylol
zzylol deleted the docs/design-edge-framework-naming-and-design-revisions branch May 2, 2026 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant