Skip to content

mvp phase ε.1: three-mode cost model with native OTLP to Prometheus (Mode 3) - #319

Merged
zzylol merged 3 commits into
mainfrom
mvp/phase-eps.1-three-mode-otlp
May 7, 2026
Merged

zzylol merged 3 commits into
mainfrom
mvp/phase-eps.1-three-mode-otlp

Conversation

@zzylol

@zzylol zzylol commented May 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Phase ε.1 collapses the planner's raw-vs-sketch + edge-vs-backend axes
into a single tri-mode BindMode selector. All three modes use OTLP
on the wire
— Mode 3 ships raw OTLP directly to Prometheus's native
OTLP receiver at /api/v1/otlp/v1/metrics (Prom 2.47+, activated with
--web.enable-otlp-receiver).

Three modes:

  • Mode 1 (SketchAtEdge) — sketch processor at edge ships sketch state
    (default for high samples_per_window).
  • Mode 2 (RawAtEdge_SketchAtBackend) — raw OTLP at edge, backend
    builds sketch at ingest (when edge_cpu_budget is tight). Phase ε.2
    implements the raw-input ingest path.
  • Mode 3 (RawAtEdge_PrometheusArchive) — raw OTLP straight to
    Prometheus; backend HTTP-forwards queries to /api/v1/query. Phase
    ε.2 wires the prometheus_remote engine on the backend.

Changes

  • planner::wire_cost — DELTA-encoded per-sketch wire-cost table,
    break-even-samples math, three-mode BindMode with the Phase-ε.1
    selection rules.
  • SketchExpr::RawAtEdgeSketchAtBackend and
    SketchExpr::RawAtEdgePrometheusArchive variants + allocator /
    emitter walks.
  • BackendAggregation::aggregation_input (sketch_envelope vs raw) and
    EdgeStageConfig::prometheus_archive_metrics.
  • emit_edge_yaml adds an otlphttp/prometheus exporter +
    routing processor when Mode 3 metrics are present.
  • emit_backend_storage_routing_with_prometheus emits
    engine: prometheus_remote for Mode 3 metrics.
  • deploy/configs/prometheus-otlp-receiver.yml and
    deploy/docker-compose/mvp-prometheus-archive.yml for the deploy
    side.

Break-even table (50 B per raw sample)

Sketch family State (B) Break-even (samples/window)
DDSketch+delta 600 16
KLL (full) 3,000 64
HLL+delta 10,000 204
Count-Min+delta 4,000 84
CountSketch+delta 250,000 5,004

KLL has no delta variant (per Implementation.tex — randomised
compaction is not additively mergeable).

Test plan

  • cargo test --release -p controller — 533 passed (vs. 516 baseline =
    17 new tests pass), 10 pre-existing failures unchanged.
  • All wire_cost / stage_config Phase ε.1 tests green
    (cargo test --release phase_eps1 → 6/6 ok;
    cargo test --release planner::wire_cost → 11/11 ok).
  • Step-2.4 demo runs verifying Mode 3 e2e against a live Prometheus
    with --web.enable-otlp-receiver (separate PR).

Notes for Phase ε.2

  • Engine ID: prometheus_remote.
  • Query URL: ${ASAP_PROMETHEUS_QUERY_URL:-http://prometheus-archive:9090}/api/v1/query.
  • Expected forwarder shape: pass-through of PromQL string + time (or
    start/end/step for range queries) → Prometheus → re-emit JSON
    result back to the backend's HTTP query handler.
  • BackendAggregation::aggregation_input == Raw requires an OTLP-raw
    ingest path on OtlpReceiver that builds the sketch from samples on
    arrival (Mode 2; out of scope for ε.1).

🤖 Generated with Claude Code

zzylol and others added 3 commits May 7, 2026 17:58
Phase ε.1 collapses the planner's raw-vs-sketch + edge-vs-backend axes
into a single tri-mode selector. Adds:

- `planner::wire_cost` — per-sketch DELTA-encoded wire-cost table,
  break-even-samples math, and `BindMode` selection rules.
  Resulting break-evens at 50 B per raw sample: DDSketch+delta=16,
  KLL=64, HLL+delta=204, Count-Min+delta=84, Count-Sketch+delta=5,004.
- `SketchExpr::RawAtEdgeSketchAtBackend` (Mode 2) and
  `SketchExpr::RawAtEdgePrometheusArchive` (Mode 3) variants.
- Allocator coloring + emitter walk for both new variants.
- `BackendAggregation::aggregation_input` (sketch_envelope vs raw)
  and `EdgeStageConfig::prometheus_archive_metrics` carried through.

KLL has no delta variant per Implementation.tex — it ships full state;
documented in the cost-table doc string.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the L5 emitters for the three-mode placements:

- `emit_edge_yaml` adds an `otlphttp/prometheus` exporter pointing at
  Prometheus's native OTLP receiver (`/api/v1/otlp/v1/metrics`) when
  `prometheus_archive_metrics` is non-empty. A `routing` processor
  dispatches per-metric on `attributes["asap.mode"]` between the
  warm-tier pipeline (sketch processors → otlp/backend) and the
  archive pipeline (passthrough → otlphttp/prometheus).
- `emit_backend_config_json` surfaces `aggregationInput` so the
  backend's StreamingConfig consumer knows whether to expect sketch
  envelopes (Mode 1) or raw OTLP samples to sketch at ingest (Mode 2).
- `emit_backend_storage_routing_with_prometheus` adds Mode-3 metrics
  with a single `engine: prometheus_remote` target — Phase ε.2
  registers the engine on the backend HTTP query path.

Edge-side YAML uses OTLP everywhere on the wire (no
prometheusremotewrite). Endpoint is `${ASAP_PROMETHEUS_OTLP_URL}`-
overridable for the deploy team.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the deploy-side artefacts for Mode 3 (RawAtEdgePrometheusArchive):

- `deploy/configs/prometheus-otlp-receiver.yml` — Prometheus 2.47+
  config for native OTLP ingest. Sets `out_of_order_time_window: 30m`
  to match the existing Thanos overlay's tolerance, and
  `promote_resource_attributes` for the standard OTel resource fields
  the `prometheus_remote` engine's PromQL queries rely on.

- `deploy/docker-compose/mvp-prometheus-archive.yml` — overlay that
  brings up Prometheus with `--web.enable-otlp-receiver` AND
  `--enable-feature=otlp-deltatocumulative`. The latter is required so
  agents don't need a separate cumulativetodelta processor before
  exporting OTel-SDK delta-encoded counters.

Step-2.4 demo runs that want to exercise Mode 3 should layer this
overlay on top of `mvp-multi-stage.yml`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 413147b into main May 7, 2026
@zzylol
zzylol deleted the mvp/phase-eps.1-three-mode-otlp branch May 9, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant