Skip to content

asap-precompute-go: emit DDSketch relative_accuracy on the wire (fixes empty warm quantile queries) - #469

Merged
zzylol merged 1 commit into
feat/sum-aggregation-typefrom
fix/ddsketch-relative-accuracy-on-wire
May 29, 2026
Merged

zzylol merged 1 commit into
feat/sum-aggregation-typefrom
fix/ddsketch-relative-accuracy-on-wire

Conversation

@zzylol

@zzylol zzylol commented May 29, 2026

Copy link
Copy Markdown
Contributor

Problem

The fused asap_edge warm path emits DDSketch envelopes via oteladapter.Encode, but the encoder never set the output pmetric.DDSketch container's relative_accuracy — unlike the standalone ddsketchprocessor (shim_helpers.go SetRelativeAccuracy). So every fused-edge DDSketch frame shipped relative_accuracy=0.0, and the backend registered the sid as a degenerate ε=0 sketch.

Impact (root-caused live on the Google-cluster trace): the data-plane query router keys backend selection on the accuracy bound. An ε=0 DDSketch reports an exact accuracy bound, so quantile_over_time(...) is routed to the archive (Gorilla/Thanos) instead of the warm SketchStore — and returns empty (kind=exact, no result). The controller policy correctly declared alpha=0.02, but it never reached the wire.

Fix

Thread the alpha from the sketch instance through to the emitted frame:

  • DDSketchWrapper.RelativeAccuracy() exposes the configured alpha (sketches/ddsketch.go).
  • SketchEnvelope gains an in-process RelativeAccuracy field, populated in serializeSeries from the sketch (0 for non-DDSketch families) (envelope.go, precompute.go).
  • otel/encode.go sets dst.SetRelativeAccuracy(env.RelativeAccuracy) on the DDSketch container when > 0.

Tests

otel/relative_accuracy_integration_test.go drives the real DDSketchWrapper through observe → Drain → Encode and asserts alpha=0.02 is preserved on the envelope, the in-memory pmetric.DDSketch, AND a full OTLP proto marshal→unmarshal round-trip — on both the full-snapshot and delta-transmission paths.

End-to-end validation (Google-cluster trace, multinode)

With the fix, a warm p99 query resolves on the SketchStore (not the archive):

quantile_over_time(0.99, gct_ms_ddsketch[2m])  →  status success, data_source=asap_query, latency 2.6ms
  zone z0: 0.04878   zone z1: 0.04878   zone z2: 0.04878   zone z3: 0.05077

Accuracy vs exact offline ground truth — 1.69–2.05% relative error, matching the DDSketch ε=0.02 SLA:

zone GT p99 warm p99 rel err
z0 0.04962 0.04878 1.69%
z1 0.04981 0.04878 2.05%
z2 0.04974 0.04878 1.93%
z3 0.04993 0.05077 1.69%

Before the fix the same query returned empty (routed to archive as kind=exact).

Notes / follow-ups (out of scope)

  • Controller emit currently honors only DDSketch/KLL sketch_family_override; HLL/CMS/CountSketch workload entries are emitted to the agent as ddsketch — a separate controller-emit gap (worked around here with a static 5-family agent config for validation).
  • Stacked on feat/sum-aggregation-type; retarget to main after that merges.

🤖 Generated with Claude Code

… OTLP frame

The fused asap_edge warm path emits DDSketch envelopes via
oteladapter.Encode, but the encoder never set the output
pmetric.DDSketch container's relative_accuracy — unlike the standalone
ddsketchprocessor (shim_helpers SetRelativeAccuracy). So every fused-edge
DDSketch frame shipped relative_accuracy=0.0, and the backend registered
the sid as a degenerate ε=0 sketch. A ε=0 DDSketch can't produce bucketed
quantiles, so `quantile_over_time(...)` capability-misses to the archive
and returns empty (observed live: all google_cluster DDSketch sids
registered with relative_accuracy 0.0 while the controller policy
correctly declared alpha=0.02).

Thread the alpha from the sketch instance through to the wire:
- DDSketchWrapper.RelativeAccuracy() exposes the configured alpha.
- SketchEnvelope gains an in-process RelativeAccuracy field, populated in
  serializeSeries from the sketch (0 for non-DDSketch families).
- otel/encode.go sets dst.SetRelativeAccuracy(env.RelativeAccuracy) on the
  DDSketch container when > 0.

Verified by relative_accuracy_integration_test.go through the real
DDSketchWrapper: envelope, in-memory pmetric, AND a full OTLP proto
marshal->unmarshal round-trip all preserve alpha=0.02, on both the
full-snapshot and delta-transmission paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 0f555a8 into feat/sum-aggregation-type May 29, 2026
@zzylol
zzylol deleted the fix/ddsketch-relative-accuracy-on-wire branch May 29, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant