asap-precompute-go: emit DDSketch relative_accuracy on the wire (fixes empty warm quantile queries) - #469
Merged
zzylol merged 1 commit intoMay 29, 2026
Conversation
… OTLP frame The fused asap_edge warm path emits DDSketch envelopes via oteladapter.Encode, but the encoder never set the output pmetric.DDSketch container's relative_accuracy — unlike the standalone ddsketchprocessor (shim_helpers SetRelativeAccuracy). So every fused-edge DDSketch frame shipped relative_accuracy=0.0, and the backend registered the sid as a degenerate ε=0 sketch. A ε=0 DDSketch can't produce bucketed quantiles, so `quantile_over_time(...)` capability-misses to the archive and returns empty (observed live: all google_cluster DDSketch sids registered with relative_accuracy 0.0 while the controller policy correctly declared alpha=0.02). Thread the alpha from the sketch instance through to the wire: - DDSketchWrapper.RelativeAccuracy() exposes the configured alpha. - SketchEnvelope gains an in-process RelativeAccuracy field, populated in serializeSeries from the sketch (0 for non-DDSketch families). - otel/encode.go sets dst.SetRelativeAccuracy(env.RelativeAccuracy) on the DDSketch container when > 0. Verified by relative_accuracy_integration_test.go through the real DDSketchWrapper: envelope, in-memory pmetric, AND a full OTLP proto marshal->unmarshal round-trip all preserve alpha=0.02, on both the full-snapshot and delta-transmission paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The fused
asap_edgewarm path emits DDSketch envelopes viaoteladapter.Encode, but the encoder never set the outputpmetric.DDSketchcontainer'srelative_accuracy— unlike the standaloneddsketchprocessor(shim_helpers.goSetRelativeAccuracy). So every fused-edge DDSketch frame shippedrelative_accuracy=0.0, and the backend registered the sid as a degenerate ε=0 sketch.Impact (root-caused live on the Google-cluster trace): the data-plane query router keys backend selection on the accuracy bound. An ε=0 DDSketch reports an
exactaccuracy bound, soquantile_over_time(...)is routed to the archive (Gorilla/Thanos) instead of the warm SketchStore — and returns empty (kind=exact, no result). The controller policy correctly declaredalpha=0.02, but it never reached the wire.Fix
Thread the alpha from the sketch instance through to the emitted frame:
DDSketchWrapper.RelativeAccuracy()exposes the configured alpha (sketches/ddsketch.go).SketchEnvelopegains an in-processRelativeAccuracyfield, populated inserializeSeriesfrom the sketch (0 for non-DDSketch families) (envelope.go,precompute.go).otel/encode.gosetsdst.SetRelativeAccuracy(env.RelativeAccuracy)on the DDSketch container when > 0.Tests
otel/relative_accuracy_integration_test.godrives the realDDSketchWrapperthrough observe → Drain → Encode and asserts alpha=0.02 is preserved on the envelope, the in-memorypmetric.DDSketch, AND a full OTLP proto marshal→unmarshal round-trip — on both the full-snapshot and delta-transmission paths.End-to-end validation (Google-cluster trace, multinode)
With the fix, a warm p99 query resolves on the SketchStore (not the archive):
Accuracy vs exact offline ground truth — 1.69–2.05% relative error, matching the DDSketch ε=0.02 SLA:
Before the fix the same query returned empty (routed to archive as
kind=exact).Notes / follow-ups (out of scope)
sketch_family_override; HLL/CMS/CountSketch workload entries are emitted to the agent asddsketch— a separate controller-emit gap (worked around here with a static 5-family agent config for validation).feat/sum-aggregation-type; retarget tomainafter that merges.🤖 Generated with Claude Code