feat(sketches): per-metric sample_p knob on the CMS + HLL warm processors - #441
Merged
Merged
Conversation
…sors Plumb a per-sketch sampling probability through the agent so an operator can activate the recently added warm-sketch sampling per metric. The CMS and HLL precompute wrappers gain a WithSampleP fluent setter that calls sketchlib-go's CountMinSketch.WithSampleP / HyperLogLog.WithSampleP and preserves the probability across the Reset / Merge / ApplyDelta re-construction paths. The countminsketch and hll processors expose a sample_p config field (mapstructure: "sample_p") that the control plane sets per metric and pass it into wrapper construction. Default 1.0 everywhere (0/unset normalises to 1.0 in Config.Validate, which rejects out-of-range values) is an exact no-op, so an agent with no sample_p configured emits byte-identical wire bytes to today. Adds a sketches test proving a configured p<1 reaches the underlying sketchlib-go builder (read off the owned sketch, not just the wrapper field) and thins admitted updates, while the default 1.0 is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
3 tasks
zzylol
added a commit
that referenced
this pull request
May 26, 2026
… crash) (#444) The fused asap_edge processor's per-metric config struct did not define sample_p, so the collector's strict mapstructure decode rejected the key the control plane emits when a metric's workload sample_p < 1, crash-looping the agent. Add SampleP to MetricFamily (validated to (0,1], 0/unset => 1.0 = no sampling) and wire it into the warm sketch-build: HLL and CountMinSketch factories call sketchlib-go's WithSampleP(p). WithSampleP(1.0) is an exact no-op, so the unset/1.0 path stays byte-identical to today. DDSketch/KLL/ CountSketch (no sampling support) ignore it. Mirrors the standalone hll/countminsketch processors' sample_p wiring (#441) for the fused path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Plumb a per-sketch sampling probability through the EDGE agent so an operator can activate the recently merged warm-sketch sampling per metric.
asap-precompute-go/sketches: the CMS and HLL wrappers gain aWithSampleP(p)fluent setter that calls sketchlib-go'sCountMinSketch.WithSampleP(p, seed)/HyperLogLog.WithSampleP(p), and preserve the probability across theReset/Merge/ApplyDeltare-construction paths (centralised in anewSketch()helper).countminsketchprocessor+hllprocessor: a newsample_pconfig field (mapstructure:"sample_p");Config.Validatenormalises0/unset to1.0and rejects out-of-range values; the processor passes it into wrapper construction.Default
1.0everywhere is an exact no-op (sketchlib-go never touches the RNG atp>=1), so an agent with nosample_pconfigured emits byte-identical wire bytes to today.Sampling API used (sketchlib-go #59)
CountMinSketch.WithSampleP(p float64, seed int64)— geometric admission.HyperLogLog.WithSampleP(p float64)— hash-threshold element sampling.Test plan
go build ./...green forasap-precompute-go,countminsketchprocessor,hllprocessor.go test ./...green forasap-precompute-go(incl. newsketches/sampling_test.go).p<1reaches the underlying sketchlib-go builder (read off the ownedsk, not just the wrapper field) and thins admitted updates; default1.0unchanged.NOTE:
countminsketchprocessor/hllprocessorgo test(the full package) currently fails to compile onmaindue to a pre-existing stale-test vspdataskew (processor_test.goreferences removeddp.SampleCount/Rows/Cols/Cardinality/Precisionfields) — unrelated to this PR, which only touchesconfig.go+processor.go. The productiongo buildis green and the reaches-builder proof lives in theasap-precompute-go/sketchespackage, which tests green.Merge order
Merge this PR first. The companion control-plane PR (ProjectASAP/ASAPQuery-backend
feat/sample-p-control-plane) emits thesample_pagent config and depends on these processor fields existing.Out of scope / follow-up
Dynamic, optimizer-driven
p(tuned online against an accuracy/bandwidth budget) — this PR ships only the static per-metric knob.🤖 Generated with Claude Code