feat(gos): CountMinSketch insert-time delta gate (T=εN/k) - #525
Merged
Merged
Conversation
Converts CountMinSketch to the same insert-time GOS mechanism CountSketch already has (design-gos-unified-edge-telemetry.md §11): checks each just-touched cell's magnitude against a closed-form threshold at insert time instead of a periodic decode-prev-diff sub-window scan, resetting crossed cells in place and waking the flush loop immediately. CMS's closed form is simpler than CountSketch's: T=ε·N/k (derivations §8.2), an L1-scale, max-composition bound with no water-filling and no √(dw) factor (CMS's point query is a min over d cells, so staleness is bounded by the single worst stale cell). N (current total mass) comes from the sketch's own incrementally-tracked per-row L1 accumulator (CM_L1, O(rows)), which sketchlib-go's InsertWithHashGOS now keeps correct under in-place resets the same way L2 already was — sketchlib-go commit b28a39579eeb5447284ced3cf29574ec7f33008b (ProjectASAP/sketchlib-go@feat/gos-cms-primitive). - asap-precompute-go/sketches/cms.go: CMSWrapper gets SetGosMode, gosDirty/gosWake/ConsumeWakeSignal, drainGosDelta (the ComputeDeltaAgainst branch that drains the dirty list instead of decoding prev), and GosDeltaThreshold/currentMass (CM_L1-backed). - asap-precompute-go/sketches/gos_threshold.go: adds CMSIsotropicThreshold (T=ε·N/k), alongside CountSketch's F2IsotropicThreshold. - asap-precompute-go/precompute.go: subWindowShouldEmit/subWindowMarkEmitted get a CountMinSketch case analogous to the existing CountSketch one (applyGosMode needed no change — already generic via a structural interface assert). - opentelemetry-collector-contrib-patch/processor/asapedgeprocessor/ warm_sketch.go: the CMS factory primes SetGosMode at series creation (subWindowEnabled's GosDeltaEpsilon decoupling was already family-agnostic, no change needed there). - config_validate.go: gos_delta_epsilon now also accepts family=countminsketch (no emit_heap exclusion needed — CMS has no heap variant). - go.mod (asapedgeprocessor): repoints the sketchlib-go replace at this worktree's own sketchlib-gos-cms fork (was resolving to the shared, unconverted /mydata/sketchlib-go checkout), mirroring asap-precompute-go/go.mod's existing replace. Tests: sketchlib-go-level (gos_test.go, prior commit), wrapper-level (cms_gos_test.go: empty-drain-nil, wake-fires-once, telescoping via ComputeDeltaAgainst+ApplyDelta), and processor-level end-to-end (gos_cms_test.go: ConsumeMetrics alone, with SubWindowInterval unset and WindowDuration long, produces a flush with no manual wakeSubWindow call). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
3 tasks
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the GOS per-family stack (base: CountSketch
split/pr-gos-countsketch/ #524). Converts CMS's delta gating to insert-time detect+reset (design §11). FormulaT=εN/k— max-composition, no√(dw)(derivations §8.2); N read from the sketch's existing incremental per-rowL1accumulator (CM_L1(), min-across-rows), decremented on GOS resets — no new tracking field. Depends on sketchlib-go #74.Wiring mirrors CountSketch:
SetGosMode/dirty-list/wake-signal/drainGosDeltaincms.go,CMSIsotropicThresholdingos_threshold.go,SketchTypeCountMinSketchbranches insubWindowShouldEmit/subWindowMarkEmitted, factory priming at series creation,gos_delta_epsilonaccepted forfamily=countminsketch.applyGosModeneeded no change (already generic).Known interaction with sampling PR #518 (not a blocker for THIS PR, flagged for #518's reattachment): #518 adds row-sampled insert paths (
InsertWithHashSampledPerRowandApplyAdmittedOccurrence→InsertWithHashAtRows) that bypass the GOS threshold check, so a CMS with BOTHsample_p<1/row-admission ANDgos_delta_epsilon>0would silently skip insert-time detection. Composing them needs GOS-aware sampled-insert primitives (or explicit mutual-exclusion) — #518's job when it rebases onto this stack. Estimates stay unbiased; only the GOS mechanism is lost for sampled sketches.Test plan
ComputeDeltaAgainst+ApplyDelta.TestGOSCMSInsertWakesFlush—SubWindowIntervalunset,WindowDuration=1h,ConsumeMetricsalone produces a flush (no manualwakeSubWindow).go build/go vet/go testclean in asap-precompute-go + processor.🤖 Generated with Claude Code