fix(precompute): factory routes CountSketch[WithHeap] to CmsAccumulator - #261
Merged
Merged
Conversation
The precompute accumulator factory was silently falling through to the catch-all `tracing::warn! + SumAccumulatorUpdater` default for both `AggregationType::CountSketch` and `AggregationType::CountSketchWithHeap`. Any raw-input ingest path (precompute worker + sketch_db backfill window_builder) hitting one of those policies would produce a Sum accumulator instead of a frequency sketch — silently wrong results. Route both to `CmsAccumulatorUpdater` (which already handles the shared `(rows, cols)` matrix shape) for parity with the existing `CountMinSketch | CountMinSketchWithHeap` arm at line 729. This is the correctness floor: a registered policy yields a working accumulator, not a Sum default. Limitation called out in the doc comment: like the existing `CountMinSketchWithHeap` arm, this drops the per-policy top-k heap. Dedicated `CountSketchAccumulatorUpdater` / `CountMinSketchWithHeapAccumulatorUpdater` impls (the accumulator structs exist but lack the updater wrapper) are tracked as follow-up — the present arm only matters for Mode 2 / raw-input ingest, which the OTLP sketch-envelope path doesn't exercise. `config_is_keyed` updated correspondingly (the four new variants are all multi-population by shape, parallel to CountMinSketch), plus a test covering all four arms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes a silent correctness bug surfaced by the audit in PR #259's follow-up. The precompute accumulator factory was falling through to the catch-all
tracing::warn! + SumAccumulatorUpdaterdefault for bothAggregationType::CountSketchandAggregationType::CountSketchWithHeap. Any raw-input ingest path hitting one of those policies would produce a Sum accumulator instead of a frequency sketch.What changed
CountSketch | CountSketchWithHeaproutes toCmsAccumulatorUpdater(parallel to the existingCountMinSketch | CountMinSketchWithHeaparm). Both families share the(rows, cols)matrix shape, so the same updater works as a correctness floor.config_is_keyedlists the four CountSketch / CMS variants now (was missingCountMinSketchWithHeap,CountSketch,CountSketchWithHeap).test_config_is_keyedcovers all four arms.Limitation (called out in code comment)
This drops the per-policy top-k heap — same limitation as the existing
CountMinSketchWithHeaparm. The dedicated heap-bearing updaters (CountSketchAccumulatorUpdater,CountMinSketchWithHeapAccumulatorUpdater) are tracked as follow-up. The accumulator structs exist (count_sketch_accumulator.rs,count_min_sketch_with_heap_accumulator.rs) but lack theAccumulatorUpdaterwrapper trait impl.The OTLP modified-sketch wire-format path uses
SketchEnvelopeingest (not raw), so this arm only matters for Mode 2 / raw-input ingest — not the gateway-less path that today's tests exercise.Test plan
cargo test --workspace --lib— 1549 pass, 0 fail🤖 Generated with Claude Code