test(integration/parity): all-sketch e2e parity harness - #225
Merged
Merged
Conversation
…legacy processors
Builds a deterministic multi-metric pmetric.Metrics input covering all 5
sketch domains and feeds it through both asap-precompute-go (runtime) and
the legacy OTel processors. Compares emitted SketchEnvelopes byte-for-byte.
This is the gate before refactoring 5 processors into shims (Phase 2 steps
2.5–2.9). If parity holds, shim refactors become trivial; if not, it
surfaces runtime gaps before any production-shape change.
Results:
DDSketch PASS — 6 envelopes byte-identical between runtime and legacy
KLL SKIP — legacy batch path drops resource attrs from series
key and adds `_kll` metric-name suffix
HLL SKIP — legacy batch path drops resource attrs from series
key and adds `_hll_cardinality` metric-name suffix
CountSketch SKIP — legacy emits one global partition; runtime emits
per-(resource,labelset) series
CountMinSketch SKIP — legacy keys series by (metricName, dp-attrs) only;
runtime always includes resource attrs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…5 PASS - Add OmitResourceAttrs and GlobalAggregation config knobs to honor legacy KLL/HLL/CMS/CountSketch series-key semantics. Defaults preserve today's resource-aware behavior; shims opt in. - Route SeriesKey through PrecomputeConfig.SeriesKeyFor / -ForEntry so observe-time and flush-time agree on the bucket for a given config — single source of truth. - Bake legacy metric-name suffix (`_kll`, `_hll_cardinality`, `countsketch_partition`) into the runtime's MetricName per sketch in the harness; no AdapterConfig.MetricSuffix needed. - Strip CountSketch's operator-visibility attrs (`sample_count`, `window_duration_seconds`) from the diff comparison key — they are observability hints, not part of the routing key. - Result: HLL, CountSketch, CountMinSketch flip to PASS; DDSketch unchanged. KLL remains SKIP for an out-of-scope cause: sketchlib-go's KLLSketch coin is seeded from `time.Now()` (sketches/KLL/kll.go::newCoin), so the runtime and legacy build independent sketches whose compaction patterns diverge regardless of input. Byte-parity for KLL needs a deterministic-seed API in sketchlib-go (separate PR). Closes the gate before Phase 2 shim refactors (steps 2.5–2.9) for the 4 deterministic-emit sketches; KLL gate is parked on the sketchlib-go follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Plumb seedable KLL constructor through runtime wrapper and legacy processor (Config.Seed knob, default nil = time-based for production). - Add EmitWindowStats config to runtime; CountSketch now emits sample_count and window_duration_seconds attrs natively. - Drop diff-projection strip; CountSketch parity is now byte-identical. - All 5 sketches PASS, no SKIPs. Depends on: sketchlib-go#54 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
zzylol
added a commit
that referenced
this pull request
May 4, 2026
) KLL processor reduces from ~720 LoC to ~120 LoC shim. State machine moves to asap-precompute-go. sketch_wrapper.go implements QuantileSketch over sketchlib-go KLL. Config.Seed (added in PR #225) flows through. Public test API: Shim.ProcessBatch/ProcessMetrics/FlushWindow. Parity harness: TestParity_KLL byte-identical (3 envelopes). Phase 2 step 2.6. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
End-to-end parity harness that proves wire-format invariance between the new
asap-precompute-goruntime and the 5 legacy OTel sketch processors. This is the gate before Phase 2 steps 2.5–2.9 (refactoring each processor into a thin shim that delegates to the runtime).The harness builds a deterministic multi-metric
pmetric.Metricspayload covering all five sketch domains, feeds the same payload through both pipelines, and diffs the emittedSketchEnvelopepayloads byte-for-byte.Depends on
fix(KLL): make compaction RNG seedable for byte-determinism. This PR'sreplace github.com/ProjectASAP/sketchlib-go => ../../../sketchlib-godirectives consume the seedable constructor (NewKLLSketchWithSeed) needed for KLL byte-parity. Once sketchlib-go feat: matrix aggregation (aggregate_by + label_matchers) for KLL, HLL, DDSketch processors #54 merges, thereplacelines collapse into a normal version bump.What the harness asserts
SketchEnvelope.Payload) byte-identical for every (metric, resource-labels, dp-labels) tuple. This is the ADR-0002 §"Behavior preservation" invariant.When divergence is found, the diff reporter prints, per sketch type:
How to run locally
The harness is fully synchronous: no goroutines, no wall-clock waits, no flake-prone timers. Two runs of the same harness produce byte-identical inputs and byte-identical outputs.
Results
NewKLLSketchWithSeed(k, 42)from sketchlib-go #54 + newkllprocessor.Config.Seedknob; defaultSeed=nilkeeps production behavior time-seeded).OmitResourceAttrs=true, runtime metric name<base>_hll_cardinality).GlobalAggregation=true+ newEmitWindowStats=true; runtime emitssample_countandwindow_duration_secondsnatively, no diff-side attr strip).OmitResourceAttrs=true).Closing the last two gaps in this PR
KLL — deterministic compaction seed
Until sketchlib-go #54,
KLLSketch's compaction coin was seeded fromtime.Now().UnixNano()so the runtime and legacy processors built independent random streams. The fix exposesNewKLLSketchWithSeed(k, seed)and stores the seed soClear()re-seeds deterministically across window rotations. This PR plumbs the seed through:harness.newKLLWrapper(k, HarnessKLLSeed)inintegration/parity/harness/sketches.go.kllprocessor.Config.Seed *int64knob, set to&HarnessKLLSeedinharness/legacy.go. Production deployments leaveSeed=niland observe today's time-seeded behavior.CountSketch — true byte-parity (no diff-side strip)
Previous form of this PR strip-projected
sample_countandwindow_duration_secondsout of the legacy data point because the runtime didn't emit them. This PR addsPrecomputeConfig.EmitWindowStats(default false): when true,serializeSeriesappends those two attrs onto the envelope'sLabelsat flush time, flowing throughotel/encode.go::KeyValuesToAttributesto the output data point. Only the CountSketch sketch descriptor flips it on; the other four are unchanged so their parity is unaffected.The harness's legacy
WindowDurationis now set tocfg.WindowSize(was 24h) so the legacy and runtime stamp the samewindow_duration_secondsvalue — the test runs sub-second so the legacy ticker never fires beforeShutdown.What changed in this PR
asap-precompute-go/config.go— addsOmitResourceAttrs,GlobalAggregation, andEmitWindowStatsflags toPrecomputeConfig, plusSeriesKeyFor(obs)/SeriesKeyForEntry(resourceLabels, labels)helpers (single call site for observe-time / flush-time).asap-precompute-go/precompute.go::serializeSeries— appendssample_countandwindow_duration_secondsto envelopeLabelswhenEmitWindowStatsis set; usescfg.SeriesKeyForEntryfor snapshot-cache consistency.asap-precompute-go/window.go— observe routes through the new helpers;seriesEntrystripsResourceLabels(andLabelsunderGlobalAggregation).opentelemetry-collector-contrib-patch/processor/kllprocessor/config.go— newSeed *int64config (default nil = time-based; only the parity harness sets it).opentelemetry-collector-contrib-patch/processor/kllprocessor/processor.go—newKLLSketchaccepts a*Configand dispatches toNewKLLSketchWithSeedwhencfg.Seed != nil.integration/parity/harness/runtime.go—HarnessKLLSeedconstant; per-sketch descriptor carries the config flags and nowemitWindowStats.integration/parity/harness/sketches.go—newKLLWrapper(k, seed)uses the seedable sketchlib-go constructor.integration/parity/harness/legacy.go— setskllproc.Config.Seed = &HarnessKLLSeed; alignsWindowDurationbetween CountSketch / CMS legacy and runtime.integration/parity/harness/diff.go— drops the CountSketch attr-strip (still strips legacy KLL'skll.koperator-only attribute, which lives in the data point but not the runtime envelope).integration/parity/parity_test.go— KLL is no longer SKIP; all 5 sketches must PASS.Constraints honored
rand.Source; KLL's compaction is now seeded too via the new sketchlib-go API.consumertest.Sinkconsumers.Shutdown-driven flush (CountSketch, CountMinSketch).go test -race ./integration/parity/...clean;go test -count=3 ./...clean.cd asap-precompute-go && go test ./...clean.opentelemetry-collectorandopentelemetry-gois intentionally not committed.EmitWindowStatswas added beyond the existing two flags (OmitResourceAttrs,GlobalAggregation).What this PR does NOT do
Seedknob onkllprocessor.Config(production-default unchanged).Test plan
go build ./...— clean.cd integration/parity && go test -v ./...— 5 PASS, 0 SKIP.cd integration/parity && go test -race ./...— clean.cd integration/parity && go test -count=3 ./...— clean (no flake).cd asap-precompute-go && go test ./...— clean.cd opentelemetry-collector-contrib-patch/processor/kllprocessor && go test ./...— clean (existing tests still pass after thenewKLLSketch(*Config)refactor).