fix(processors): unbreak baselines - DataDog->sketchlib-go fixtures + post-#211 sink assertions - #220
Merged
Merged
Conversation
…post-#211 sink assertions Why: - DataDog/sketches-go is no longer a runtime dep (production switched to sketchlib-go in #210). Tests that still imported DataDog/sketches-go to build sketch fixtures were leftover. - After #211, window-mode ConsumeMetrics forwards raw inputs through nextConsumer so chained windowed sketch processors can see the original payload. Tests written before #211 expected sink.AllMetrics() to be empty after ConsumeMetrics; they now observe both the input pass-through AND the synthesized output emitted by flushWindow. Update assertions to count both and scan-by-name where ordering is non-deterministic. - KLL/CountSketch/CountMinSketch processors imported go.opentelemetry.io/collector/processor/selfmonitor (the local fork's selfmonitor package) but the corresponding go.mod was missing the `replace` directive that points go.opentelemetry.io/collector/processor at the local fork. Add the replace; drop the now-unused gob deserializer in two test sites where the test was reading proto-encoded emit bytes. Per-processor: - ddsketchprocessor: fix 5 window-mode tests + concurrent test; treat sink as [forwarded inputs..., synthesized flush output]. - kllprocessor: add `replace go.opentelemetry.io/collector/processor` to go.mod, fix 5 window-mode test assertions, swap DeserializeKLLSketchFromBytes (gob) → DeserializeKLLSketchFromProtoBytes (proto) in TestBatchModeTransmitSketch to match emit format. - hllprocessor: fix 2 window-mode test assertions (selfmonitor was already resolvable). - countsketchprocessor: add the `replace` directive only — tests already accounted for pass-through. - countminsketchprocessor: add the `replace` directive, swap DeserializeCountMinSketchFromBytes (gob) → DeserializeCountMinSketchFromProtoBytes (proto) to match SerializeProtoBytesFO emit path. All 5 packages pass `go test ./... -count=1` and `go build ./...`. Full sketchcollector build via build_sketchcollector.sh succeeds. gofmt is clean for all files modified by this change; pre-existing gofmt debt in unrelated files left untouched. Submodule pointer drift on opentelemetry-collector and opentelemetry-go was left unstaged (pre-existing noise). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Restores green baselines across all 5 OTel sketch processors (ddsketch, kll, hll, countsketch, countminsketch) so refactor work can proceed without inheriting pre-existing debt.
Why
Three classes of pre-existing test failure, one per cluster of processors:
DataDog/sketches-go fixture leftover (ddsketch). Production switched to sketchlib-go in fix(ddsketch): make delta_transmission actually work end-to-end #210; tests still imported DataDog/sketches-go to build sketch fixtures. Done in a prior commit (already on this branch): import swap to
github.com/ProjectASAP/sketchlib-go/sketches/DDSketchandproto/sketch_envelope;setSketchPayloadusesddsketch.NewDDSketch + Update + SerializeStateProtoBytes;decodeSketchtries envelope first then bare-state fallback.Post-fix(processors): forward inputs through windowed sketch processors #211 window-mode pass-through. After PR fix(processors): forward inputs through windowed sketch processors #211, window-mode
ConsumeMetricsforwards raw inputs throughnextConsumerso chained windowed sketch processors (processors: [kll, hll, batch]) can see the original payload. Tests written before fix(processors): forward inputs through windowed sketch processors #211 expectedsink.AllMetrics()to be empty afterConsumeMetrics; they now observe both the input pass-through AND the synthesized output emitted byflushWindow. Updated assertions to countN inputs + 1 flush = N+1entries and scan-by-name where ordering is non-deterministic (concurrent test).selfmonitorimport + go.mod replace missing (kll, countsketch, countminsketch). These processors importgo.opentelemetry.io/collector/processor/selfmonitor, a package that lives in the local fork atopentelemetry-collector/processor/selfmonitorbut is not in any upstream tag. The corresponding go.mod was missing thereplace go.opentelemetry.io/collector/processor => ../../../opentelemetry-collector/processordirective that ddsketchprocessor already had. Added the replace;go mod tidyresolves cleanly afterward.A bonus issue surfaced for kll/countminsketch: production emits sketch payloads via proto envelopes (
SerializePortable + proto.Marshalfor KLL,SerializeProtoBytesFOfor CMS), but two tests calledDeserialize<sketch>FromBytes(gob-encoded) — which mismatches the wire format and panics withgob: duplicate type received. Switched the tests toDeserialize<sketch>FromProtoBytesto match the emit path.Scope
Per the user-supplied plan, this PR does NOT touch:
opentelemetry-collector-contrib/(unmodified upstream mirror)telegraf-patch/plugins/aggregators/ddsketch/(separate DataDog cleanup; out of scope)cmd/telemetrygen/(separate DataDog cleanup; out of scope)asap-precompute-go/(already merged in feat(asap-precompute-go): bootstrap module + types + window + snapshot cache + OTel adapter (Phase 2 steps 2.1-2.4) #219)Submodule pointer drift on
opentelemetry-collectorandopentelemetry-gowas left unstaged (pre-existing noise).Per-processor results
(gofmt issues remain in files I did not touch — pre-existing debt, out of scope.)
Full collector build via
./build_sketchcollector.shsucceeds end-to-end: builder producescmd/sketchcollector/sketchcollectorbinary.Files modified per processor
processor_test.go(5 window-mode tests + concurrent test → expect input pass-through + flush; index synthesized output by name where ordering is non-deterministic),go.mod/go.sum(DataDog dep removed in prior commit on this branch).processor_test.go(5 window-mode tests fixed; gob→proto deserializer inTestBatchModeTransmitSketch),go.mod(addedreplace go.opentelemetry.io/collector/processor).processor_test.go(2 window-mode tests fixed),go.mod/go.sum(no replace needed;go mod tidyrebalanced direct vs. indirect).go.mod/go.sumonly — addedreplace go.opentelemetry.io/collector/processor. Tests already accounted for pass-through.processor_test.go(gob→proto deserializer inTestProcessor_TumblingWindow_Correctness),go.mod(added thereplace).Test plan
cd <each-processor>thengo mod tidy && go build ./... && go test ./... -count=1— all 5 pass../build_sketchcollector.sh— succeeds.out[0]).