Skip to content

refactor(ddsketchprocessor): thin shim delegating to asap-precompute-go (Phase 2 step 2.5) - #227

Merged
zzylol merged 1 commit into
mainfrom
phase2/ddsketch-shim
May 4, 2026
Merged

zzylol merged 1 commit into
mainfrom
phase2/ddsketch-shim

Conversation

@zzylol

@zzylol zzylol commented May 2, 2026

Copy link
Copy Markdown
Contributor

Summary

DDSketch processor reduces from ~942 LoC to a ~178 LoC shim
(processor.go) that delegates the windowing, snapshot caching, and
delta encoding state machine to the asap-precompute-go runtime
extracted in PR #219 (Phase 2 steps 2.1–2.4). The shim's responsibility
shrinks to what it actually owns:

  • OTel-side lifecycle (Start / Shutdown / ConsumeMetrics) and
    the window-mode tick goroutine.
  • Translating Configprecompute.PrecomputeConfig.
  • Quantile materialization when TransmitSketch=false (decode the
    runtime's envelope back to a DDSketch and emit Gauge per-quantile
    data points). The Phase-2.5 spec keeps this on the shim because
    otel.Encode does not reconstruct sketches from payload bytes.

Auxiliary helpers (sketch wrapper, mergeAppend, metadata propagation,
config translation, self-monitor wiring) live in sibling files in the
processor package so processor.go stays focused on lifecycle +
pipeline.

LoC delta

File Lines
processor.go 178
sketch_wrapper.go 188
shim_helpers.go 280
config_translate.go 85
monitor.go 57
Shim total 788

processor.go itself is the lifecycle + Test API surface; the
remaining files are reusable adapter glue. Pre-refactor processor.go
was 942 LoC and held all four concerns (binding, decode, runtime,
encode) in one file.

Parity test result

TestParity_DDSketch is byte-identical pre/post refactor:

=== RUN   TestParity_DDSketch
    parity_test.go:97: parity OK: 6 envelopes matched byte-for-byte
--- PASS: TestParity_DDSketch (0.45s)

All five parity subtests still pass at HEAD:

TestParity_AllSketches/TestParity_DDSketch       PASS (6 envelopes)
TestParity_AllSketches/TestParity_KLL            PASS (3 envelopes)
TestParity_AllSketches/TestParity_HLL            PASS (3 envelopes)
TestParity_AllSketches/TestParity_CountSketch    PASS (1 envelope)
TestParity_AllSketches/TestParity_CountMinSketch PASS (602 envelopes)

go test -race ./... passes for the processor package
(TestShutdownDuringConsume, TestWindowModeFlushDuringConsume,
TestWindowModeConcurrentConsume all clean).

Public API surface confirmation

ADR-0002 §"Test API contract" lists three test-friendly public methods
that today's tests directly exercise. The shim ships exactly these:

  • Shim.ProcessBatch(ctx, md) (pmetric.Metrics, error) — synchronous
    decode → observe → tick → encode → append-into-md → return md (the
    legacy processBatch shape).
  • Shim.ProcessMetrics(ctx, md) error — observe-only; window-mode
    ConsumeMetrics calls this; tests use it directly.
  • Shim.FlushWindow(ctx context.Context) error — force-tick + encode +
    forward via nextConsumer. Concurrency-safe: swaps the per-metric
    Precompute map under p.mu so the flush loop owns the snapshot
    exclusively.

processor_test.go mechanical rename: processBatchProcessBatch,
flushWindowFlushWindow. No test logic changed.

Divergence from the execution plan

Two adapter-side accommodations were necessary because the otel.Decode
/ otel.Encode helpers in asap-precompute-go (frozen for this PR)
don't propagate certain typed pmetric fields:

  1. Count / AggregationTemporality propagation.
    SketchEnvelope.Count and .AggregationTemporality are typed Go
    fields, not on the proto wire, so otel.Decode doesn't read them
    from input DDSketch DPs and otel.Encode doesn't write them onto
    output DPs. Two helpers in shim_helpers.go
    (patchEnvelopeMetadata, stampDPMetadata) thread these typed
    fields through the runtime. A future PR can fold this into
    otel.Decode/Encode once all five shims need the same plumbing.

  2. Per-input-metric Precompute. The legacy DDSketch processor
    emits one synthesized output metric per input metric name; the
    runtime models one Precompute per (sketch type, agg_id). The
    shim runs one Precompute per distinct input metric name, keyed on
    getOrCreate(batch, observation.Metric, cfg). This keeps existing
    multi-metric tests (TestBatchModeDualInput,
    TestWindowModeDualInput) working without touching the runtime.

Both accommodations are isolated to the shim and don't leak into the
runtime contract.

Test plan

  • go test ./opentelemetry-collector-contrib-patch/processor/ddsketchprocessor/... clean (no -race issues)
  • go test -v -run TestParity_DDSketch ./integration/parity/... byte-identical 6 envelopes
  • go test -v ./integration/parity/... all 5 sketch subtests still byte-identical
  • go vet ./opentelemetry-collector-contrib-patch/processor/ddsketchprocessor/... clean
  • No submodule pointer drift committed
  • No changes to asap-precompute-go source (runtime frozen for this PR)
  • No changes to other processor directories

🤖 Generated with Claude Code

DDSketch processor reduces from ~942 LoC to a ~178 LoC shim. The
windowing, snapshot caching, and delta encoding state machine moves to
asap-precompute-go (already extracted). DDSketch sketch_wrapper.go
implements QuantileSketch over sketchlib-go.

Public test API: Shim.ProcessBatch/ProcessMetrics/FlushWindow per
ADR-0002 §"Test API contract".

Parity harness: TestParity_DDSketch byte-identical pre/post refactor.

Phase 2 step 2.5.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant