fix(asap-precompute-go): add Drain() and remove 6 pseudo-timestamp workarounds - #239
Merged
Merged
Conversation
Tick(nowMs) only rotates when nowMs >= activeEndMs, so calling Tick(time.Now()) on a shutdown path mid-window silently drops pending observations. All 6 callers worked around this with hacky pseudo-timestamps (1<<50 stride, now+2*window, etc.). Add Precompute.Drain() — unconditional rotation regardless of timestamp — and route all 5 OTel shim Shutdown paths plus the Telegraf allsketches plugin Stop path through it. Pseudo-timestamp workarounds removed. Parity harness: still all-PASS byte-identical. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Precompute.Tick(nowMs)only rotates the active window whennowMs >= activeEndMs. Callers that wanted to drain pending dataon shutdown naturally passed
time.Now()— but if shutdown happensmid-window,
nowMs < activeEndMsandTickreturns nil. Mid-windowobservations were silently dropped on the lifecycle exit.
All 6 callers worked around this with one of three hacky
pseudo-timestamp patterns:
nextFlushTick()→1<<50 base + 1<<32 stride per callforceTickMs = 1<<62 - 1time.Now() + 2*windowtime.Now()(silently buggy on shutdown — noworkaround, just the original bug)
Same fix written 6 different ways. Solve it once at the API level.
The fix
Add
Precompute.Drain()— a sibling ofTick(nowMs)thatunconditionally rotates the active window regardless of wall-clock
time and returns the resulting envelopes. After Drain, the next
active window starts at the same boundary Tick would have used at
natural rotation (
activeStartMs := old activeEndMs; activeEndMs += windowSize).Internals:
windowState.rotateand the newwindowState.drainboth delegate to a shared
rotateLockedhelper. Drain skips thenowMs < activeEndMsgate, and is a no-op on an already-emptyactive window so callers can invoke it multiple times on shutdown
without fast-forwarding through empty buckets.
Affected callers (all on main)
opentelemetry-collector-contrib-patch/processor/hllprocessor/processor.gonextFlushTick()(1<<50 + 1<<32 stride, plusflushSeq atomic.Uint64field)Drain()opentelemetry-collector-contrib-patch/processor/ddsketchprocessor/shim_helpers.goTick(1<<62 - 1)Drain()opentelemetry-collector-contrib-patch/processor/countsketchprocessor/shim_helpers.goTick(1<<62 - 1)Drain()opentelemetry-collector-contrib-patch/processor/countminsketchprocessor/{processor,shim_helpers}.goTick(1<<62 - 1)(batch) +Tick(time.Now())(window)Drain()(both paths)opentelemetry-collector-contrib-patch/processor/kllprocessor/processor.goTick(time.Now())(silently buggy on shutdown)Drain()telegraf-patch/processors/allsketches/allsketches.goTick(time.Now() + 2*window)on StopDrain()on StopPR #237 (Telegraf allsketches plugin) was merged shortly before
this PR was opened, so the Telegraf fix lands here on main rather
than as a follow-up commit on #237's branch.
Validation
Parity harness: all-PASS byte-identical, all 5 sketches.
New unit tests
TestPrecompute_DrainFlushesMidWindowObservations— exercisesthe bug directly: observe 3 values early in window 0, confirm
Tick(2_000)(mid-window) returns 0 envelopes, confirmDrain()returns 1 envelope with Count=3 and the natural[0, 10_000)window range. Also confirms the next activewindow starts at the natural rotation boundary so a follow-up
Tick(20_000)flushes window 1 cleanly.TestPrecompute_DrainEqualsTickAtBoundary— pins the contractthat Drain emits the same envelope payload Tick would emit if
called precisely at activeEndMs (same range, same count, same
bytes).
LoC removed across the 6 caller sites
~47 LoC of workaround code (constants, helpers, pseudo-timestamp
arithmetic, and their explanatory comments) removed across the 6
caller sites. Replaced by direct
Drain()calls.🤖 Generated with Claude Code