mvp_report: fix DDSketch family metric name (http_latency_ms → http_requests_total_latency_ms) - #346
Merged
Merged
Conversation
…equests_total_latency_ms) PR #337's SKETCH_FAMILIES table used `http_latency_ms` as the DDSketch metric, but the actual workload-spec entry (and fake-exporter emit) is `http_requests_total_latency_ms`. Result: post-PR-#345 demo runs reported DDSketch row n=0 even though warm tier was answering quantile queries — the rel-err values were attributed to no family. This aligns the table with the live workload-spec. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 26, 2026
…457) DDSketch already opts in to per-window delta-against-empty (#456). This extends the same opt-in to CountMinSketch, CountSketch, and HLL on both edge runtimes; KLL stays full-only and DDSketch is unchanged. asap-precompute-go (sketches/{cms,countsketch,hll}.go): each wrapper now implements DeltaAgainstEmptyBase(), returning the encoded empty-sketch snapshot the SnapshotCache caches as the outbound base after each window-close emit. The next window's ComputeDeltaAgainst (already wired to the sketchlib-go ComputeDelta for each family) then diffs against an empty base, so each window's delta carries that window's own per-window state — no cross-window subtraction. CMS msgpack mode (which cannot carry deltas) returns a nil base to keep legacy always-refresh; the SnapshotCache now treats a nil/empty empty-base as "fall through to legacy refresh". asap-precompute-rs (src/sketches/{cms,countsketch,hll}.rs): each wrapper implements delta_against_empty_base() and a real compute_delta_against that decodes the prior envelope and diffs via the asap_sketchlib compute_delta for that family (CMS/CountSketch cell deltas, HLL register delta), with a full-snapshot fallback on empty/undecodable prior. Each apply_delta now dispatches on payload shape (full-state envelope vs raw delta proto) via apply_delta_bytes, mirroring the DDSketch wrapper. The stale sketches/mod.rs "always-full" caveat is updated. Tests (mirroring DDSketch's): for each family, applying the emitted delta to an empty base reconstructs the window's state, and consecutive windows emit their own state (no cross-window subtraction) — Go in sketches/*_test.go, Rust in snapshot_cache.rs. HLL's disjoint-window test pins that the empty-base reset defeats register-MAX leakage. Depends on asap_sketchlib#59 (CMS/CountSketch/HLL compute_delta / apply_delta_bytes; the underlying codec is the byte-parity twin of the sketchlib-go delta codec). The backend base rotation (#346) is already sketch-agnostic, so no backend change is needed. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Jun 9, 2026
Rewrites #458's static fixed-interval sub-window producer into a dynamic, divergence-gated one: a series emits a sub-window delta only when its sketch has diverged from the backend's last-acked copy by >= ε in the family's norm, so the backend can answer the OPEN (incomplete) window's query to relative ε at minimal bandwidth — the right behavior when tumbling windows are large (a single boundary emit leaves the open window stale; a blind fixed cadence is wasteful or insufficient). This is the CDM functional-tracking (ε-approximation) branch (Cormode-2013), distinct from the alert coordinator's threshold-exceedance path, and the per-series LOCAL case (no coordinator) of the aggregation taxonomy. Mechanism (kept from #458): EmitSubWindow / ComputeSubWindowDelta (no boundary empty-base reset) / subWindowVisit / serializeSubWindowSeries / the flush check ticker. New: a per-family divergence gate (tight for every family): - Sum: |sum_now − sum_acked| ≥ ε·|sum_now| (value, L1) - CMS: ΔN ≥ ε·N (count, f̂≤f+εN) - KLL/DDSketch: ΔN ≥ ε·N (rank staleness ≤ ΔN) - Count-Sketch: ‖f_now − f_acked‖₂ ≥ ε·‖f_now‖₂ (Frobenius cell-diff) - HLL: |card_now − card_acked| ≥ ε·card_now (relative) Scalar families track the divergence reference on seriesEntry; Count-Sketch snapshots its cell matrix in the wrapper. ε=0 ⇒ emit every tick (#458's fixed mode, backward compatible); first emit of a window always ships full state. Config: SubWindowInterval (check cadence) + SubWindowEpsilon on PrecomputeConfig and the asap_edge processor (sub_window_interval / sub_window_epsilon), validated 0<interval<window and 0≤epsilon<1. Composes with the merged backend per- window_start base rotation (#346): sub-window deltas carry the open window's range and accumulate until the boundary advances window_start. Scope: Go runtime + asap_edge processor (the deployable path). Rust mirror deferred. KLL is not delta-capable, so sub-window emission applies to the delta-capable families. Tests: threshold gating, fixed-mode, disabled-without- delta, Count-Sketch L2 divergence — all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Jun 9, 2026
Sum's ComputeDeltaAgainst was a stub returning full state, which made Sum unsafe
for sub-window emission (additive backend would over-count). Make Sum a proper
per-window-reset (PWR) incremental delta family, mirroring DDSketch/CMS/etc.:
- ComputeDeltaAgainst now returns {Δsum, Δcount} = current − prev (incremental),
not full state; nil when unchanged.
- SumWrapper implements emptyBaseDeltaSketch.DeltaAgainstEmptyBase (16-byte
{0,0}), so the boundary resets the cached base to empty each window and the
next window's emits are deltas from zero — without this, window N+1's first
delta would be (currentₙ₊₁ − fullₙ), a bogus cross-window subtraction.
Because the boundary ComputeDelta and the sub-window ComputeSubWindowDelta share
the outbound base cache, both now produce deltas against the last emit, so a
window's sub-window emits + boundary emit accumulate additively to the exact
window total (no over-count). Sum is removed from the sub-window exclusion;
subWindowEmissionSafe now gates only KLL (a full-state merge that cannot
subtract). Sum's per-series divergence gate (value: |Δsum| ≥ ε·|sum|) is restored.
Sum metrics WITHOUT delta are unchanged (boundary still serializes full via
Snapshot, never ComputeDelta). Backend compatibility: Sum delta envelopes are
applied via the same additive ApplyDelta + window_start-triggered PWR reset the
backend already uses for the other delta families (#346).
Test: TestSubWindow_Sum_IncrementalReconstruction — three sub-window emits + a
boundary emit reconstruct the window total (175) with no over-count, and a new
window's first emit is a delta from zero (40), proving PWR. Full module green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Small fix: SKETCH_FAMILIES table had legacy metric name. Aligns with live workload-spec.