feat(gos): HLL register-change delta adapter (|2^C'−2^C|≥2^τ, never-reset) - #529
Merged
Merged
Conversation
Convert HLL to the GOS insert-time delta model (design-gos-unified-edge-
telemetry.md §11, sampling-cdm-gos-derivations.md §8.7), the structurally
odd-one-out family: its state is a vector of max-registers merged by
register-wise MAX (idempotent, monotone), so — unlike every additive family
in this workstream — HLL registers are NEVER reset on send. Re-sending an
unchanged/already-known register value is harmless (max(x,x)=x), which is
exactly what makes the never-reset rule correct.
HLLWrapper (asap-precompute-go/sketches/hll.go):
- SetGosMode(tau, _): τ is REINTERPRETED onto the existing generic
PrecomputeConfig.GosDeltaEpsilon knob (a count of "doublings", not
CountSketch's ε budget) — HLL and CountSketch never share a live sketch,
so overloading the one float64 field avoids a second family-specific
config field. The uint32 sites arg is accepted for a uniform SetGosMode
shape but unused (HLL's formula has no multi-site k term).
- Insert path routes UpdateValue/UpdateBytes through sketchlib-go's new
InsertWithHashReportingChange; a register that mechanically changed is
checked against |2^C'-2^C| >= 2^τ (exact integer shifts, not math.Pow)
and, if crossed, queued in gosDirty with its CURRENT value. gosLastSent
tracks the per-register last-sent baseline the sketch itself cannot
(registers are never reset, so "since last sent" isn't readable off the
register).
- First-nonzero-write mitigation (design doc §12 open item 4): a register's
first-ever nonzero write (last=0, cur>0) is sent unconditionally
regardless of τ. This is EXPLICITLY UNVERIFIED for the small-cardinality
regime per the design doc — implemented because it's cheap and proposed,
not because it's proven.
- ComputeDeltaAgainst drains gosDirty into a sparse RegisterDelta WITHOUT
touching a single underlying register (the critical correctness
difference from CountSketch, which resets cells to 0 on send); dedups to
each register's latest value and sorts ascending for the varint wire form.
- gosWake/ConsumeWakeSignal wire the insert-time crossing to the runtime's
out-of-cycle sub-window flush. Reset clears per-window GOS state but keeps
gosTau (mode config, not accumulation state).
Runtime (precompute.go, config.go): applyGosMode + the sub-window Gate-1
bypass now cover SketchTypeHLLSketch when GosDeltaEpsilon>0; the pre-existing
non-GOS EstimateCardinality() divergence path is left fully intact when GOS
is off.
Processor (asapedgeprocessor): the dense-HLL factory primes SetGosMode at
series creation; config_validate accepts gos_delta_epsilon (τ, no (0,1)
upper bound) on family=hll but rejects it with hll_sparse (the sparse base
has no per-register GOS path — this PR is scoped to dense HLL). CountSketch's
ε (0,1) bound is preserved.
Tests: sketchlib-level detection + first-nonzero mitigation + sampling;
wrapper-level never-reset assertion (verifies raw registers unchanged after a
drain — the opposite of every other family's test), MAX-merge idempotency,
wake-signal, Reset semantics; processor-level end-to-end
TestGOSHLLInsertWakesFlush (ConsumeMetrics alone produces a flush with no
SubWindowInterval and no manual wakeSubWindow) plus config-validation cases.
Uses sketchlib-go feat/gos-hll-primitive @ bf7828357a8125413d905a22c4a0446cf642ac76
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
zzylol
changed the base branch from
split/pr-gos-countsketch
to
split/pr-gos-kll
July 17, 2026 02:48
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the GOS per-family stack (base: CountSketch
split/pr-gos-countsketch/ #524). Depends on sketchlib-go #76 (InsertWithHashReportingChange). The structurally-odd family: registers are NEVER reset (MAX-merge idempotency), unlike every additive family.Send when a register's linearized value crosses
|2^C'−2^C|≥2^τ(derivations §8.7), computed with exact integer shifts (1<<C), nevermath.Pow.drainGosDeltaserializes the pending dirty-list into a sparseRegisterDeltaand touches ZERO underlying registers.GosDeltaEpsilon/gos_delta_epsilonreinterpreted as τ (a doublings count); validation allows τ>0 unbounded for HLL while keeping CountSketch's ε∈(0,1).SetGosMode(tau, sites)keeps CountSketch's signature soapplyGosModedrives both via one assert;sitesunused for HLL.sparseInsert(no per-register change accessor), soconfig_validaterejectsgos_delta_epsilon+hll_sparsewith a boot error — a follow-up gap.The pre-existing non-GOS
EstimateCardinality()divergence path is left fully intact when GOS is off. HLL's hash-threshold sampling is kept out of the coordinated path, so no #518 interaction.Test plan
2^τboundary.TestGOSHLLInsertWakesFlush+TestConfigValidate_GosHLL.🤖 Generated with Claude Code