Conversation
Edge side of the F2 whole-sketch monitor and the unified GOS framework doc. - f2engine.go: F2Engine — local safe-zone test (mirrors Rust f2.rs), ship/silent gating, RefBroadcast handling; safe radius uses (1-ε)τ. - types.go: FunctionalF2, F2Mode, Sketch report payload, RefBroadcast directive. - grpcclient: sketch payload + RefBroadcast wiring; regenerated stubs. - countsketch.go: CellMatrix() accessor for the whole-sketch monitor. - cmd/f2driver: multi-edge eval driver; deploy/.../f2_monitor_eval.sh. - docs/design-gos-unified-edge-telemetry.md: the full GOS framework (error bound, water-filling thresholds, tunable mem/compute/comm objective, WZ optimality); docs/f2-geometric-monitoring.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The edge-side twin of the Rust control_plane threshold_alloc: compute the F2 GOS delta threshold from local sketch state instead of a fixed configured value. - gos_threshold.go: Go mirror of threshold_alloc — F2 isotropic closed form T = ε‖Ĉ‖/(2k√(dw)) + box-constrained water-filling AllocateThresholds (T_j ∝ √(V_j/|g_j|), sampling floor, query/freshness caps). - CountSketchWrapper.ComputeGosDelta: emits a per-cell delta using the norm- relative GOS threshold via the EXISTING sparse-delta path (which already gates each cell by |ΔS[r][c]| ≥ threshold) — no wire/serialization change. The threshold is adaptive (scales with ‖Ĉ‖) so whole-sketch relative error stays within ε as the sketch grows. The anisotropic (gradient-weighted) per-cell variant needs a vector-threshold delta in sketchlib-go (follow-up); the F2 isotropic case is uniform so the scalar path suffices. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ig-gated) Route the sub-window and full-window delta emit paths through the GOS F2 per-cell threshold when PrecomputeConfig.GosDeltaEpsilon > 0, else the fixed DeltaThreshold (default → unchanged behavior). - config.go: GosDeltaEpsilon + GosSites knobs. - countsketch.go: GosDeltaThreshold(ε, k) → ceil(ε‖Ĉ‖/(2k√(dw))). - precompute.go: gosDeltaThreshold() gate at both ComputeSubWindowDelta and ComputeDelta call sites (structural interface assert; no Sketch iface change). Backend apply_delta is unchanged — the wire format (sparse cells) is identical; only which cells cross the (now norm-relative) threshold differs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ry toggle)
Make the per-cell threshold shape a config knob so the edge-memory vs
communication tradeoff is tunable:
- GosAnisotropic=false (default): ISOTROPIC scalar T=ε‖Ĉ‖/(2k√(dw)), O(1) mem.
- GosAnisotropic=true: gradient-weighted per-cell {T_j} (water-filling
T_j∝√(V_j/|g_j|), g_j=2|Ĉ_j|), O(d·w) mem, less communication on skewed data.
- config.go: GosAnisotropic knob.
- countsketch.go: SetGosMode + ComputeDeltaAgainst branch; computeAnisotropicDelta
+ gosThresholdMatrix (AllocateThresholds → sketchlib ComputeDeltaPerCell).
Heap-msgpack path falls back to isotropic.
- precompute.go: applyGosMode() sets the mode per emit (structural assert; no
Sketch iface change); delta call passes the fixed threshold, wrapper overrides.
DEPENDS ON sketchlib-go#69 (ComputeDeltaPerCell). Backend apply_delta unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OnRef now handles both reference forms: a Full frame replaces the cached C_ref; a sparse delta (IsDelta) applies [rowIdx,colIdx,vals] cell-wise onto it. A delta with no cached base is dropped and counted (a Full keyframe follows). Wire codec is sketchlib-go asapmsgpack.UnmarshalCountSketchDeltaSparse (byte-parity with the Rust rmp_serde 5-tuple producer — verified). DEPENDS ON sketchlib-go#69 (sparse delta codec commit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mpling
Refine the sampling model to the SDK↔collector split (§3.1): the SDK does
row-admission only — by NitroSketch geometric skip-sampling (O(1) RNG per
admitted update, not d coins per item; whole-item skip is O(1)) — and never
hashes; the agent collector computes the row hashes and updates counters ONLY
for admitted rows, with 1/p inverse-probability weighting. A sample admitting no
row is dropped at the source (collector skips its deserialization + hashing).
Per-row knob p_{i,r}; unbiasedness + variance/ε_sa; cost model split into
Comp_sdk (RNG) + Comp_coll (hash) both ∝ (Σ_r p_{i,r})·rate; wire volume ×
(1−∏(1−p)). This matches the existing sketchlib-go GeometricSampler.
Ref: NitroSketch (Liu et al., SIGCOMM 2019).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SDK needs per-series (sketch type, dims, {p_{i,r}}) to make the row/counter
admission decision — pushed by the controller over the same config channel as
the collector plan. Update-sampling with 1/p weighting is unbiased only for
ADDITIVE counters: CMS & CountSketch (per-row, d counters/item), DDSketch
(per-item, the d=1 case). KLL (non-linear random compaction) and HLL (idempotent
register MAX — a dropped max is unrecoverable) are NOT sampled. Matches the edge
runtime's applyGrantedSampleP wiring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verification against the code: geometric skip-sampling, 1/p weighting, and the
additive-families-only rule MATCH, but the current runtime samples collector-side
and per-item (whole-sketch, scalar sampleP) — not SDK-side per-row (p_{i,r}). So
the source-drop / admitted-rows-only savings aren't realized yet. Marks §3.1 as
the target design and records the gap + what closing it needs (move admission to
the OTLP SDK aggregators; generalize sampleP to a per-row vector).
Corrects the earlier over-claim that it "matches the existing edge runtime".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ampling
Analyze the estimator/variance/error-bound under the SDK-side per-row admission,
before changing code:
- Per-row estimator (independent Z_{x,r}), unbiased; Var = F₂/w (collisions) +
(1−p_r)/p_r·S₂ (sampling).
- KEY result — median decorrelation: per-row admission makes the d row estimates
INDEPENDENT, so the median-of-rows concentrates the sampling error into the
(1−δ) guarantee; whole-item admission shares one Z_x → the sampling error is
common-mode and SURVIVES the median (Θ(√((1−p)/p·F₂)), not reduced by d). At
equal edge CPU (E[rows]=d·p), per-row is the strictly better estimator — the
quantitative reason to move admission into the SDK.
- ε_sa composes in quadrature with ε_sk (per-row) vs linearly (whole-item).
- CDM threshold coupling: the sampling floor becomes row-indexed
T_j ≥ √(V_j(1−p_r)/p_r); GOS water-filling unchanged in form.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CountSketchWrapper.UpdateString now uses UpdateStringSampledPerRow (per-row geometric admission + 1/p weight) instead of the whole-item admit. This decorrelates the d row estimates so the median concentrates the sampling error into the (1−δ) guarantee (design §3.2) at equal edge CPU. Depends on sketchlib-go#69. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update the implementation-status table: the code is now per-row (UpdateStringSampledPerRow), so the §3.2 tight estimator (median decorrelation) is realized. Remaining gap is location only (move admission into the OTLP SDK to drop unadmitted samples before collector deserialization). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- TestGosAnisoSavingsRatio: sweeps cell skew, confirms measured ρ == predicted Cauchy–Schwarz factor exactly (Zipf: 50% saving; heavy: 97%), and ρ≤1. - docs/gos-eval-results.md: the ρ table; geometric vs distributed F2 (stable 4.0×, ramp 1.74× less, both alert in-band); the C_ref delta-broadcast effect (egress 5.3× down, flips ramp from loss to win); WZ k/ε² normalization (geometric stable ≈5× the k·S unit vs distributed 20×); per-row unbiasedness. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…row, WZ)
Doc-review corrections:
- H1/H2: unify the error composition — sketch+sampling (random) combine in
QUADRATURE via one median tail; staleness (deterministic worst-case) adds
LINEARLY. Rewrite Theorem 1 to the median-combined form (drop the standalone
z_δ term, 1−3δ → 1−δ) and §7 Layer B to peel ε_st linearly then split the
random residual ε_sk²+ε_sa²=(ε_q−ε_st)². Note split_budget uses the quadrature
approximation.
- H3/M1: reconcile per-site rate vs per-row admission (p_{i,r}=p_i uniform,
AllocateSampleRates is per-site); per-row rate differentiation is future. Mark
the per-row estimator benefit as already realized (code), only the SDK
relocation pending.
- M2: SDK CPU is Θ(d) skip decrements/item (RNG ∝ Σp); the O(1) whole-item skip
is not shipped.
- M4: WZ rate-vs-total — compare over a bounded-change horizon; point to the k·S
normalization in gos-eval-results.md.
- M5: the common-mode term is the queried key's own contribution ∝ f(y)√((1−p)/p),
not √F₂ (cross-key noise partially decorrelates).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… median The geometric safe-zone ball ‖C‖ ≤ √(d(1−ε)τ) is inherently the mean-of-rows estimator ‖C‖²/d; make explicit that this is distinct from §3.2's median-of-rows point-query estimator so the two readouts (robust key location vs aggregate energy) aren't conflated. Matches the coordinator switch to mean_f2(). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Relocate the per-row NitroSketch admission decision into the SDK-build path so sampling happens at the source (§3.1 of the GOS design), not only in the collector wrapper. The SDK CountSketch aggregator now carries a per-series GeometricSampler (seeded deterministically from the attribute key via FNV-1a-64) and routes measurements through UpdateStringSampledPerRow with 1/p weighting when 0 < sample_p < 1; sample_p<=0 or >=1 keeps the exact prior full-update behavior. Threaded a new `sample_p` knob: PipelineCountSketchParams (YAML) → AggregationCountSketch (+[0,1] validation) → pipeline → Builder.CountSketch → newCountSketchAgg. Default 0 is a no-op, so existing configs/tests are unchanged. Doc: §3.1 status table "Where" flips to realized on the SDK-build path; spell out what each deployment mode saves (SDK-build = source CPU + one-sketch-per-series deserialization; collector-build = otlpfilter whole-datapoint wire-thinning). CMS/DDSketch SDK-build hosting noted as follow-up (CMS needs a sketchlib per-row update; DDSketch is the d=1 WithSampleP case). Note: opentelemetry-go-patch is a partial checkout (sibling `metric` module absent via replace directive) so it does not compile in this tree; verified by gofmt + sketchlib-go API parity (NewGeometricSampler/Reset/UpdateStringSampledPerRow). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…aggregators Extend the SDK-build sampling relocation (previously CountSketch-only) to the other two sampleable families, all behind the same `sample_p` knob threaded Pipeline*Params (YAML) -> Aggregation* (+[0,1] validation) -> pipeline -> Builder -> agg constructor. Default 0 is a no-op, so existing configs/tests are unchanged. - CountMinSketch: per-series external GeometricSampler routed through the new sketchlib-go InsertWithHashSampledPerRow (per-row admission, 1/p in-place, wire stays exact). sampleCount still counts raw items (metadata, not a rescale factor), so it stays unconditional. - DDSketch: the d=1 whole-item case — enable the sketch's built-in WithSampleP at series creation (raw counts, wire stamps p, consumer rescales x1/p). measure() needs no change; Update() already respects the internal sampler. - Generalized the per-series RNG seed helper csSamplerSeed -> samplerSeedForAttrs (FNV-1a-64 over the attribute key), shared by all three aggregators. Updated the 7 DDSketch builder test call-sites for the new sampleP arg and the design doc §3.1 (CMS/DDSketch follow-up now resolved). Same partial-checkout caveat: opentelemetry-go-patch does not compile in this tree (sibling `metric` module absent); verified by gofmt + sketchlib-go API parity, and the new sketchlib-go CMS method is unit-tested in its own repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…fallback Review fixes F1/F2/F4/F5 on the SDK sampling relocation: - F4 (single-location sampling): CS/CMS aggregators switch from the stateful GeometricSampler to sketchlib-go's new ConsistentSampler — a stateless hash decision per (series-seed, occurrence, row). The admitted-row set is a pure function of shared inputs, so any pipeline stage (SDK, otlpfilter, collector wrapper) recomputing it agrees exactly: one sampling owner regardless of location, and double evaluation is idempotent instead of double-diluting. Collector-build stages derive occ from the wire-visible TimeUnixNano (design §3.1.1); plan-level sample_at stays as the enforcement bit but correctness no longer depends on it. - F5 (window-start seed): per-series seeds are XOR-salted with the window start (seedSalt refreshed in delta() for CS/CMS/DDSketch), so recreated series draw a fresh admission pattern each window instead of repeating it — sampling errors decorrelate across windows. - F1 defense (never wedge a series): CS/CMS payloadFor now falls back to a full frame on any ComputeDelta/SerializeDelta error and still refreshes the snapshot, mirroring the DDSketch payloadFor contract. With sketchlib's new float64 delta wire this path should not trigger, but the emit path must always produce a valid payload. - Doc §3.1.1: the consistent-sampling scheme, occurrence-id rules (per occurrence, never per key), window salt, and the float64 wire note (F2). Pairs with sketchlib-go b3661f6 (lossless float64 wire) + f264598 (ConsistentSampler + RowSampler interface). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ent sampling)
Closes the collector-build half of design §3.1.1: the wire filter and the
sketch wrappers now evaluate the SAME stateless decision per datapoint —
seed = common.SeedForMetric(name), occ = time_unix_nano/1e6 (ms, the exact
integer the decoded Observation.TimestampMs carries) — so the filter's
whole-datapoint drop and the wrapper's per-row admissions never compound.
- otlpfilter: replaced the stateful per-metric GeometricSampler machinery
(samplers map, mutexes) with stateless ConsistentAdmit decisions.
SetParams(name -> {P, Rows}) configures per-metric (p, d); the filter reads
time_unix_nano (field 3, fixed64) from the opaque datapoint bytes and drops
iff no row admits (R(x)=∅, the (1-p)^d fast path). Zero/absent timestamp →
fail-open passthrough. SetP kept as the Rows=1 convenience.
- runtime: new optional precompute.SampleIdentitySetter interface; the window
threads (obs.Metric, obs.TimestampMs) in recordLocked with a per-entry
cached assert. The runtime stays sketchlib-free (wrappers derive the seed).
- wrappers (sketches): SetSampleIdentity on all three sampleable families.
CountSketch routes UpdateString through the consistent per-row path
(1/p in place, wire exact); CMS switches OFF its internal whole-item
sampler and moves to per-row in-place weighting via
InsertWithHashSampledPerRow — envelope becomes exact (no downstream
rescale; empty-base memo invalidated; newSketch/WithSampleP re-apply paths
gated); DDSketch (d=1) moves admission out of the sketch into the wrapper's
consistent decision with SetWireSampleP keeping the raw-counts + envelope-p
convention (Reset re-stamps). Zero timestamp → per-series occurrence-counter
fallback, so such points are still sampled exactly once end-to-end.
Contract tests pin cross-stage agreement item-for-item: filter survivors ≡
stateless recomputation (otlpfilter), CS wrapper touches the sketch iff the
filter keeps (2000/2000), DDSketch admissions == filter row-0 decisions,
CMS envelope exact + unbiased with no rescale, counter fallback unbiased.
Doc §3.1.1 updated to implemented status (occ-ms convention, ts==0 semantics,
same-ms correlation caveat, Rows-mismatch asymmetry).
Pairs with sketchlib-go: ConsistentSampler pin/Rebind + SeedForMetric +
DDSketch.SetWireSampleP.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ault) Precompute-go half of the §3.1.1 production wiring: - monitor.Engine.SetSampleGrantHook: fired for ACCEPTED grants only (stale-epoch/unknown dropped, mirroring grantedSampleP), outside the engine mutex on the transport goroutine, with (aggID, Grant.SampleP). - otlpfilter: Upsert (copy-on-write single-entry update; p<=0/p>=1 withdraws; identical re-grants are no-ops so per-round grants cost one map lookup), Params (read-only introspection), Default() — the process-wide shared state the asap_edge grant hook writes and the asap_otlp receiver reads; the build-tagged receiver sketch defaults to it (NewFactory(nil)). README rewritten for the consistent-sampling semantics. - Doc §3.1.1: grant-plumbing paragraph (flow, family gating, AggID == SeedForMetric identity). The processor-side hook installation lands in the opentelemetry-collector- contrib submodule (asapedgeprocessor/warm_sketch.go). Tests: engine hook contract; Upsert grant lifecycle against FilterRequest (install → thin, identical re-grant → same survivors, withdraw → byte-identical passthrough, foreign-metric isolation); Default() shared instance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Processor half of the §3.1.1 grant plumbing (captured into the tracked
opentelemetry-collector-contrib-patch tree via backup_otel_collector_contrib_
patches.sh; the submodule itself stays on the upstream remote, untouched):
- warm_sketch.go: next to SetMonitorEngine, install Engine.SetSampleGrantHook
forwarding accepted grants to otlpfilter.Default().Upsert(inputMetricName,
{P, Rows}). Rows from the new wireSampleRows(fam): CS/CMS matrix rows
(csmDims), DDSketch d=1; Sum/KLL/HLL never installed (same gating as
applyGrantedSampleP). Foreign-agg grants filtered by pcfg.AggID; the key is
the INPUT metric name (what the pre-decode filter matches on raw OTLP
bytes), not the suffixed output name.
- grant_wire_test.go: wireSampleRows mapping across all six families, plus a
composition test driving the exact installed closure end-to-end — accepted
grant lands in Default() under the input name with the family fan-out,
foreign-agg grants are ignored, and a p=1 re-grant withdraws the entry.
Pairs with asap-precompute-go 2bdf039 (Engine.SetSampleGrantHook +
otlpfilter Upsert/Params/Default).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e state DDSketchState dropped its DataPoint-level scalars (count/sum/min/max, fields 4-7 reserved) in sketchlib-go, so the envelope round-trip test must derive the count from the store buckets via NewFromState + GetCount instead of the removed state.Count field. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-u trip
monitor_e2e.sh: Sum monitors now register under the series-group key
(groupKeyBytes: 'k=v;k2=v2'), so the coordinator harness must be
configured with the same key or every registration is rejected as
unconfigured and no slack is ever granted — the e2e timed out with the
edge silent. Pass the key as the harness's new 6th argument.
f2_monitor_eval.sh: ${pat} referenced inside the same 'local' statement
that assigns it expands before the assignment lands, which is an unbound
variable under set -u. Split into two local statements.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documentation sweep from the doc-vs-code review — comment/prose only, no
behavior change:
Retired sampling law. The coordinator uses the whole-sketch epsilon-floor
p_i = 1/(1+eps^2*rate_i), not the per-key sqrt(f_i/rate_i) KKT water-filling.
Correct the stale sqrt(f/rate) references in monitor.proto (+ generated
pb.go), monitor/types.go, monitor/engine.go; banner monitor/sampling_alloc.go
as a retired reference impl with no production caller; and update design
sec 3.1/7C, distributed-nitrosketch (top banner), eval-query-plan,
evaluation-plan-figures, use-case-dataset-survey.
F2 doc regenerated. f2-geometric-monitoring.md was a generation behind:
the C_ref delta broadcast is implemented and winning (ramp geometric
531,132 bytes, not the stale 1,384,920); the live coordinator uses the
mean-of-rows estimator (not estimate_f2/median); deltas are a 5-element
msgpack frame; and F2Engine is not yet reachable from the production edge
runtime (monitorValue drops FunctionalF2) — noted honestly.
Design sec 11/12 status. Record that the GOS threshold control loop is not
wired end-to-end (control plane emits gos_* knobs no collector processor
parses; coupling floor inert at SampleP=1); fix the stale three-way
quadrature (it is linear-peel-then-quadrature per split_budget); note the
edge computes {T_j} locally (not shipped via OpAMP); split open-problem #1
into done (sparse delta) vs open (anisotropic broadcast gate); fix the
dangling design-section-D cross-reference in gos-eval-results.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gosThresholdMatrix hardcoded GosParams.SampleP=1.0, so the sampling to threshold coupling floor T_j >= sqrt(V_j(1-p)/p) was permanently inert even when a coordinator grant set p<1 -- the anisotropic delta gate could transmit finer than the sampling noise (the spurious-emit jitter the design warns about). Pass w.SampleP() instead; it returns 1.0 when unsampled, which GosParams.floor treats as no floor, so the unsampled path is byte-identical to before. Add TestGosThresholdMatrixHonorsSampleFloor: with p=0.3 every active cell's threshold respects its floor, and with p=1 at least one active cell sits below that same floor -- proving the wiring is observable, not a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Close the last hop of the GOS delta-gating control loop. The control plane
derives and emits per-metric gos_delta_epsilon / gos_sites / gos_anisotropic
(emit/agent.rs, countsketch only), and precompute's applyGosMode consumes
PrecomputeConfig.GosDeltaEpsilon at series creation + window advance -- but
no collector processor parsed those YAML keys, so GosDeltaEpsilon was always
0 and applyGosMode was a production no-op (the anisotropic delta path was
reachable only from tests/eval).
Thread the three knobs through all three layers: MetricFamily config struct
(mapstructure gos_delta_epsilon/gos_sites/gos_anisotropic) -> sketchOpts ->
PrecomputeConfig. With a non-zero epsilon now reaching applyGosMode, the
norm-adaptive GOS delta gate (isotropic, or anisotropic per-cell {T_j})
activates end-to-end from a pushed streaming-config.
TestGosKnobsDecode guards the strict-mapstructure-decode regression (the same
crash-loop class the sample_p keys hit) and asserts the values land on the
family fields, including the gos_anisotropic-omitted -> isotropic default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FunctionalF2 was silently dropped by the edge runtime: precompute's scalar monitorValue switch handles only Sum/CMSPoint/LinearBuckets, so an F2 spec fell through to ok=false and disabled monitoring -- F2Engine was reachable only from the eval driver, not a pushed streaming-config. F2 is whole-sketch and non-linear, so it does not fit the scalar per-observation hook. Add a parallel path: - precompute: SetF2Engine + DriveF2Monitor. DriveF2Monitor feeds each active series' current cumulative cell matrix to F2Engine.OnWindow (whole-stream = one series, empty group key matching Configure(aggID, nil)). window.f2Visit iterates series under the lock. - adapter (asapedgeprocessor): a FunctionalF2 branch builds + Configures an F2Engine (dims from the family's CountSketch rows/cols, f2_mode from config) with its own gRPC client, and emitSubWindow drives DriveF2Monitor once per sub-window tick -- the continuous cadence the geometric safe-zone needs. New config: threshold.functional=f2, threshold.f2_mode=geometric|distributed. The scalar monitorValue still returns ok=false for F2 -- now correct, since F2 is handled by the separate path, not a silent drop. Test: TestMonitor_F2_CountSketch_DriveShipsMatrix (Observe -> DriveF2Monitor -> register + ship the serialized matrix, distributed ships every tick). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add an env-gated delta-loss injector to the f2driver (F2_INJECT=corrupt|drop, F2_INJECT_NTH; edge-0 only, default OFF so a normal eval is byte-identical) and a companion runner f2_deltaloss_demo.sh that runs ramp/geometric three ways — no loss / corrupt delta / dropped delta — and checks the alert still fires. Recorded results (gos-eval-results.md §2): alert=1 in all three cases (safety preserved). A corrupt delta is detected at the edge (ref_errs=1 → needFull force-ship); a silently dropped delta is undetected by that edge (ref_errs=0) and recovered by the coordinator's periodic keyframe + the other edges' exact sketches in the global merge. Normal f2_monitor_eval.sh unchanged (923280/230820/923280/531132). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…_f2.csv Add f2_wholesketch_cluster.sh — the distributed version of f2_monitor_eval.sh: the f2_monitor_harness coordinator runs on the WARM node and the f2driver edges on a source node, so every sketch ship and C_ref broadcast crosses the 8-node cluster's 10GbE fabric instead of loopback. Measured (recorded at eval-8node/f2_wholesketch_cluster.csv): byte-identical to the local eval (deterministic protocol) — stable geometric 230,820 vs distributed 923,280 (4.0x), ramp geometric 531,132 (1.74x), same alert decisions. Retire eval-8node/fig9_f2.csv: it measured the SCALAR coordinated-sampling F2 variant (otel-app local_value path), which is not the whole-sketch geometric protocol this eval axis is about; the stale baseline predated the whole-sketch coordinator routing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add mode=raw to the f2driver: every sample ships as a real serialized msgpack [ts,key,value] record (bytes counted from the frames, no coordinator), with the alert decided from the EXACT global F2 -- the complete-information ground truth. Wire raw rows into both eval scripts and the cluster CSV. Honesty note recorded in gos-eval-results.md: on the default tiny-H protocol-stress workload (4 keys) raw is far CHEAPER than sketches by construction (a sketch ship is a fixed d*w ~= 11.5 KB; raw scales with H). F2_KEYS sweeps the cardinality: measured H=2048 raw ramp = 4,007,440 B vs sketch 923,280/531,132 (sketch wins 4.3x/7.5x); crossover H* ~= 470 (vs distributed) / ~270 (vs ramp geometric) on this run shape. Measured (local + cluster over the real NIC, byte-identical): stable raw 3,956 / distributed 923,280 / geometric 230,820; ramp raw 7,120 / distributed 923,280 / geometric 531,132 -- all alert decisions correct. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ment
Replace the circular closed-form rho check with an end-to-end cluster
measurement: gos_aniso_cluster.sh runs a real asap-otel agent (CountSketch +
gos_delta_epsilon, iso vs gos_anisotropic:true -- the NEW config keys, first
exercised on a live agent) fed by the otel-app five-sketch Zipf(s) producer,
counting delta bytes at the sink's :4317 with an iptables counter.
Honest result (recorded, NOT oversold): aniso saves ~40% at low skew
(s=1.1, rho=0.60) but is a wash at s=1.5/2.0 -- the OPPOSITE of the
closed-form 'savings grow with skew' story. Two confounds flagged for
follow-up (a ~303KB per-window floor swamping the delta; countsketch hashing
spreading a heavy endpoint across d random-sign cells so input-skew != cell
skew). Mechanism verified live (config parse -> applyGosMode -> per-cell
{T_j}); payoff unconfirmed on real workloads.
Adds gos-aniso-agent.yaml.tmpl + gos-eval-sink.yaml (minimal agent/sink
configs). Also documents the original rho table as a closed-form check, not
a measurement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ries sampling_alloc.go (AllocateSampleRates, the per-key sqrt(f/rate) KKT water-filling) was retired earlier this session -- the live allocation is the whole-sketch epsilon-floor in data_plane. It had no callers outside its own test (grep-confirmed). Delete both files. gitignore the compiled f2driver/e2edriver eval binaries and datasets_eval/debs/data -- untracked artifacts that kept showing in git status. Removed the stray f2driver ELF. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Fig 9 header listed fig9_f2.csv, which was retired when the F2 eval axis moved to the whole-sketch geometric protocol. The granted-p values it fed are preserved inline in the section's table; drop the dead filename and note the provenance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
nopprocessor had zero references: not in any OCB builder-config manifest, no pipeline config, no components.go registration, no tidylist entry (grep across patch tree, deploy configs, and both asap-otel manifests). Remove the patch dir + the submodule working-tree overlay copy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The telegraf / otel-arrow / vector submodules are uninitialized and no build
or eval arm exercises these integrations. Move design-asap-{telegraf,
otap-rust,vector}-integration.md to docs/dormant/ with a DORMANT banner, and
fix inbound refs (system-overview.md + the cross-links between them). Content
preserved verbatim; only relocated + banner-marked.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ellites Verified the 3 satellites are COMPLEMENTARY, not duplicate: each carries a derivation the canonical doc omits (nitrosketch: the SDK->collector split survival proof + per-family applicability + empirical; tumbling-cost: the per-family two-disciplines cost tables; taxonomy: the full 2D grid + (agg_id,group_key) keying). So deleting would lose content -- instead unify the reader's entry point: each now banners sampling-cdm-gos-derivations.md as canonical and states what is unique here. Navigation consolidated, no content dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
phase-2-{execution-plan,perf-bench-go,perf-deployment}.md all record
COMPLETED phase-2 work (runtime-extraction map, Go perf audit, perf
deployment). Concatenate into a single phase-2.md history archive (each
former doc demoted to a ## section with a provenance comment), delete the 3
originals, and repoint the ADR-0002 inbound link. 3 files → 1; content
preserved verbatim.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…es; archive resolved investigation eval-query-plan.md (0 inbound, the per-dataset query list) and eval-instrumentation-notes.md (the sweep-CSV column reference) are companions to the §6 figure plan -> merge both into evaluation-plan-figures.md as sections; repoint the runbook's inbound link. eval-label-axis-cpu-rootcause.md is a COMPLETED 2026-05 investigation -> move to docs/archive/ with a resolved banner. sdk-cost-evaluation.md kept (3 inbound, distinct topic). 5 eval docs -> 2 (+1 archived); content preserved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The *mergeprocessor Go processors (countsketch/countminsketch/ddsketch/hll/kll) were the gateway-side sketch accumulators. The asap-gateway hop was retired (#400) and cross-edge merge now runs on the backend data_plane (Rust); no pipeline ever wired them and nothing imports them (grep-confirmed across configs, code, and both manifests). They were dead weight compiled into every asap-otel binary. Remove their gomod+path entries from all OCB manifests (builder-config.yaml, builder-config-sketches.yaml, asap-otel-opamp), delete the 5 patch processor dirs, and note the removal in delta-transmission-design.md (design of record kept; mechanism now realized backend-side). Verified: OCB rebuild succeeds (exit 0) — the manifest resolves and the collector compiles without them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ale too small?') The default whole-sketch F2 workload has only H=4 distinct keys, so raw (~7KB) beats both sketch modes -- a fixed d*w~=11.5KB/ship cost cannot win at tiny cardinality. Replace the prose estimate with the measured F2_KEYS sweep: raw scales linearly with H while the sketch cost is H-independent, so the crossover is H~=500 (vs distributed) / ~280 (vs geometric); at H=32768 sketch wins 73x/127x. Clarify the two orthogonal axes: sketch-vs-raw is a cardinality question (crossover + C1-wire), geometric-vs-distributed is an H-independent monitoring question (ship-on-violation), so the 4.0x/1.74x headline holds at any scale; the H=4 run isolates the latter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e-sketch broadcast limit Add tau=250000*H auto-scaling to f2_wholesketch_cluster.sh (keeps the ramp crossing at the same fractional step at any cardinality) and default it to H=2048 realistic distinct-key count, F2_KEYS threaded to the driver over ssh. Measured at H=2048 (recorded in *_h2048.csv), two honest findings: - Sketch beats raw at scale: ramp distributed 923,280 (fixed d*w) is 4.3x smaller than raw 4,007,440 — resolves the 'raw is cheapest' concern once H is realistic. - Geometric's ramp win does NOT survive a dense sketch: geometric ramp rose 531,132 (H=4) -> 1,647,966 (H=2048) and now LOSES to distributed. Egress decomposes to ~4*28*11,541 = near-full-matrix broadcasts: 2048 keys saturate the 1280-cell sketch, so sparse C_ref deltas degenerate to full matrices and the O(k) amplification returns (design sec 12 #1 + small-norm limit, now measured). Stable geometric still wins 4x (only 4 ships). The protocol pays off when the sketch is sparse relative to its cell budget. Corrects the earlier 'geometric ratio is H-independent' claim: the ship COUNT is ~H-independent, the BYTE count is not (broadcast grows with density). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t result f2driver gains a 'zipf' ramp pattern (per-key drift weighted by Zipf(1.2), Σ weight = H·drift so τ stays comparable) to probe whether input skew helps the thresholded broadcast gate. It does not (Count-Sketch homogenizes cell magnitudes), which is now documented in gos-eval-results.md §2 alongside the density sweep: the gate is implemented, safe (H=4 byte-identical), and the right mechanism, but its Count-Sketch payoff is capped; the effective high-cardinality lever is sizing w to keep fill H/(d·w) ≪ 1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… workload) Add a real-data F2 path: debs_f2_trace.py bins the DEBS 2022 trading-day CSV (symbol = key) into step/edge per-symbol event counts, f2driver gains an F2_TRACE replay mode (sketch + raw), and f2_debs_eval.sh runs the raw / distributed / geometric matrix against the real trace. Measured (H=5493 real symbols, d=5 w=4096, tau=2.5e10, first 4M events; eval-8node/f2_debs.csv): raw 99,122,250 B (ship every event) distributed 14,747,280 B (6.7x less than raw) geometric 13,033,555 B (7.6x less than raw; beats distributed, 40/80 ships) all fire the alert at the real F2 crossing. On real data both claims hold by construction: sketch >> raw (real cardinality) and geometric > distributed (w=4096 sketch is sparse, no dense-broadcast amplification). Resolves the 'raw is cheapest / scale too small' thread with a real workload. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per the directive that reported metrics must be freshly-run, not looked up: delete all prior (June 14-20) result data — datasets_eval/multisketch/results/ *.json (perfamily accuracy, c1-wire, baselines, gt-*, guard, resource, compound), and the eval-8node June snapshots (RESULTS.md, PHASE2_INTEGRATED_ RESULTS.md, FINDINGS.md, accuracy-raw.csv, fig3/6b/7/8/9/10/11*, costmodel, basesweep, figs/). Kept: the fresh this-session CSVs (f2_debs, f2_wholesketch_cluster*, gos_aniso_cluster) and the run-it-yourself README. The consolidated e2e metrics script (next) regenerates everything fresh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-component resources) e2e_metrics.sh merges the two existing harnesses into one run against the REAL multinode ASAPQuery-backend, every number produced fresh: - run_demo.sh lib brings up the asap stack (cold node1 / warm node2 / agents node0+node3) and arm_measure captures per-component CPU/mem + per-node bandwidth + query latency (MetricsQL replay vs node2:9091); - run_perfamily.py (now env-parametrized: E2E_BACKEND / E2E_AGENT_METRICS / E2E_REPLAY_ENDPOINT / --external-stack) replays GT-known slices through the SAME warm backend and scores query ACCURACY vs exact ground truth per family (quantile rel-err / topk recall / freq envelope / cardinality rel-err); - aggregate_report.py + an inline accuracy table → one fresh E2E_METRICS.md. run_perfamily gains env overrides so its scorers run against a remote backend without standing up its own single-host stack. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ne fresh report run_demo's arm_measure references measure_per_edge_bandwidth.py/measure_stages.py which no longer exist (stale), so e2e_metrics.sh now uses the WORKING snapshot_resources.sh (per-container CPU/mem + per-node NIC, multinode) + metricsql_replay.py (query latency vs node2:9091). The report is an inline aggregator over this run's snapshot CSVs + replay.jsonl + accuracy JSONs, and degrades gracefully (notes when the family-accuracy prep did not run). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-cell sketch pipeline ships CMS/CountSketch/DDSketch cells to the backend, which reconstructs the full sketch and answers any query — F2 (‖f‖₂²) is just one such query. The dedicated geometric-CDM F2 monitor (safe-zone gating to save bandwidth) is redundant with that design, since the sketch is shipped anyway for the other queries. Remove it entirely. Edge (asap-precompute-go): delete f2engine.go (+tests), f2driver; unwire FunctionalF2/F2Mode/ParseF2Mode/Spec F2 dims, SetF2Engine/DriveF2Monitor, window.f2Visit, engine.OnRef, and the transport DTOs (Report.Sketch, RefBroadcast). Edge processor: drop functional=f2 + f2_mode. Regenerate monitorpb (drop MonitorReport.sketch, RefBroadcast, CoordToEdge.ref). Docs: delete f2-geometric-monitoring.md + retired ADRs/archive; scrub refs. Per-cell sketches never depended on F2Engine, so the excision is clean. All Go modules build; go vet clean; all tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y types debs_otlp_map.py maps DEBS 2022 trading events → OTLP for topk/freq/distinct/ quantile with exact offline ground truth; debs_backend_accuracy.sh replays through the real all-families stack and scores each query vs GT. run.py: add --anchor-span-s N — spread the wall-clock-anchored replay across N seconds (distinct ns per point) inside one window, so count-type sketches (value==1.0 per event) keep per-key multiplicity instead of deduping every event of a key onto one identical (series,ts,value) sample. e2e_metrics.sh: consolidated multinode e2e (fresh accuracy + per-component resources), swapping in the working snapshot_resources.sh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 8, 2026
Contributor
Author
|
Superseded by a clean 3-way split (F2/geometric-CDM monitoring removed as redundant with the per-cell sketch → backend query design):
Review/merge in order #509 → #510 → #511 (sketchlib-go #69 first, which #510 needs). Backend: F2 removal folded into #383, plus standalone reducer fix #384. |
This was referenced Jul 16, 2026
zzylol
added a commit
that referenced
this pull request
Jul 16, 2026
Evaluation and documentation for the GOS + NitroSketch sampling work in this stack: - DEBS 2022 real-backend query-accuracy harness (all 4 sketch query types), Google cluster trace integrated ε-sweep (accuracy·latency·freshness·resources, single-node + cluster), C1 bandwidth/encoding-factor experiments (DDSketch/HLL vs raw+gzip), anisotropic-vs-isotropic per-cell delta measurement, raw-vs-sketch cardinality crossover. - otel_collector_benchmark/epsilon_floor: ε-floor vs NitroSketch benchmark, in-tree. - docs/design-gos-unified-edge-telemetry.md: the canonical GOS design doc (unifies error-bounded sketching, coordinated sampling, Geometric Monitoring, and OctoSketch-style change transmission into one per-cell decision). - docs/sampling-cdm-gos-derivations.md: the NitroSketch/CDM sampling math derivations. - docs/gos-eval-results.md, docs/phase-2.md: recorded eval results and phase-2 implementation history. Design docs are bundled with eval/results here rather than split alongside their respective code PRs (2/3) — same "benchmarks + docs together" shape the original #508 split used for its eval(#508c) PR. Sweep-run output (raw CSVs under deploy/mvp-multinode/eval-8node/, and the INTEGRATED_SWEEP_RESULTS.md results dump) is not committed — those are eval-run artifacts, not source. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Edge side of the F2 whole-sketch monitor and the full GOS design doc.
Pairs with ASAPQuery-backend PR (coordinator + threshold allocation).
Edge runtime
f2engine.go:F2Engine— local safe-zone test (byte-mirror of Rustf2.rs/geo.rs), ship/silent gating,RefBroadcasthandling; safe radiususes
(1-ε)τ.types.go:FunctionalF2,F2Mode,Sketchreport payload,RefBroadcast.grpcclient: sketch payload +RefBroadcastwiring; regenerated stubs.countsketch.go:CellMatrix()accessor.cmd/f2driver+deploy/mvp-multinode/scripts/f2_monitor_eval.sh: multi-edgecross-language eval.
Design docs
docs/design-gos-unified-edge-telemetry.md— the complete GOS framework:unified relative error bound, freshness, tunable memory/compute/communication
objective, per-cell water-filling thresholds,
k/ε²Woodruff–Zhang optimality,attribution (GM/AutoMon/OctoSketch/sampling/WZ), implementation map.
docs/f2-geometric-monitoring.md— the F2 wiring + eval results.Tests
go test ./monitor/...green, including theis_locally_safegolden-vectorparity with the Rust side.
Eval
Geometric ≈ 4× less communication than the ship-every-window baseline in the
stable regime; ramp regime motivates the delta-broadcast follow-up.
🤖 Generated with Claude Code