Conversation
…/wire in one run Extends pareto_sweep.py: per arm also measures query latency (perf_counter p50/p99), data freshness (emit→queryable poll from producer end), and per-process RSS, on top of the existing accuracy + cpu + wire. Sweeps the admission p (= the ε-floor the autonomous coordinator sets) and reports the implied ε. Phase-1 single-node table + honest caveats (accuracy ~13% calibration; freshness ~19ms is a compressed-replay artifact — real freshness/per-component/cold-tier = Phase-2 cluster). See INTEGRATED_SWEEP_RESULTS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…onent + cold-tier Thread -warm-sample-p through run_demo.sh producers (default 1.0, backward-compat). Add epsilon_cluster_sweep.sh: per admission p bring up the warm+cold asap stack (wall-clock paced) and capture per-tier freshness, per-container CPU/mem, per-node NIC bandwidth, and warm query latency. Baseline p=1.0 result in PHASE2_INTEGRATED_RESULTS.md: real freshness warm ~1.0s/archive ~0.66s (the column single-node returned None for), and the per-component split (warm sketch backend = 2% CPU / 117MiB for 5000+ sketches; cost at edge + cold archiver). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… invariant to p 5-arm cluster sweep (p 1.0→0.05). Clean result: query latency (~1.1ms p50) and warm sketch-backend cost (~1% CPU / 68MiB) are FLAT across the full sampling range — the ε-floor keeps one sketch per series so the serving path is p-independent. Freshness/edge-CPU/NIC show no systematic p-trend (expected: window-seal latency and producer-side work are independent of admission sampling); per-arm scatter is short-soak + multi-series probe noise, baseline run is the reliable freshness point. Adds aggregate_phase2_sweep.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…99, robust to ε Thread trace-replay (OTELAPP_TRACE_MOUNT/OTELAPP_TRACE_ARGS) through run_demo.sh producers so the real google-cluster-2019 cpu_rate trace flows through the distributed stack with -warm-sample-p admission sampling applied. epsilon_accuracy_sweep.sh sweeps p and queries the warm DDSketch via quantile_over_time(q,metric[5m]) (the correct form; histogram_quantile routes to thanos). accuracy=1-|sketch-GT|/GT vs exact offline GT (p99=0.043274). Result: p99 accuracy ~90% flat across p 1.0->0.05 (ε 0->0.056) — sampling is accuracy-robust; cluster DDSketch ~1-8% err vs single-node's 13% (better-calibrated window). Completes the integrated table: accuracy+latency+freshness+per-component, one stack. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zzylol
changed the title
Cold query explicitly tested (not inferred): count_over_time[10m] -> thanos_query -> 24957/25395 archived samples; timestamp-consistency via window-scaling (1m/2m empty matching the 2m+1m archive lag, 5m=16.7k, 10m=25k), cross-producer agreement ~3%, value distribution matches GT (warm p99 0.042-0.049 vs 0.0433, archived values in trace domain). Honest limit recorded: cold values confirmed indirectly (value funcs route to warm frontend). gorilla-merger: RestartCount=0 OOMKilled=false, 21MiB/32GiB, no OOM; MinIO S3 upload not observed completing in-window. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Earlier total_objs=0 was a pre-compaction-cycle reading, not a gap. Watching across the 5m compactor cycle: 'shipper uploaded blocks uploaded=1' to bucket=asap-gorilla-tsdb, MinIO holds a real TSDB block (chunks/index/meta.json, erasure-coded). Cold tier durable. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lets run_demo producers run as coordinated CDM edges (-coordinator-url / -monitor-config-url / -monitor-key) via env, alongside the existing OTELAPP_WARM_SAMPLE_P / OTELAPP_TRACE_ARGS knobs. Used to validate the autonomous /plan/auto → coordinator → ε-floor-p loop end-to-end. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… fix retired √(f/rate) ref Consolidated comparative-rigor plan: 5 claims + Pareto + autonomous-allocation + ablations, each crossed with real datasets (Google-cluster-2019 staged, DEBS-2022 downloading, 3rd gap) and real baselines (raw+codec, Prometheus/Thanos/VM, NitroSketch/OmniSketch, Cormode-CDM, ASAP ablations). Priority gaps: real baselines, statistical rigor (trials+CI), all-6-family accuracy, DEBS e2e, autonomous-alloc quality vs oracle, scale. Also corrected the headline's stale per-key sampling law to the unified whole-sketch ε-floor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…g not engaged, KLL/CMS) Fresh run on main: DDSketch p50 0.72% + HLL 0.33% solid (natural-fit gauge families); KLL p50 regressed 0.0007→0.0597, CMS one-sided violated (143<244), agent shipped 57MB raw not ~1MB sketched. Flags gate C1/C4. Data-fitness finding: frequency families belong on DEBS (symbol-trade freq), not gct gauge data. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…perating-point) 57MB wire = per-family slices all contain 8 metric aliases (agent sketches 1, forwards 7 raw); accuracy is valid. KLL median = small-N rank coarseness (DDSketch buckets win small-N); CMS one-sided = gauge data has no count to over-estimate (frequency families → DEBS). C1 wire to be re-measured with true family slices. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…C1 clean-wire harness fix identified C4 cell ✅ for the natural-fit gauge families (DDSketch p50 0.72%, HLL 0.33%) after root-cause; C1 cell notes the make_perfamily un-slice + ship-wait coupling that blocks clean per-family wire. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…→ clean sketch-vs-raw wire
Fixes the two gaps that polluted per-family wire: (1) c1_wire.py slices the data to
{cpu_rate Sum-anchor, memory_usage, ONE family metric} (drops the 6 noise aliases that
forwarded raw → 57MB), (2) agent-raw-coldoff.yaml (no asap_edge) replays the same slice
as the raw baseline. Captures W_sketch vs W_raw over N trials with 95% CI.
Validated (ddsketch, real gct, 1 trial): W_sketch 0.99MB vs W_raw 33.7MB = 34.1x reduction,
accuracy intact (p50 0.72%). Confirms the prior 57MB was an un-sliced-data artifact, not a
regression (W_sketch=0.99MB matches the prior good run).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…(real gct, 3 trials+CI) Matrix C1 cell → measured ✅. Fixed harness (true family slice + raw-forward arm) gives clean sketch-vs-raw wire with tight CIs; W_sketch reproduces the prior committed numbers exactly, settling the '57MB' question (un-sliced data, not a regression). Compression/Prom baselines remain the open comparative gap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…2.7x over raw+gzip Compression baseline (b0-gzip/zstd): gzip on the actual raw payload = 12.3x encoding factor; ASAP's 33.8x decomposes as encoding(12.3x) x residual-aggregation(2.7x) — internally consistent (12.3*2.7≈33.8). Sketching still beats a gzip-compressed raw baseline by ~2.7x. Honest caveat: --network host single-node can't isolate the agent→backend wire (loopback method failed, all arms ~35MB); clean on-wire compression belongs on the cluster NIC. Configs+driver ready for it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…60x smaller; X1 ✅ Updated Fig 9 to the unified ε-floor (retired √(f/rate)); added the standalone algorithm result (real DEBS rates, equal bandwidth): ε-floor max rel-err 0.06-0.41 vs fixed-p 3.9-40.9. First real baseline vs academic prior art (NitroSketch). Matrix X1 → ✅. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…hole-sketch findings Retract the 40-60x per-key claim (circular oracle f_k + retired law). Corrected: whole-sketch ε-floor on real CMS — 17x insert-throughput at bounded L2, memory constant, p derived from ε+observable R. Honest: ε-floor bounds L2 not per-key (rare-key accuracy inherently limited under sampling); right metric = L2/heavy-hitter (TODO). Fleet modest (1.3x, 3 exchanges). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t 1) asap-precompute-rs hardcoded sample_p=0.0 in all wrappers — it could decode a sampled envelope (query-side rescale) but never PRODUCE one, so a Rust edge under the CDM coordinator's ε-floor grant would ship unsampled full-rate sketches. Mirror Go's coordinated path (precompute.SampleSetter, applyGrantedSampleP): - New sampling.rs: GeometricSampler (NitroSketch geometric skip-sampling, splitmix64) + wire_sample_p (stamp 0.0 when exact for byte-parity, else p). - SampleSetter trait in precompute.rs — implemented by exactly CMS, CountSketch, DDSketch (the additive families). Sum/KLL/HLL deliberately excluded, matching Go's switch in applyGrantedSampleP (HLL keeps its own hash-threshold path). - Each of the 3 wrappers: sample_p field + GeometricSampler, with_sample_p builder + set_sample_p (reseed), admit-gate in update(), stamp wire_sample_p in the envelope, reseed on reset(). Seeds mirror Go (cms ZMPC / cs 0x5a3e06d / dd DDSP). - Count sketches rescale by 1/p on query; DDSketch is scale-invariant (samples to shed work, no rescale) — both still stamp p. Exact path unchanged: sample_p stays 0.0 → #243 byte-parity preserved (all 57+11+27 tests green). New test asserts admit≈p fraction + envelope stamp + SetSampleP. Part 2 (runtime hook: derive p from the engine's granted ε-floor and install the window sample-hook, mirroring Go's applyGrantedSampleP wiring) is a follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…antile (#30) Locks in the quantile case: sampled DDSketch admits ~p of updates (total_count ~p×) but its quantile shape survives (scale-invariant, no 1/p rescale), and the envelope stamps p while the exact path stamps 0.0 for byte-parity. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…+ skewed fleet) F1-aggregate held at ε (0.050@ε=.05); point-query degrades as ε·√(R/f_k) (heavy survive, rare lost); 22× insert-tput; synthetic fleet cold-edge 7× better than fixed-p. Fig 9 → ✅ algorithm. X1 cell updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ollector_benchmark Relocate the standalone ε-floor benchmark INTO ASAPCollector (its own Go module, relative replace → ../../../sketchlib-go, mirroring cardinality_crossover) so the paper-eval code lives with the rest of the eval and sketchlib-go stays unmodified. Whole-sketch ε-floor p=1/(1+ε²R) on real CMS over real DEBS: - 20× insert throughput (183 vs 8.9 Mupd/s), CMS memory constant - accuracy LAW: F1-aggregate held at ≈ε (0.050@ε=.05); point query degrades as ε·√(R/f_k) (heavy survive, rare lost) - synthetic skewed fleet (rate-CV=2.9): ε-floor equalizes per-edge err at ε; fixed-p under-protects the cold edge 7× (0.148 vs 0.020) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Superseded by |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
End-to-end integrated evaluation of coordinated ε-floor sampling, measuring all four metric classes together on one stack, swept over ε (p = 1/(1+ε²·rate)): query accuracy, query latency, data freshness (emit→queryable), and per-component resources (CPU/mem/bandwidth) — plus warm/cold two-tier routing.
Phase-1 — single-node (
datasets_eval/google_cluster/e2e/)epsilon_e2e_sweep.py+INTEGRATED_SWEEP_RESULTS.md. google-cluster-2019 cpu_rate (pooled). Accuracy flat ~0.85–0.87 across p, latency ~2.5 ms, CPU 0.10→0.08 with sampling, wire ~40× below raw, RSS bounded. (Freshness is degenerate single-node — compressed replay seals windows instantly → measured on the cluster instead.)Phase-2 — 8-node cluster (
deploy/mvp-multinode/)PHASE2_INTEGRATED_RESULTS.md+epsilon_cluster_sweep.sh(freshness/resources/latency) +epsilon_accuracy_sweep.sh(accuracy) +aggregate_phase2_sweep.py. Threaded-warm-sample-pand trace-replay (OTELAPP_TRACE_MOUNT/OTELAPP_TRACE_ARGS) throughrun_demo.shproducers so the same google_cluster trace flows through the distributed warm+cold stack with admission sampling applied.Results, swept over ε 0→0.056 (p 1.0→0.05):
Headline: on real hardware, coordinated ε-floor sampling buys ~40× volume reduction while accuracy, latency, freshness, and backend cost all stay essentially invariant to the sampling rate — the point of the unified law.
Honest caveats documented in each results doc (single-node accuracy ~13% calibration vs Fig 3; freshness/edge-CPU/NIC per-arm scatter is short-soak + multi-series-probe noise, not a p-trend).
🤖 Generated with Claude Code