Skip to content

benchmark: q5 ground truth - #102

Closed
GnaneshGnani wants to merge 8 commits into
debs_q4from
debs_q5
Closed

GnaneshGnani wants to merge 8 commits into
debs_q4from
debs_q5

Conversation

@GnaneshGnani

@GnaneshGnani GnaneshGnani commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Q5 — Realized volatility σ proxy (DDSketch IQR/1.349)

Adds ground truth and 60-minute accuracy benchmark for Q5: realized volatility approximated as IQR/1.349 from DDSketch quantiles.

Benchmark results

Setup: Day 08-11-21, data_filtered, accuracy_sla=0.01, 12 × 5-min windows
Sketch: DDSketch, relative_accuracy=0.01, quantiles [0.25, 0.75]
Metric: fraction of symbols where |sketch_σ − exact_σ| / max(...) < 10%.

Windows Pass rate Verdict
12 (09:00–10:00 CET) 3 / 12 ⚠️ Fails early market-open; passes from ~09:45

Clear warm-up pattern: at market open intraday price moves are larger and more skewed, violating the near-normal IQR/1.349 assumption. By 09:45 the distribution settles and the fraction climbs above 90%. Windows 09:45–10:00 score 0.909, 0.916, 0.910.

Cumulative: Q1: 11/12 ⚠️ | Q3: not run | Q4: 12/12 ✅ | Q5: 3/12 ⚠️

See results/benchmarking.md for per-window breakdown.

@zzylol

zzylol commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Closing — this branch is conflicting with main (badly diverged, ~32k-line deletions) and predates the consolidated multi-sketch + latency eval harness merged in #494, which redid this ground-truth work. Please reopen and rebase if any ground truth here is still needed.

@zzylol zzylol closed this Jun 14, 2026
@zzylol
zzylol deleted the debs_q5 branch July 17, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants