Measure MetricsQL candidates against VictoriaMetrics - #626
Merged
Merged
Conversation
# Conflicts: # data_plane/src/query_engines/asap_query_engine/post_asap_readout.rs
This was referenced Sep 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VictoriaMetrics evaluation now uses the shared MetricsQL compiler entry and native exact process with matching cache policy, per-query accuracy units, and complete candidate CPU accounting. Accuracy calibration uses production accumulators and a disjoint key namespace; it does not substitute HLL standard error for measured per-query error.
The normal control-plane compiler selected HLL p12 from aligned measured candidate quotes. An independent fresh-process evaluation of the generated 40-series, one-hour, 100ms dataset completed 60/60 warm queries with complete results and maximum relative error 4.557%, within the empirical 5% target. Median latency was 13.174ms versus native VM 62.564ms; combined backend+externalVM lifecycle CPU was 15.264s versus 16.735s. Memory was higher, not lower. Full scope, provenance, and limitations are in
docs/developer_docs/hll-cost-selected-victoriametrics-evaluation.md.Verification: MetricsQL entry and HLL ERP regression passed; workflow Python tests passed; release accuracy calibration, complete cost selection, and independent selected-plan/native process runs completed. The predeclared p10 failure remains recorded. CI is pending for the latest documentation update.
Stacked on #619. The generated sensitivity case does not replace original o11ybench/YAML workload coverage, larger-scale measurements, or moving-window/concurrent-throughput experiments; these remain pending. Native VM v1.126.0 rejects entropy_over_time and l2_over_time, so this PR claims no native comparison for those functions. Finite ingest feedback is implemented separately in #638; autonomous continuous replanning is outside this PR.