eval(inference): expand warm-tier PromQL pattern coverage - #79
Merged
Merged
Conversation
Expands the canonical inference YAMLs from one entry per metric to cover multi-quantile, wider ranges (`[2m]` / `[5m]`), rate / increase, sum / count over time, spatial aggregations, and top-K. Each new pattern family is pinned by an integration test that exercises the YAML → `find_query_config` → `query_statistic` routing end-to-end against an appropriately-typed accumulator. Pre/post: PromQL YAML 1 → 33 entries, SQL YAML 1 → 8 entries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
asap-query-engine/examples/promql/inference_config.yaml(and the SQL twin) from a single entry per metric to cover multi-quantile, wider ranges, rate / increase, sum / count over time, spatial aggregations, and top-K. Each entry binds a(metric, PromQL pattern) → (agg_id, plan)tuple consumed byfind_query_config's exact-string match.asap-query-engine/tests/inference_yaml_pattern_coverage.rs(10 tests) pinning the YAML →find_query_config→parse_and_match_promql→query_statisticrouting for every new family with synthetic ingest, asserting non-empty / correct-shape vector results.Pre/post coverage
examples/promql/inference_config.yamlquantile_over_timeφ × range, 6sum_over_time/count_over_time× range, 5rate/increase× range, 3 spatial aggregations, 3topk(N, ...))examples/sql/inference_config.yamlPer-pattern wiring confirmation
quantile_over_time(φ, m[r])Statistic::QuantileDDSketchAccumulator,DatasketchesKLLAccumulatorsum_over_time(m[r])Statistic::SumDDSketchAccumulator,SumAccumulator,CountSketchAccumulator,CountMinSketchAccumulatorSumAccumulatoris the simplest backing; sketches answer with no-key totalscount_over_time(m[r])Statistic::CountDDSketchAccumulator,SumAccumulator,HllSketchAccumulator(HLL aliases Count → Cardinality)rate(m[r])/increase(m[r])Statistic::Rate/Statistic::IncreaseIncreaseAccumulator,MultipleIncreaseAccumulatorcount(m)Statistic::Countsum(m)/avg(m)Statistic::Sum/(Sum, Count)SumAccumulator/DDSketchAccumulatorquantile by (lbl) (φ, m)Statistic::Quantile(spatial)DatasketchesKLLAccumulatortopk(N, m)Statistic::TopkCountSketchAccumulator(row-mean total when no key supplied)Patterns deliberately NOT added (would require code changes)
histogram_quantile(φ, ...). Not incontroller_patterns—parse_and_match_promqlonly recognisesquantile_over_timefor the temporal-quantile shape andsum / count / avg / quantile / min / max / topkfor the spatial shape.histogram_quantilequeries reach the warm tier today only via the controller's stage-split path (which decomposes them intorate(...)precompute jobs); listinghistogram_quantile(...)in the inference YAML would silently fail-to-match against incoming canonical-AST strings.topk(N, ...)over CountSketch. Thequery_statisticno-key fallback returns row-mean total volume, not enumerated heavy-hitters. Per-key enumeration needs a pairedSetAggregator/DeltaSetAggregatoron the agent (already tracked as a follow-up inPROGRESS.md).Open question
The task references
backend-inference.yamlby that exact filename. That file does NOT live in this repo — it's a deployment-side overlay atASAPCollector/deploy/configs/backend-inference{,-cms,-hll,-kll,-cs}.yaml, which theasap/query-backend:devimage volume-mounts into the container. The canonical schema source for the inference YAML format is theexamples/promql/inference_config.yamltemplate in this repo (consumed viaread_inference_config→InferenceConfig::from_yaml_file), which this PR expands. A separate PR againstASAPCollectorwill mirror the same pattern families into the per-sketch deployment overlays.Test plan
cargo test --release -p query_engine_rust --test inference_yaml_pattern_coverage— 10/10 pass.cargo test --release -p query_engine_rust capability_matching— 5/5 pass (no regression to capability-fallback path).cargo test --release -p query_engine_rust structural_matching— 5/5 pass (no regression to AST-canonical match path).cargo test --release -p query_engine_rust file_io::tests::test_read_inference_config— passes (YAML schema unchanged).cargo clippy --release -p query_engine_rust --testsclean for the new test file (5 pre-existing-D warningserrors inengines/simple_engine.rsandcount_sketch_accumulator.rsare unrelated to this PR's domain — not touched per file-domain constraints).