Observability: /plan/:metric/diff, /cost-model, and HTTP integration tests - #84
Merged
Merged
Conversation
Adds a `query_parser` module that implements SP-1 workload extraction from raw query strings, covering DEBS 2022 financial queries and ClickBench SQL patterns, following the SQL-to-sketch mapping rules in the design doc. ## query_parser/promql.rs Regex-based PromQL parser recognising: quantile_over_time, avg/min/max/ sum/count_over_time, histogram_quantile, topk(k, count_over_time), and count(count_over_time ... by (dims)) for cardinality. Bare selectors (no agg fn) are marked exact_required for RSI/MACD/stochastic passthrough. ## query_parser/sql.rs sqlparser-AST traversal implementing the doc's Generate_SQL_Aggregation_ Sketch_Mapping rules: COUNT(*)+GROUP BY → Frequency; COUNT(DISTINCT) → Cardinality (HLL); AVG → Quantile p50; MIN/MAX → Quantile p0/p100; ORDER BY DESC LIMIT k → heavy-hitter CountSketch; SUM → exact. ## query_parser/mod.rs QueryHint enum for named DEBS patterns (DebsEma, DebsTopK, DebsPriceStats, DebsVolatility, DebsCardinality, DebsTwap, DebsAnomaly, ExactRequired). debs_hint() classifies financial.last_trade_price queries by quantile set. ## analyzer.rs Adds optional query_string field to QuerySpec. When provided, the parsed result populates metric_name, aggregations, time_window, group_by_labels, and label_filters; explicit fields override parsed values. ## types.rs / rules.rs Extends QueryWorkload with exact_required and quantiles fields. The RulesPlanner returns a raw-passthrough plan when exact_required=true and seeds DDSketch/KLL quantile params from parsed φ values. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…delivery (#80) - SP-5 (online_cost_model): expose OnlineMetricsStore from planner; wire into CostModelPlanner via with_online_store() so scoring uses EMA-blended costs - SP-8 (feedback loop): Scraper gains dynamic add/remove/set_sketch_type endpoints backed by Arc<RwLock<>>; new on_metrics callback feeds observed bandwidth + CPU/sample deltas into the EMA store after each scrape - OpAMP role tracking: X-Agent-Role header distinguishes agent vs backend collectors; push_to_role() delivers role-appropriate configs; on_connect / on_disconnect callbacks register/deregister scrape endpoints automatically - main.rs: initialise EMA store, wire Scraper + OpAMP callbacks, start scraper background loop, push agent config to Agent-role and backend config to Backend-role collectors on every plan Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
) Enumerates all Pareto-optimal collection plans across three objectives (bandwidth bytes/sec, CPU µs/sample, memory bytes). A plan is included only if no other plan is strictly better on all three dimensions and it meets the accuracy SLA. The frontier is sorted by caller-supplied ObjectiveWeights so select_best() returns the single preferred plan. New HTTP endpoint POST /api/v1/plan/pareto accepts the same QuerySpec payload plus optional weights JSON, and returns frontier + recommended sketch. Uses the live EMA cost table when available. 7 new tests: frontier non-empty, all points meet SLA, weight-based selection, tight-SLA filtering, no dominated points invariant. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- WorkloadStore (store/workload.rs): persists (QueryWorkload, WorkloadCharacteristics)
per metric so the replanner can call planner.plan() without the original QuerySpec
- Replanner (replan.rs): closes the SP-8 feedback loop with two triggers:
- violation-triggered: scraper SLA violation → agent_id→metric lookup → replan_metric()
- expiry-triggered: background ticker every 5 min calls replan_expired() for any
metric whose valid_until has passed
After re-planning: pushes updated YAML to role-appropriate collectors via OpAMP,
updates scraper endpoint sketch types for correct EMA attribution
- main.rs: builds WorkloadStore + Replanner, late-binds into violation callback via
Arc<RwLock<Option<Replanner>>> cell, starts expiry-ticker background task,
persists workload to WorkloadStore in handle_plan, registers agent→metric mapping
- 9 new tests in replan.rs and workload.rs covering all re-plan paths
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New endpoints: - GET /api/v1/plan/:metric/diff — returns sketch_type, delta_transmission and mode changes between current and previous plan; 404 if metric unknown, has_diff=false if only one plan version exists - GET /api/v1/cost-model — returns EMA-blended cost table per sketch type with observation count so operators can see how much live data has influenced the planner HTTP integration tests (api_tests in main.rs, 12 new tests): - POST /api/v1/plan: happy path, invalid metric name (422), unknown agg (422) - GET /api/v1/plan/:metric: 404, returns data after POST - POST /api/v1/plan/:metric/rollback: no previous (400), not found (400) - GET /api/v1/plan/:metric/diff: no previous (has_diff=false), 404 - GET /api/v1/cost-model: all sketch types present with observation counts - POST /api/v1/plan/pareto: frontier non-empty, best sketch set - GET /api/v1/agents: empty map initially Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace regex-based query parsing with full AST parsers: - PromQL: promql-parser 0.8 (GreptimeTeam) — walks all *_over_time, histogram_quantile, topk/count/sum/avg/stddev aggregate operators - SQL: sqlparser 0.61 (apache/datafusion-sqlparser-rs, already in use) — implements the Top-Down SQL-to-sketch mapping algorithm with WHERE push-down, HAVING filters, top-K detection, JOIN push-down, UNION ALL New shared sketch algebra IR (SketchExpr) with 9 operators (Source, Filter, Window, Partition, Agg, Dedup, TopK, Merge, JoinSketch) and 8 algebraic rewrite rules (filter push-down, sketch linearity, Hydra multi-key, HLL dedup elimination, etc.). Backward-compatible parse_query() → ParsedQuery shim preserved for existing analyzer. Docs: docs/sketch-algebra-query-mapping.md covers all operator→sketch mappings, rewrite rules, mergeability reference, and DEBS/ClickBench case studies. 180 tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- query_parser/*, replan.rs: take main's versions (BaselinePlanner, full SP-1 query parser) - planner/mod.rs: keep both baseline_planner and pareto modules - main.rs: keep BaselinePlanner (main) + observability's online_cost_model import and test helper; fix test AppState to use BaselinePlanner Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SieDeta
pushed a commit
that referenced
this pull request
Apr 17, 2026
…tests (#84) * SP-1: PromQL and SQL query parsing with DEBS Q1–Q12 and sketch mapping Adds a `query_parser` module that implements SP-1 workload extraction from raw query strings, covering DEBS 2022 financial queries and ClickBench SQL patterns, following the SQL-to-sketch mapping rules in the design doc. ## query_parser/promql.rs Regex-based PromQL parser recognising: quantile_over_time, avg/min/max/ sum/count_over_time, histogram_quantile, topk(k, count_over_time), and count(count_over_time ... by (dims)) for cardinality. Bare selectors (no agg fn) are marked exact_required for RSI/MACD/stochastic passthrough. ## query_parser/sql.rs sqlparser-AST traversal implementing the doc's Generate_SQL_Aggregation_ Sketch_Mapping rules: COUNT(*)+GROUP BY → Frequency; COUNT(DISTINCT) → Cardinality (HLL); AVG → Quantile p50; MIN/MAX → Quantile p0/p100; ORDER BY DESC LIMIT k → heavy-hitter CountSketch; SUM → exact. ## query_parser/mod.rs QueryHint enum for named DEBS patterns (DebsEma, DebsTopK, DebsPriceStats, DebsVolatility, DebsCardinality, DebsTwap, DebsAnomaly, ExactRequired). debs_hint() classifies financial.last_trade_price queries by quantile set. ## analyzer.rs Adds optional query_string field to QuerySpec. When provided, the parsed result populates metric_name, aggregations, time_window, group_by_labels, and label_filters; explicit fields override parsed values. ## types.rs / rules.rs Extends QueryWorkload with exact_required and quantiles fields. The RulesPlanner returns a raw-passthrough plan when exact_required=true and seeds DDSketch/KLL quantile params from parsed φ values. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * SP-5/SP-8: runtime wiring — EMA cost model, dynamic scraper, backend delivery (#80) - SP-5 (online_cost_model): expose OnlineMetricsStore from planner; wire into CostModelPlanner via with_online_store() so scoring uses EMA-blended costs - SP-8 (feedback loop): Scraper gains dynamic add/remove/set_sketch_type endpoints backed by Arc<RwLock<>>; new on_metrics callback feeds observed bandwidth + CPU/sample deltas into the EMA store after each scrape - OpAMP role tracking: X-Agent-Role header distinguishes agent vs backend collectors; push_to_role() delivers role-appropriate configs; on_connect / on_disconnect callbacks register/deregister scrape endpoints automatically - main.rs: initialise EMA store, wire Scraper + OpAMP callbacks, start scraper background loop, push agent config to Agent-role and backend config to Backend-role collectors on every plan Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * SP-6: Pareto frontier planner and POST /api/v1/plan/pareto endpoint (#82) Enumerates all Pareto-optimal collection plans across three objectives (bandwidth bytes/sec, CPU µs/sample, memory bytes). A plan is included only if no other plan is strictly better on all three dimensions and it meets the accuracy SLA. The frontier is sorted by caller-supplied ObjectiveWeights so select_best() returns the single preferred plan. New HTTP endpoint POST /api/v1/plan/pareto accepts the same QuerySpec payload plus optional weights JSON, and returns frontier + recommended sketch. Uses the live EMA cost table when available. 7 new tests: frontier non-empty, all points meet SLA, weight-based selection, tight-SLA filtering, no dominated points invariant. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * SP-8: automated re-planning on violation and plan expiry (#83) - WorkloadStore (store/workload.rs): persists (QueryWorkload, WorkloadCharacteristics) per metric so the replanner can call planner.plan() without the original QuerySpec - Replanner (replan.rs): closes the SP-8 feedback loop with two triggers: - violation-triggered: scraper SLA violation → agent_id→metric lookup → replan_metric() - expiry-triggered: background ticker every 5 min calls replan_expired() for any metric whose valid_until has passed After re-planning: pushes updated YAML to role-appropriate collectors via OpAMP, updates scraper endpoint sketch types for correct EMA attribution - main.rs: builds WorkloadStore + Replanner, late-binds into violation callback via Arc<RwLock<Option<Replanner>>> cell, starts expiry-ticker background task, persists workload to WorkloadStore in handle_plan, registers agent→metric mapping - 9 new tests in replan.rs and workload.rs covering all re-plan paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Observability endpoints and HTTP integration tests (#84) New endpoints: - GET /api/v1/plan/:metric/diff — returns sketch_type, delta_transmission and mode changes between current and previous plan; 404 if metric unknown, has_diff=false if only one plan version exists - GET /api/v1/cost-model — returns EMA-blended cost table per sketch type with observation count so operators can see how much live data has influenced the planner HTTP integration tests (api_tests in main.rs, 12 new tests): - POST /api/v1/plan: happy path, invalid metric name (422), unknown agg (422) - GET /api/v1/plan/:metric: 404, returns data after POST - POST /api/v1/plan/:metric/rollback: no previous (400), not found (400) - GET /api/v1/plan/:metric/diff: no previous (has_diff=false), 404 - GET /api/v1/cost-model: all sketch types present with observation counts - POST /api/v1/plan/pareto: frontier non-empty, best sketch set - GET /api/v1/agents: empty map initially Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * SP-1: full AST-based PromQL/SQL parsers with sketch algebra IR Replace regex-based query parsing with full AST parsers: - PromQL: promql-parser 0.8 (GreptimeTeam) — walks all *_over_time, histogram_quantile, topk/count/sum/avg/stddev aggregate operators - SQL: sqlparser 0.61 (apache/datafusion-sqlparser-rs, already in use) — implements the Top-Down SQL-to-sketch mapping algorithm with WHERE push-down, HAVING filters, top-K detection, JOIN push-down, UNION ALL New shared sketch algebra IR (SketchExpr) with 9 operators (Source, Filter, Window, Partition, Agg, Dedup, TopK, Merge, JoinSketch) and 8 algebraic rewrite rules (filter push-down, sketch linearity, Hydra multi-key, HLL dedup elimination, etc.). Backward-compatible parse_query() → ParsedQuery shim preserved for existing analyzer. Docs: docs/sketch-algebra-query-mapping.md covers all operator→sketch mappings, rewrite rules, mergeability reference, and DEBS/ClickBench case studies. 180 tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New endpoints
GET /api/v1/plan/:metric/diff— shows what changed between the current and previous plan: sketch_type, delta_transmission, and processor mode. Returns{"has_diff": false}when only one plan version exists; 404 when the metric is unknown.GET /api/v1/cost-model— returns the EMA-blended cost table for every sketch type, includingobservationscount so operators know how much live scrape data has influenced plan selection.HTTP integration tests (12 new)
Tests live in
main.rs::api_testsusingtower::ServiceExt::oneshotagainsttest_app()— a minimal in-processAppState+Routerwith no real network connections:plan_happy_pathplan_invalid_spec_returns_422plan_invalid_aggregation_returns_422get_plan_not_found_returns_404get_plan_after_postrollback_no_previous_returns_400rollback_not_found_returns_400diff_no_previous_returns_has_diff_falsediff_not_found_returns_404cost_model_returns_all_sketch_typespareto_returns_frontier_for_quantileagents_returns_empty_map_initiallyTest plan
cargo test— all 156 tests passStacks on #83 (
controller-replan).🤖 Generated with Claude Code