diff --git a/control_plane/docs/design.md b/control_plane/docs/design.md index 48cf2243..40164d2f 100644 --- a/control_plane/docs/design.md +++ b/control_plane/docs/design.md @@ -43,8 +43,8 @@ DataCollector/controller already documents its query→sketch translation as a 5 | # | Layer | What it does | Today's locations | |---|-------|--------------|-------------------| -| 1 | **Query Language** | Parse raw strings (PromQL, SQL, DataFusion, ElasticDSL, …) into a language-specific AST | DC `controller/src/query_parser/{promql,sql}.rs` + the per-language façade `controller/src/query_parser/language/{promql,sql,elastic_dsl}/`; asap-planner-rs pulls `promql-parser` + `sqlparser` directly; asap-fusion consumes a pre-built DataFusion `LogicalPlan` (its L1 happens upstream). *Refactor 2026-05 absorbed `controller/src/query_language/` into `query_parser::language/`.* | -| 2 | **Language Logical Plan** | Per-language algebra tree (`Aggregate` / `Window` / `Filter` / `Sort` / `Limit`) preserving language semantics, **no sketch names, no sketch binding** | DC `controller/src/language_logical_plan/{lower,plan}.rs` + `controller/src/intent_algebra/legacy_expr.rs` (the legacy L2 relational IR the `query_parser` front ends emit); asap-fusion inherits DataFusion's `LogicalPlan` as its L2; asap-planner-rs has no L2 today (uses a template-pattern catalogue) — **Phase 4 builds one** | +| 1 | **Query Language** | Parse raw strings (PromQL, SQL, DataFusion, ElasticDSL, …) into a language-specific AST | DC `controller/src/query_parser/{promql,sql}.rs` (each parses straight to the L2 relational tree below); asap-planner-rs pulls `promql-parser` + `sqlparser` directly; asap-fusion consumes a pre-built DataFusion `LogicalPlan` (its L1 happens upstream) | +| 2 | **Language Logical Plan** | Per-language algebra tree (`Aggregate` / `Window` / `Filter` / `Sort` / `Limit`) preserving language semantics, **no sketch names, no sketch binding** | DC `controller/src/intent_algebra/legacy_expr.rs` — the L2 relational `QueryExpr` tree the `query_parser` front ends emit (one shared tree for PromQL + SQL today; a per-language split is future work); asap-fusion inherits DataFusion's `LogicalPlan` as its L2; asap-planner-rs has no L2 today (uses a template-pattern catalogue) — **Phase 4 builds one** | | 3 | **Intent algebra** | Language- and deployment-independent IR: `QueryExpr` + `AggIntent`. Describes **intent only** — *what* to compute, with accuracy target. **No sketch type, no sketch parameters, no sketch-bound nodes** (`SketchAgg` / `SketchJoin` / `SketchSubtract` etc. live in the L4 IR `SketchExpr`). **No language-shaped operators** (no `HistogramQuantile`, no `PromQLSubquery` — those are PromQL L2 nodes that lower to data-model-agnostic shapes here). **One canonical form per plan** — no `WindowedAgg` (use `Window` over `Aggregate`). Heavy-hitter intents are first-class (`AggIntent::TopK`) so heavy-hitter sketches bind directly on the intent rather than on a generic `Sort + Limit` shape; generic `Sort + Limit` survives in `QueryExpr` for non-heavy-hitter cases (e.g. `ORDER BY name LIMIT 10`). Every edge carries a typed `Schema`. Data-model-agnostic — `QueryExpr::Scan` wraps a `Source` sum with `TimeSeries` / `Table` / `Join` variants so the same L3 IR covers ASAPQuery's time-series queries and asap-fusion's tabular queries. | DC `controller/src/intent_algebra/{agg_intent,query_expr,schema,lower,cse}.rs` (canonical L3) + `controller/src/intent_algebra/legacy_to_canonical.rs` (the L2→L3 converter: folds the legacy `legacy_expr` relational tree straight to the canonical IR, single-statistic sketchable `Aggregate` fusion included — no intermediate fused L3 IR); asap-fusion's 3-variant `SubPopulationAnalyticsType` maps to a subset; asap-planner-rs's 9-variant `Statistic` maps to a subset — **Phase 4 splits sketch binding out of planner's current fused L3+L4**. *Refactor 2026-05 absorbed `controller/src/algebra/expr.rs` into `intent_algebra/legacy_expr.rs`.* | | 4 | **Sketch algebra + optimizer** | Cost-aware algebraic rewrite rules under deployment constraints. **This is where sketch binding happens** — L4 rules take intent-only L3 (`QueryExpr`) and emit the sketch-bound IR (`SketchExpr`). ~12 rules in DC; a smaller targeted subset in planner; `SketchConfigRule` + `HashModeRule` in fusion. | Core provides the **rule engine driver** + `OptimizerRule` trait + a shared rule library + the sketch-bound IR `core::sketch_algebra::SketchExpr`; deployment models **pick** which rules to enable + supply their own deployment constraints. DC `controller/src/sketch_algebra/` (IR + binding rules) + `controller/src/optimizer/{engine,trait_def,baseline,rules/,cost/}.rs` (rule engine + cost models); fusion `src/optimizer/rules/`; planner's `map_statistic_to_precompute_operator`. *Refactor 2026-05 absorbed `controller/src/algebra/optimizer.rs` (→ `optimizer/engine.rs`) and `controller/src/planner/{cost_model,delta_cost_model,online_cost_model,pareto,tco,wire_cost,rules,baseline_planner}.rs` (→ `optimizer/{cost/,rules/,baseline.rs}`).* | | 5 | **Physical Execution Plan** | Assign ops to pipeline stages (edge / gateway / backend / object store); produce the deployment-specific artifact (OpAMP YAML, `streaming_config.yaml`, rewritten DataFusion `LogicalPlan`). **Sketch binding is already committed by L4**; L5 is about stage allocation + emission. | Core provides the **stage allocator framework** + `PhysicalPlanner` trait + the sketch catalogue; deployment models supply their own **topology** (3-stage / 1-stage / 0-stage) + their own **emitter** for the output format. DC `controller/src/physical/{allocator,planner,plan,sketch_catalog,stage_split,topology,colored_dag/}.rs` (allocator framework + sketch catalogue + typed three-stage colouring) + `controller/src/emit/{stage_config,otap,telegraf,agent,backend,asapquery_backend,precompute,trait_def}.rs` (per-deployment-model emitters + `PlanEmitter` trait) + `controller/src/pipeline.rs` (L1→…→L5 driver, formerly `analyzer.rs`); asap-planner-rs `output/generator.rs`; asap-fusion `src/executor/`. *Refactor 2026-05 absorbed `controller/src/algebra/{physical,allocator,plan,directory}.rs` and `controller/src/planner/stage_split.rs` and the legacy `controller/src/stage_split/` framework into `physical/`; absorbed `controller/src/config/` into `emit/` + `workload.rs`; renamed `analyzer.rs` to `pipeline.rs`.* | @@ -167,9 +167,9 @@ If something must be optional (e.g. OpAMP for deployments that don't run OTel co > > | Target §5 path | Current single-crate path | > |---|---| -> | `crates/core/query_language/` | `controller/src/query_parser/language/` | -> | `crates/core/logical_plan/` | `controller/src/language_logical_plan/` | -> | `crates/core/intent_algebra/` | `controller/src/intent_algebra/` (with `legacy_expr` carrying the legacy L2 relational IR and `legacy_to_canonical` converting it to the canonical L3 types) | +> | `crates/core/query_language/` | `controller/src/query_parser/{promql,sql}.rs` (L1 parsers — emit the L2 tree directly) | +> | `crates/core/logical_plan/` | `controller/src/intent_algebra/legacy_expr.rs` (the L2 relational `QueryExpr` tree) | +> | `crates/core/intent_algebra/` | `controller/src/intent_algebra/` (with `legacy_expr` carrying the L2 relational IR and `legacy_to_canonical` lowering it to the canonical L3 types) | > | `crates/core/sketch_algebra/` | `controller/src/sketch_algebra/` | > | `crates/core/optimizer/{engine,trait,rules,cost}/` | `controller/src/optimizer/{engine.rs,trait_def.rs,rules/,cost/,baseline.rs}` | > | `crates/core/physical/{planner_trait,stage_allocator,topology,executor,sketch_catalog}/` | `controller/src/physical/{planner,allocator,plan,stage_split,sketch_catalog,topology,colored_dag/}.rs` | @@ -285,11 +285,11 @@ Core is not a trait-stubs library. It ships real L1/L2/L3 code lifted from DC's > > | `core::*` reference in §6 | Current single-crate path | > |---|---| -> | `core::query_language` | `controller/src/query_parser/language/` | -> | `core::logical_plan` | `controller/src/language_logical_plan/` | -> | `core::intent_algebra` | `controller/src/intent_algebra/` (canonical) + `controller/src/intent_algebra/legacy_{expr,lower}.rs` (legacy IR pending unification) | +> | `core::query_language` | `controller/src/query_parser/{promql,sql}.rs` | +> | `core::logical_plan` | `controller/src/intent_algebra/legacy_expr.rs` (the L2 relational `QueryExpr` tree) | +> | `core::intent_algebra` | `controller/src/intent_algebra/` (canonical `query_expr` / `agg_intent`) + `controller/src/intent_algebra/legacy_expr.rs` (the L2 relational IR) | > | `core::sketch_algebra` | `controller/src/sketch_algebra/` | -> | `core::lower` | per-layer: `controller/src/language_logical_plan/lower.rs` + `controller/src/intent_algebra/{lower,legacy_to_canonical}.rs` + `controller/src/sketch_algebra/lower.rs` | +> | `core::lower` | per-layer: `controller/src/intent_algebra/{lower,legacy_to_canonical}.rs` + `controller/src/sketch_algebra/lower.rs` | > | `core::optimizer::engine` | `controller/src/optimizer/engine.rs` | > | `core::optimizer::trait` | `controller/src/optimizer/trait_def.rs` (placeholder) | > | `core::optimizer::rules` | `controller/src/optimizer/rules/` | @@ -322,26 +322,24 @@ Per-language parsers, one module each: Each returns a language-flavored AST type. No sketch awareness. -> **Implementation status (Phase D).** L1 lives in -> `controller/src/query_language/` with a `Language` trait -> (`fn id() -> QueryLanguage; fn parse(&str) -> Result`) -> + a `LanguageAst` sum type (one variant per `QueryLanguage`). The -> PromQL backend (`query_language::promql::PromQLLanguage`) wraps the -> existing `query_parser::{parse_query, parse_query_expr}` entry points -> — no parsing logic was duplicated; `PromQLAst` bundles the algebra -> tree (`QueryExpr`) and the flat `ParsedQuery` summary the legacy -> analyzer expects. The `Sql`, `DataFusion`, and `ElasticDsl` -> backends are stubbed: they implement `Language` but `parse` returns -> `ParseError::Unimplemented(...)`. This keeps the type system uniform -> for the orchestrator while the DC build ships PromQL only. L2 lives -> in `controller/src/language_logical_plan/`: `LanguageLogicalPlan` -> mirrors `LanguageAst` with one variant per language; PromQL's L2 IS -> the existing `QueryExpr` tree (rebadged + paired with a flat -> `LanguageLogicalPlanSummary` projection). `lower_to_logical_plan` -> is the L1 → L2 pass; non-PromQL variants surface -> `LoweringError::UnsupportedLanguage` cleanly. The legacy -> `query_parser::parse_query` / `parse_query_expr` entry points stay -> unchanged for back-compat. +> **Implementation status.** L1 + L2 are realised concretely, without +> the `Language`-trait / `LanguageAst` indirection the target design +> sketches. L1 lives in `controller/src/query_parser/{promql,sql}.rs` — +> each parser walks its language's AST and emits the **L2 relational +> tree directly** (`intent_algebra::legacy_expr::QueryExpr`: `Aggregate` +> / `Window` / `Filter` / `Join` / `Sort` / `Limit` / …). That tree +> *is* L2. `query_parser::parse_query_expr_canonical` then lowers it to +> the canonical L3 IR via `intent_algebra::legacy_to_canonical` +> (sketch-fusion + the Binder folded in); `parse_query` projects the +> flat `ParsedQuery` summary the legacy analyzer / pipeline consume. +> +> An earlier `Language`-trait + `LanguageAst` + `LanguageLogicalPlan` +> scaffold (a per-language enum mirroring the target design) was built +> ahead of its call site, never wired into the pipeline, and drifted +> out of sync — its "L2" actually held the L3 tree. It was removed; the +> per-language `Language`-trait extension point gets rebuilt when a +> second real language backend (SQL beyond the current direct parser, +> DataFusion, ElasticDSL) actually needs it. ### `core::logical_plan` — Layer 2 @@ -1872,7 +1870,7 @@ The pre-refactor layout grew organically as the controller absorbed three legacy | `controller/src/planner/stage_split.rs` | `controller/src/physical/stage_split.rs` | | `controller/src/analyzer.rs` | `controller/src/pipeline.rs` | | `controller/src/stage_split/` | `controller/src/physical/colored_dag/` | -| `controller/src/query_language/` | `controller/src/query_parser/language/` | +| `controller/src/query_language/` | *(deleted)* — was moved to `controller/src/query_parser/language/`, then removed as unwired scaffolding; the real L1 parsers are `query_parser/{promql,sql}.rs` | | `controller/src/config/workloads.rs` | `controller/src/workload.rs` | | `controller/src/config/{stage_config*,otap,telegraf,agent,backend,asapquery_backend,precompute}.rs` | `controller/src/emit/{stage_config,otap,telegraf,agent,backend,asapquery_backend,precompute}.rs` | | (new) | `controller/src/emit/trait_def.rs` — `PlanEmitter` trait placeholder | @@ -1885,6 +1883,6 @@ The pre-refactor layout grew organically as the controller absorbed three legacy **TODOs left from the refactor:** - `controller/src/emit/stage_config.rs` (3,020 lines, formerly `config/stage_config.rs`) was moved whole rather than split into `emit/opamp.rs` + `emit/streaming_config.rs` + `emit/inference_config.rs` per design.md §5. The monolith mixes OTel-collector YAML emit, ASAPQuery-backend JSON emit, storage-routing JSON emit, and shared internals; a clean split needs ownership reorganisation, not file renames. Tracked for a follow-up. -- `controller/src/intent_algebra/legacy_expr.rs` carries the legacy **L2 relational** `QueryExpr` the `query_parser` front ends emit; `legacy_to_canonical.rs` is the single pass that converts it (sketch-fusion folded in) to the canonical `intent_algebra::{query_expr,agg_intent}` L3 types. The sketch-fused `SketchAgg` / `WindowedAgg` legacy variants and the standalone `legacy_lower` pass have been retired. Several controller modules (query_parser, language_logical_plan, physical/, optimizer/, emit/) still reference `legacy_expr` for shared leaf types; fully deleting the legacy tree is a follow-up gated on the canonical `Predicate` covering the remaining `ScalarExpr` variants. +- `controller/src/intent_algebra/legacy_expr.rs` carries the legacy **L2 relational** `QueryExpr` the `query_parser` front ends emit; `legacy_to_canonical.rs` is the single pass that converts it (sketch-fusion folded in) to the canonical `intent_algebra::{query_expr,agg_intent}` L3 types. The sketch-fused `SketchAgg` / `WindowedAgg` legacy variants and the standalone `legacy_lower` pass have been retired. Several controller modules (query_parser, physical/, optimizer/, emit/) still reference `legacy_expr` for shared leaf types; fully deleting the legacy tree is a follow-up gated on the canonical `Predicate` covering the remaining `ScalarExpr` variants. The `legacy_*` naming is itself a misnomer — `legacy_expr` is the real, current L2 IR — and an honest rename is the planned follow-up. - `controller/src/optimizer/trait_def.rs` (`OptimizerRule`), `controller/src/emit/trait_def.rs` (`PlanEmitter`), `controller/src/deployment_model.rs` (`DeploymentModelRegistry`) ship as placeholders — the existing free-function emitters and concrete rule loops still drive behaviour. Migrating them onto the trait surfaces lands when the per-deployment-model crate split lands. diff --git a/control_plane/src/intent_algebra/mod.rs b/control_plane/src/intent_algebra/mod.rs index f37afa04..4d0ef4fd 100644 --- a/control_plane/src/intent_algebra/mod.rs +++ b/control_plane/src/intent_algebra/mod.rs @@ -83,9 +83,8 @@ pub mod schema; // pre-existing legacy L2 relational IR formerly at // `controller/src/algebra/expr.rs` lives here while a separate follow-up // unifies it with the canonical `query_expr` module above. It still -// carries the heavy `QueryExpr` type the `query_parser` modules emit and -// the planner / allocator / physical planner / language_logical_plan -// modules consume today. +// carries the `QueryExpr` type the `query_parser` modules emit and the +// planner / allocator / physical planner modules consume today. pub mod column_resolution; pub mod legacy_expr; diff --git a/control_plane/src/language_logical_plan/lower.rs b/control_plane/src/language_logical_plan/lower.rs deleted file mode 100644 index 67349e51..00000000 --- a/control_plane/src/language_logical_plan/lower.rs +++ /dev/null @@ -1,41 +0,0 @@ -//! L1 → L2 lowering — produces a [`LanguageLogicalPlan`] from a -//! [`crate::query_parser::language::LanguageAst`]. -//! -//! Today only the PromQL backend has a working L1, so only the PromQL -//! variant of `LanguageAst` lowers to a real L2 tree. Other variants -//! return [`LoweringError::UnsupportedLanguage`] cleanly so callers -//! can surface a uniform error. - -use crate::query_parser::language::{language_ast::LanguageAst, promql::PromQLAst}; -use crate::types_v2::QueryLanguage; - -use super::plan::{LanguageLogicalPlan, LanguageLogicalPlanSummary}; - -/// Errors surfaced by the L1 → L2 lowering pass. -#[derive(Debug, thiserror::Error)] -pub enum LoweringError { - /// The L1 AST is for a language whose L2 lowering is not yet implemented. - #[error("L2 lowering not implemented for {0:?}")] - UnsupportedLanguage(QueryLanguage), -} - -/// Lower an L1 [`LanguageAst`] into an L2 [`LanguageLogicalPlan`]. -/// -/// The PromQL backend's L2 representation is the existing -/// `algebra::expr::QueryExpr` tree (already produced by -/// `query_parser::parse_query_expr` and stashed inside [`PromQLAst`]). -/// We just re-tag it as the PromQL L2 variant and project the flat -/// summary; no re-parsing or re-walking is required. -pub fn lower_to_logical_plan(ast: &LanguageAst) -> Result { - match ast { - LanguageAst::PromQL(p) => Ok(lower_promql(p)), - } -} - -fn lower_promql(ast: &PromQLAst) -> LanguageLogicalPlan { - LanguageLogicalPlan::PromQL { - source: ast.source.clone(), - tree: ast.expr.clone(), - summary: LanguageLogicalPlanSummary::from_parsed_query(&ast.summary), - } -} diff --git a/control_plane/src/language_logical_plan/mod.rs b/control_plane/src/language_logical_plan/mod.rs deleted file mode 100644 index a5531a42..00000000 --- a/control_plane/src/language_logical_plan/mod.rs +++ /dev/null @@ -1,36 +0,0 @@ -// Layer 2 scaffolding ships ahead of any in-tree call site (the -// downstream `pipeline` driver lands in a later phase). Dead-code -// warnings are silenced here, not on individual items, so the public -// surface is uncluttered. -#![allow(dead_code, unused_imports)] - -//! Layer 2 — `language_logical_plan`. -//! -//! Per `control_plane/docs/design.md` §6 `core::logical_plan`, L2 is the -//! per-language algebra tree that preserves language-specific -//! semantics (PromQL instant vs range vector, SQL window frames, -//! Elastic buckets) before the L3 normalisation pass collapses -//! everything into the language-orthogonal [`crate::intent_algebra::legacy_expr::QueryExpr`]. -//! -//! # DC deployment scope -//! -//! - PromQL → real L2 (carries the existing `QueryExpr` tree from -//! `query_parser::parse_query_expr` plus a flat summary projection). -//! - Other languages → not yet implemented; lowering errors cleanly. -//! -//! # Layer split -//! -//! - L1 ([`crate::query_parser::language`]) — `&str` → [`crate::query_parser::language::LanguageAst`]. -//! - L2 ([`Self`]) — [`crate::query_parser::language::LanguageAst`] → -//! [`LanguageLogicalPlan`]. -//! - L3 (`crate::intent_algebra`, Phase B) — [`LanguageLogicalPlan`] -//! → language-orthogonal [`crate::intent_algebra::legacy_expr::QueryExpr`]. - -pub mod lower; -pub mod plan; - -pub use lower::{lower_to_logical_plan, LoweringError}; -pub use plan::{LanguageLogicalPlan, LanguageLogicalPlanSummary}; - -#[cfg(test)] -mod tests; diff --git a/control_plane/src/language_logical_plan/plan.rs b/control_plane/src/language_logical_plan/plan.rs deleted file mode 100644 index 894bb20e..00000000 --- a/control_plane/src/language_logical_plan/plan.rs +++ /dev/null @@ -1,123 +0,0 @@ -//! [`LanguageLogicalPlan`] — the L2 canonical representation. -//! -//! Per `control_plane/docs/design.md` §6 `core::logical_plan`, L2 is a -//! **per-language algebra tree** preserving language-specific -//! semantics (PromQL instant vs range vector, SQL window frames, -//! Elastic buckets) — `Aggregate { AggFunc }`, `Window`, `Filter`, -//! `Sort`, `Limit`. **No sketch names yet**. -//! -//! The PromQL variant carries the canonical -//! [`crate::intent_algebra::QueryExpr`] tree (produced by -//! `query_parser::parse_query_expr_canonical` and stashed inside -//! `PromQLAst`) plus the flat `ParsedQuery` summary downstream -//! analyzer / planner consumers want. -//! -//! Step γ7: this used to carry the legacy `legacy_expr::QueryExpr`; now -//! that the producers route through the L3 Binder + canonical converter, -//! L2 carries the canonical IR directly. Other languages get their own -//! variants when implemented. - -use std::collections::HashMap; -use std::time::Duration; - -use crate::intent_algebra::QueryExpr; -use crate::query_parser::{ParsedQuery, QueryHint}; -use crate::types::AggType; -use crate::types_v2::QueryLanguage; - -/// L2 canonical container — one variant per source language. -/// -/// Each variant carries: -/// - the raw source string (for diagnostics + replan correlation), -/// - the language-specific algebra tree, -/// - a flat [`LanguageLogicalPlanSummary`] view that downstream -/// non-tree consumers (analyzer / planner / replan) can read -/// without walking the tree themselves. -#[derive(Debug, Clone)] -pub enum LanguageLogicalPlan { - /// PromQL L2. Tree = canonical `QueryExpr` from - /// `query_parser::parse_query_expr_canonical`. - PromQL { - /// Original PromQL source. - source: String, - /// Algebra tree — the PromQL-flavored L2 shape. - tree: QueryExpr, - /// Flat summary projection of the tree. - summary: LanguageLogicalPlanSummary, - }, - // Future variants (kept commented to surface intent): - // Sql { source: String, tree: SqlTree, summary: ... }, - // ElasticDsl { source: String, tree: EsTree, summary: ... }, -} - -impl LanguageLogicalPlan { - /// The originating [`QueryLanguage`] for this plan. - pub fn language(&self) -> QueryLanguage { - match self { - LanguageLogicalPlan::PromQL { .. } => QueryLanguage::PromQL, - } - } - - /// Original source string this plan was lowered from. - pub fn source(&self) -> &str { - match self { - LanguageLogicalPlan::PromQL { source, .. } => source, - } - } - - /// Borrow the flat summary projection. - pub fn summary(&self) -> &LanguageLogicalPlanSummary { - match self { - LanguageLogicalPlan::PromQL { summary, .. } => summary, - } - } - - /// Borrow the algebra tree if the language is PromQL. - pub fn as_promql_tree(&self) -> Option<&QueryExpr> { - match self { - LanguageLogicalPlan::PromQL { tree, .. } => Some(tree), - } - } -} - -/// Flat, language-orthogonal projection of the L2 plan. This is the -/// shape downstream non-tree consumers (the legacy analyzer + planner -/// in DC) want. -/// -/// It mirrors the fields on [`crate::query_parser::ParsedQuery`] so -/// adapters between the two layers stay trivial. -#[derive(Debug, Clone, Default)] -pub struct LanguageLogicalPlanSummary { - /// Metric / table name extracted from the L2 tree. - pub metric_name: String, - /// Aggregation types present in the tree. - pub aggregations: Vec, - /// `GROUP BY` / `by (dims)` keys. - pub group_by_labels: Vec, - /// Equality label / `WHERE` filters. - pub label_filters: HashMap, - /// Window size (PromQL `[w]` / SQL `TUMBLE`). - pub time_window: Duration, - /// Set when the query needs per-sample exact computation. - pub exact_required: bool, - /// Quantile φ values implied by the query. - pub quantiles: Vec, - /// Optional named pattern hint (DEBS classifier). - pub hint: Option, -} - -impl LanguageLogicalPlanSummary { - /// Lift a legacy [`ParsedQuery`] into a [`LanguageLogicalPlanSummary`]. - pub fn from_parsed_query(pq: &ParsedQuery) -> Self { - Self { - metric_name: pq.metric_name.clone(), - aggregations: pq.aggregations.clone(), - group_by_labels: pq.group_by_labels.clone(), - label_filters: pq.label_filters.clone(), - time_window: pq.time_window, - exact_required: pq.exact_required, - quantiles: pq.quantiles.clone(), - hint: pq.hint.clone(), - } - } -} diff --git a/control_plane/src/language_logical_plan/tests.rs b/control_plane/src/language_logical_plan/tests.rs deleted file mode 100644 index c6583d7b..00000000 --- a/control_plane/src/language_logical_plan/tests.rs +++ /dev/null @@ -1,89 +0,0 @@ -//! Tests for L2 lowering (`language_logical_plan`). - -use super::*; -use crate::intent_algebra::{AggIntent, QueryExpr}; -use crate::query_parser::language::{Language, LanguageAst, PromQLLanguage}; -use crate::types::AggType; -use crate::types_v2::QueryLanguage; - -fn parse_promql(src: &str) -> LanguageAst { - PromQLLanguage - .parse(src) - .expect("PromQL parse should succeed") -} - -#[test] -fn lower_promql_to_logical_plan_basic() { - let ast = parse_promql("up"); - let plan = lower_to_logical_plan(&ast).expect("lowering should succeed"); - assert_eq!(plan.language(), QueryLanguage::PromQL); - assert_eq!(plan.source(), "up"); - let s = plan.summary(); - assert_eq!(s.metric_name, "up"); -} - -#[test] -fn lower_promql_quantile_preserves_summary() { - let ast = parse_promql("quantile_over_time(0.99, http_request_duration{env=\"prod\"}[5m])"); - let plan = lower_to_logical_plan(&ast).unwrap(); - let s = plan.summary(); - assert_eq!(s.metric_name, "http_request_duration"); - assert!(s.aggregations.contains(&AggType::Quantile)); - assert_eq!(s.quantiles, vec![0.99]); - assert_eq!(s.label_filters.get("env").map(|v| v.as_str()), Some("prod")); - assert_eq!(s.time_window, std::time::Duration::from_secs(5 * 60)); -} - -#[test] -fn lower_promql_keeps_algebra_tree_for_l3() { - // The PromQL L2 tree is the canonical `QueryExpr`. `quantile_over_time` - // lowers (via the legacy `WindowedAgg`) into the canonical - // `Window { child: Aggregate { aggs: [Quantile] } }` fold. - let ast = parse_promql("quantile_over_time(0.99, http_request_duration{env=\"prod\"}[5m])"); - let plan = lower_to_logical_plan(&ast).unwrap(); - let tree = plan.as_promql_tree().expect("PromQL plan"); - match tree { - QueryExpr::Window { child, .. } => assert!(matches!( - child.as_ref(), - QueryExpr::Aggregate { aggs, .. } - if matches!(aggs.as_slice(), [AggIntent::Quantile { .. }]) - )), - other => panic!("expected canonical Window{{Aggregate}}, got {other:?}"), - } -} - -#[test] -fn lower_promql_topk_extracts_groupby() { - let ast = parse_promql("topk by (service) (10, count_over_time(requests{env=\"prod\"}[1m]))"); - let plan = lower_to_logical_plan(&ast).unwrap(); - let s = plan.summary(); - assert_eq!(s.metric_name, "requests"); - assert!(s.group_by_labels.contains(&"service".to_string())); -} - -#[test] -fn lower_unsupported_language_errors_cleanly() { - // Build a stub LanguageAst variant by hand — we cannot construct - // SQL/ElasticDsl variants because LanguageAst doesn't - // expose those yet (PromQL is the only variant). So the equivalent - // smoke test is: stub backends fail at L1 with `Unimplemented`, and - // the type system rules out passing them to L2 lowering. We assert - // that contract here. - let err = crate::query_parser::language::SqlLanguage - .parse("SELECT 1") - .unwrap_err(); - assert!(matches!( - err, - crate::query_parser::language::ParseError::Unimplemented(_) - )); -} - -#[test] -fn lowering_error_for_unsupported_language_is_displayable() { - // Direct construction of the error variant — verifies the Display - // impl renders something useful. Future variants (Sql/etc.) will - // exercise this path organically once their L1 backends ship. - let err = LoweringError::UnsupportedLanguage(QueryLanguage::Sql); - let msg = format!("{err}"); - assert!(msg.contains("Sql"), "got: {msg}"); -} diff --git a/control_plane/src/lib.rs b/control_plane/src/lib.rs index 401091fb..497c848f 100644 --- a/control_plane/src/lib.rs +++ b/control_plane/src/lib.rs @@ -41,12 +41,10 @@ //! - `sketch_algebra` — `Capability` enum + `capability_for(agg_intent: &AggIntent)` //! lookup table (Phase 4 / 5 use this to route raw-name PromQL). //! - `intent_algebra` — `AggIntent` + `QueryExpr` DAG (canonical L3 IR; -//! `legacy_expr` / `legacy_lower` carry the older `algebra::expr`-flavored -//! IR pending full migration). -//! - `language_logical_plan` — PromQL → AST → logical plan. -//! - `query_parser` — front-end parsers (Layer 1). -//! The former top-level `query_language/` module was folded into -//! `query_parser::language` by this refactor. +//! `legacy_expr` carries the L2 relational IR the parsers emit and +//! `legacy_to_canonical` lowers it to the canonical L3 types). +//! - `query_parser` — front-end parsers (Layer 1): `promql.rs` / `sql.rs` +//! emit the L2 relational `legacy_expr::QueryExpr` tree. //! - `physical` — L5 framework (allocator, planner, plan, sketch_catalog, //! colored_dag, stage_split, topology). //! - `optimizer` — L4 rule engine + cost model traits/impls + baseline @@ -64,7 +62,6 @@ pub mod backend_client; pub mod deployment_model; pub mod emit; pub mod intent_algebra; -pub mod language_logical_plan; pub mod metrics_exposer; pub mod monitor; pub mod opamp; diff --git a/control_plane/src/main.rs b/control_plane/src/main.rs index cc8f2c09..9a8df34b 100644 --- a/control_plane/src/main.rs +++ b/control_plane/src/main.rs @@ -2,7 +2,6 @@ use control_plane::accuracy; use control_plane::backend_client; use control_plane::emit; use control_plane::intent_algebra; -use control_plane::language_logical_plan; use control_plane::metrics_exposer; use control_plane::monitor; use control_plane::opamp; diff --git a/control_plane/src/query_parser/language/elastic_dsl/mod.rs b/control_plane/src/query_parser/language/elastic_dsl/mod.rs deleted file mode 100644 index 67694449..00000000 --- a/control_plane/src/query_parser/language/elastic_dsl/mod.rs +++ /dev/null @@ -1,25 +0,0 @@ -//! ElasticDSL L1 backend — stub. -//! -//! Reserved for the future ElasticDSL deployment model -//! (`control_plane/docs/design.md` §3 row 1). No L1 parser is shipped in -//! the DC deployment build. - -use super::language_ast::LanguageAst; -use super::{Language, ParseError}; -use crate::types_v2::QueryLanguage; - -/// ElasticDSL implementation of the [`Language`] trait. Currently stubbed. -#[derive(Debug, Default, Clone, Copy)] -pub struct ElasticDslLanguage; - -impl Language for ElasticDslLanguage { - fn id(&self) -> QueryLanguage { - QueryLanguage::ElasticDsl - } - - fn parse(&self, _source: &str) -> Result { - Err(ParseError::Unimplemented( - "ElasticDSL parser not implemented in DC deployment mode", - )) - } -} diff --git a/control_plane/src/query_parser/language/language_ast.rs b/control_plane/src/query_parser/language/language_ast.rs deleted file mode 100644 index b1ec8e55..00000000 --- a/control_plane/src/query_parser/language/language_ast.rs +++ /dev/null @@ -1,35 +0,0 @@ -//! [`LanguageAst`] — the discriminated union over per-language ASTs. -//! -//! One variant per [`crate::types_v2::QueryLanguage`]. The PromQL variant -//! is the only one carrying real data today; the rest are reserved for -//! future deployment models (asap-fusion, ElasticDSL). - -use super::promql::ast::PromQLAst; - -/// Tagged union of language-flavored ASTs returned by [`super::Language::parse`]. -/// -/// Adding a new language is mechanical: add a new variant here and wire -/// up a backend in `query_language::::`. -#[derive(Debug, Clone)] -pub enum LanguageAst { - /// PromQL AST — wraps the existing `query_parser::ParsedQuery` plus - /// the full `QueryExpr` from the legacy parser. - PromQL(PromQLAst), - // Future backends (kept commented to surface intent): - // Sql(SqlAst), - // ElasticDsl(EsAst), -} - -impl LanguageAst { - /// Return `true` when this AST belongs to the PromQL backend. - pub fn is_promql(&self) -> bool { - matches!(self, LanguageAst::PromQL(_)) - } - - /// Borrow the inner `PromQLAst` if the variant is PromQL. - pub fn as_promql(&self) -> Option<&PromQLAst> { - match self { - LanguageAst::PromQL(a) => Some(a), - } - } -} diff --git a/control_plane/src/query_parser/language/mod.rs b/control_plane/src/query_parser/language/mod.rs deleted file mode 100644 index 5713ec9c..00000000 --- a/control_plane/src/query_parser/language/mod.rs +++ /dev/null @@ -1,94 +0,0 @@ -// L1 scaffolding ships ahead of any in-tree call site (the -// downstream `pipeline` driver lands in a later phase). Dead-code -// warnings are silenced here, not on individual items, so the public -// surface is uncluttered. -#![allow(dead_code, unused_imports)] - -//! Layer 1 — `query_parser::language`. -//! -//! Per-language parser façade. See `control_plane/docs/design.md` §6 -//! `core::query_language` for the design contract. Refactor 2026-05: -//! the former top-level `query_language/` module was folded into -//! `query_parser/language/` so the L1 parser frontend lives under one -//! module path. -//! -//! # Module layout -//! -//! ```text -//! query_parser/language/ -//! ├── mod.rs — Language trait + ParseError (this file) -//! ├── language_ast.rs — LanguageAst sum type -//! ├── promql/ — PromQL backend (active; wraps query_parser::promql) -//! ├── sql/ — SQL backend (stub; Unimplemented) -//! └── elastic_dsl/ — ElasticDSL backend (stub; Unimplemented) -//! ``` -//! -//! # DC deployment scope -//! -//! Only [`promql::PromQLLanguage`] is implemented. The other backends -//! return [`ParseError::Unimplemented`] so the type system is uniform; -//! adding a real backend later is a localised change to the relevant -//! sub-module. - -pub mod elastic_dsl; -pub mod language_ast; -pub mod promql; -pub mod sql; - -pub use elastic_dsl::ElasticDslLanguage; -pub use language_ast::LanguageAst; -pub use promql::PromQLLanguage; -pub use sql::SqlLanguage; - -#[cfg(test)] -mod tests; - -use crate::types_v2::QueryLanguage; - -// ── Errors ──────────────────────────────────────────────────────────────────── - -/// Errors a [`Language`] backend can return from `parse`. -/// -/// `Unimplemented` is used by the stubbed-out non-PromQL backends so the -/// type system stays uniform without committing to a parser implementation. -#[derive(Debug, thiserror::Error)] -pub enum ParseError { - /// The underlying language parser rejected the source string. - #[error("parse failed for {language:?}: {source_err}")] - Backend { - /// Which backend rejected the source. - language: QueryLanguage, - /// Free-form parser-specific message. - source_err: String, - }, - - /// The backend is registered but not implemented in this build (DC - /// deployment mode ships PromQL only). - #[error("{0}")] - Unimplemented(&'static str), -} - -impl ParseError { - /// Convenience: wrap any `Display` parser error into [`ParseError::Backend`]. - pub fn backend(language: QueryLanguage, e: impl std::fmt::Display) -> Self { - ParseError::Backend { - language, - source_err: e.to_string(), - } - } -} - -// ── Trait ───────────────────────────────────────────────────────────────────── - -/// Per-language L1 parser façade. One impl per [`QueryLanguage`] variant. -/// -/// `parse` is the only required method; `id` is a static tag so callers -/// can cross-check the variant returned by [`LanguageAst`] without -/// downcasting. -pub trait Language: Send + Sync { - /// The [`QueryLanguage`] variant this backend implements. - fn id(&self) -> QueryLanguage; - - /// Parse `source` into a language-flavored AST. - fn parse(&self, source: &str) -> Result; -} diff --git a/control_plane/src/query_parser/language/promql/ast.rs b/control_plane/src/query_parser/language/promql/ast.rs deleted file mode 100644 index 4803ea1e..00000000 --- a/control_plane/src/query_parser/language/promql/ast.rs +++ /dev/null @@ -1,46 +0,0 @@ -//! `PromQLAst` — the L1 output of the PromQL backend. -//! -//! Thin wrapper that bundles the parser's two outputs (the canonical -//! `QueryExpr` algebra tree and the flat `ParsedQuery` summary) so -//! downstream L2 lowering can pick whichever shape it needs without -//! re-parsing. - -use crate::intent_algebra::QueryExpr; -use crate::query_parser::ParsedQuery; - -/// PromQL parse output. Carries both the algebra tree and the flat -/// summary the analyzer consumes. -/// -/// We bundle both because the parser already produces them, and L2 -/// lowering wants the structured tree while back-compat callers -/// (`Analyzer`) still want the flat summary. -#[derive(Debug, Clone)] -pub struct PromQLAst { - /// Original PromQL source string (preserved for debugging / errors). - pub source: String, - /// Algebra tree — the canonical `parse_query_expr_canonical` output. - pub expr: QueryExpr, - /// Flat summary — the `parse_query` output. - pub summary: ParsedQuery, -} - -impl PromQLAst { - /// Construct from the legacy parser's two outputs. - pub fn new(source: String, expr: QueryExpr, summary: ParsedQuery) -> Self { - Self { - source, - expr, - summary, - } - } - - /// Borrow the algebra tree. - pub fn expr(&self) -> &QueryExpr { - &self.expr - } - - /// Borrow the flat summary. - pub fn summary(&self) -> &ParsedQuery { - &self.summary - } -} diff --git a/control_plane/src/query_parser/language/promql/mod.rs b/control_plane/src/query_parser/language/promql/mod.rs deleted file mode 100644 index e2c395c2..00000000 --- a/control_plane/src/query_parser/language/promql/mod.rs +++ /dev/null @@ -1,44 +0,0 @@ -//! PromQL L1 backend — wraps the existing `query_parser::promql` parser -//! behind the [`super::Language`] trait. -//! -//! No new parsing logic lives here; this is purely an adapter that -//! collects the legacy parser's two outputs into a [`PromQLAst`] and -//! tags it as [`LanguageAst::PromQL`]. - -pub mod ast; - -use super::language_ast::LanguageAst; -use super::{Language, ParseError}; -use crate::query_parser::{parse_query, parse_query_expr_canonical}; -use crate::types_v2::QueryLanguage; - -pub use ast::PromQLAst; - -/// PromQL implementation of the [`Language`] trait. -/// -/// Construct with `PromQLLanguage::default()`; the type is unit so it -/// is cheap to clone / store in a registry. -#[derive(Debug, Default, Clone, Copy)] -pub struct PromQLLanguage; - -impl Language for PromQLLanguage { - fn id(&self) -> QueryLanguage { - QueryLanguage::PromQL - } - - fn parse(&self, source: &str) -> Result { - // Delegate to the existing parser — both entry points re-parse the - // same string today; the cost is negligible (microseconds) and we - // get the flat `ParsedQuery` for free for back-compat callers. - // `PromQLAst.expr` is the canonical L3 `QueryExpr`. - let expr = parse_query_expr_canonical(source) - .map_err(|e| ParseError::backend(QueryLanguage::PromQL, e))?; - let summary = - parse_query(source).map_err(|e| ParseError::backend(QueryLanguage::PromQL, e))?; - Ok(LanguageAst::PromQL(PromQLAst::new( - source.to_string(), - expr, - summary, - ))) - } -} diff --git a/control_plane/src/query_parser/language/sql/mod.rs b/control_plane/src/query_parser/language/sql/mod.rs deleted file mode 100644 index b8ca1c70..00000000 --- a/control_plane/src/query_parser/language/sql/mod.rs +++ /dev/null @@ -1,26 +0,0 @@ -//! SQL L1 backend — stub for the DC deployment mode. -//! -//! The DC deployment ships PromQL only; the SQL parser path remains -//! addressable through the legacy `query_parser::sql` entry point but -//! is not exposed via the new [`Language`] trait yet. A real impl wraps -//! `sqlparser` and emits a `SqlAst` analogous to [`super::promql::PromQLAst`]. - -use super::language_ast::LanguageAst; -use super::{Language, ParseError}; -use crate::types_v2::QueryLanguage; - -/// SQL implementation of the [`Language`] trait. Currently stubbed. -#[derive(Debug, Default, Clone, Copy)] -pub struct SqlLanguage; - -impl Language for SqlLanguage { - fn id(&self) -> QueryLanguage { - QueryLanguage::Sql - } - - fn parse(&self, _source: &str) -> Result { - Err(ParseError::Unimplemented( - "SQL parser not implemented in DC deployment mode", - )) - } -} diff --git a/control_plane/src/query_parser/language/tests.rs b/control_plane/src/query_parser/language/tests.rs deleted file mode 100644 index 1fd5e8dd..00000000 --- a/control_plane/src/query_parser/language/tests.rs +++ /dev/null @@ -1,64 +0,0 @@ -//! Tests for the L1 `query_language` façade. - -use super::*; -use crate::types_v2::QueryLanguage; - -#[test] -fn promql_language_id_is_promql() { - let l = PromQLLanguage; - assert_eq!(l.id(), QueryLanguage::PromQL); -} - -#[test] -fn promql_language_parses_basic() { - let ast = PromQLLanguage - .parse("up") - .expect("PromQL parse should succeed"); - assert!(ast.is_promql(), "expected PromQL variant"); - let promql = ast.as_promql().unwrap(); - assert_eq!(promql.summary().metric_name, "up"); - assert_eq!(promql.source, "up"); -} - -#[test] -fn promql_language_parses_quantile_over_time() { - let ast = PromQLLanguage - .parse("sum by (host) (quantile_over_time(0.99, latency[5m]))") - .expect("PromQL quantile parse should succeed"); - let promql = ast.as_promql().unwrap(); - assert!(promql.summary().quantiles.contains(&0.99)); - assert_eq!(promql.summary().metric_name, "latency"); -} - -#[test] -fn promql_language_surfaces_parse_error_as_backend_error() { - let err = PromQLLanguage.parse("@!not promql@!").unwrap_err(); - match err { - ParseError::Backend { language, .. } => { - assert_eq!(language, QueryLanguage::PromQL); - } - other => panic!("expected ParseError::Backend, got {other:?}"), - } -} - -#[test] -fn sql_language_returns_unimplemented() { - let err = SqlLanguage.parse("SELECT 1").unwrap_err(); - assert!(matches!(err, ParseError::Unimplemented(_))); - assert_eq!(SqlLanguage.id(), QueryLanguage::Sql); -} - -#[test] -fn elastic_dsl_language_returns_unimplemented() { - let err = ElasticDslLanguage.parse("{}").unwrap_err(); - assert!(matches!(err, ParseError::Unimplemented(_))); - assert_eq!(ElasticDslLanguage.id(), QueryLanguage::ElasticDsl); -} - -#[test] -fn language_trait_object_is_object_safe() { - // Compile-time check: the trait can be used as `dyn Language` so - // future code can store a `Vec>` registry. - let l: Box = Box::new(PromQLLanguage); - assert_eq!(l.id(), QueryLanguage::PromQL); -} diff --git a/control_plane/src/query_parser/mod.rs b/control_plane/src/query_parser/mod.rs index cb19ba3f..3aac597b 100644 --- a/control_plane/src/query_parser/mod.rs +++ b/control_plane/src/query_parser/mod.rs @@ -29,12 +29,6 @@ pub mod promql; pub mod sql; -pub mod language; - -// Re-export the L1 language façade at this module's top so call sites -// that historically used `crate::query_language::*` (folded in by the -// 2026-05 refactor) keep a one-level import. -pub use language::{Language, LanguageAst, ParseError, PromQLLanguage, SqlLanguage, ElasticDslLanguage}; use std::collections::HashMap; use std::time::Duration;