Skip to content

feat(cost): lower and estimate PromQL physical operators - #328

Merged
zzylol merged 21 commits into
feat/cost-query-dag-stackfrom
feat/cost-promql-stack-linear
Sep 3, 2026
Merged

zzylol merged 21 commits into
feat/cost-query-dag-stackfrom
feat/cost-promql-stack-linear

Conversation

@zzylol

@zzylol zzylol commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Why

PromQL range evaluation, subqueries, vector matching, relabeling, series sampling, scalar/vector bridges, info enrichment, per-series functions, and presence functions have different physical work and cardinality rules. Modeling them as relational pass-through nodes either undercounts work or accepts impossible statistics.

The statistics layer also needs the PromQL shape of each physical edge. Rows and decoded bytes alone cannot distinguish a scalar from an instant vector or a range vector, prove evaluation-step consistency, or bound series cardinality.

What

  • Adds explicit physical operators for:
    • range selectors and subqueries;
    • scalar/vector and vector/vector binary evaluation;
    • label rewrites and info enrichment;
    • limitk and limit_ratio series sampling;
    • scalar-to-vector, vector-to-scalar, and source-free scalar leaves;
    • fixed-state per-series functions; and
    • absence and per-series presence evaluation.
  • Extends physical edges with typed PromQL evidence: series, evaluation_steps, and Scalar | Vector | RangeVector.
  • Keeps algorithm configuration on PhysicalOperator and workload/catalog facts on the matching OperatorStatistics variant.
  • Recursively lowers supported QueryExpr nodes into the physical DAG and preserves operation class, operand mode, vector-match cardinality, range/resolution, sampling configuration, and physical child order.
  • Adds every PromQL physical node, its query example, evidence contract, CPU formula, memory formula, and failure behavior to the physical integration design.

Correctness mechanisms

  • Every PromQL parent input must exactly equal its child output in rows, bytes, series, steps, and value kind. A parent cannot silently drop PromQL metadata.
  • Range selectors require positive duration and per-series window-sample evidence. Subquery child steps must equal outer steps multiplied by realized subquery steps; an explicit resolution must agree with the range.
  • Binary operators preserve scalar/vector operand direction. Vector/vector matching requires key width and a selected build side. and, or, unless, one-to-one, many-to-one, and one-to-many each enforce their own output bounds.
  • Instant-vector results cannot contain more rows than series × evaluation_steps.
  • Scalar/vector bridges validate direction; scalar leaves have zero children and emit exactly one scalar row per step.
  • Relabeling preserves rows, series, steps, and value kind. Info enrichment preserves the data-side cardinality and charges info-side hash state.
  • limitk is bounded per group and is kept distinct from ordered heap Top-K. limit_ratio preserves its signed ratio exactly.
  • absent/absent_over_time use the global synthesized-series bound. present_over_time uses a separate per-series bound.
  • Info-metric selectors are part of SourceCoverage; two candidates with different symbolic InfoMatcher values are not treated as the same comparison scope.
  • Generic HashAggregate is rejected when it attempts to stand in for an unmodeled PromQL cross-series aggregation.
  • Missing configuration, missing statistics, inconsistent arity, impossible cardinality, or unsupported algorithms make the complete candidate unavailable instead of contributing zero cost.

Resource formulas

The estimator keeps CPU operations, peak memory, and source reads separate. PromQL nodes add only local work; DAG composition charges shared physical identity once and applies the node's execution multiplicity.

Examples:

  • range memory = input_series × max_window_samples_per_series × sample_width;
  • per-series state = input_series × accumulator_bytes_per_series;
  • vector/vector binary memory = selected build-side series × (matching_key_bytes + hash metadata);
  • series-sample memory = selected series × (key_bytes + hash metadata);
  • source/disk bytes remain charged only by physical Scan nodes.

The query-time range buffer above is not a tumbling, sliding, pane, or exponential-histogram summary-maintenance window.

Before

PromQL nodes were either unavailable, approximated by generic relational shapes, or validated from a flat set of optional facts. Several paths could lose series/step metadata or apply the wrong cardinality rule.

After

Each supported PromQL algorithm has a typed physical operator, matching statistics shape, lowering rule, semantic validator, and resource formula. Unsupported PromQL shapes fail closed.

Validation

  • cargo test -p asap-aware-mapping — 281 passed
  • cargo clippy -p asap-aware-mapping --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

Depends on #327.

@zzylol
zzylol force-pushed the feat/cost-query-dag-stack branch from f1a650a to 3d0702a Compare September 2, 2026 19:28
@zzylol
zzylol force-pushed the feat/cost-promql-stack-linear branch 2 times, most recently from c402f0b to f6a62ad Compare September 3, 2026 02:54
@zzylol
zzylol force-pushed the feat/cost-query-dag-stack branch from 3d0702a to fa09dcf Compare September 3, 2026 02:54
@zzylol
zzylol marked this pull request as ready for review September 3, 2026 03:17
@zzylol
zzylol force-pushed the feat/cost-query-dag-stack branch from fa09dcf to 2c86288 Compare September 3, 2026 16:07
@zzylol
zzylol force-pushed the feat/cost-promql-stack-linear branch from f6a62ad to 9f48885 Compare September 3, 2026 16:11
@zzylol
zzylol force-pushed the feat/cost-query-dag-stack branch from 2c86288 to 0a91fbc Compare September 3, 2026 16:29
@zzylol
zzylol force-pushed the feat/cost-promql-stack-linear branch from 9f48885 to 72bb94d Compare September 3, 2026 16:29
@zzylol zzylol changed the title feat(cost): model PromQL physical operators feat(cost): model and validate PromQL physical operators Sep 3, 2026
@zzylol
zzylol force-pushed the feat/cost-query-dag-stack branch 2 times, most recently from 1307e34 to 228ea1e Compare September 3, 2026 18:47
@zzylol zzylol changed the title feat(cost): model and validate PromQL physical operators feat(cost): lower and estimate PromQL physical operators Sep 3, 2026
@zzylol
zzylol merged commit dcd7683 into feat/cost-query-dag-stack Sep 3, 2026
2 checks passed
@zzylol
zzylol deleted the feat/cost-promql-stack-linear branch September 7, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant