feat(cost): lower raw query DAGs into evidence-backed physical plans - #327
Merged
Merged
Conversation
zzylol
force-pushed
the
feat/cost-statistics-scope
branch
from
September 2, 2026 19:28
0176c51 to
50b2a80
Compare
zzylol
force-pushed
the
feat/cost-query-dag-stack
branch
from
September 2, 2026 19:28
f1a650a to
3d0702a
Compare
zzylol
force-pushed
the
feat/cost-statistics-scope
branch
from
September 3, 2026 02:54
50b2a80 to
fa19c73
Compare
zzylol
force-pushed
the
feat/cost-query-dag-stack
branch
from
September 3, 2026 02:54
3d0702a to
fa09dcf
Compare
zzylol
marked this pull request as ready for review
September 3, 2026 03:17
zzylol
force-pushed
the
feat/cost-statistics-scope
branch
from
September 3, 2026 16:05
fa19c73 to
e1411fd
Compare
zzylol
force-pushed
the
feat/cost-query-dag-stack
branch
from
September 3, 2026 16:07
fa09dcf to
2c86288
Compare
zzylol
force-pushed
the
feat/cost-statistics-scope
branch
from
September 3, 2026 16:29
e1411fd to
0b03b6e
Compare
zzylol
force-pushed
the
feat/cost-query-dag-stack
branch
from
September 3, 2026 16:29
2c86288 to
0a91fbc
Compare
This was referenced Sep 3, 2026
zzylol
force-pushed
the
feat/cost-query-dag-stack
branch
from
September 3, 2026 18:38
0a91fbc to
1307e34
Compare
zzylol
force-pushed
the
feat/cost-query-dag-stack
branch
from
September 3, 2026 18:47
1307e34 to
228ea1e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The analytical estimator can compose a physical DAG only after a selected raw query has been lowered completely. Shape-specific handling of
Scan -> Aggregateor one Count/Top-K example misses nested operators, can lose edge cardinality, and cannot distinguish repeated logical occurrences from deliberately shared physical work.The lowering must also preserve the work that changes a physical algorithm's cost. Two filters with different predicate work, a one-key and a three-key join, or a global and partitioned sort must not receive the same CPU/memory estimate merely because they share an operator name.
What
QueryExproperator DAG into the existingPhysicalOperatorvocabulary.OperatorStatisticsfor every node.PhysicalOperator.InMemoryOrderedWindowtoInMemoryAnalyticWindowto distinguish SQL analytic execution from streaming tumbling/sliding/exponential-histogram windows.offset/@as unavailable until temporal context can be propagated into descendant Scan evidence.This PR does not lower
SummaryExpr. Post-ASAP summary physical lowering and complete-alternative ranking remain separate stack layers.How
Every logical occurrence is independent by default. The evidence provider owns physical identity; two occurrences are charged once only when it deliberately returns the same non-empty
physical_idwith identical operator, children, source coverage, statistics, and buffer evidence. The lowerer does not equateRcaddress identity with physical sharing.Parent input rows/bytes must match the corresponding child output. Source storage bytes remain separate from decoded logical edge bytes. Evidence is snapshotted into the returned
PhysicalDag, so costing does not reread a live provider.For ordered operators,
PartitionStatisticssupplies the rows and logical bytes of every independently processed partition. The checked sum must equal the input edge. Global ordering requires exactly one partition; partitioned ordering computes:The physical integration design now includes a definition, query example, required evidence, and CPU/memory/I/O behavior for every physical node.
Supported raw mappings
QueryExprshapewithoutUNION ALLUnsupported aggregate algorithms, HAVING,
withoutgrouping with runtime-unknown keys, cross/non-equi joins, distinct set operations, non-identity temporal shifts, PromQL range/subquery/vector operators, and post-ASAP summary operations remain unavailable until their physical implementations and evidence are explicit.Before this PR
Only a few hand-written raw query shapes could reach analytical costing. Operator-local expression/key work and partition distribution were erased. A non-empty Scan could report zero disk reads, and ungrouped or empty-input aggregation evidence could not be represented correctly.
After this PR
For example:
lowers recursively to:
Each edge has its own rows/bytes, only Scan charges physical source reads, the aggregate retains every group, and Top-K retains only its bounded heap. Any missing physical fact makes the complete query unavailable.
Validation
cargo test -p asap-aware-mapping— 270 tests passedcargo clippy -p asap-aware-mapping --all-targets -- -D warningscargo fmt --all -- --checkgit diff --checkDepends on #326. #328 adds PromQL-specific physical operators and validation.