Skip to content

feat(cost): rank complete physical plan alternatives - #330

Merged
zzylol merged 50 commits into
mainfrom
feat/cost-planner-ranking
Sep 4, 2026
Merged

zzylol merged 50 commits into
mainfrom
feat/cost-planner-ranking

Conversation

@zzylol

@zzylol zzylol commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Why

Logical candidate ranking is valid only when raw and post-ASAP alternatives cover the same target, source contents, predicates, event-time range, recurrence, and workload horizon. Partial plans or statistics from different snapshots must not enter one comparison.

The implementation must also distinguish the physical plan being estimated from the analytical method used to estimate it. Physical operator identities and evidence are inputs; CPU, memory, and I/O formulas are analytical predictions rather than runtime measurements.

What

  • Add PhysicalPlanCostModel at the final planner-selection boundary.
  • Capture one immutable, versioned PhysicalEvidenceSnapshot per target; blank or whitespace-only versions fail before any evidence lookup.
  • Lower the raw target and every rewrite into complete physical DAGs; require a complete provider-bound physical DAG for summary candidates.
  • Compare separate CPU operations, peak-memory bytes, and source-read bytes with one validated calibration.
  • Cache the raw baseline by retained target identity and consumer count.
  • Make missing, stale, inconsistent, unsupported, or more-expensive candidates unavailable so the raw plan remains selected.
  • Bypass local CSE structural heuristics when a complete-plan model is active.

Mechanism

The modules expose four distinct responsibilities:

  • query_physical_lowering recursively maps QueryExpr and logical rewrites to physical operators.
  • physical_operator_statistics defines the per-operator catalog/runtime evidence contract.
  • analytical_cost applies algorithmic formulas to an EvidenceBackedPhysicalDag and composes CPU, peak-memory, and source-I/O estimates.
  • physical_plan_cost_model compares complete raw and replacement physical alternatives at the planner-ranking boundary.

SummaryExpr candidates are bound by PlannerPhysicalPlanProvider, because implementation-specific summary operators and retained state require physical evidence. Raw and replacement DAGs are estimated against the exact same ComparisonScope and snapshot version. EvidenceBackedPhysicalDag is the single shared physical DAG representation; query lowering does not publish a second PhysicalDag alias. No structural node count or optimistic zero is used by this analytical selection path.

The self-contained design and formula contract are documented in docs/design_docs/asap-aware-mapping/analytical-resource-cost.md; physical lowering integration is documented separately in physical-plan-integration.md.

Validation

  • cargo test -p asap-aware-mapping
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check

Stack: based on #328; #331 adds streaming-summary resource modeling.

@zzylol
zzylol force-pushed the fix/cost-estimator-invariants branch from 44d8dab to e32d209 Compare September 2, 2026 19:28
@zzylol
zzylol force-pushed the feat/cost-planner-ranking branch 2 times, most recently from 9bc1660 to 39417cd Compare September 3, 2026 02:54
@zzylol
zzylol force-pushed the fix/cost-estimator-invariants branch from e32d209 to 9d35b6d Compare September 3, 2026 02:54
@zzylol
zzylol marked this pull request as ready for review September 3, 2026 03:17
@zzylol
zzylol force-pushed the fix/cost-estimator-invariants branch from 9d35b6d to 9d7dfc2 Compare September 3, 2026 16:13
@zzylol
zzylol force-pushed the feat/cost-planner-ranking branch from 4356990 to 2886b84 Compare September 3, 2026 16:13
@zzylol
zzylol force-pushed the feat/cost-planner-ranking branch from 86025bb to fa499a1 Compare September 3, 2026 23:54
@zzylol
zzylol changed the base branch from fix/cost-estimator-invariants to feat/cost-promql-stack-linear September 3, 2026 23:54
@zzylol
zzylol changed the base branch from feat/cost-promql-stack-linear to main September 4, 2026 11:25
@zzylol
zzylol merged commit 5e86bea into main Sep 4, 2026
3 checks passed
@zzylol
zzylol deleted the feat/cost-planner-ranking branch September 7, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant