Skip to content

feat(cost): estimate physical DAG resources analytically - #332

Merged
zzylol merged 21 commits into
mainfrom
feat/analytical-resource-cost-323
Sep 3, 2026
Merged

zzylol merged 21 commits into
mainfrom
feat/analytical-resource-cost-323

Conversation

@zzylol

@zzylol zzylol commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Why

The planner needs resource estimates derived from physical work, rather than the previous structural node count. CPU work, peak memory, and source scan bytes must remain separate so later ranking can apply an explicit calibration without mixing rows, groups, and bytes.

What

  • Introduces the generic physical operator DAG and per-edge cardinality/width evidence.
  • Estimates CPU operations, peak memory, and source scan bytes for scan, filter, project, aggregate, deduplicate, sort, Top-K, limit, window, and concat operators.
  • Deduplicates shared physical sub-DAGs by identity.
  • Rejects cycles, missing inputs, duplicate identities, invalid widths, and contradictory generic operator cardinalities.
  • Replaces structural node counting in the analytical estimator.

How

Each operator owns an algorithmic resource formula. Child output statistics flow across edges; source bytes are charged only at physical scans; peak memory follows concurrent state requirements; shared nodes are charged once. Generic semantic checks reject evidence such as expanding filters, aggregate outputs larger than the group count, or Top-K outputs larger than k.

Before

A larger expression tree could appear more expensive even when it avoided most scans, and contradictory statistics could silently produce a favorable estimate.

After

Every supported physical DAG produces independently inspectable CPU, memory, and I/O estimates, or is unavailable with a concrete validation error.

Validation

  • cargo test -p asap-aware-mapping
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check

Stack base. #326 adds comparison-scope and evidence-provider validation.

@zzylol
zzylol marked this pull request as ready for review September 3, 2026 03:08
@zzylol
zzylol force-pushed the feat/analytical-resource-cost-323 branch from 87e8e32 to 8fdfdbf Compare September 3, 2026 16:02
@zzylol

zzylol commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

@milindsrivastava1997 @Selvomega Just labeling you as reviewers for this feature and future check

@zzylol
zzylol merged commit f44e64e into main Sep 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant