Skip to content

feat(asap-aware-mapping): select CSE using effective cross-group costs - #272

Merged
zzylol merged 1 commit into
mainfrom
feat/whole-plan-cost-selection-271
Aug 25, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/whole-plan-cost-selection-271

Conversation

@zzylol

@zzylol zzylol commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Closes #271. Part of #33.

Why this PR is needed

PlanSpace::cost_sorted ranks each memo group independently. For CSE choices, it passes the group's structural consumer_count to CostModel::cse_share_decision.

That count is correct only when ancestor choices do not change how often the group executes. If a shared ancestor is instead selected as RecomputeIndependently, every execution of that ancestor also executes its descendants. A descendant can therefore run more times than its number of direct incoming edges suggests.

Concrete example:

Assume three dashboard queries run over the same request table:

-- Dashboard 1: enterprise tenants
SELECT tenant_id
FROM (SELECT DISTINCT tenant_id FROM requests) AS active_tenants
WHERE tenant_id LIKE 'enterprise-%';

-- Dashboard 2: the same result for another dashboard
SELECT tenant_id
FROM (SELECT DISTINCT tenant_id FROM requests) AS active_tenants
WHERE tenant_id LIKE 'enterprise-%';

-- Dashboard 3: all active tenants
SELECT DISTINCT tenant_id FROM requests;

After CSE, the repeated work is represented as:

c = Dedup tenant_id from requests
a = Filter c to enterprise tenants

Dashboard 1 --\
                +--> a --> c
Dashboard 2 --/
Dashboard 3 ------------> c

The two identical enterprise queries share one structural node a. Therefore the graph contains only one a -> c edge. Structurally, the planner sees:

a.consumer_count = 2   (Dashboard 1 and Dashboard 2)
c.consumer_count = 2   (one edge from a and one edge from Dashboard 3)

Now use concrete cost estimates:

recomputing a or c once = 40 cost units
maintaining one shared result = 100 cost units

2 executions: recompute cost = 2 x 40 = 80   -> recompute is cheaper
3 executions: recompute cost = 3 x 40 = 120  -> sharing is cheaper

The cost model first selects a = RecomputeIndependently, because its two executions cost 80 instead of 100. Dashboard 1 and Dashboard 2 therefore execute separate copies of a, and each copy also executes c. Dashboard 3 executes c once more:

c through Dashboard 1's copy of a = 1
c through Dashboard 2's copy of a = 1
c for Dashboard 3                   = 1
                                      -
c.effective_consumer_count          = 3

A local decision uses c.consumer_count = 2 and chooses recomputation at cost 80. The whole-plan decision uses c.effective_consumer_count = 3 and chooses sharing at cost 100 instead of recomputation at cost 120.

Independent group ranking still costs c as two executions and can choose the wrong CSE plan. The planner needs to carry an ancestor's selected multiplicity into its descendants before deciding them.

How this PR solves it

This PR adds PlanSpace::global_selection, a parent-before-child dynamic-programming pass over the complete candidate space.

1. Identify candidate roles explicitly

ReplacementSubDAG now carries ReplacementProvenance:

SummaryImplementation
CseShare
CseRecompute
LogicalRewrite

The planner can locate the exact CSE share/recompute pair even when summary, Hydra, roll-up, or semantic-rewrite candidates coexist in the same memo group. It no longer infers CSE semantics from rationale text, replacement shape, or pointer identity alone.

2. Build the possible reference graph

The planner builds a ReferenceGraph from:

  • workload roots;
  • each target's original child edges; and
  • child edges introduced by logical rewrite candidates.

Kahn's algorithm produces a real parent-before-child topological order. Discovery order is insufficient for shared diamonds because a later-discovered parent can reference an earlier-discovered child.

3. Propagate effective execution counts

For each group in topological order, selection computes:

effective uses = direct root uses + contributions from selected parents

A selected parent contributes:

parent chose Share       -> edge_count x 1
parent chose Recompute   -> edge_count x parent_effective_uses
ordinary selected plan   -> edge_count x parent_effective_uses

Only the selected rewrite's child edges propagate counts. Edges belonging to unselected alternatives affect topological ordering but do not inflate execution counts.

4. Re-cost CSE with the corrected count

When a group has a CSE pair and at least two effective uses, global_selection calls the existing CostModel::cse_share_decision with the corrected count. It does not introduce a second cost interface.

If a shared ancestor collapses a descendant to one effective execution, the descendant still receives a valid selected plan: an unrelated non-CSE candidate when available, otherwise the original/share candidate as the single-execution fallback.

Before and after

For the example above:

Before: independent local ranking
  a: raw count 2 -> RecomputeIndependently
  c: raw count 2 -> RecomputeIndependently   (incorrect whole-plan choice)

After: global_selection
  a: effective count 2 -> RecomputeIndependently
  c: effective count 3 -> Share              (corrected choice)

PlanSpace::cost_sorted remains available for inspecting all locally ranked alternatives. PlanSpace::global_selection returns one SelectedGroup per site with both its structural and effective consumer counts.

Interaction with current strategies

  • CSE share/recompute decisions use the propagated effective count.
  • Logical rewrites propagate edges from the selected rewrite, not the original target.
  • Summary-family and Hydra ranking remain local because their existing cost hooks do not consume execution multiplicity.
  • The pass retains unrelated candidates in the memo even when it selects the explicit CSE pair.

Scope

This is not an exhaustive search across every possible plan combination or a global resource-budget optimizer. It corrects the ancestor-dependent execution count used by the existing pairwise CSE cost decision.

Tests

Coverage includes:

  • a recomputed ancestor increasing a descendant from two to three effective uses and changing its decision to Share;
  • a shared ancestor collapsing a CSE descendant to one use while still producing a selected plan;
  • propagation through the selected logical rewrite;
  • mixed CSE and unrelated rewrite candidates;
  • materializing a CSE choice for a descendant with one structural edge but multiple effective executions;
  • a shared-diamond whose discovery order is not topological;
  • local-ranking fallback for sketch-family groups; and
  • unchanged behavior for non-interacting workloads.

Validation

  • cargo test -p asap-types -p asap-aware-mapping -p asap-integration-tests
  • cargo clippy -p asap-aware-mapping --all-targets -- -D warnings
  • cargo fmt --all -- --check

All local validation passes, including 142 asap-aware-mapping tests.

@zzylol
zzylol force-pushed the feat/whole-plan-cost-selection-271 branch from 3c10e84 to a42d3d0 Compare August 25, 2026 22:50
@zzylol
zzylol changed the base branch from feat/cascades-search-252 to main August 25, 2026 22:50
@zzylol
zzylol force-pushed the feat/whole-plan-cost-selection-271 branch from a42d3d0 to b9b0e90 Compare August 25, 2026 22:51
@zzylol zzylol changed the title feat(asap-aware-mapping): whole-plan (cross-group) cost-based selection over PlanSpace feat(asap-aware-mapping): select CSE using effective cross-group costs Aug 25, 2026
@zzylol
zzylol force-pushed the feat/whole-plan-cost-selection-271 branch from b9b0e90 to de880da Compare August 25, 2026 23:09
@zzylol
zzylol merged commit 107b08a into main Aug 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

asap-aware-mapping: whole-plan (cross-group) cost-based selection over the search's candidate-plan space

1 participant