feat(pre-asap): structural CSE via bottom-up hash-consing (stages 1+2) - #235
Merged
Merged
Conversation
Rebased onto main now that #226 (the QueryExpr rename) and #227 (the SQL function catalog) have landed there. This PR's own design and content are unchanged; adapted the new cse.rs module's exhaustive QueryExpr match to the current post-rename variant names (Merge -> Concat, Distinct -> Dedup, Scalar -> PromqlScalar, EvalTime -> QueryTimestamp, WindowFunc -> SQLWindowFunc, Arith -> Arithmetic, VectorFromScalar/ScalarFromVector/Relabel/InfoJoin/Sample/Subquery -> their Promql*-prefixed names, ArithOp/CompareOp -> ArithmeticOpKind/ CompareOpKind) and reapplied mod.rs's doc/pub-use additions around the renamed expr_ir re-export. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
zzylol
force-pushed
the
feat/pre-asap-cse
branch
from
August 22, 2026 20:59
a400d1f to
eba4fbe
Compare
This was referenced Aug 22, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements stages 1 and 2 of #223's 4-stage landing plan — the pre-ASAP
structural common-subexpression elimination pass, and a real caller wired
alongside it so the pass never becomes unwired dead code (the same fate
asap-plan::cse::dedupe_subtreesmet in #192).Stage 1 —
asap_types::pre_asap::cse::share_common_subtrees+InternTable(new module,crates/types/src/pre_asap/cse.rs): bottom-uphash-consing over an already-
resolve_root'dQueryExprtree.DefaultHasher, thesame trick
dag_export.rs's ownstructural_hashuses, deliberatelykept independent for now — stage 3 unifies them) only narrows the
candidate set;
PartialEqis the actual, non-negotiable sharingdecision, every time.
Schema::has_unique_key(): a subtree with no provable rowidentity is never hoisted, even when structurally identical to
something already interned.
of the same
share_common_subtreescall for free — no separatemechanism.
canonicalize.rs'schildren_mutexactly:only the relational-skeleton children participate; a scalar
sub-expression reachable only through a wrapper position (
Predicate,ProjectItem.expr,Aggregate.having, …) stays embedded, compared aspart of its owning node's
PartialEq— the same scopecanonicalize.rsalready settled on for its own bottom-up walk.Quantiles don't merge; nounique_keys⇒no merge (including a
without(...)grouping sanity check);medianvs. an explicit
q=0.5percentile do merge (both already lower tothe identical
AggIntent::Quantile, persql_lowering.rs'smedian_is_the_same_intent_as_an_explicit_half_percentile);single-query CSE; and
Distinctgated the same way asAggregate.Stage 2 —
asap_aware_mapping::implement_workload/implement_workload_with(crates/asap-aware-mapping/src/bind.rs): areal caller for stage 1. Memoizes on
Rc::as_ptridentity overimplement_tree/implement_tree_with, so two workload roots sharing anRc<QueryExpr>aftershare_common_subtreesbind to one sharedRc<SummaryNode>instead of being bound twice. Newcrates/integration-tests/tests/cse.rsdrives real PromQL text throughthe full
lower_promql → share_common_subtrees → implement_workloadpipeline and asserts the sharing survives end-to-end via
Rc::ptr_eqonthe bound result (plus a negative control for non-identical queries, and
a single-query-repeated-subexpression case).
Adaptations from the issue's illustrative pseudocode
The issue's own text says its code sketch is illustrative, not verified
against current code, and current
mainhas drifted since it was written:QueryIddoesn't exist anywhere in the codebase.share_common_subtreesand
implement_workloadare generic over a caller-chosenIdinstead ofinventing a new pub type.
Schema::is_reusable()doesn't exist; the real method ishas_unique_key().implement_tree/implement_tree_withreturnResult<Rc<SummaryNode>, ImplementError>, not an infallibleRc<SummaryNode>, soimplement_workloadreturnsVec<(Id, Result<Rc<SummaryNode>, ImplementError>)>rather than the sketch'sVec<(QueryId, Rc<SummaryNode>)>. Only successful binds are memoized; a failed bind isretried (not cached) if the same
Rcreappears.Merge/Distinct/Scalar/etc. rename in refactor: rename PromQL-specific QueryExpr nodes (and Merge/Distinct/WindowFunc/EvalTime) for clarity #226 had not landed onmainas of this branch, so this PR uses the pre-rename variant namesthat are actually on
maintoday.Deliberately deferred (per the issue's own stage sequencing)
Not attempted in this PR, matching #223's landing plan:
dag_export::structural_hashwith this pass'shashing, and updating
tools/dag-viewer's "shared subtree" highlightingto reflect real CSE output instead of the current hash-proxy.
CostModel("onlythen," per the issue).
Also out of scope, per the issue: subsumption reasoning
(
asap_aware_mapping::boundary::Matcher) and cross-batch/persistentsharing.
Test plan
cargo build --workspace --all-targetscargo test --workspace(all green, including the 6 newcseunit tests and 3 new integration tests)
cargo fmt --all -- --checkcargo clippy --workspace --all-targets --all-features -- -D warningsPart of #223.
🤖 Generated with Claude Code