feat(asap-aware-mapping): generalize summary/CSE selection into a ReplacementStrategy trait

crates/asap-aware-mapping/src/lib.rs

@@ -8,12 +8,11 @@
 //! **Common sub-expression elimination (CSE) is not this crate's job.**
 //! Detection is a primary pass over the pre-ASAP `QueryExpr` IR itself
 //! (`asap_types::pre_asap`, design tracked in issue #223), run before a
-//! tree ever reaches [`bind::implement_tree`] — see issue #222 for why
-//! (batch query optimization needs to see shared work across a
+//! tree ever reaches [`replacement::SketchAlgorithmStrategy`] — see issue #222
+//! for why (batch query optimization needs to see shared work across a
 //! `QueryWorkload` before summary binding, not after). This crate may
 //! eventually run a second, narrower CSE pass of its own over an
-//! already-[`implement_tree`](bind::implement_tree)'d
-//! `SummaryExpr`/`SummaryNode` DAG, recognizing sharing that's invisible
+//! already-bound `SummaryExpr`/`SummaryNode` DAG, recognizing sharing that's invisible
 //! at the pre-ASAP level by construction — e.g. `Quantile(x, 0.99)` and
 //! `Quantile(x, 0.95)` are structurally distinct `AggIntent`s but can
 //! still share one built sketch, read out twice. That post-ASAP pass is
@@ -30,53 +29,105 @@
 //!
 //! ## Status
 //!
-//! Two real occupants and one stub:
+//! This crate has two replacement/search capabilities — and deliberately no
+//! third one that commits to a single, final, physically-materialized
+//! answer for a whole workload:
+//!
+//! - [`replacement::ReplacementStrategy`] — per-target, never prunes,
+//!   exhaustive. The `TargetSubDAG`/`ReplacementSubDAG`/
+//!   `ReplacementStrategy` vocabulary `docs/design_docs/asap_aware_mapping.md` stubs out
+//!   under "Key concepts (not yet implemented)", implemented for real (issue
+//!   #251, part of #33). [`replacement::SketchAlgorithmStrategy::replacements`]
+//!   both *decides* what an `AggIntent` may become
+//!   ([`replacement::implementations_for_with`], exhaustive and ranked via a
+//!   `CostModel`, sized to the `AccuracyTarget`) and *constructs* each
+//!   candidate's bound [`SummaryNode`](asap_types::post_asap::SummaryNode) —
+//!   every candidate comes back, not just one.
+//!   [`replacement::SharedSubtreeStrategy`] does the analogous job for the
+//!   build-independently-vs-build-once-and-share choice at a CSE-detected
+//!   shared subtree.
+//! - [`replacement::search_workload`]/[`replacement::PlanSpace::cost_sorted`]
+//!   — workload-wide, every candidate + cost, never materializes one
+//!   physical answer. Merged into the same module (issue #252, part of
+//!   #33): [`replacement::search_workload`]/[`replacement::search_workload_with`]
+//!   *search* — discover every candidate `TargetSubDAG` across a whole
+//!   workload (not just one target in isolation) and run every registered
+//!   strategy against each one, to a fixpoint, without ever materializing a
+//!   flat `2^N`-sized candidate-plan list: [`replacement::PlanSpace`] holds
+//!   one Cascades-style [`replacement::MemoGroup`] per distinct
+//!   `TargetSubDAG`, each carrying every alternative discovered for it.
+//!   [`replacement::PlanSpace::cost_sorted`] is the final
+//!   `sorted_by(cost_model)` step, ranking each group's candidates
+//!   best-first via the same [`CostModel`](cost_model::CostModel) the
+//!   single-target steps above already consult — see [`replacement`]'s own
+//!   module docs for the full design (MEMO groups vs. flat plans, dedup
+//!   discipline, termination, cost-based ranking).
+//!
+//! **Picking *which* candidate, and materializing one final answer, is a
+//! downstream deployment's job, out of this crate's scope.** This crate's
+//! output boundary is [`replacement::PlanSpace`]: every candidate
+//! replacement plus its cost, meant for a downstream consumer (e.g. a
+//! DAG-visualization view, or a deployment's own physical binder). Which
+//! sketch to commit to *and* where to place it are a joint decision only a
+//! deployment can see the full picture for — picking one in isolation, with
+//! no real consumer of that single materialized answer inside this crate,
+//! is out of scope. (A prior workload-wide "keep first/cost-preferred
+//! candidate per node, memoized by `Rc` identity" entry point —
+//! `bind::implement_workload`/`implement_workload_with` — used to live here
+//! and was removed for exactly this reason; see the terminology table below
+//! for where that "one answer" step now belongs, downstream.)
 //!
-//! - [`boundary`] — the per-intent sketch-vs-exact (accuracy) decision:
-//!   `AggIntent → Implementation` (a summary family's own `(Kind, Params)`,
-//!   or an exact accumulator) sized to the `AccuracyTarget` (issue #98).
-//!   [`boundary::implementation_for`] is the per-node decision;
-//!   [`bind::implement_tree`] drives it over a whole tree, and
-//!   [`bind::implement_workload`] drives it over a whole workload's roots —
-//!   memoized on `Rc` identity so two roots that
-//!   `asap_types::pre_asap::cse::share_common_subtrees` already collapsed
-//!   onto one shared subtree bind to one shared `SummaryNode` too (issue
-//!   #212, #222, #223) — see the terminology section below for why these are
-//!   named around "implementation" rather than "bind".
 //! - [`cost_model`] — the [`CostModel`](cost_model::CostModel) trait every
 //!   deployment's cost-based sketch selection plugs into (issues #6, #33).
 //!   `asap-plan` itself only ships [`DefaultCostModel`](cost_model::DefaultCostModel),
-//!   which preserves [`boundary`]'s built-in static preference order.
+//!   which preserves [`replacement`]'s built-in static preference order and
+//!   — via [`CostModel::estimate_cost`](cost_model::CostModel::estimate_cost)
+//!   — exposes an actual numeric cost per candidate, not just a relative
+//!   rank, for a caller (e.g. a DAG-visualization view) that wants to show
+//!   "candidate A costs ≈ X" next to "candidate B costs ≈ Y".
+//! - [`explanation`] — this crate's explanation of a replacement: a
+//!   reporting *view* over [`replacement`]'s candidate-plan space (issue
+//!   #257, part of #33) that translates every discovered `TargetSubDAG` with
+//!   a non-trivial candidate list into an
+//!   [`explanation::ReplacementExplanation`] (why a replacement exists,
+//!   where, reusing the candidate's own rationale rather than inventing new
+//!   prose), meant for the same downstream consumer (e.g. a
+//!   DAG-visualization view) the crate doc's `## Status` section above
+//!   already names for [`replacement::PlanSpace`] itself. Superseded PR
+//!   #247's own rule-based traversal, which re-walked the tree once per
+//!   optimization before [`replacement::search_workload`] existed to read
+//!   from instead — see that module's docs for the full reframing.
 //!
 //! ## Terminology — "bind" already means three different things nearby;
 //! this crate's own logical→physical step is named "implementation" instead
 //!
 //! `asap-plan` and its downstream consumers (e.g. `ASAPQuery-backend`'s
 //! `control_plane`) independently reused the word "bind" for three
 //! *different*, layer-specific meanings — none of which is what this
-//! crate's [`boundary`]/[`bind`] modules do. To avoid becoming a fourth,
-//! colliding sense of the same word, this crate names its own logical
-//! intent → physical realization step after the term the query-optimization
-//! literature already uses for exactly that step: **implementation**
-//! (Cascades/Volcano's "implementation rule", logical → physical, as
-//! distinct from a *transformation rule*, logical → logical — see Graefe,
-//! *The Cascades Framework for Query Optimization*):
+//! crate's own [`replacement`] module does. To avoid becoming a
+//! fourth, colliding sense of the same word, this crate names its own
+//! logical intent → physical realization step after the term the
+//! query-optimization literature already uses for exactly that step:
+//! **implementation** (Cascades/Volcano's "implementation rule", logical →
+//! physical, as distinct from a *transformation rule*, logical → logical —
+//! see Graefe, *The Cascades Framework for Query Optimization*).
 //!
 //! | Term | Stage | Meaning | Lives in |
 //! |---|---|---|---|
 //! | **Parse** | parse | text (PromQL/SQL) → AST | `asap-frontend-promql` / `asap-frontend-sql` |
 //! | **Bind #1** | name resolution | `ColumnRef` (a name) → `ColumnId` (a concrete schema column) — the classic RDBMS "Parse → **Bind** → Optimize" pipeline sense (e.g. SQL Server's query-processor terminology) | [`asap_types::pre_asap::binder::Binder`](https://docs.rs/asap-types) |
-//! | **Implementation** — [`boundary::implementation_for`] | pre-ASAP → post-ASAP, *one node* | choosing a concrete physical realization (a sketch family, an exact accumulator, or pass-through) for one [`AggIntent`](asap_types::pre_asap::agg_intent::AggIntent) | [`boundary`] |
-//! | **`implement_tree`** — [`bind::implement_tree`] | pre-ASAP → post-ASAP, *whole tree* | walk a whole `QueryExpr` tree, calling [`boundary::implementation_for`] per node, and emit the complete post-ASAP [`SummaryExpr`](asap_types::post_asap::SummaryExpr)/`SummaryNode` DAG — named after "implementation" too rather than reusing "bind" a second time | [`bind`] |
-//! | **Bind #2** (downstream, not in this crate) | post-ASAP → deployment placement | a *deployment's* own physical binder, additionally deciding **placement** (edge vs. backend, wire format, …) — a genuinely different, deployment-specific decision this crate doesn't model at all | e.g. `control_plane::sketch_algebra::rules::bind_*` (as of this writing; expected to fold into that deployment's cost-model layer rather than stay a separate "bind" concept) |
+//! | **Implementation** — `replacement::implementations_for_with` | pre-ASAP → post-ASAP, *one node* | enumerating every concrete physical realization (a sketch family, an exact accumulator, or pass-through) for one [`AggIntent`](asap_types::pre_asap::agg_intent::AggIntent) | [`replacement`] |
+//! | **Replacement** — [`replacement::SketchAlgorithmStrategy::replacements`] | pre-ASAP → post-ASAP, *one target, every candidate* | wrap each `implementations_for_with` candidate into its own bound [`SummaryNode`](asap_types::post_asap::SummaryNode), ranked — a caller wanting one answer takes the first entry itself | [`replacement`] |
+//! | **Search** — [`replacement::search_workload`]/[`replacement::search_workload_with`] | pre-ASAP → post-ASAP, *whole workload, every candidate* | a Cascades/Volcano-style MEMO search: discover every candidate `TargetSubDAG` across a whole workload (not just one target in isolation), run every registered `ReplacementStrategy` against each to a fixpoint, and dedup into a [`replacement::PlanSpace`] — one [`replacement::MemoGroup`] per distinct `TargetSubDAG` holding every alternative discovered for it, never a flat `2^N`-sized list of whole candidate plans | [`replacement`] |
+//! | **Bind #2** (downstream, not in this crate) | post-ASAP → deployment placement | a *deployment's* own physical binder, deciding **which** candidate to commit to *and* **placement** (edge vs. backend, wire format, …) for a whole workload — a genuinely different, deployment-specific decision this crate doesn't model at all (this is also where a prior workload-wide "keep first/cost-preferred candidate per node" step, `bind::implement_workload`/`implement_workload_with`, would belong if a deployment still wants that exact behavior — it isn't shipped by this crate) | e.g. `control_plane::sketch_algebra::rules::bind_*` (as of this writing; expected to fold into that deployment's cost-model layer rather than stay a separate "bind" concept) |
 //!
 //! A related question (tracked alongside issues #6/#33): whether this
 //! crate should also own a **matching** predicate — "does an already
 //! *available* `Implementation` satisfy a *required* one" — the way a
 //! database's materialized-view matching / "answering queries using
 //! views" layer does. It owns the *question*, not an *answer*:
-//! [`boundary::Matcher`] is a trait with no default implementation and no
-//! shipped instance, the same shape as [`cost_model::CostModel`] and for
+//! [`replacement::Matcher`] is a trait with no default implementation and
+//! no shipped instance, the same shape as [`cost_model::CostModel`] and for
 //! the same reason — which `Implementation`s are actually *available*
 //! anywhere is entirely a downstream deployment's concern (an inventory
 //! this crate has no way to see), and even the pure sketch-algebra
@@ -87,15 +138,17 @@
 //! `sketch_algebra::capability::Capability`/`is_satisfied_by` is the
 //! reference downstream implementation.
 
-pub mod bind;
-pub mod boundary;
 pub mod cost_model;
+pub mod explanation;
+pub mod replacement;
 
-pub use bind::{
-    implement_tree, implement_tree_with, implement_workload, implement_workload_with,
-    ImplementError,
+pub use cost_model::{CostModel, DefaultCostModel};
+pub use explanation::{
+    explain_replacements, explain_replacements_with, ExplanationKind, ReplacementExplanation,
 };
-pub use boundary::{
-    implementation_for, implementation_for_with, summary_candidates, Implementation, Matcher,
+pub use replacement::{
+    default_strategies, default_strategies_with, search_workload, search_workload_with,
+    summary_candidates, ImplementError, Implementation, Matcher, MemoGroup, PlanSpace, RankedGroup,
+    Replacement, ReplacementStrategy, ReplacementSubDAG, SharedSubtreeStrategy,
+    SketchAlgorithmStrategy, TargetSubDAG, MAX_SEARCH_ITERATIONS,
 };
-pub use cost_model::{CostModel, DefaultCostModel};