refactor(types): collapse QueryExpr::PromqlScalar into Literal (#220) - #246
Merged
Merged
Conversation
Instance 1 of #220: `QueryExpr::PromqlScalar(f64)` held exactly the same value `QueryExpr::Literal(ScalarValue::Float64(_))` does, just at a different tree position (a `BinaryOp` operand / query root, vs. a `Compare`/`Arithmetic` operand). Collapses the two into one variant, `Literal`, plus a new marker wrapper `PromqlScalarBridge(Rc<QueryExpr<C>>)` that carries the tree-position distinction the old variant tag used to: wrapping a `Literal` in `PromqlScalarBridge` is what now says "this scalar sub-expression sits at an operator-tree position and has its own row schema" (`output_schema`'s `BinaryOp`/root arms), vs. a bare `Literal` in a scalar-sub-language position, which still has none (`QueryExprError::ScalarHasNoRowSchema`). - `query_expr.rs`: removes `PromqlScalar`, adds `PromqlScalarBridge`, and the `promql_scalar`/`as_promql_scalar` constructor/accessor every call site now uses in place of the old bare variant. - `canonicalize.rs`, `resolve.rs`, `binder.rs`: the bridge's child is a genuine scalar-sub-language node now, routed through `resolve_expr`/ `columns_referenced` like every other scalar position, instead of being an opaque `f64` leaf. - `cse.rs`, `dag_export.rs`: minimal surgical updates (kept out of scope otherwise per the CSE work happening elsewhere) — the bridge is still treated as a single opaque unit, matching the old `PromqlScalar` leaf's treatment exactly. - Both front ends' test suites, `devtools`, `integration-tests`: every call site updated; behavior preserved exactly (all existing tests pass unchanged). Adds unit tests in `query_expr.rs` and `resolve.rs` pinning: the bridge/bare-literal value equivalence, that the row-schema/no-row- schema split now rides on the wrapper rather than the variant, and that `BinaryOp`'s `vector_match` semantics are unaffected. Instance 2 (`BinaryOp` vs `Compare`/`Arithmetic`) is scoped out as a separate, larger change — tracked in #245. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Aug 23, 2026
…dge rename Rebase fixup: #246 (merged after this branch was forked) collapsed QueryExpr::PromqlScalar(f64) into PromqlScalarBridge(Rc<QueryExpr>), matching rebuild_children's existing treatment (never descended into, interned as a single unit). dag_node_count's count_unique needed the same update - it still referenced the removed PromqlScalar variant, which fails to compile against current main. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Aug 23, 2026
…haring (#237, #223, #212) (#243) * docs(asap-aware-mapping): decide rule-based vs. cost-based CSE sharing (#237) Records the decision for issue #237: how CostModel's still-deferred stage-4 CSE-credit wiring (issue #223's landing plan) should decide whether a detected, legality-gated common subexpression is actually worth sharing. Decision: a hybrid, per #237's own suggestion — unconditional sharing below a cheap-recompute threshold, a real cost comparison above it. Neither pure Volcano/Cascades (this repo has no plan-enumeration/DP search anywhere; new infrastructure disproportionate to a binary share-or-don't decision) nor pure System R (ignores the real cost of continuously maintaining a shared summary for a rarely-read or cheap-to-recompute subtree) fits on its own. The hybrid also matches this crate's own existing pattern for the structurally analogous sketch-vs-exact decision (rank_candidates/ size_params: a cheap built-in default, overridable by a deployment's real cost knowledge) and is layering-forced, not just preferred: detection (asap-types::pre_asap::cse) sits below this crate and cannot consult a CostModel, so it must stay cost-agnostic (System R-style default), pushing the cost-aware override downstream into this crate. Also sketches the concrete shape for a future stage-4 PR: a new CostModel::cse_share_decision trait method (default preserves today's unconditional-share behavior byte for byte), a CseCandidate input carrying the shared subtree/bound summary/consumer count, and the call site (implement_workload_with's memo-hit branch in bind.rs). Documentation only — no CSE-credit logic implemented, per #237's own scope ("not asking for an implementation yet"). Also updates cse.rs's landing-plan note to point at this decision instead of just flagging stage 4 as deferred. Related to #223, #212. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(asap-aware-mapping): implement CSE stage 4 (Volcano/Cascades cost-based sharing) Replaces the previous doc-comment-only decision for #237 with a real, working implementation, per review feedback: - The design discussion moves out of code comments into docs/cse-cost-model-decision.md. - CostModel gains CseCandidate/ShareDecision/cse_share_decision, plus two overridable cost-estimation hooks (cse_recompute_cost, cse_shared_maintenance_cost) with concrete default heuristics. The default cse_share_decision body performs a genuine Volcano/Cascades- style cost comparison — share iff the estimated cost of maintaining one shared summary is no greater than the estimated total cost of recomputing it independently at every consumer — rather than a fixed System R-style rule. - bind::implement_workload_with now computes each shared subtree's true consumer_count via a whole-workload pre-pass, decides ShareDecision once per shared subtree (from full knowledge of the workload's sharing structure), and applies it consistently to every occurrence. This closes out #223's stage 4 and #212's original "add CSE" tracking issue. Stage 3 (structural-hash unification) landed separately in #244. New tests: cost_model.rs unit tests pinning the default cost comparison's behavior in both directions and confirming custom CostModel overrides are actually consulted; a bind.rs test confirming a CostModel that declines sharing produces genuinely independent SummaryNodes even for two roots that are the same Rc<QueryExpr>. All pre-existing CSE tests (including the DefaultCostModel-sharing integration tests) still pass unchanged — the default cost weights were calibrated against a real lowered query so today's sharing behavior for realistic subtrees is preserved. Verified: cargo build --workspace --all-targets, cargo test --workspace, cargo fmt --all -- --check, cargo clippy --workspace --all-targets --all-features -- -D warnings, cargo doc --workspace --no-deps (no new broken-link warnings). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(asap-aware-mapping): CSE recompute-cost proxy must be DAG-aware, not tree-shaped default_cse_recompute_cost measured a subtree's cost via serde_json::to_string(subtree).len() — but a CseCandidate's subtree, by definition, is something CSE already found sharing in, so it's generally a DAG, not a tree. Rc's Serialize impl serializes the pointee's value at every occurrence, not once by identity, so a naive full serialization re-counts any descendant the subtree already shares internally (e.g. from single-query CSE, x op x collapsing both branches onto one Rc) once per parent that references it — silently inflating the cost estimate relative to the DAG's real size. Adds asap_types::pre_asap::cse::dag_node_count(root) -> usize: counts unique nodes reachable from root, deduplicated by Rc pointer identity, using the same operator-child traversal scope share_common_subtrees itself uses (mirrors rebuild_children's field enumeration, as a separate read-only traversal since rebuild_children consumes and rebuilds its input). default_cse_recompute_cost now calls this instead of serde_json. Recalibrates default_cse_shared_maintenance_cost's UNIT constant (60.0 -> 1.0) to match the new magnitude: node counts are small integers (typically 1-20), not byte lengths (typically 200-500) — the weight table's relative ordering (ExactAggregate cheapest, StatModel priciest) is unchanged, only the absolute scale. Verified this preserves existing default-sharing behavior: all pre-existing CSE tests, including crates/integration-tests/tests/cse.rs's DefaultCostModel-sharing assertions, pass unchanged. New tests: - cse.rs: dag_node_count is the naive count when nothing is shared; correctly dedupes an internally-shared subtree (BinaryOp with both branches pointing at the same Rc: 3 unique nodes, not 5); dedupes correctly across two workload roots sharing one subtree. - cost_model.rs: default_recompute_cost_does_not_double_count_an_internally_shared_descendant — a Join with two independent Scan children costs 3, the same Join with both children pointing at one shared Scan costs 2, not 3 (the bug this fix addresses, demonstrated directly at the layer that consumes dag_node_count). Verified: cargo build --workspace --all-targets, cargo test --workspace (444 passed, 0 failed), cargo fmt --all -- --check, cargo clippy --workspace --all-targets --all-features -- -D warnings, cargo doc --workspace --no-deps (no new broken-link warnings). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(types): update dag_node_count for PromqlScalar -> PromqlScalarBridge rename Rebase fixup: #246 (merged after this branch was forked) collapsed QueryExpr::PromqlScalar(f64) into PromqlScalarBridge(Rc<QueryExpr>), matching rebuild_children's existing treatment (never descended into, interned as a single unit). dag_node_count's count_unique needed the same update - it still referenced the removed PromqlScalar variant, which fails to compile against current main. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
4 tasks
zzylol
added a commit
that referenced
this pull request
Aug 23, 2026
…rBridge rename #246 (merged after this branch was forked) renamed QueryExpr::PromqlScalar(f64) to PromqlScalarBridge(Rc<QueryExpr>). walk_rc_children and for_each_operator_child still referenced the removed variant. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves Instance 1 of #220 (
Scalar(f64)vsLiteral(ScalarValue)duplication). Instance 2 (BinaryOpvsCompare/Arithmetic) is scoped out — see below.What #220 observed
#205 folded the old, separate
Expr<C>scalar-expression tree intoQueryExpritself, and in doing so left two node shapes representing the same relational-vs-scalar-sub-language split for what's semantically the same value:QueryExpr::Scalar(f64)(renamedPromqlScalarby Consider mapping SQL now()/CURRENT_TIMESTAMP (incl. ClickHouse now()) to EvalTime #184, merged in this PR) — a PromQL number literal / folded constant, appearing only as aBinaryOpoperand or a bare-scalar query root.QueryExpr::Literal(ScalarValue)— a typed scalar constant used inside the scalar-expression sub-language (Compare/Arithmetic/Case/InListoperands, etc.).Every PromQL scalar is
f64, soLiteral(ScalarValue::Float64(v))held exactly the valuePromqlScalar(v)did — the two variants existed side by side purely because of which tree position each was allowed to appear in, not because they carried different data.The fix
Removed
PromqlScalar(f64). Added a new marker variant instead:This is the "wrapper/marker at the bridge points instead of on the leaf itself" option #220 called out. Every construction site that used to write
QueryExpr::PromqlScalar(v)now writesQueryExpr::promql_scalar(v), which buildsPromqlScalarBridge(Rc::new(Literal(ScalarValue::Float64(v))))— so the value now always lives in the oneLiteralvariant, andPromqlScalarBridgecarries only the fact that this particularLiteralsits at an operator-tree position (aBinaryOpoperand, aPromqlVectorFromScalarchild, or a whole query's scalar-typed root) rather than in a scalar-sub-language position (Compare/Arithmetic/Case/… operand).How the row-schema-position distinction is now tracked
Previously,
output_schema(),canonicalize.rs'schildren_mut, andresolve.rs's match all used "is this node taggedPromqlScalarorLiteral" to tell "this operand has its own row schema" from "this is a nested scalar leaf with none" (QueryExprError::ScalarHasNoRowSchema). Now that both cases can hold the identicalLiteral(ScalarValue::Float64(_))node, that split rides on the wrapper, not the variant tag:output_schema()has a dedicated arm forPromqlScalarBridge(_)(same "singlevaluecolumn,Float64, closed" schemaPromqlScalarused to return) and no longer special-casesLiteralat all — a bareLiteralstill falls through to the scalar-leaf catch-all and still returnsErr(ScalarHasNoRowSchema), exactly as before.canonicalize.rs'schildren_mutandcse.rs'srebuild_childrentreatPromqlScalarBridgeas a single opaque unit (no recursion into its child) — the same treatmentPromqlScalargot as a leaf, now applied to the wrapper instead.resolve.rsroutes the bridge's child throughresolve_expr(the scalar-sub-language resolver), the same function every other scalar position already uses, rather than the old no-opPromqlScalar(v) => PromqlScalar(*v)passthrough. In practice the child is always aLiteral(noColumnRefto resolve), so this is behavior-preserving, but it's now structurally consistent with how every other scalar position resolves instead of being a special case.binder.rs'scollect_referenced_columnssimilarly now peels column refs off the bridge's child via the samenamedhelper every other scalar-typed field uses (a no-op today, since the child is always aLiteral, but no longer hand-waved as "never has columns" by construction).Added
QueryExpr::promql_scalar(v: f64) -> Self(constructor) andQueryExpr::as_promql_scalar(&self) -> Option<f64>(accessor,Someonly for a bridge wrapping a plainLiteral(Float64)) to keep call sites — construction and pattern-matching alike — as close to the old one-variant ergonomics as possible.Call sites updated
Everywhere that constructed or matched on
QueryExpr::PromqlScalar:crates/types/src/pre_asap/{query_expr.rs, canonicalize.rs, resolve.rs, binder.rs, cse.rs, dag_export.rs}crates/types/src/dag_export.rsandcrates/types/src/pre_asap/cse.rs— both touched by the currently in-flight CSE work (New feature: add common sub expr elimination #212/Design: pre-ASAP structural CSE via bottom-up hash-consing over Rc<QueryExpr> #223/CSE: decide rule-based vs. cost-based framework for whether to actually share a detected common subexpression #237) in other worktrees; kept to the minimal surgical edit needed to keep them compiling and behaviorally identical (no structural changes to either module's own logic).crates/frontend-promql/src/promql.rs(the only front end that constructs it) and its test suites (promql_conformance.rs,observability/awesome_prometheus_alerts.rs)crates/frontend-sqltest suites that pattern-match generically over everyQueryExprvariant (netflow.rs,synthetic_packet_trace.rs)crates/integration-tests/tests/binary_op.rscrates/devtools/src/bin/variant_coverage.rs,crates/devtools/examples/canonical_examples.rsdocs/pre-asap-ir.mdBehavior preservation
This is a pure representation refactor — no semantics change. All existing tests pass unchanged (
cargo test --workspace: every crate green, including the full PromQL conformance suite, the SQL lowering suite, and the CSE/binding integration tests). Added new unit tests:query_expr.rs::promql_scalar_bridges_a_literal_float_at_an_operator_position— the constructor/accessor round-trip, and that a bridge and an unwrappedLiteralof the same value are distinct nodes.query_expr.rs::row_schema_rides_on_the_bridge_wrapper_not_the_literal_variant— pins the tree-position distinction directly: the identicalLiteral(Float64(42.0))value has a row schema when bridged, and still errors withScalarHasNoRowSchemawhen bare.query_expr.rs::binary_op_schema_follows_the_vector_side_over_a_scalar_bridge_with_vector_match_intactandresolve.rs::resolve_root_threads_a_scalar_bridge_operand_and_preserves_vector_match— confirmBinaryOp'svector_matchsemantics (Instance 2's territory) are untouched by this change, both pre- and post-resolve_root.What's scoped out
Instance 2 (
BinaryOpvsCompare/Arithmetic) is not addressed here.BinaryOpKindalready just wrapsArithmeticOpKind/CompareOpKind, so the same duplication pattern exists one level up — butBinaryOpcarriesvector_match: Option<VectorMatch>, whichCompare/Arithmeticdon't and which only makes sense at the relational-tree position, and the blast radius is roughly 60 call sites (vs. ~25 forPromqlScalar) including several PromQL-specific vector-match assertions that need care to preserve exactly. That's large enough to want its own PR. Filed as a tracking issue: #245.Closes: part of #220 (Instance 1 only — Instance 2 tracked separately in #245)
🤖 Generated with Claude Code