Skip to content

refactor(types): collapse QueryExpr::PromqlScalar into Literal (#220) - #246

Merged
zzylol merged 1 commit into
mainfrom
refactor/queryexpr-scalar-relational-dedup-220
Aug 23, 2026
Merged

zzylol merged 1 commit into
mainfrom
refactor/queryexpr-scalar-relational-dedup-220

Conversation

@zzylol

@zzylol zzylol commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Resolves Instance 1 of #220 (Scalar(f64) vs Literal(ScalarValue) duplication). Instance 2 (BinaryOp vs Compare/Arithmetic) is scoped out — see below.

What #220 observed

#205 folded the old, separate Expr<C> scalar-expression tree into QueryExpr itself, and in doing so left two node shapes representing the same relational-vs-scalar-sub-language split for what's semantically the same value:

Every PromQL scalar is f64, so Literal(ScalarValue::Float64(v)) held exactly the value PromqlScalar(v) did — the two variants existed side by side purely because of which tree position each was allowed to appear in, not because they carried different data.

The fix

Removed PromqlScalar(f64). Added a new marker variant instead:

PromqlScalarBridge(Rc<QueryExpr<C>>)

This is the "wrapper/marker at the bridge points instead of on the leaf itself" option #220 called out. Every construction site that used to write QueryExpr::PromqlScalar(v) now writes QueryExpr::promql_scalar(v), which builds PromqlScalarBridge(Rc::new(Literal(ScalarValue::Float64(v)))) — so the value now always lives in the one Literal variant, and PromqlScalarBridge carries only the fact that this particular Literal sits at an operator-tree position (a BinaryOp operand, a PromqlVectorFromScalar child, or a whole query's scalar-typed root) rather than in a scalar-sub-language position (Compare/Arithmetic/Case/… operand).

How the row-schema-position distinction is now tracked

Previously, output_schema(), canonicalize.rs's children_mut, and resolve.rs's match all used "is this node tagged PromqlScalar or Literal" to tell "this operand has its own row schema" from "this is a nested scalar leaf with none" (QueryExprError::ScalarHasNoRowSchema). Now that both cases can hold the identical Literal(ScalarValue::Float64(_)) node, that split rides on the wrapper, not the variant tag:

  • output_schema() has a dedicated arm for PromqlScalarBridge(_) (same "single value column, Float64, closed" schema PromqlScalar used to return) and no longer special-cases Literal at all — a bare Literal still falls through to the scalar-leaf catch-all and still returns Err(ScalarHasNoRowSchema), exactly as before.
  • canonicalize.rs's children_mut and cse.rs's rebuild_children treat PromqlScalarBridge as a single opaque unit (no recursion into its child) — the same treatment PromqlScalar got as a leaf, now applied to the wrapper instead.
  • resolve.rs routes the bridge's child through resolve_expr (the scalar-sub-language resolver), the same function every other scalar position already uses, rather than the old no-op PromqlScalar(v) => PromqlScalar(*v) passthrough. In practice the child is always a Literal (no ColumnRef to resolve), so this is behavior-preserving, but it's now structurally consistent with how every other scalar position resolves instead of being a special case.
  • binder.rs's collect_referenced_columns similarly now peels column refs off the bridge's child via the same named helper every other scalar-typed field uses (a no-op today, since the child is always a Literal, but no longer hand-waved as "never has columns" by construction).

Added QueryExpr::promql_scalar(v: f64) -> Self (constructor) and QueryExpr::as_promql_scalar(&self) -> Option<f64> (accessor, Some only for a bridge wrapping a plain Literal(Float64)) to keep call sites — construction and pattern-matching alike — as close to the old one-variant ergonomics as possible.

Call sites updated

Everywhere that constructed or matched on QueryExpr::PromqlScalar:

Behavior preservation

This is a pure representation refactor — no semantics change. All existing tests pass unchanged (cargo test --workspace: every crate green, including the full PromQL conformance suite, the SQL lowering suite, and the CSE/binding integration tests). Added new unit tests:

  • query_expr.rs::promql_scalar_bridges_a_literal_float_at_an_operator_position — the constructor/accessor round-trip, and that a bridge and an unwrapped Literal of the same value are distinct nodes.
  • query_expr.rs::row_schema_rides_on_the_bridge_wrapper_not_the_literal_variant — pins the tree-position distinction directly: the identical Literal(Float64(42.0)) value has a row schema when bridged, and still errors with ScalarHasNoRowSchema when bare.
  • query_expr.rs::binary_op_schema_follows_the_vector_side_over_a_scalar_bridge_with_vector_match_intact and resolve.rs::resolve_root_threads_a_scalar_bridge_operand_and_preserves_vector_match — confirm BinaryOp's vector_match semantics (Instance 2's territory) are untouched by this change, both pre- and post-resolve_root.

What's scoped out

Instance 2 (BinaryOp vs Compare/Arithmetic) is not addressed here. BinaryOpKind already just wraps ArithmeticOpKind/CompareOpKind, so the same duplication pattern exists one level up — but BinaryOp carries vector_match: Option<VectorMatch>, which Compare/Arithmetic don't and which only makes sense at the relational-tree position, and the blast radius is roughly 60 call sites (vs. ~25 for PromqlScalar) including several PromQL-specific vector-match assertions that need care to preserve exactly. That's large enough to want its own PR. Filed as a tracking issue: #245.

Closes: part of #220 (Instance 1 only — Instance 2 tracked separately in #245)

🤖 Generated with Claude Code

Instance 1 of #220: `QueryExpr::PromqlScalar(f64)` held exactly the
same value `QueryExpr::Literal(ScalarValue::Float64(_))` does, just at
a different tree position (a `BinaryOp` operand / query root, vs. a
`Compare`/`Arithmetic` operand). Collapses the two into one variant,
`Literal`, plus a new marker wrapper `PromqlScalarBridge(Rc<QueryExpr<C>>)`
that carries the tree-position distinction the old variant tag used to:
wrapping a `Literal` in `PromqlScalarBridge` is what now says "this
scalar sub-expression sits at an operator-tree position and has its
own row schema" (`output_schema`'s `BinaryOp`/root arms), vs. a bare
`Literal` in a scalar-sub-language position, which still has none
(`QueryExprError::ScalarHasNoRowSchema`).

- `query_expr.rs`: removes `PromqlScalar`, adds `PromqlScalarBridge`,
  and the `promql_scalar`/`as_promql_scalar` constructor/accessor every
  call site now uses in place of the old bare variant.
- `canonicalize.rs`, `resolve.rs`, `binder.rs`: the bridge's child is a
  genuine scalar-sub-language node now, routed through `resolve_expr`/
  `columns_referenced` like every other scalar position, instead of
  being an opaque `f64` leaf.
- `cse.rs`, `dag_export.rs`: minimal surgical updates (kept out of
  scope otherwise per the CSE work happening elsewhere) — the bridge
  is still treated as a single opaque unit, matching the old
  `PromqlScalar` leaf's treatment exactly.
- Both front ends' test suites, `devtools`, `integration-tests`: every
  call site updated; behavior preserved exactly (all existing tests
  pass unchanged).

Adds unit tests in `query_expr.rs` and `resolve.rs` pinning: the
bridge/bare-literal value equivalence, that the row-schema/no-row-
schema split now rides on the wrapper rather than the variant, and
that `BinaryOp`'s `vector_match` semantics are unaffected.

Instance 2 (`BinaryOp` vs `Compare`/`Arithmetic`) is scoped out as a
separate, larger change — tracked in #245.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@zzylol
zzylol merged commit 7b6aa55 into main Aug 23, 2026
3 checks passed
zzylol added a commit that referenced this pull request Aug 23, 2026
…dge rename

Rebase fixup: #246 (merged after this branch was forked) collapsed
QueryExpr::PromqlScalar(f64) into PromqlScalarBridge(Rc<QueryExpr>),
matching rebuild_children's existing treatment (never descended into,
interned as a single unit). dag_node_count's count_unique needed the
same update - it still referenced the removed PromqlScalar variant,
which fails to compile against current main.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Aug 23, 2026
…haring (#237, #223, #212) (#243)

* docs(asap-aware-mapping): decide rule-based vs. cost-based CSE sharing (#237)

Records the decision for issue #237: how CostModel's still-deferred stage-4
CSE-credit wiring (issue #223's landing plan) should decide whether a
detected, legality-gated common subexpression is actually worth sharing.

Decision: a hybrid, per #237's own suggestion — unconditional sharing below
a cheap-recompute threshold, a real cost comparison above it. Neither pure
Volcano/Cascades (this repo has no plan-enumeration/DP search anywhere; new
infrastructure disproportionate to a binary share-or-don't decision) nor
pure System R (ignores the real cost of continuously maintaining a shared
summary for a rarely-read or cheap-to-recompute subtree) fits on its own.
The hybrid also matches this crate's own existing pattern for the
structurally analogous sketch-vs-exact decision (rank_candidates/
size_params: a cheap built-in default, overridable by a deployment's real
cost knowledge) and is layering-forced, not just preferred: detection
(asap-types::pre_asap::cse) sits below this crate and cannot consult a
CostModel, so it must stay cost-agnostic (System R-style default), pushing
the cost-aware override downstream into this crate.

Also sketches the concrete shape for a future stage-4 PR: a new
CostModel::cse_share_decision trait method (default preserves today's
unconditional-share behavior byte for byte), a CseCandidate input carrying
the shared subtree/bound summary/consumer count, and the call site
(implement_workload_with's memo-hit branch in bind.rs).

Documentation only — no CSE-credit logic implemented, per #237's own scope
("not asking for an implementation yet"). Also updates cse.rs's landing-plan
note to point at this decision instead of just flagging stage 4 as
deferred.

Related to #223, #212.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(asap-aware-mapping): implement CSE stage 4 (Volcano/Cascades cost-based sharing)

Replaces the previous doc-comment-only decision for #237 with a real,
working implementation, per review feedback:

- The design discussion moves out of code comments into
  docs/cse-cost-model-decision.md.
- CostModel gains CseCandidate/ShareDecision/cse_share_decision, plus
  two overridable cost-estimation hooks (cse_recompute_cost,
  cse_shared_maintenance_cost) with concrete default heuristics. The
  default cse_share_decision body performs a genuine Volcano/Cascades-
  style cost comparison — share iff the estimated cost of maintaining
  one shared summary is no greater than the estimated total cost of
  recomputing it independently at every consumer — rather than a fixed
  System R-style rule.
- bind::implement_workload_with now computes each shared subtree's true
  consumer_count via a whole-workload pre-pass, decides ShareDecision
  once per shared subtree (from full knowledge of the workload's
  sharing structure), and applies it consistently to every occurrence.

This closes out #223's stage 4 and #212's original "add CSE" tracking
issue. Stage 3 (structural-hash unification) landed separately in #244.

New tests: cost_model.rs unit tests pinning the default cost
comparison's behavior in both directions and confirming custom
CostModel overrides are actually consulted; a bind.rs test confirming
a CostModel that declines sharing produces genuinely independent
SummaryNodes even for two roots that are the same Rc<QueryExpr>. All
pre-existing CSE tests (including the DefaultCostModel-sharing
integration tests) still pass unchanged — the default cost weights
were calibrated against a real lowered query so today's sharing
behavior for realistic subtrees is preserved.

Verified: cargo build --workspace --all-targets, cargo test --workspace,
cargo fmt --all -- --check, cargo clippy --workspace --all-targets
--all-features -- -D warnings, cargo doc --workspace --no-deps (no new
broken-link warnings).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(asap-aware-mapping): CSE recompute-cost proxy must be DAG-aware, not tree-shaped

default_cse_recompute_cost measured a subtree's cost via
serde_json::to_string(subtree).len() — but a CseCandidate's subtree, by
definition, is something CSE already found sharing in, so it's
generally a DAG, not a tree. Rc's Serialize impl serializes the
pointee's value at every occurrence, not once by identity, so a naive
full serialization re-counts any descendant the subtree already shares
internally (e.g. from single-query CSE, x op x collapsing both
branches onto one Rc) once per parent that references it — silently
inflating the cost estimate relative to the DAG's real size.

Adds asap_types::pre_asap::cse::dag_node_count(root) -> usize: counts
unique nodes reachable from root, deduplicated by Rc pointer identity,
using the same operator-child traversal scope share_common_subtrees
itself uses (mirrors rebuild_children's field enumeration, as a
separate read-only traversal since rebuild_children consumes and
rebuilds its input). default_cse_recompute_cost now calls this instead
of serde_json.

Recalibrates default_cse_shared_maintenance_cost's UNIT constant
(60.0 -> 1.0) to match the new magnitude: node counts are small
integers (typically 1-20), not byte lengths (typically 200-500) — the
weight table's relative ordering (ExactAggregate cheapest, StatModel
priciest) is unchanged, only the absolute scale. Verified this
preserves existing default-sharing behavior: all pre-existing CSE
tests, including crates/integration-tests/tests/cse.rs's
DefaultCostModel-sharing assertions, pass unchanged.

New tests:
- cse.rs: dag_node_count is the naive count when nothing is shared;
  correctly dedupes an internally-shared subtree (BinaryOp with both
  branches pointing at the same Rc: 3 unique nodes, not 5); dedupes
  correctly across two workload roots sharing one subtree.
- cost_model.rs: default_recompute_cost_does_not_double_count_an_internally_shared_descendant
  — a Join with two independent Scan children costs 3, the same Join
  with both children pointing at one shared Scan costs 2, not 3 (the
  bug this fix addresses, demonstrated directly at the layer that
  consumes dag_node_count).

Verified: cargo build --workspace --all-targets, cargo test --workspace
(444 passed, 0 failed), cargo fmt --all -- --check, cargo clippy
--workspace --all-targets --all-features -- -D warnings, cargo doc
--workspace --no-deps (no new broken-link warnings).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(types): update dag_node_count for PromqlScalar -> PromqlScalarBridge rename

Rebase fixup: #246 (merged after this branch was forked) collapsed
QueryExpr::PromqlScalar(f64) into PromqlScalarBridge(Rc<QueryExpr>),
matching rebuild_children's existing treatment (never descended into,
interned as a single unit). dag_node_count's count_unique needed the
same update - it still referenced the removed PromqlScalar variant,
which fails to compile against current main.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Aug 23, 2026
…rBridge rename

#246 (merged after this branch was forked) renamed
QueryExpr::PromqlScalar(f64) to PromqlScalarBridge(Rc<QueryExpr>).
walk_rc_children and for_each_operator_child still referenced the
removed variant.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant