Skip to content

refactor(legacy_expr): batch 2 — lift 10 A-variants into canonical L3 + typed Predicate - #136

Merged
zzylol merged 1 commit into
mainfrom
refactor/legacy-expr-batch-2-a-lifts
May 11, 2026
Merged

zzylol merged 1 commit into
mainfrom
refactor/legacy-expr-batch-2-a-lifts

Conversation

@zzylol

@zzylol zzylol commented May 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Additive lift of all 10 A-classified legacy variants into the canonical intent_algebra::query_expr::QueryExpr, plus the minimal typed Predicate IR for Filter.pred. Zero consumer-site redirects — those happen in subsequent C-reshape batches.

What landed

Canonical QueryExpr grew 5 → 15 variants

Pre-existing (PR #130) Added in this PR
Scan, Window, Aggregate, LetBinding, Ref Filter, Project, Partition, Distinct, Merge, Join, SetOp, Sort, Limit, BinaryOp

Naming: single-input variants use child: (canonical convention). Multi-input variants follow design.md §6 shapes verbatim.

Supporting types

ColumnRef, PartitionKeys, BinaryOpKind, JoinKind, SetOpKind, SortKey, VectorMatch{,Kind}, VectorGrouping, GroupSide, LiteralValue, ProjectItem — all lifted into canonical + re-exported from intent_algebra::mod.rs.

New typed Predicate

pub enum Predicate {
    Column(ColumnRef),
    Literal(LiteralValue),
    BinaryOp { op: BinaryOpKind, lhs: Box<Predicate>, rhs: Box<Predicate> },
    IsNull { expr: Box<Predicate>, negated: bool },
}

Plus Predicate::from_legacy_scalar(&legacy::ScalarExpr) -> Result<Predicate, QueryExprError>. Translates the 4 supported variants; returns QueryExprError::UnsupportedLegacyScalar(name) for the 4 E-deferred (FunctionCall / ScalarSubquery / InList / Between) per user decision.

Handles LiteralValue::Duration → Int(nanos) fold (canonical LiteralValue deliberately narrower).

Why zero consumer redirects in this PR

Agent's honest finding: every legacy-A-variant consumer (query_parser/, physical/{stage_split,planner,allocator}.rs, optimizer/engine.rs, legacy_lower.rs) simultaneously matches A-variants AND C-variants (Source, Aggregate, Window, SketchAgg, WindowedAgg, TopK, etc.) AND constructs ScalarExpr predicates with E-deferred variants. Migrating them mid-pipeline would require lifting C-variants and E-variants in the same PR — out of scope per the batch plan.

Consumers migrate one C-batch at a time:

Batch What
3a (next) AggFunc → AggIntent (medium)
3b TopK split into AggIntent::TopK vs Sort+Limit (medium)
3c WindowedAgg un-fusion — 20+ consumers; HIGH risk
3d SketchAgg → Aggregate@L3 + L4 binding reads canonical; HIGH risk
3e PromQL parser-side histogram_quantile → Quantile substitution + drop legacy HistogramQuantile/PromQLSubquery (low/mechanical)
13 Retire legacy_expr.rs + legacy_lower.rs

Catch-all arms in 6 canonical-side consumers

Files: optimizer/cost/mod.rs, physical/colored_dag/{allocator,emitter}.rs, sketch_algebra/lower.rs, warm_tier_analysis.rs. These previously matched canonical QueryExpr exhaustively over the 5 pre-existing variants; now they handle the 10 new ones conservatively (Edge stage / Logical wrap / child-walk / zero-cost) with TODO markers.

Serde caveat

Initially used #[serde(tag = "kind")] on ColumnRef/LiteralValue/PartitionKeys/Predicate, but serde rejects internally-tagged newtype variants containing primitives. Switched to externally-tagged #[serde(rename_all = "snake_case")]. Caught by a_variant_serde_roundtrip_filter.

Build + test

  • cargo build --release -p controller — clean
  • cargo build --release -p query_engine_rust — clean
  • cargo test -p controller --lib666 passed (was 655 after Batch 1; +11 new tests covering schema pass-through + Predicate translation + serde round-trip)
  • cargo test -p query_engine_rust --lib -- engines::warm_tier — 13/13 pass

Diff: 8 files, +794 / -8

🤖 Generated with Claude Code

… + typed Predicate

Additive lift of all 10 A-classified legacy variants into the canonical
`intent_algebra::query_expr::QueryExpr`, plus the minimal typed
`Predicate` IR for `Filter.pred` covering the 4 used-and-cleanly-shaped
ScalarExpr variants.

## What landed

### Canonical `QueryExpr` grew from 5 → 15 variants

Pre-existing (PR #130 / canonical): `Scan`, `Window`, `Aggregate`,
`LetBinding`, `Ref`.

New (this PR): `Filter`, `Project`, `Partition`, `Distinct`, `Merge`,
`Join`, `SetOp`, `Sort`, `Limit`, `BinaryOp`.

Naming convention: single-input variants use `child:` (matches existing
canonical `Window`/`Aggregate`/`LetBinding`). Multi-input variants
follow design.md §6 shapes exactly: `Merge { children }`,
`Join { kind, pred, left, right }`, `SetOp { kind, all, left, right }`,
`BinaryOp { op, lhs, rhs, vector_match }`.

### Supporting types lifted alongside

`ColumnRef`, `PartitionKeys`, `BinaryOpKind`, `JoinKind`, `SetOpKind`,
`SortKey`, `VectorMatch`/`VectorMatchKind`/`VectorGrouping`/`GroupSide`,
`LiteralValue`, `ProjectItem`. All re-exported from
`intent_algebra::mod.rs`.

### New typed `Predicate`

```rust
pub enum Predicate {
    Column(ColumnRef),
    Literal(LiteralValue),
    BinaryOp { op: BinaryOpKind, lhs: Box<Predicate>, rhs: Box<Predicate> },
    IsNull { expr: Box<Predicate>, negated: bool },
}
```

Plus `Predicate::from_legacy_scalar(&legacy::ScalarExpr) ->
Result<Predicate, QueryExprError>`. Translates the 4 supported
variants (Column / Literal / BinaryOp / IsNull); returns
`QueryExprError::UnsupportedLegacyScalar(name)` for the 4 E-deferred
(FunctionCall / ScalarSubquery / InList / Between). Per user decision,
those stay in legacy until a real consumer demands the typed shape.

Translation handles `LiteralValue::Duration → Int(nanos)` fold (canonical
LiteralValue is deliberately narrower).

### Consumer migration deferred to subsequent batches

**Zero consumer-site redirects in this PR** — and that's the right
call. Every legacy-A-variant consumer (`query_parser/`, `physical/
{stage_split,planner,allocator}.rs`, `optimizer/engine.rs`,
`legacy_lower.rs`) simultaneously matches A-variants AND C-variants
(`Source`, `Aggregate`, `Window`, `SketchAgg`, `WindowedAgg`, `TopK`,
`HistogramQuantile`, `PromQLSubquery`) AND constructs `ScalarExpr`
predicates with E-deferred variants. Migrating them mid-pipeline would
require lifting C-variants and E-variants in the same PR — explicit
out-of-scope per the batch plan.

The consumers migrate one C-batch at a time:
- Batch 3a (PR #11): `AggFunc → AggIntent`
- Batch 3b (PR #10): `TopK` split
- Batch 3c (PR #8): `WindowedAgg` un-fusion
- Batch 3d (PR #9): `SketchAgg → Aggregate@L3`
- Batch 3e (PR #12): PromQL parser `histogram_quantile → Quantile`
- Batch 13: retire `legacy_expr.rs` + `legacy_lower.rs`

### Catch-all arms added in 6 canonical-side consumers

Files: `optimizer/cost/mod.rs`, `physical/colored_dag/{allocator,emitter}.rs`,
`sketch_algebra/lower.rs`, `warm_tier_analysis.rs`. These previously
matched canonical `QueryExpr` exhaustively over the 5 pre-existing
variants; now they need conservative handling for the 10 new ones
(Edge stage / Logical wrap / child-walk / zero-cost) with TODO
markers for the follow-up reshape batches.

### Serde tag note

Initially gave `ColumnRef`/`LiteralValue`/`PartitionKeys`/`Predicate`
internally-tagged `#[serde(tag = "kind", rename_all = "snake_case")]`
attrs, but serde rejects internally-tagged newtype variants containing
primitives (e.g. `Named(String)`). Switched to externally-tagged
`#[serde(rename_all = "snake_case")]`. Caught by the round-trip test.

## Build + test

- `cargo build --release -p controller` — clean
- `cargo build --release -p query_engine_rust` — clean
- `cargo test -p controller --lib` — **666 passed** (was 655 after
  Batch 1; +11 from new typed-Predicate + schema tests)
- `cargo test -p query_engine_rust --lib -- engines::warm_tier` — 13/13

## Diff: 8 files, +794 / -8

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 16f78c5 into main May 11, 2026
@zzylol
zzylol deleted the refactor/legacy-expr-batch-2-a-lifts branch July 17, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant