Summary
GROUP BY ROLLUP(…) / CUBE(…) / GROUPING SETS (…) are rejected. Split out of #110, whose expression-GROUP BY work landed without them.
Before #110 they failed with a misleading error — non-column GROUP BY expression: ROLLUP (…) — which suggested they were the same problem as GROUP BY date_trunc(…). They are not. They now fail with multi-level grouping: ROLLUP.
Why they are different
GROUP BY date_trunc('minute', ts) is one grouping level over a derived column, so materializing the expression beneath the aggregate is enough (that is what #110 does).
Multi-level grouping emits several grouping levels from one scan, plus a __grouping_id discriminator column that says which level produced each row. DataFusion models this as Expr::GroupingSet in group_expr and adds __grouping_id to the aggregate's output schema:
SELECT g, SUM(v) FROM t GROUP BY ROLLUP(g)
group_expr: [GroupingSet(Rollup([Column(t.g)]))]
agg schema: [g, __grouping_id, sum(t.v)]
QueryExpr::Aggregate.by is a single GroupKeys — one key set, one output row shape. There is nothing to lower the extra levels onto.
Options
- Expand to a
Merge of one Aggregate per grouping level, with a synthesized __grouping_id literal column per branch. Faithful and needs no new IR, but multiplies the scan unless CSE hoists it — plan::cse would need to do so for this to be worth anything.
- A
grouping_sets: Vec<GroupKeys> field on Aggregate. One node, one scan, but every consumer (plan::bind, plan::boundary, schema derivation) has to learn that the node emits several row shapes.
Option 1 is strictly easier and reuses machinery that exists; option 2 is what a sketch-aware executor would actually want, since one pass over the data can feed all levels.
Reproduction
crates/frontend-sql/tests/sql_lowering.rs::multi_level_grouping_is_rejected
pins the current rejection.
Notes
Summary
GROUP BY ROLLUP(…)/CUBE(…)/GROUPING SETS (…)are rejected. Split out of #110, whose expression-GROUP BYwork landed without them.Before #110 they failed with a misleading error —
non-column GROUP BY expression: ROLLUP (…)— which suggested they were the same problem asGROUP BY date_trunc(…). They are not. They now fail withmulti-level grouping: ROLLUP.Why they are different
GROUP BY date_trunc('minute', ts)is one grouping level over a derived column, so materializing the expression beneath the aggregate is enough (that is what #110 does).Multi-level grouping emits several grouping levels from one scan, plus a
__grouping_iddiscriminator column that says which level produced each row. DataFusion models this asExpr::GroupingSetingroup_exprand adds__grouping_idto the aggregate's output schema:QueryExpr::Aggregate.byis a singleGroupKeys— one key set, one output row shape. There is nothing to lower the extra levels onto.Options
Mergeof oneAggregateper grouping level, with a synthesized__grouping_idliteral column per branch. Faithful and needs no new IR, but multiplies the scan unless CSE hoists it —plan::csewould need to do so for this to be worth anything.grouping_sets: Vec<GroupKeys>field onAggregate. One node, one scan, but every consumer (plan::bind,plan::boundary, schema derivation) has to learn that the node emits several row shapes.Option 1 is strictly easier and reuses machinery that exists; option 2 is what a sketch-aware executor would actually want, since one pass over the data can feed all levels.
Reproduction
pins the current rejection.
Notes
lower_sqlwhile working on SQL: expression GROUP BY (date_trunc time bucketing), aggregates over expressions, and GROUPING SETS/ROLLUP/CUBE are rejected #110.crates/frontend-sql/src/sql/mod.rs, theas_grouping_setguard at the top oflower_aggregate.