feat(sql): expand GROUPING SETS / ROLLUP / CUBE into merged per-level aggregates (#118)

[zzylol]

Closes #118. Takes option 1 from the issue — expand to a Merge of one Aggregate per grouping level — which needs no IR change and reuses the Merge path #109 opened.

Shape

SELECT service, SUM(bytes) FROM metrics GROUP BY ROLLUP(service)

Merge {
  Project { service := service,          sum := sum,  Aggregate{by:[service]} }
  Project { service := CAST(NULL, Utf8), sum := sum,  Aggregate{by:[]}        }
}

Three things that were not obvious

A level that omits a key must still emit it, as NULL. So each level's Aggregate is wrapped in a Project that reinstates the omitted keys and restores the canonical column order. Without it the levels are not union-compatible — and Merge derives its schema from the first child, so every other branch would be silently misdescribed. Same trap as the histogram_quantiles branches in #121.

The nulls have to be cast. infer_expr_type maps a bare L3Scalar::Null to Float64, so the () level of ROLLUP(service) would have emitted service: Float64. They are cast to the key's declared type instead, and a test asserts Utf8.

__grouping_id is safe to drop. DataFusion always adds it to the aggregate schema, but the enclosing Projection never references it. Its only purpose is to distinguish a subtotal's NULL from a data NULL, which is observable solely through GROUPING(col) — and GROUPING is an aggregate this front end rejects (unsupported aggregate: grouping). grouping_function_is_rejected pins that, since it is precisely what makes dropping the column sound.

Also verified

• DataFusion normalizes every mixed form (GROUP BY g, ROLLUP(d)) into a single GroupingSets, so group_expr is always exactly one grouping expression. One shape to handle, not three.
• distinct_expr() is ordered like the aggregate's leading schema fields, which is the column order the enclosing Projection expects.
• Composes with SQL: expression GROUP BY (date_trunc time bucketing), aggregates over expressions, and GROUPING SETS/ROLLUP/CUBE are rejected #110: SUM(bytes * 8) materializes in a derived-column Project beneath every level's Aggregate.
• WHERE still folds onto the Scan; HAVING / ORDER BY / LIMIT sit above the Merge and work.
• A non-column key inside a grouping set (ROLLUP(date_trunc(...))) is rejected — it would also need to be reinstatable as a typed null — with a message that says so rather than the generic SQL: expression GROUP BY (date_trunc time bucketing), aggregates over expressions, and GROUPING SETS/ROLLUP/CUBE are rejected #110 one.

Cost

The scan is duplicated per level; plan::cse hoists it back into a single producer, the same trade histogram_quantiles makes. CUBE(n) is 2^n levels by definition, so the IR tree grows with it — no artificial cap added, since DataFusion builds the same power set.

Tests (8 new, 1 rewritten)

multi_level_grouping_is_rejected (added by #110) is replaced by multi_level_grouping_lowers_to_one_aggregate_per_level. The rest pin prefix expansion, the power set, mixed-form normalization, typed nulls, union compatibility, the GROUPING() rejection, the expression-key rejection, and composition with #110.

Verification

cargo test --workspace      # 387 passed, 0 failed
cargo clippy --all-targets  # clean
cargo fmt --all --check     # clean

🤖 Generated with Claude Code