Skip to content

SQL: expression GROUP BY (date_trunc time bucketing), aggregates over expressions, and GROUPING SETS/ROLLUP/CUBE are rejected #110

Description

@zzylol

Summary

The SQL lowerer requires bare column references in both GROUP BY keys and aggregate arguments. Any expression in either position is rejected. This blocks the single most common shape in time-series and netflow SQL: time bucketing.

Not caught by the existing corpus tests — netflow.rs and synthetic_packet_trace.rs both pass, because neither corpus contains an expression in a grouping or aggregate position.

Reproduction

Against the netflow_table catalog from crates/frontend-sql/tests/netflow/netflow.rs:

-- 1. time bucketing: expression GROUP BY
SELECT date_trunc('minute', time) AS m, SUM(pkt_len)
FROM netflow_table GROUP BY m;
-- unsupported feature: non-column GROUP BY expression: date_trunc(Utf8("minute"), netflow_table.time)

-- 2. aggregate over an expression (unit conversion — bytes to bits)
SELECT SUM(pkt_len * 8) FROM netflow_table;
-- unsupported aggregate: sum over a non-column expression

-- 3. multi-level aggregation
SELECT srcip, dstip, SUM(pkt_len) FROM netflow_table GROUP BY GROUPING SETS ((srcip),(dstip));
SELECT srcip, dstip, SUM(pkt_len) FROM netflow_table GROUP BY ROLLUP(srcip, dstip);
SELECT srcip, dstip, SUM(pkt_len) FROM netflow_table GROUP BY CUBE(srcip, dstip);
-- unsupported feature: non-column GROUP BY expression: GROUPING SETS (...) / ROLLUP (...) / CUBE (...)

Location

  • crates/frontend-sql/src/sql/mod.rs:528non-column GROUP BY expression: {other}
  • crates/frontend-sql/src/sql/mod.rs:510{name} over a non-column expression

Both reject anything that is not a plain Expr::Column after DataFusion planning.

Why this matters

GROUP BY date_trunc(...) / time_bucket(...) is the canonical windowed-aggregation idiom — the SQL equivalent of PromQL's range selector, which we already lower. Its absence means the SQL front end cannot express the query family the sketch layer exists to accelerate. GroupKeys is positional (GroupKeys::by(vec![usize])), so a grouping expression has no slot to live in; this likely needs a projected/derived grouping column, or a Relabel/Project inserted beneath the Aggregate.

GROUPING SETS/ROLLUP/CUBE land in the same rejection because DataFusion models them as a grouping expression, but they are semantically distinct (they emit multiple grouping levels in one pass) and could reasonably be split into their own issue if the expression-GROUP BY work lands first.

Notes

  • SUM(pkt_len * 8) and date_trunc bucketing are independent of each other; the aggregate-argument fix is the smaller of the two.
  • Suggest extending crates/frontend-sql/tests/netflow/data/netflow.sql with a date_trunc bucketing query once supported — its absence is why this went unnoticed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestsqlSQL front-end lowering (DataFusion → L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions