feat(ir): canonicalize SQL ROW_NUMBER partitioned top-k (#24) - #93
Merged
Merged
Conversation
The "unsupported subquery" error #24 reported is already gone (derived-table + WindowFunc support landed), so S8/S9 lowered — but only to the raw `Filter(rn <= k) over WindowFunc(RowNumber PARTITION BY p ORDER BY o)` shape, structurally unlike the PromQL `topk by (…)` they mirror. Add a `canonicalize` rewrite (issue #34's pass) that recognises the `ROW_NUMBER()`-partitioned top-k idiom and rewrites it to the generic partitioned `Limit{k}{ Sort{ o, partition_by: p } }`. Because the pass runs its rules to a fixpoint, the existing heavy-hitter rule then promotes the count-ranked case to an `Aggregate([TopK])`, so: - S8 (ROW_NUMBER … ORDER BY COUNT(*) DESC) → `Aggregate{ by: [region], [TopK{k}], child: Aggregate([Count], by: [service, region]) }` — the same partitioned heavy-hitter shape as PromQL P10. - S9 (… ORDER BY AVG(latency) DESC) → `Limit{k}{ Sort{ partition_by: [region] } { Aggregate([Avg]) } }` — the generic partitioned form, mirroring PromQL P9. The match is conservative: it fires only when the filtered column is exactly the `ROW_NUMBER` window output (the appended last column, verified via the inner schema) ranked with `<= k`; any other predicate/column is left alone. Tests: - `asap-l2` unit tests: count case → partitioned heavy-hitter; avg case → partitioned Sort+Limit; a filter on a non-row-number column is untouched. - `crates/lower/tests/cross_language.rs`: end-to-end, S8 reaches the same heavy-hitter shape as PromQL P10 and S9 the same generic partitioned form as P9. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #24.
State of the issue
The "unsupported subquery (inline view / derived table)" error #24 originally reported is already gone — derived-table +
WindowFuncsupport landed since it was filed, so S8/S9 lower. What remained was the second half of the ask: recognition of theROW_NUMBER() OVER (PARTITION BY …)pattern. Without it, S8/S9 produced the rawFilter(rn ≤ k) over WindowFunc(RowNumber …)shape — structurally unlike the PromQLtopk by (…)they mirror.Fix
Add one rewrite to the shared
canonicalizepass (built in #34): recognise the ROW_NUMBER-partitioned top-k idiom and rewrite it to the generic partitionedLimit{k}{ Sort{ o, partition_by: p } }. The pass applies its rules to a fixpoint, so the existing heavy-hitter rule then composes on top and promotes the count-ranked case:ROW_NUMBER() … ORDER BY COUNT(*) DESC,WHERE rn ≤ 5Aggregate{ by:[region], [TopK{5}], child: Aggregate([Count], by:[service,region]) }topk by (…) (5, count_over_time(…))… ORDER BY AVG(latency) DESCLimit{5}{ Sort{ partition_by:[region] }{ Aggregate([Avg]) } }topk by (…) (5, rate(…))So a SQL
ROW_NUMBERtop-k and the PromQLtopk byit mirrors now converge on the same canonical IR.The match is conservative: it fires only when the filtered column is exactly the
ROW_NUMBERwindow output — the single column the WindowFunc appends, verified against the inner relation's schema — ranked with<= k. Any other predicate/column/window function is left untouched.Tests
asap-l2unit tests: count-ranked ROW_NUMBER → partitioned heavy-hitter; avg-ranked → partitionedSort+Limit; a filter on a non-row-number column is left as aFilter.crates/lower/tests/cross_language.rs: end-to-end vialower_sql— S8 reaches the same heavy-hitter shape as PromQL P10, S9 the same generic partitioned form as P9.Full workspace suite green; clippy clean.