Do we need AggIntent::TopK?

[milindsrivastava1997]

This is based on the assumption that L3 IR should only have one representation possible for a given query.

TopK can also be represented as order by + limit. In that case, do we need AggIntent::TopK?

[zzylol]

Resolution: keep AggIntent::TopK — it isn't redundant with Sort + Limit. Closing as working-as-designed.

The premise ("topk = order by + limit, so it's a second representation") doesn't hold, for two reasons:

1. They're different operators, not two spellings of one.

• AggIntent::TopK { k, accuracy } is an intent carrying an accuracy target → it maps to a single-pass heavy-hitter sketch (SpaceSaving / CMS-with-heap). It never materialises-then-sorts.
• Sort + Limit is exact: materialise all rows, sort, cut. No accuracy, no sketch affordance.

A sketch top-k does not decompose into sort+limit, so collapsing them would erase the only signal L4 has to bind a heavy-hitter sketch.

2. The lowering is deterministic and language-agnostic — one canonical form per query. Each query maps to exactly one shape; you never get to "pick":

• frequency-ranked, descending (topk(k, count_over_time(…)) / ORDER BY COUNT(*) DESC LIMIT k) → AggIntent::TopK
• everything else (bottomk, topk over avg/quantile, ORDER BY name LIMIT 10, ranking by a group key) → Sort + Limit

Verified on both paths: PromQL promql.rs (heavy_hitter check) and SQL sql/mod.rs (heavy_hitter_topk → lower_as_topk), the latter pinned by sql_lowering.rs::count_ranked_topk_is_heavy_hitter. So "one canonical form per query" — the assumption this issue is built on — is upheld; TopK is the canonical form for the heavy-hitter shape.

This is the design's explicit exception to "one canonical form per plan" (design.md §6.2): the rule forbids variants that decompose into others — TopK doesn't, because the sketch primitive is one physical operation.

Note (by design): the Sort+Limit → TopK fold is partial — only the sketchable shape (DESC by COUNT) folds; generic/exact rankings legitimately remain Sort + Limit because no heavy-hitter sketch can serve them. So a "no order-by+limit survives in L3" invariant would be too strong; the correct (narrow) invariant is "no DESC-by-COUNT Sort+Limit survives un-folded", which the existing tests already pin.

This dovetails with #12: grouping is now unified under one GroupKeys type (PR #18), and TopK's grouping rides on Aggregate.by (heavy-hitter) while generic topk/bottomk grouping rides on Sort.partition_by — both GroupKeys. So every grouping concept has one spelling, and TopK vs Sort+Limit stays a clean intent-vs-exact-operator distinction.

Conclusion: keep TopK. No code change needed — canonicalization + tests already exist on both language paths.