Skip to content

feat(ir): shared L3 canonicalization pass for heavy-hitter topk (#34) - #91

Merged
zzylol merged 1 commit into
mainfrom
feat/34-l3-canonicalization
Jul 4, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/34-l3-canonicalization

Conversation

@zzylol

@zzylol zzylol commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Closes #34. Fixes #20 (SQL alias gate miss) and #25 (SQL/PromQL heavy-hitter tree divergence) as a side effect.

Problem

Semantically identical "top-k by count" queries produced different L3 depending on source language, and each front end had its own heavy-hitter recognition gate with subtly different logic:

Fix: one shared pass

asap_l2::canonicalize — a bottom-up L3 rewrite run at the end of the shared convert_root, so both front ends (and any future language) get it for free. It promotes a count-ranked

Limit { Sort DESC-by-count { [Project] Aggregate([Count], by: G) } }

into the canonical

Aggregate { by: <sort partition>, aggs: [TopK{k}],
            child: Aggregate { by: G, aggs: [Count], … } }

The match is positional — the DESC key must resolve (through an optional passthrough projection) to the Count's output column — so it doesn't care whether the count was aliased. That kills #20 without any SQL-plan-specific alias handling.

The SQL front-end gate is removed (heavy_hitter_topk / lower_as_topk + helpers): SQL now emits a plain Sort+Limit and lets canonicalize do the promotion — the same path PromQL's generic topk takes. PromQL's topk(k, count_over_time(…)) gate still emits the canonical shape directly (canonicalize is idempotent on it).

Result: SQL and PromQL heavy-hitters are now structurally identical above the leaf — TopK over an explicit Count — differing only in what they count (SQL rows vs. windowed samples), which is the genuinely intentional part of #25. The outer TopK.by is the ranking partition ([] for a global ORDER BY … LIMIT k); the grouping lives on the inner Count.

Tests

Verified via the topk_ir example: S1≡S2 now, S3 (OFFSET) / S4 (SUM) / S5 (AVG) stay Sort+Limit, P1/P10 unchanged.

Full workspace suite green; clippy clean.

Not in scope

min_of/max_of etc. are unrelated. Partitioned SQL topk (S8/S9, #24) now lowers on its own (derived-table support landed since #24 was filed) and is untouched here.

Semantically equivalent SQL and PromQL queries produced structurally
different L3, and the heavy-hitter gate was duplicated across both front
ends with divergent recognition logic. This adds a single post-lowering
canonicalization pass both languages run through.

- `asap_l2::canonicalize` — a bottom-up L3 rewrite, run at the end of the
  shared `convert_root` so both front ends get it for free. It promotes a
  count-ranked `Limit{Sort{[Project] Aggregate([Count])}}` into the canonical
  heavy-hitter shape: an outer `Aggregate([TopK{k}])` (grouped by the sort's
  partition) over the *explicit* inner `Aggregate([Count])` — the shape
  PromQL's `topk(k, count_over_time(...))` already produced. The match is
  positional (the DESC key must land on the Count's output column), so it is
  oblivious to whether the count was aliased.

- Remove the SQL front-end heavy-hitter gate (`heavy_hitter_topk` /
  `lower_as_topk` and their helpers). SQL now emits a plain Sort+Limit and
  lets canonicalize recognise the count-ranked shape. This fixes the alias
  blind spot (#20 — `ORDER BY cnt DESC` now promotes like `ORDER BY COUNT(*)`)
  and closes the SQL/PromQL structural gap (#25 — SQL's implicit-count
  `TopK{Scan}` is gone; both languages now emit `TopK` over an explicit
  `Count`). The outer `TopK.by` is the ranking partition ([] for a global
  `ORDER BY … LIMIT k`), with the grouping on the inner `Count`.

Tests:
- `asap-l2` unit tests for the pass: promotion (with/without a passthrough
  projection), idempotency, and the negative cases (ascending, OFFSET,
  ranking a group key, non-Count aggregate).
- `crates/lower/tests/cross_language.rs` — the executable spec: SQL S2 and
  PromQL P1 reach the same canonical shape; aliased ≡ inline (#20);
  non-count / OFFSET stay generic in both languages.
- Updated the SQL `count_ranked_topk_is_heavy_hitter` expectation to the
  canonical two-level form.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@zzylol
zzylol merged commit 673c483 into main Jul 4, 2026
1 check passed
@zzylol
zzylol deleted the feat/34-l3-canonicalization branch July 4, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

L3 IR standardization: cross-language correctness and canonicalization SQL heavy-hitter gate misses count-ranked topk when COUNT is aliased

1 participant