tracking(query-engine): audit top-k and CountMinSketchWithHeap behavior

[milindsrivastava1997]

What to build

Track the investigation and resolution of the top-k and CountMinSketchWithHeap audit across the planner, query engine, and common query/configuration code.

The audit identified 18 findings. Existing related issues already cover:

• Capability-matching fallback can't resolve self-keyed CountMinSketchWithHeap top-k #501 — self-keyed heap top-k capability fallback
• CountMinSketchWithHeap still costs at the flat stub — cms-heap wrapper exists but isn't wired up #651 and docs(optimizer): reconcile optimization and sketch-bench formulations #652 — optimizer parameter grids, cost modeling, and formulation/default reconciliation
• fix(query-engine): reject sum-weighted heap CMS for COUNT matching #666 and Reject aggregation parameters that silently fall back or change semantics #674 — heap weighting and invalid aggregation parameters
• Make planner sketch parameters explicit instead of silently using accuracy defaults #675 — planner defaults and top-k parameter validation
• Fail loudly when accumulator serialization fails #673 — accumulator serialization failures
• refactor(asap-types): model aggregation kinds and subtypes with typed enums #670 — typed aggregation kinds and subtypes

This tracking issue covers the remaining follow-up work and coordinates those existing tickets.

Acceptance criteria

• Every audit finding is mapped to an existing or follow-up issue.
• High-risk behavior mismatches have either a code fix or an explicit documented limitation.
• Each behavior change has regression coverage, including failure paths.
• Related issues are cross-referenced and closed only after their acceptance criteria are met.
• The editable audit checklist is kept current.

Blocked by

None - can start immediately.

References

• TOPK_CMS_HEAP_AUDIT.md
• Capability-matching fallback can't resolve self-keyed CountMinSketchWithHeap top-k #501
• CountMinSketchWithHeap still costs at the flat stub — cms-heap wrapper exists but isn't wired up #651
• docs(optimizer): reconcile optimization and sketch-bench formulations #652
• fix(query-engine): reject sum-weighted heap CMS for COUNT matching #666
• refactor(asap-types): model aggregation kinds and subtypes with typed enums #670
• Fail loudly when accumulator serialization fails #673
• Reject aggregation parameters that silently fall back or change semantics #674
• Make planner sketch parameters explicit instead of silently using accuracy defaults #675

[milindsrivastava1997]

Audit mapping

The 18 findings map to the following issues:

• Finding 1: Capability-matching fallback can't resolve self-keyed CountMinSketchWithHeap top-k #501
• Finding 2: fix(query-engine): reconcile CountMinSketchWithHeap merge semantics #680
• Finding 3: fix(sql-parser): preserve SQL top-k secondary ordering semantics #682
• Findings 4 and 13: fix(query-engine): preserve SUM semantics for heap top-k #681
• Findings 5 and 6: CountMinSketchWithHeap still costs at the flat stub — cms-heap wrapper exists but isn't wired up #651, docs(optimizer): reconcile optimization and sketch-bench formulations #652, and Make planner sketch parameters explicit instead of silently using accuracy defaults #675
• Finding 7: fix(query-engine): reject sum-weighted heap CMS for COUNT matching #666 and Reject aggregation parameters that silently fall back or change semantics #674
• Finding 8: Make planner sketch parameters explicit instead of silently using accuracy defaults #675
• Finding 9: fix(precompute): harden legacy CountMinSketch deserialization #683
• Findings 10 and 15: Fail loudly when accumulator serialization fails #673 and refactor(query-engine): remove obsolete heap top-k APIs and paths #685
• Finding 11: fix(query-engine): make heap top-k keys collision-safe #679
• Finding 12: fix(query-engine): account for CountMinSketchWithHeap memory usage #684
• Finding 14: refactor(asap-types): model aggregation kinds and subtypes with typed enums #670 and refactor(query-engine): remove obsolete heap top-k APIs and paths #685
• Findings 16 through 18: refactor(query-engine): remove obsolete heap top-k APIs and paths #685

Representative cross-crate failure:

planner: CountMinSketchWithHeap, sub_type=topk
fallback: requires paired SetAggregator/DeltaSetAggregator
result: no compatible aggregation

The seven new follow-up issues cover the findings not already tracked by the existing issues above.