feat(engine): serve from SummaryExecutor when it is provably safe

[zzylol]

Summary

Phase 2 of the SummaryExecutor rollout (Phase 1: shadow-mode comparison, #419, merges the new path's answer alongside the legacy one and logs a diff, never affecting what's served).

This PR adds the actual serving cutover, gated behind ASAP_SUMMARY_EXECUTOR_LIVE (default off): when a query lowers and executes cleanly through SummaryExecutor and isn't the known ASAPController#163 grouping-ambiguity shape, engine.rs now serves the answer directly from SummaryExecutor and skips the legacy SketchReducer call entirely for that candidate. Every other outcome falls back to the legacy path exactly as it does today — None from the new gate is indistinguishable from Phase 1's shadow-only behavior.

What "safe to serve" means

A query is served from the new path only when:

1. lower_promql_to_l4node succeeds (excludes rate()/irate(), unparseable queries, anything that didn't realize to a concrete binding).
2. asap_sketch::exec::execute() returns Ok(...).
3. The grouping-ambiguity gate: for a sketch-family (ExecOutcome::Value) outcome, if the tree's root SummaryAgg's by was empty AND more than one group came back, this is exactly the ambiguous shape ASAPController#163 describes — decline to serve, fall back to legacy (which resolves it today via its own bespoke special case).

Changes

• summary_executor.rs: GroupState::exact_coverage gives ExactAgg the same coverage story SummaryValue already has — needed so the live-serve path can report ASAPTierResult.coverage regardless of which family (sketch vs. exact) answered.
• l4_readout.rs (new): shared lowering + execution + conversion into ASAPTierResult's (series, coverage) shape, with the mechanical ambiguous_merge_risk gate. Both shadow_compare.rs and live_serve.rs now call this instead of duplicating the conversion logic (de-dupe, no behavior change to Phase 1's shadow logic).
• live_serve.rs (new): the actual cutover module — flag check (ASAP_SUMMARY_EXECUTOR_LIVE), ambiguity gate, Some(...) means "use this instead of the legacy reducer."
• engine.rs: wired into the range-query and instant-query dispatch loops at the point where the legacy reducer is called; skips the now-redundant apply_outer_agg_fold and shadow-mode comparison call when a candidate was already served live (there's no separate legacy answer left to diff against).

Explicitly NOT in this round

• Retiring sketch_reducer.rs — the ambiguous-by shapes, rate, and topk-over-rate families still need it.
• Resolving the grouping ambiguity itself (ASAPController#163) — waiting on the upstream conversation; this PR's ambiguity gate is a correct, conservative fallback, not a resolution.
• Flipping ASAP_SUMMARY_EXECUTOR_LIVE's default to on.

Test plan

• cargo build --workspace — clean.
• cargo test -p data_plane --lib — 955 passed, 0 failed (flag off, no behavior change).
• cargo test -p data_plane --test e2e_controller_plans_and_backend_serves — 12 passed / 2 known pre-existing failures (cms_with_heap_topk, count_sketch_with_heap_topk), identical result with ASAP_SUMMARY_EXECUTOR_LIVE=1 set.
• New e2e test live_serve_actually_answers_ddsketch_quantile — proves the flag-on path genuinely serves (not just shadow-compares) a correct DDSketch quantile answer.
• New e2e test live_serve_ambiguous_hll_global_count_falls_back_correctly — two distinct-service HLL sids, ungrouped count(), flag on — proves the ambiguity gate declines to serve and the legacy fallback still answers correctly.
• cargo clippy -p data_plane --lib — identical warning count to the base branch (no new warnings introduced).
• cargo fmt on touched files only.

🤖 Generated with Claude Code

[zzylol]

Rebased onto the updated #419 and removed the grouping-ambiguity gate entirely — it is now obsolete rather than merely simplifiable. (0f96520)

Why it can go

L4ReadoutOutcome::ambiguous_merge_risk existed purely as a workaround for ProjectASAP/ASAPPlanner#163: an empty by: Vec<ColumnId> was indistinguishable between "no grouping concept applies" (the group split is correct) and "an aggregation operator asked to reduce everything" (the groups should have merged). Unable to tell which, try_serve_from_summary_executor declined to serve any empty-by shape producing >1 group and fell back to the legacy path.

#165 made the reduction kind explicit, and #419 made resolve_group_key act on it, so both branches are already resolved correctly before the gate ran:

• PerEntity — the multi-group split is definitionally correct (one row per entity, never merged). Never a "risk"; here the gate could only ever decline a correct answer.
• Reduce([]) — every candidate shares one group key, so the outcome has exactly one group and the values.len() > 1 trigger cannot fire at all.

The flag would be unconditionally false today. Rewriting it against Reduction would preserve a heuristic whose only remaining effect is spurious fallback, so I removed it: the field, its computation, the root_summary_agg_by_is_empty tree walk feeding it, and the live_serve.rs early-return.

Tests inverted, not dropped

These covered exactly the behavior that changed, so each now asserts the opposite:

before	after
flag_on_ambiguous_shape_falls_back — asserted is_none()	flag_on_global_merge_shape_is_served_merged_not_declined — asserts it is served, as one merged series, cardinality ~6 not ~3
ambiguous_global_merge_shape_is_flagged — asserted the flag + two unmerged series	global_merge_shape_now_merges_instead_of_being_declined — asserts one merged series
e2e live_serve_ambiguous_hll_global_count_falls_back_correctly	live_serve_hll_global_count_merges_across_sids

The e2e assertion was also tightened: it previously accepted "any positive value," which a legacy fallback also satisfied — so it could not actually observe whether the new path served the query. It now requires >= 1.5, distinguishing a real cross-sid merge (~2) from serving only one sid's registers (~1).

Verification

• cargo check -p data_plane --all-targets — clean
• cargo test -p data_plane --lib — 956 passed / 0 failed

Note this PR now serves a shape it previously declined, so the live-serve blast radius is slightly larger than when it was first opened — that is the intended effect of the upstream fix (the shape is now provably correct rather than provably ambiguous), but worth a look during review given ASAP_SUMMARY_EXECUTOR_LIVE changes what is actually served.

Depends on #419.

---

🤖 Generated with Claude Code

[zzylol]

Rebased onto the updated #419 to pick up ProjectASAP/ASAPPlanner#170 (the sketch/sketch_input → summary/summary_input rename + Implementation::Summary merge — see #419's comment for the details).

No logic in this branch's own two commits touched a renamed field or the Implementation enum, so the rebase (c29d3b1, 5b6c639 — new SHAs from the rebase, content otherwise unchanged from before) required zero additional fixes beyond what landed in #419.

Verification

• cargo check -p control_plane --all-targets — clean
• cargo check -p data_plane --all-targets — clean
• cargo test -p data_plane --lib — 956 passed / 0 failed
• cargo test -p control_plane --lib — 766 passed / 1 pre-existing failure (same as feat(engine): shadow-mode SummaryExecutor comparison (no serving change) #419, unrelated)

Depends on #419.

---

🤖 Generated with Claude Code