-
Notifications
You must be signed in to change notification settings - Fork 0
SummaryExecutor: permanent self-exclusions + upstream composition gaps leave some PromQL shapes always on the legacy/archive path #436
Copy link
Copy link
Open
Labels
area: data planeRuntime ingestion, storage, and query-serving data pathsRuntime ingestion, storage, and query-serving data pathsarea: query enginePromQL lowering, execution, result shaping, and fallback behaviorPromQL lowering, execution, result shaping, and fallback behaviorarea: summary executionWarm-summary binding, readout, composition, and result decodingWarm-summary binding, readout, composition, and result decodingenhancementNew feature or requestNew feature or requestfeature: PromQLPromQL compatibility and execution semanticsPromQL compatibility and execution semanticsfeature: warm summariesASAP warm-tier materialization, readiness, and servingASAP warm-tier materialization, readiness, and servingintegration: ASAPPlannerASAPPlanner API, IR, candidate, and selected-plan integrationASAPPlanner API, IR, candidate, and selected-plan integrationmeta: trackingUmbrella issue coordinating multiple implementation gapsUmbrella issue coordinating multiple implementation gaps
Description
Activity
Metadata
Metadata
Assignees
Labels
area: data planeRuntime ingestion, storage, and query-serving data pathsRuntime ingestion, storage, and query-serving data pathsarea: query enginePromQL lowering, execution, result shaping, and fallback behaviorPromQL lowering, execution, result shaping, and fallback behaviorarea: summary executionWarm-summary binding, readout, composition, and result decodingWarm-summary binding, readout, composition, and result decodingenhancementNew feature or requestNew feature or requestfeature: PromQLPromQL compatibility and execution semanticsPromQL compatibility and execution semanticsfeature: warm summariesASAP warm-tier materialization, readiness, and servingASAP warm-tier materialization, readiness, and servingintegration: ASAPPlannerASAPPlanner API, IR, candidate, and selected-plan integrationASAPPlanner API, IR, candidate, and selected-plan integrationmeta: trackingUmbrella issue coordinating multiple implementation gapsUmbrella issue coordinating multiple implementation gaps
Summary
Several PromQL query shapes can never be served from a summary/sketch via
SummaryExecutorand permanently fall back to the legacy dispatch / archive path — some by this repo's own deliberate design, some because of gaps upstream in ASAPController. This issue consolidates them in one place since they share a root cause ("can't be answered via summary today") but are currently scattered across separate issues/comments.This repo's own permanent self-exclusions
data_plane/src/query_engines/asap_query_engine/l4_lowering.rs'sLoweringSkipenum documents two shapeslower_promql_to_l4nodeexcludes before ever attempting to bind, by design — not bugs, not TODOs:LoweringSkip::RateShape(l4_lowering.rs:69-78) —rate()/irate()queries.lower.rs'sbind_recursiverewritesAggIntent::Rate→Increasebefore binding, which WOULD bind successfully to a validSummaryAgg{Increase}tree, butsummary_executor.rshas no rate-division logic (dividing by a coverage-clamped range), so comparing against it would produce a spurious mismatch. Must be excluded before ever calling intocontrol_plane's binder.LoweringSkip::NotRealized(l4_lowering.rs:88-98) — outer fold over an aggregate, e.g.topk(K, sum by(...)(rate(m[r]))).implement_tree_in_withonly recurses throughAggregatenodes, so hitting the outerSort/Limitwraps the WHOLE tree as one opaqueLogicalblob even though the inner aggregate would bind fine on its own.Neither has a workaround today; both always fall back to the legacy
SketchReducer/archive path.Related, already-filed issues (not duplicated here, just linked for context)
accuracy: Exact) has no warm-tier materialization, always falls to archive (partially blocked on ASAPController#151's missingexact_realizationaccumulator form for TopK).Why this issue exists separately
None of the above already frame "why does this query never get served from summary" as one topic across both this repo's own permanent exclusions and the upstream composition gaps. Confirmed via
gh issue listthat #348 is a different, narrower topic (activating the exact→archive routing gate), not this.Ask
Track this as the umbrella "queries that can't be summary-served" issue. Close (or don't) independently as upstream/local gaps are addressed — this issue is for visibility, not meant to gate other work.