backend-plan: serving-time cutover -- prefer plan over SketchStore reconstruction (Phase 4) - #441
Merged
Merged
Conversation
…ore reconstruction (Phase 4) Completes the BackendPlan wire-format cutover (control_plane/docs/design-backend-plan-wire-format.md). Serving time now reads planning's real family/params decision directly off an installed BackendPlan when one covers the query's metric, instead of always reconstructing it from SketchStore metadata. - l4_lowering.rs: new observed_family_for_metric_from_plan() reads Materialization.kind/.params directly (no AggregationConfig reconstruction needed -- they already are the (SummaryKind, SummaryParams) pair this needs). lower_promql_to_l4node() now takes an Option<&BackendPlan> and tries the plan first, falling back to the existing SketchStore-reconstruction path (observed_family_for_metric / ObservedFamilyCostModel) when no plan is installed or it doesn't cover the metric. Three new unit tests prove: (1) SketchStore reconstruction still works with no plan, (2) a plan materialization WINS over a disagreeing SketchStore registration for the same metric, (3) a plan that doesn't cover the metric falls through to SketchStore reconstruction rather than silently failing to observe anything. - l4_readout.rs / live_serve.rs: thread the same Option<&BackendPlan> parameter through unchanged otherwise. - engine.rs: ASAPQueryEngine gains a HotReloadBackendPlan handle (with_hot_reload_backend_plan) and a backend_plan_snapshot() helper; both live-serving call sites (range + instant query) pass the current snapshot through. - main.rs: the HotReloadBackendPlan handle is now shared between the query engine and the HTTP server (previously only the latter), so a POST /api/v1/backend-plan is observable by the next query, same sharing contract as hot_reload_config. - cost_model.rs: ObservedFamilyCostModel's doc updated to reflect its new fallback status. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Closed
5 tasks
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recreated after #439 got tangled in the base-branch chain when #438 was recreated as #440 (see #440's description). Same content, now rebased directly onto
main.Summary
Final phase of the BackendPlan cutover (
control_plane/docs/design-backend-plan-wire-format.md): serving time (l4_lowering.rs) now reads planning's real family/params decision directly off an installedBackendPlanwhen it covers the query's metric, instead of always reconstructing it fromSketchStoremetadata viaObservedFamilyCostModel.observed_family_for_metric_from_plan— readsMaterialization.kind/.paramsdirectly off the plan; noAggregationConfigreconstruction needed (they already are the pairObservedFamilyCostModelwants).lower_promql_to_l4nodetries the plan first, falls back to the existingSketchStore-reconstruction path when no plan is installed or it doesn't cover the metric — additive, not a replacement.ASAPQueryEnginegets aHotReloadBackendPlanhandle, shared with the HTTP server inmain.rs(same sharing contract ashot_reload_config), so aPOST /api/v1/backend-planis observable by the next query.ObservedFamilyCostModel's doc updated: fallback status, not dead code.Test plan
l4_lowering.rs::tests::backend_plan_cutover— the load-bearing one (a_plan_materialization_wins_over_sketchstore_reconstruction) registers a Kll sid inSketchStorebut installs a plan declaring DDSketch for the same metric, and asserts the boundL4Nodeuses DDSketch. Two companion tests confirm the no-plan and plan-doesn't-cover-this-metric fallback paths are unchanged.cargo test --release --lib -p control_plane— 732/732.cargo test --release --lib -p data_plane— 915/915 (2 pre-existing ignores).cargo build --release --workspace— clean.