Conversation
Combines the wire push and endpoint pieces of the BackendPlan cutover (control_plane/docs/design-backend-plan-wire-format.md): control_plane side: - BackendClient::post_backend_plan_typed posts encoded BackendPlan bytes to POST /api/v1/backend-plan (sibling endpoint derived from the configured streaming-config URL, same convention as storage_routing). - emit::backend_push::push_cumulative_entries now also builds a BackendPlan from the same cumulative_be snapshot used for the legacy streaming-config/storage-routing documents (via backend_plan::from_stage_config from the prior PR) and fires a best-effort, single-attempt push alongside the existing coupled push. This is the real plan-time call site (Phase 3) -- the only place that builds BackendStageConfig for a live push. - The plan push's outcome never affects PushOutcome, which existing callers key real (legacy-path) behavior on -- nothing consumes BackendPlan yet, so its failure must stay invisible to them. No in-function retry loop (unlike the coupled documents): the next replan cycle is the retry backstop, matching push_or_log's existing fire-and-forget contract for the legacy YAML path. data_plane side: - HotReloadBackendPlan (storage_engines/types/hot_reload_config.rs): same ArcSwap-snapshot/swap shape as HotReloadStreamingConfig, applied to control_plane::backend_plan::BackendPlan. - GET/POST /api/v1/backend-plan handlers, registered alongside (not replacing) /api/v1/streaming-config. POST decodes protobuf bytes and atomically swaps; does not touch the sid catalog or SketchStore reconciliation. - main.rs installs an empty handle at startup so the endpoints don't 503 before the control plane's first push lands, mirroring the existing backend-storage-routing bootstrap pattern. Nothing consumes the installed BackendPlan yet -- that's Phase 4 (serving-time cutover), which also retires ObservedFamilyCostModel's SketchStore-reconstruction to a fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Closed
5 tasks
Replace forward references to "Phase 4"/"nothing consumes this yet" (stale now that the serving-time consumer exists) with descriptions of the actual current architecture: BackendPlan is read by ASAPQueryEngine's serving-time lookup, sits alongside (not in place of) the legacy streaming-config path, and a dropped push just falls back to SketchStore reconstruction until the next replan cycle. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #437 (Phase 1). Phases 2+3 of the BackendPlan cutover (
control_plane/docs/design-backend-plan-wire-format.md): the control plane now actually pushes an encodedBackendPlanto a newPOST /api/v1/backend-planendpoint, alongside (not instead of) the legacystreaming-config/storage_routingdocuments. Nothing consumes the installed plan yet — that's Phase 4, tracked separately.BackendClient::post_backend_plan_typed— new typed POST method, same transient/permanent classification as the existing typed methods, hitting a sibling endpoint derived from the configured streaming-config URL.emit::backend_push::push_cumulative_entries(the one real call site that buildsBackendStageConfigfor a live push) now also builds aBackendPlanviabackend_plan::from_stage_config(backend-plan: construct BackendPlan from BackendStageConfig (Phase 1) #437) from the samecumulative_besnapshot, and fires a best-effort, single-attempt push. Its outcome never affectsPushOutcome— nothing depends on it succeeding yet, and a failure must stay invisible to real callers.data_plane:HotReloadBackendPlan(sameArcSwapshape asHotReloadStreamingConfig) +GET/POST /api/v1/backend-planhandlers, wired intomain.rswith an empty bootstrap handle (mirrors the existing storage-routing bootstrap pattern).Test plan
cargo test --release -p control_plane --lib backend_client:: backend_push::— including a dedicated test proving the plan push fires alongside the coupled push, and a test proving a plan-push failure does NOT flipPushOutcome.cargo test --release -p data_plane --lib— 912/912 pass (2 pre-existing ignores), including newHotReloadBackendPlanround-trip + 503 + bad-bytes tests.cargo build --release --workspace— clean.e2e_controller_plans_and_backend_servesfailures (*_topkcases) reproduce identically onmain— not a regression from this change.