Make completed summary windows immutable after durable flush - #644
Merged
Merged
Conversation
…' into feat/maintenance-completion
zzylol
marked this pull request as ready for review
September 11, 2026 13:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completed inner windows must remain unchanged before maintenance can safely feed them into an outer sketch. Closing the Remote Write receiver did not prevent other producers from changing the same stored window, and a persisted completion marker without durable payload would prevent repair after restart.
This change enforces a per-physical-SeriesId completed-window boundary in SummaryStore. Completion holds the append fence, verifies admitted work is published, requests the existing flusher to persist the completed prefix, and only then writes the monotone boundary to the existing metadata sidecar. Future windows remain writable. Remote Write drain waits for this durability barrier; OTLP and precompute publication propagate rejected late writes.
Before this PR: a window could be called complete, then changed by a late sketch or exact-state append; restarting did not preserve a write-admission boundary.
After this PR: a completed
[0,30000)window rejects late writes both before and after restart, while[30000,60000)can still accept state. Stale metadata flushes cannot reopen it. Failed completion persistence leaves the live revision and boundary unchanged.Verification: 303 storage tests passed after integration with current main and #645, including real payload/manifest flush with no ordinary hot-tier deadline, shutdown/restart, late sketch/exact rejection, failed metadata persistence, stale metadata flush, and future-window isolation. All nine real production-process compatibility tests passed (22.77s), including persisted-state restart and Remote Write execution. Performance measurements are not applicable.
This is the immutable input boundary for maintenance, not a complete summary-over-summary executor. Durable atomic publication of derived outputs, continuous producer watermarks, and physical-lifetime metadata reclamation remain follow-ups. In-memory deployments have no restart guarantee. Includes independently reviewed #645; merge that dependency first.