Rotate retired physical series on catalog reactivation - #640
Merged
Merged
Conversation
zzylol
marked this pull request as ready for review
September 11, 2026 04:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A retired summary must be able to reappear in a later catalog without reusing its old physical samples. Permanent physical tombstones alone prevent that lifecycle; resolving delayed outputs against the latest catalog can instead contaminate a replacement.
This change keeps the logical definition/group stable and allocates a fresh physical
SeriesIdonly when a different installed catalog generation authorizes reactivation. The resolver persists the rotation before publishing its cache entry. Worker inputs and outputs carry the generation and physical series captured during routing; a delayed output cannot resolve itself into the replacement. Remote Write, configured OTLP, and backfill use the same fallible resolver.Before: a removed series could not safely be activated again, and resolver persistence failures could leave an in-memory identity available to catalog-bound producers.
After: catalog v1's removed physical series remains tombstoned; catalog v2 can publish the same logical summary under a new physical series. An old v1 output is rejected, and restarting restores the new resolver mapping without including the old series' disk payload. Corrupt complete WAL records fail closed; only incomplete tails may be discarded.
Verification: on the submitted source tree, 65 ingest tests, 325 precompute tests, and 115 backfill tests passed. These include strict persistence failures, corrupt WAL rejection, and stale physical output rejection before and after replacement. All eight production-process compatibility tests also passed (21.82s), including Remote Write warm/fallback and persisted-state restart. A real disk restart regression also passed: the replacement retains only its new payload while the original physical series stays absent. Performance measurements are not applicable.
Builds on merged #633. Independent source review passed; final CI is pending. The final main merge has an identical tree to the locally tested and previously CI-green revision. Legacy records without catalog provenance remain unbound. Cross-generation reuse of arbitrary existing summary state and physical tombstone reclamation are outside this change. Old queued inputs without a first published storage instance are conservatively rejected after a catalog switch; retained registered series may drain their birth generation or accept the current generation.