Skip to content

Rotate retired physical series on catalog reactivation - #640

Merged
zzylol merged 10 commits into
mainfrom
feat/catalog-series-reactivation
Sep 11, 2026
Merged

zzylol merged 10 commits into
mainfrom
feat/catalog-series-reactivation

Conversation

@zzylol

@zzylol zzylol commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

A retired summary must be able to reappear in a later catalog without reusing its old physical samples. Permanent physical tombstones alone prevent that lifecycle; resolving delayed outputs against the latest catalog can instead contaminate a replacement.

This change keeps the logical definition/group stable and allocates a fresh physical SeriesId only when a different installed catalog generation authorizes reactivation. The resolver persists the rotation before publishing its cache entry. Worker inputs and outputs carry the generation and physical series captured during routing; a delayed output cannot resolve itself into the replacement. Remote Write, configured OTLP, and backfill use the same fallible resolver.

Before: a removed series could not safely be activated again, and resolver persistence failures could leave an in-memory identity available to catalog-bound producers.

After: catalog v1's removed physical series remains tombstoned; catalog v2 can publish the same logical summary under a new physical series. An old v1 output is rejected, and restarting restores the new resolver mapping without including the old series' disk payload. Corrupt complete WAL records fail closed; only incomplete tails may be discarded.

Verification: on the submitted source tree, 65 ingest tests, 325 precompute tests, and 115 backfill tests passed. These include strict persistence failures, corrupt WAL rejection, and stale physical output rejection before and after replacement. All eight production-process compatibility tests also passed (21.82s), including Remote Write warm/fallback and persisted-state restart. A real disk restart regression also passed: the replacement retains only its new payload while the original physical series stays absent. Performance measurements are not applicable.

Builds on merged #633. Independent source review passed; final CI is pending. The final main merge has an identical tree to the locally tested and previously CI-green revision. Legacy records without catalog provenance remain unbound. Cross-generation reuse of arbitrary existing summary state and physical tombstone reclamation are outside this change. Old queued inputs without a first published storage instance are conservatively rejected after a catalog switch; retained registered series may drain their birth generation or accept the current generation.

@zzylol
zzylol changed the base branch from fix/durable-summary-lifecycle to main September 11, 2026 04:01
@zzylol
zzylol marked this pull request as ready for review September 11, 2026 04:01
@zzylol
zzylol merged commit f00b562 into main Sep 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant