Add a shared authoritative SummaryCatalog - #536
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SDS descriptors were backend-local runtime types, so the control plane, Collector, and backend could not reference one authoritative materialization definition. This change moves the shared SDS descriptor contract into
asap_typesand makes physical compilation produce a versionedSummaryCatalog.The catalog contains deterministic Summary Descriptor and Data Descriptor tables plus materialization records that reference both IDs. It validates schema versions, canonical content-derived IDs, foreign keys, duplicate conflicts, and materialization identity.
PhysicalPlancarries the catalog alongside the execution plans whileBackendPlanremains temporarily available for the next migration step.Fidelity is algorithm-specific and parameter-bound: exact summaries declare exactness; KLL, DDSketch, HLL, CMS, and CountSketch carry their relevant configuration and a versioned guarantee model. Complete configured operator identity preserves heap size, Hydra tiling, subtype, and other state-affecting parameters. Legacy data-plane metadata is explicitly marked as partial and cannot collide with authoritative configured identities.
Validation:
asap_types: 52 tests passed, including 9 shared SDS tests.control_plane: 700 library tests passed in CI, plus integration suites.This implements the shared descriptor portion and control-plane catalog construction. Follow-up stacked PRs make PrecomputePlan, QueryPlan, CollectorPlan, and TransmissionPlan consume catalog references before removing BackendPlan. The catalog wire type itself will move to
asap_typesduring the instance/inventory lifecycle consolidation so the data plane does not depend on a control-plane-owned contract.