Skip to content

Add a shared authoritative SummaryCatalog - #536

Merged
zzylol merged 5 commits into
mainfrom
feat/shared-summary-catalog
Sep 9, 2026
Merged

zzylol merged 5 commits into
mainfrom
feat/shared-summary-catalog

Conversation

@zzylol

@zzylol zzylol commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

SDS descriptors were backend-local runtime types, so the control plane, Collector, and backend could not reference one authoritative materialization definition. This change moves the shared SDS descriptor contract into asap_types and makes physical compilation produce a versioned SummaryCatalog.

The catalog contains deterministic Summary Descriptor and Data Descriptor tables plus materialization records that reference both IDs. It validates schema versions, canonical content-derived IDs, foreign keys, duplicate conflicts, and materialization identity. PhysicalPlan carries the catalog alongside the execution plans while BackendPlan remains temporarily available for the next migration step.

Fidelity is algorithm-specific and parameter-bound: exact summaries declare exactness; KLL, DDSketch, HLL, CMS, and CountSketch carry their relevant configuration and a versioned guarantee model. Complete configured operator identity preserves heap size, Hydra tiling, subtype, and other state-affecting parameters. Legacy data-plane metadata is explicitly marked as partial and cannot collide with authoritative configured identities.

Validation:

  • asap_types: 52 tests passed, including 9 shared SDS tests.
  • control_plane: 700 library tests passed in CI, plus integration suites.
  • data-plane SDS registry tests: 4 passed.
  • data-plane SDS persistence tests: 8 passed.
  • MVP CI passed.

This implements the shared descriptor portion and control-plane catalog construction. Follow-up stacked PRs make PrecomputePlan, QueryPlan, CollectorPlan, and TransmissionPlan consume catalog references before removing BackendPlan. The catalog wire type itself will move to asap_types during the instance/inventory lifecycle consolidation so the data plane does not depend on a control-plane-owned contract.

@zzylol
zzylol merged commit 7082a5c into main Sep 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant