feat(maintenance): activate complete canonical populations - #679
Merged
Merged
Conversation
…ped-rows # Conflicts: # data_plane/src/precompute_engine/maintenance_runtime.rs
…ntenance-complete-population-cohort
zzylol
marked this pull request as ready for review
September 11, 2026 20:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finite remote-write maintenance can now install an actual selected global summary over complete, matching populations from multiple raw sources. Previously the canonical population transport and transactional consumer existed, but control-plane installation still rejected their configs; distinct-group testing also exposed a DDS quantile readout that used a lower-rank convention instead of the typed PromQL interpolation contract.
The compiler marks the actual selected global target and raw frontiers canonical before computing fingerprints and derived-input identity. Shared installation validation restricts activation to backend-local remote write, matching canonical members, aligned stored full nonoverlapping windows and global reduction. The existing runtime requires complete matching group sets and publishes one aggregate transaction. No synthetic winner, raw substitute, grouping DTO or inferred watermark is introduced.
Typed DDS quantile readout uses the additive sketch-library interpolation API (ProjectASAP/asap_sketchlib#140); legacy library quantile behavior remains unchanged. Maintenance validates finite, positive and representable DDS inputs before update. KLL readout is unchanged.
Before: a selected
quantile(0.9, sum_over_time(m[1m]) + sum_over_time(n[1m]))over two matching populations could not use this installed canonical path, and lower-rank DDS readout failed the distinct-value oracle. After: supported complete populations produce one immutable global result; missing source/group coverage fails closed, retries do not add parts and same-generation restart preserves the result.Verification: final integrated three scoped compiler and installation tests passed; independent CP/shared, readout and merge reviews passed. Runtime evidence includes 28 summary-executor tests, the DDS domain regression, 14 maintenance tests and the real adapter's nonpositive/unrepresentable-population regression. Clean actual process at d930876 passed in 41.00s with actual selected DDS alpha 0.01, distinct group values 20/40 versus strict PromQL reference 38, missing-source/group rejection, repeated completion, restart and generation transition. The main-integrated 4263323 process also passed in 40.79s (build 2m03s); dependencies are clean Collector 1d8efd07 and sketch library 8c03d7c, with no Cargo overrides. Test and production binary/log hashes and retained artifacts are recorded in the local validation manifest. Subsequent clean main merge f8b6d45 adds only the independently tested #678 producer roster contract and its documentation; activation and readout behavior are unchanged.
Scope remains finite completion, full nonoverlapping windows, supported arithmetic without vector matching, raw source frontiers and global reduction. This is not continuous-watermark or sliding-window activation, a native-engine differential benchmark, or a performance claim. Dependency #140 must merge before this PR.