feat(sketch-db): Phase 3a — §7 schema-timeline read API - #20
Merged
Merged
Conversation
Adds `SchemaRegistry::timeline_for_metric(metric, t1_ms, t2_ms)` returning the non-overlapping, time-ordered `(agg_id, clipped_range)` segments that cover the query range — the key primitive the query engine needs to stitch results across reconfigure boundaries instead of seeing a data cliff at the moment of the schema change (design doc §7). Each `TimelineSegment` carries the owning schema's `AggStatus` plus a `TimelineCoverage` enum (`Sketch` / `Purged`) so the query path can decide per-segment whether to read from the sketch store or fall back to the exact DB per §7.3. Ownership is computed on-demand from the registry state (start = created_at_ms, end = min(next.created_at_ms, retired_at_ms) or open-ended for the active schema) — no separate index to keep consistent with the lifecycle map. Gaps between a retired schema's expiry and the successor's creation correctly leave the gap unowned, producing zero segments there so the caller knows to fall back. Pure data-structure work; SimpleEngine is not yet wired to use this (Phase 3b, next PR). Phase 3b will add `combine_statistic()` for additive stats and `PartialResult` for non-combinable quantile/topk across schema boundaries. 10 new unit tests — single-active, two-segment reconfigure, retired-without-successor, expired-schema marking, range-outside, inverted range, ignores-other-metrics, many-schema non-overlap invariant, and the gap case. 547 lib tests total (up from 537); workspace clippy clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
zzylol
added a commit
that referenced
this pull request
Apr 20, 2026
…able stats (#48) Task #34 gap #1 of 3. The §7 schema-timeline primitive (`SchemaRegistry::timeline_for_metric`) and the cross-schema combiner (`engines::timeline_dispatch::combine_statistic`) landed in PRs #20 / #22 / #25, but `SimpleEngine::handle_query_promql` was still resolving a single `agg_id` via `resolve_agg_info_promql` and running the full query against it. A query whose time range spans a reconfigure boundary (old `agg_id` retired, new `agg_id` created) saw a data cliff for the pre-boundary slice. This PR wires the dispatcher: * New `SimpleEngine::try_handle_query_promql_via_timeline`: 1. Parse + pattern-match the query, extract metric name. 2. Build a probe `QueryExecutionContext` to read the resolved `[t1, t2]` + `Statistic`. 3. Call `timeline_for_query(metric, t1, t2)`. Bail out with `None` (fall-through to default single-agg path) if fewer than two segments, or if the statistic is non-combinable (quantile / topk / cardinality / rate / increase — those follow in PR B2 with a Partial HTTP response surface). 4. Per segment: reuse `build_query_execution_context_promql_for_agg_id` from PR #37 (the extracted forced-agg-id entry point), clip the store plan's `[start, end]` to the segment's bounds, execute, collect results. 5. Group by label-tuple and fold per-group per-segment scalars through `combine_statistic`. Emit the combined scalar as an `InstantVectorElement`. Purged segments or segments whose `agg_id` is no longer in the config go into `unresolved` so the combiner sees them. * `handle_query_promql` now tries the timeline path first; returns immediately on `Some`, falls through to the existing single-agg path on `None`. Zero behavior change when the timeline has 0–1 segments for the query's metric (the common case today). ## Scope Combinable stats only: Count / Sum / Min / Max. Non-combinable stats still take the single-agg path — PR B2 will surface `CombinedResult::Partial` on the HTTP response so users see `{covered, missing: [segments]}` explicitly instead of a silent data cliff. ## Validation - `cargo test -p query_engine_rust --lib` — 728 pass (baseline unchanged; the dispatcher stays dormant when tests only register one schema per metric). - `cargo clippy --all-targets -- -D warnings` — clean - `cargo fmt --all -- --check` — clean ## Follow-ups (explicit non-scope here) - **Integration test** seeding two agg_ids + cross-boundary Sum query. Requires the full `PrecomputeEngine` setup harness the existing e2e tests use; deferred as a dedicated PR so this one stays a focused dispatcher patch. - **PR B2**: Partial response surface for non-combinable stats on the HTTP adapter. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SchemaRegistry::timeline_for_metric(metric, t1_ms, t2_ms)returning the non-overlapping, time-ordered(agg_id, clipped_range)segments that cover the query range.TimelineSegmentcarries the owning schema'sAggStatusplus aTimelineCoverageenum (Sketch/Purged) so the query path can fall back to the exact DB per §7.3 when a segment's data has been purged.combine_statistic()/PartialResult.Why
This is the primitive the query engine needs so a query spanning a reconfigure boundary does not see a data cliff at the moment of the schema change. Design doc §7 describes the full model; this PR lands the read API so Phase 3b has something to dispatch against.
How ownership is computed
For each schema of a given metric, ordered by
created_at_ms:created_at_msmin(successor.created_at_ms, self.retired_at_ms), oru64::MAXif neither applies (open-ended active schema)Gaps between a retired schema's
retired_at_msand the successor'screated_at_mscorrectly produce zero segments — the caller's cue to fall back to the exact DB.Phase 3a scope constraints
created_at_msis wall-clock at observation time. After restart without on-disk schema persistence, timelines reflect only the post-restart history. Phase 2c closes that gap.Expiredones markedPurged. The caller inspectscoverageto decide.Test plan
stores::sketch_db::schema::tests: single active, two-segment reconfigure, retired-without-successor, expired-marking, range-outside, inverted range, other-metric filtering, many-schema non-overlap invariant, coverage gap (both straddling and fully-inside).cargo clippy --workspace --all-targets --tests -- -D warningsclean.cargo fmt -- --checkclean.🤖 Generated with Claude Code