Skip to content

feat(http): surface §7 Partial results on Prometheus warnings - #49

Merged
zzylol merged 1 commit into
mainfrom
feat/phase3-partial-http-surface
Apr 20, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/phase3-partial-http-surface

Conversation

@zzylol

@zzylol zzylol commented Apr 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Task #34 gap #2 of 3. #48's dispatcher activated per-segment execution for combinable stats (Count/Sum/Min/Max) but refused to run for non-combinable stats (Quantile/Topk/Cardinality/Rate/Increase) — those still fell through to the single-agg path and saw the same data cliff the dispatcher was written to prevent. Reason: there was no way to tell the HTTP caller "this answer is Partial because the query spans a reconfigure boundary and the statistic can't be combined scalarly."

This PR adds the Partial surface.

What's in this PR

  • QueryResult::vector_with_warnings + warnings() accessor. InstantVector / RangeVector gain a #[serde(default, skip_serializing_if = \"Vec::is_empty\")] warnings: Vec<String> field. Default constructors (QueryResult::vector / matrix) stay wire-compatible — they serialise without the field when empty, so every existing caller and snapshot test holds.
  • Prometheus adapter: top-level warnings: [] on PrometheusResponse (also skip-if-empty), matching upstream's native API. format_success_response + format_range_success_response thread through any warnings the engine populated.
  • Dispatcher: drops the combinable-only early return. The full loop runs for every statistic whenever the timeline has ≥2 segments. combine_statistic still returns Full(v) for cleanly combinable inputs and Partial { covered, missing } everywhere else — on Partial we accumulate the covered scalar (when present), flag any_partial, and build a human-readable warnings list carrying metric, range, statistic, dropped-group count, and up to three unresolved segments. Overflow summarised; full set still inspectable via GET /api/v1/db/timeline.

Example response shape

```json
{
"status": "success",
"data": { "resultType": "vector", "result": [...] },
"warnings": [
"partial result: query spans 2 schemas for metric 'latency' over [1000, 9000] and the requested statistic Quantile is not cleanly combinable across schema boundaries — see GET /api/v1/db/timeline?metric=latency&start_ms=1000&end_ms=9000 for the full segment map",
"segment agg_id=1 [1000, 5000) status=Retired coverage=Sketch unresolved"
]
}
```

Validation

  • `cargo test -p query_engine_rust --lib` — 734 pass (was 728; +5 new + 1 that was previously filtered became visible)
  • `cargo clippy --all-targets -- -D warnings` — clean
  • `cargo fmt --all -- --check` — clean
  • New tests: 3 in engines::query_result::tests (default-empty / with-warnings / matrix wire contract) + 2 in Prometheus adapter (response-side serialisation contract)

Follow-ups

🤖 Generated with Claude Code

Task #34 gap #2 of 3. #48's dispatcher activated per-segment
execution for combinable stats (Count/Sum/Min/Max) but refused
to run for non-combinable stats (Quantile/Topk/Cardinality/
Rate/Increase) — those still fell through to the single-agg
path and saw the same data cliff the dispatcher was written to
prevent. Reason: there was no way to tell the HTTP caller
"this answer is Partial because the query spans a reconfigure
boundary and the statistic can't be combined scalarly."

Changes:

- **`QueryResult::vector_with_warnings` + `warnings()` accessor.**
  `InstantVector` and `RangeVector` gain a
  `#[serde(default, skip_serializing_if = "Vec::is_empty")]`
  `warnings: Vec<String>` field. Default constructors
  (`QueryResult::vector`, `QueryResult::matrix`) stay wire-
  compatible — they still serialise without the field when
  empty, so every existing caller and snapshot test holds.

- **Prometheus adapter** adds a top-level `warnings: []` on
  `PrometheusResponse` (also skip-if-empty), matching
  upstream's native API. `format_success_response` +
  `format_range_success_response` thread through any
  warnings the engine populated; absent → no field.

- **Dispatcher** drops the combinable-only early return.
  The full loop now runs for every statistic whenever the
  timeline has ≥2 segments. `combine_statistic` still
  returns `Full(v)` for cleanly-combinable inputs and
  `Partial { covered, missing }` everywhere else — on
  Partial we accumulate the `covered` scalar (when present),
  flag `any_partial`, and build a human-readable warnings
  list with the metric, range, statistic, dropped-group
  count, and up to three unresolved segments (agg_id,
  clipped range, status, coverage). Over-three are summarised;
  the full set is still inspectable via
  `GET /api/v1/db/timeline`.

5 new unit tests: 3 in `engines::query_result::tests` covering
the default-empty / with-warnings / matrix wire contract;
2 in the Prometheus adapter covering the response-side
serialisation contract. 734 lib tests pass (+5), clippy clean,
fmt clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 4bf3ce8 into main Apr 20, 2026
@zzylol
zzylol deleted the feat/phase3-partial-http-surface branch April 20, 2026 16:39
zzylol added a commit that referenced this pull request Apr 21, 2026
)

Addresses TODO.md blocker #2 subitem #3. Queries answered by the
sketch DB now carry a theoretical accuracy bound back to the
client as two fields on the Prometheus HTTP response:

* **A — structured `accuracy` (top-level)**:
  ```json
  "accuracy": {
    "epsilon": 0.008125,
    "delta":   0.0,
    "kind":    "relative_cardinality",
    "per_segment": [...]   // populated on schema-timeline crossings
  }
  ```
* **C — human-readable `infos` (Prometheus 3.0 / Grafana 11+)**:
  ```json
  "infos": ["accuracy: ε=0.008125, δ=0, kind=relative_cardinality"]
  ```

Both are standard Prometheus-tolerant extensions — unknown
top-level fields are ignored by the upstream client/Grafana 10.
A regression test confirms a stripped-down "standard Prometheus"
decoder round-trips through our extended response.

## Wire shape

`warnings` stays reserved for partial-result / fallback
advisories (PR #49's schema-timeline dispatcher still uses it);
accuracy gets its own dedicated field so the semantics don't
mix.

When a query crosses a schema-timeline boundary, `per_segment`
lists each segment's `(agg_id, range_ms, profile)`. The
top-level `profile` is the max-ε, max-δ envelope across
segments — a conservative upper bound.

## Changes

* `stores/sketch_db/accuracy.rs`:
  * `AccuracyEnvelope { profile, per_segment }` + builders
    (`single`, `from_segments`)
  * `PerSegmentAccuracy { agg_id, range_ms, profile }`
  * `AccuracyProfile::summary()` / `AccuracyEnvelope::summary()`
    emit the `infos` one-liner.
* `engines/query_result.rs`: `InstantVector` / `RangeVector`
  carry `accuracy: Option<AccuracyEnvelope>`; new
  `QueryResult::{accuracy(), with_accuracy()}`.
* `drivers/query/adapters/prometheus_http.rs`:
  `PrometheusResponse::{infos, accuracy}` fields +
  `with_accuracy()` builder.
  `format_success_response` / `format_range_success_response`
  thread `result.accuracy()` onto the response.
* `engines/simple_engine.rs`:
  * `SimpleEngine::accuracy_envelope_for(agg_id)` — single-
    aggregation helper.
  * `execute_context` attaches single-agg accuracy.
  * Timeline dispatch builds per-segment accuracy list and
    attaches the multi-segment envelope.

## Tests

777 → 784 (+5 green):
* `prometheus_response_carries_accuracy_top_level_and_infos_mirror`
* `prometheus_response_without_accuracy_skips_both_fields`
* `prometheus_response_per_segment_contains_all_segments_with_worst_case_top`
* `accuracy_coexists_with_warnings_without_interference`
* `promql_standard_client_can_decode_response_ignoring_extensions`

clippy + fmt clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant