fix: IncreaseAccumulator implements Statistic::Sum (sum-instant of counters now works) - #109
Merged
Merged
Conversation
…unters now works) Counters are ingested by the warm tier as `IncreaseAccumulator` (and `MultipleIncreaseAccumulator` for keyed variants). Pre-fix neither trait `query` answered `Statistic::Sum`, so an instant `sum by (zone) (http_requests_total)` capability-missed even though the matcher and OnlySpatial dispatch path were correct (PR #108 regression test pinned the matcher contract; the actual data-flow gap is here). Fix: * `IncreaseAccumulator::query(Sum, ..)` returns `last_seen_measurement.value` — the latest cumulative counter value of that series, matching Prometheus' `sum(<counter>)` instant semantics. * `MultipleIncreaseAccumulator::query` already delegates to the inner `IncreaseAccumulator`, so per-key Sum follows automatically. * `compatible_agg_types(Statistic::Sum)` now lists `Increase` / `MultipleIncrease` so capability matching accepts counter-shaped configs. * Updated the two unit tests that previously asserted Sum errors. * Added a new warm-tier regression test (`sum_by_zone_instant_over_increase_accumulator_does_not_error`) that builds a `SimpleEngine` with two `IncreaseAccumulator` series under different zone labels and asserts `sum by (zone) (http_requests_total)` returns `QueryResult::Vector` with the per-zone latest cumulative values. `rate(<counter>[5m])` and `increase(<counter>[5m])` are unaffected — those statistics resolve to `Statistic::Rate` / `Statistic::Increase` which already worked, and the changed match arm only adds a new case for `Sum`. Refs: ProjectASAP/ASAPCollector#46 Refs: PR #108 (warm-tier replay regression diagnosis) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Issue: ProjectASAP/ASAPCollector#46. PR #108 pinned the warm-tier matcher contract for
sum by (zone) (http_requests_total)and concluded:This PR closes that data-flow gap.
A failing replay.jsonl line of the form (paraphrased):
{"type":"query","query":"sum by (zone) (http_requests_total)","instant":true,"expected_status":"success","actual_status":"error"}would, pre-fix, hit
EngineError::CapabilityMissbecause:compatible_agg_types(Statistic::Sum)did not includeIncrease/MultipleIncrease, so capability matching rejected counter-shaped configs.IncreaseAccumulator::query(Sum, ..)errored withUnsupported statistic.Approach
Picked the simplest correct fix per the task description's first option: implement
Statistic::Sumdirectly onIncreaseAccumulator.IncreaseAccumulator::query(Sum, ..)returnslast_seen_measurement.value— the latest cumulative counter value of that series. This is exactly what Prometheus does forsum(<counter>)instant: per-series take the latest cumulative; the engine's outersum byaggregation groups + sums those across keys.MultipleIncreaseAccumulator::queryalready delegates to the innerIncreaseAccumulator, so per-key Sum on the multi-population variant follows automatically.compatible_agg_types(Statistic::Sum)extended withAggregationType::IncreaseandAggregationType::MultipleIncreaseso capability matching accepts counter-shaped configs.rate(<counter>[5m])/increase(<counter>[5m])are unaffected — those resolve toStatistic::Rate/Statistic::Increasewhich already worked; the changed match arm only adds a new case forSum.New test
Two unit tests inside
precompute_operators/{increase_accumulator,multiple_increase_accumulator}.rsthat previously asserted Sum errored were updated to assert Sum returns the latest cumulative value, plus a new positive test for each.Test plan
cargo build --release -p query_engine_rustcargo test --release --lib -p query_engine_rust -- sum_by_zone_instant_over_increase_accumulator_does_not_error sum_by_zone_instant_does_not_error test_increase_accumulator_query test_increase_accumulator_sum_is_latest_cumulative_value test_multiple_increase_accumulator_query test_multiple_increase_accumulator_sum_per_key— all 6 passcargo test --release --lib -p query_engine_rust— 914 passed, 33 failed (same 33 pre-existing failures asorigin/main; baseline comparison confirms no new regressions)cargo test --release -p asap_types— 38/39 pass; the one failure isavg_finds_sum_and_count, which is a pre-existing flaky test relying onHashMapiteration order (fails sporadically onorigin/maintoo — confirmed via 5x run on stashed clean tree, 1/5 failures)Honest notes
last_seen_measurement.value. This matches Prometheus' instantsum(<counter>)semantics. It does not also exposeStatistic::CountfromIncreaseAccumulator(the trait still errors there), socount(http_requests_total)against an Increase-only config will still capability-miss — out of scope for this PR.compatible_agg_types(Sum)ordering putsIncrease/MultipleIncreaselast, after the existingSum/MultipleSum/CountMinSketch. Withaggregation_priority's descending-window-size sort, this means a true Sum config wins over a co-deployed Increase config of equal window — preserving today's planner picks for explicitly-plannedsum_over_timeprecomputes.Refs: ProjectASAP/ASAPCollector#46
Refs: PR #108 (warm-tier replay regression diagnosis)