Skip to content

feat: execute TopK exact values through candidate-filtered Prometheus - #558

Merged
zzylol merged 1 commit into
mainfrom
feat/topk-candidate-exact
Sep 10, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/topk-candidate-exact

Conversation

@zzylol

@zzylol zzylol commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

A hybrid topk(2, sum by (job) (rate(...))) plan previously maintained a local exact counter subtree alongside its CMS candidate heap. That duplicated exact computation and made the warm path state cost scale with the raw series count.

This change gives the installed typed DAG a candidate-dependent Prometheus leaf. The backend reads the shared CMS candidate node once, injects an anchored job matcher into the exact inner PromQL through the PromQL AST, asks Prometheus only for those candidates, and then intersects/reranks the normalized exact vector. Empty candidate sets return immediately without an RPC. Candidate requests use form POSTs so larger heaps do not hit URL limits, and candidate/query size limits turn unsafe requests into the existing whole-query capability fallback.

The compiler now installs only the CMS-with-heap for this hybrid branch; it does not provision an Increase/exact-counter materialization. Existing non-hybrid plans keep their prior local exact behavior. Successful summary-candidate plus remote-exact execution reports x-asap-execution=hybrid; exact_fallback remains reserved for whole-query Prometheus fallback. The calibration guard distinguishes both plan shapes and fail-closes the new branch unless it records one CMS read, one candidate-filtered exact RPC/evaluation, zero local exact state, and explicit hybrid provenance.

Validation:

  • cargo fmt -p control_plane -p data_plane -- --check
  • cargo test -p control_plane --lib (708 passed)
  • cargo test -p data_plane query_engines::asap_query_engine --lib (122 passed)
  • cargo test -p data_plane logical_provenance_tests --lib (5 passed)
  • python3 -m unittest discover -s tools/o11y-execution -p 'test_*.py' (46 passed)
  • evaluation-11 on the preceding implementation commit completed 20/20 correctly with one global CMS, no local exact counter, one shared summary readout, and one candidate-filtered exact RPC; the final classification-only SHA is being rerun

@zzylol
zzylol force-pushed the feat/topk-candidate-exact branch from 2af96e0 to a44133b Compare September 10, 2026 05:50
@zzylol zzylol changed the title Execute TopK exact values through candidate-filtered Prometheus feat: execute TopK exact values through candidate-filtered Prometheus Sep 10, 2026
@zzylol
zzylol force-pushed the feat/topk-candidate-exact branch 2 times, most recently from 92bf53e to f421aa2 Compare September 10, 2026 05:55
@zzylol
zzylol force-pushed the feat/topk-candidate-exact branch from f421aa2 to 3c50288 Compare September 10, 2026 06:03
@zzylol
zzylol merged commit d17d18d into main Sep 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant