Skip to content

Prevent warm queries from reading partially published input - #627

Merged
zzylol merged 4 commits into
mainfrom
fix/continuous-summary-completeness
Sep 11, 2026
Merged

zzylol merged 4 commits into
mainfrom
fix/continuous-summary-completeness

Conversation

@zzylol

@zzylol zzylol commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Warm queries could return one group while another accepted group was still queued, and counter readout treated an inactive series as an incomplete pane. This adds store-owned admission receipts and publication checks across Remote Write, workers, maintenance outputs and the complete query DAG.

Before this PR, a fast worker could publish a partial population or advance replay retention past a slower worker; branch-local checks could also mix publication revisions. After this PR, overlapping unpublished input forces fallback, accepted slow-worker outputs remain publishable, and finite drain can prove a known series has no input in a retained window. For example, the two-group regression holds one worker and rejects warm readout until both groups publish.

Queue capacity is reserved before admission. Outputs carry their actual first/last consumed revisions, corrections are combined before acknowledging a receipt, and catalog generation is checked before state mutation. Startup installs the catalog before recovery. Accepted-sample and materialized-output throughput counters are connected to successful admission/publication boundaries.

Verification: seven production compatibility process tests pass, including the complete counter/fallback matrix, ERP, CMS/CS TopK and repeated dashboards; 322 precompute tests and six admission tests pass. The Remote Write suite also covers a real two-worker partial-publication race. Performance measurement is not applicable to this correctness change.

Stacked on #616. The revision fence is currently global, so unrelated continuous updates can cause conservative fallback. Absence proof requires finite drain; live event-time completeness is not inferred. Receipts are in memory, and durable series identity migration remains separate. This does not add general multi-input maintenance transforms.

@zzylol
zzylol changed the base branch from feat/erp-per-series-execution to main September 11, 2026 01:43
@zzylol
zzylol marked this pull request as ready for review September 11, 2026 02:01
@zzylol
zzylol merged commit 3dac125 into main Sep 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant