Skip to content

fix(precompute): preserve deadline corrections - #485

Merged
zzylol merged 1 commit into
mainfrom
fix/precompute-deadline-corrections
Sep 3, 2026
Merged

zzylol merged 1 commit into
mainfrom
fix/precompute-deadline-corrections

Conversation

@zzylol

@zzylol zzylol commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Before

An absolute wall-clock deadline could close an active event-time pane while the default late-data policy still dropped later inputs. That bounded freshness by silently losing data. There was also no metric separating dropped late inputs from appended corrections.

After

  • Enable the absolute deadline only when ForwardToStore guarantees mergeable correction output.
  • Make ForwardToStore the runtime/default policy and use a 5-second maximum-open grace.
  • Preserve post-deadline raw Sum and prebuilt DDSketch inputs as mergeable corrections.
  • Export asap_precompute_late_inputs_total{action,input_kind} for drop/correction observability.
  • Document the corrected deadline and late-data contract.

Validation

  • cargo fmt -p data_plane -- --check
  • CARGO_TARGET_DIR=/mydata/ASAPQuery-backend/target cargo test -p data_plane precompute_engine:: (265 passed)
  • Regression tests cover Sum correction merge, DDSketch correction merge, and disabling deadline closure under Drop.

Stacked on #484; merge after #483 and #484.

@zzylol
zzylol changed the base branch from feat/precompute-window-close-policy to main September 3, 2026 18:33
@zzylol
zzylol merged commit 46199bc into main Sep 3, 2026
1 check passed
@zzylol
zzylol deleted the fix/precompute-deadline-corrections branch September 3, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant