Skip to content

feat: enforce plan-scoped sampling, GOS, delta, and adaptation - #477

Merged
zzylol merged 9 commits into
mainfrom
feat/runtime-adaptation-sampling-delta
Sep 4, 2026
Merged

zzylol merged 9 commits into
mainfrom
feat/runtime-adaptation-sampling-delta

Conversation

@zzylol

@zzylol zzylol commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #476.

Adds the final physical-runtime policy layer on top of TransmissionPlan.

  • binds typed sampling, delta, and CountSketch GOS policies to exact materialization fingerprints
  • validates family/estimator/mode combinations and periodic full checkpoints
  • scopes sequence/checkpoint lineage by plan, materialization, concrete series, producer and producer epoch; continuity crosses windows while each frame retains its own window identity
  • authorizes telemetry-driven changes only as a staged next plan_version with exact fresh evidence and bounded guardrails
  • enforces authorization at the atomic backend physical-plan endpoint
  • uses one canonical concrete-series identity; grouped frames retain labels and ungrouped frames use
  • preflights complete batch checkpoint/sequence lineage against a scratch snapshot before applying the first frame
  • serializes physical frame batches, decodes payloads before committing lineage, then synchronously updates snapshot/store
  • ignores exact retries, rejects gaps, and makes reads overlapping an incomplete window fail closed until exact-tier fallback
  • treats HTTP 2xx / gRPC OK as the ACK; no custom checksum/ACK/WAL transport protocol. A restart forgets receiver lineage and safely requires a new full checkpoint.

Strict runtime contract follow-up (2026-09-03):

  • modified-OTLP ingest now takes one ActivePhysicalPlan snapshot and derives both runtime materializations and the transmission contract from that same generation
  • PrecomputePlan rejects empty or duplicate schema IDs, and TransmissionPlan rejects rule encodings not advertised by the bound schema
  • every non-bootstrap active generation authorizes its successor, so changing plan_id cannot bypass runtime-adaptation guardrails
  • TopK, window-cost, and runtime-adaptation evidence dated after the validation clock is rejected rather than treated as age zero

Validation after the latest update:

  • control_plane physical compiler tests: 16 passed
  • cargo check -p data_plane
  • cargo fmt --all
  • git diff --check

This PR is intentionally open and must follow #476.

@zzylol
zzylol force-pushed the feat/transmission-plan-frame-identity branch from cacf70a to e095862 Compare September 3, 2026 04:57
@zzylol
zzylol force-pushed the feat/runtime-adaptation-sampling-delta branch from 82e6bf4 to 2c480fe Compare September 3, 2026 04:57
@zzylol
zzylol force-pushed the feat/transmission-plan-frame-identity branch from e095862 to dafdd92 Compare September 3, 2026 14:27
@zzylol
zzylol force-pushed the feat/runtime-adaptation-sampling-delta branch from 2c480fe to 03664da Compare September 3, 2026 14:30
@zzylol
zzylol force-pushed the feat/transmission-plan-frame-identity branch from dafdd92 to 9613c54 Compare September 3, 2026 20:08
@zzylol
zzylol force-pushed the feat/runtime-adaptation-sampling-delta branch from 66bd009 to e7db4a1 Compare September 3, 2026 20:17
@zzylol
zzylol force-pushed the feat/transmission-plan-frame-identity branch from 9613c54 to 0018db5 Compare September 3, 2026 20:36
@zzylol
zzylol force-pushed the feat/runtime-adaptation-sampling-delta branch from 8918d87 to 3f853e7 Compare September 3, 2026 20:37
@zzylol
zzylol force-pushed the feat/transmission-plan-frame-identity branch from 0018db5 to da0ada4 Compare September 3, 2026 20:41
@zzylol
zzylol force-pushed the feat/runtime-adaptation-sampling-delta branch from 3f853e7 to 5d59617 Compare September 3, 2026 20:41
@zzylol
zzylol force-pushed the feat/transmission-plan-frame-identity branch from da0ada4 to e79d0dd Compare September 4, 2026 02:57
@zzylol
zzylol force-pushed the feat/runtime-adaptation-sampling-delta branch from 5d59617 to 2377a02 Compare September 4, 2026 03:03
@zzylol
zzylol force-pushed the feat/transmission-plan-frame-identity branch from e79d0dd to 56e8a87 Compare September 4, 2026 03:25
@zzylol
zzylol force-pushed the feat/runtime-adaptation-sampling-delta branch from 2377a02 to 4ff7891 Compare September 4, 2026 03:25
@zzylol
zzylol changed the base branch from feat/transmission-plan-frame-identity to main September 4, 2026 03:42
@zzylol
zzylol merged commit b4d2e7d into main Sep 4, 2026
1 check passed
@zzylol
zzylol deleted the feat/runtime-adaptation-sampling-delta branch September 12, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant