Schedule complete finite source cohorts through immutable maintenance - #667
Merged
Merged
Conversation
zzylol
marked this pull request as ready for review
September 11, 2026 19:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A derived maintenance program must wait for every source at the same window. Publishing one source or pairing different windows would materialize a partial result.
What
The finite scheduler can route a complete cohort of raw input definitions into the existing immutable maintenance executor. Each source must have exactly one physical SID and group, all explicit groups must match, and windows must be aligned, stored in full, and non-overlapping.
How
Use the shared source-window validator, inspect durable population inventories, and select the full window intersection. Prove every requested source before resolving an output, then reuse the existing all-source lifetime transaction, lineage digest, and durable retry path. Single- and multi-source scheduling pass their captured catalog generation through execution instead of refreshing it at the publication boundary.
Before this PR
The public finite scheduler rejected multiple source definitions, even though the immutable cohort executor could execute a bound multi-frontier program.
After this PR
A bound two-source fixture with completed values 2 and 7 produces one durable result 9 and reuses it after repeat scheduling and restart. If the sources complete different windows, no target SID, metadata receipt, or part is created. A stale generation is rejected before another generation's state can be substituted.
Verification
64079926.675f9d1badds only the previously reviewed shared population codec and explicit ReadTime in an offline fixture; maintenance source is identical.Limits
No group shuffle, continuous watermark, sliding execution, or new-generation seal liveness claim. Restored historical writes without a new generation's completion proof remain conservatively rejected. Compiler/install capability enablement and the real selected-plan process test remain separate.
Stacked on #666 and depends on Planner #402.