Validate immutable publication against every consumed input - #664
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A multi-input immutable output must not commit after any consumed source changes lifetime, or reuse a receipt that omits an input.
What
Extend existing publication and recovery to validate an entire frozen cohort, and hash canonical complete input lineage.
How
Hold admission and instance guards across all-source validation, existing target registration, and the durable transaction. Registration reuses the existing index helper. Multi-input lineage includes definition, SID, group, generation, window bounds, payload, and program identity in canonical order. Existing single-input receipt bytes remain compatible.
Before this PR
Publication validated one source; recovery did not validate source lifetime.
After this PR
The storage transaction can reject a cohort when its second source expires, including recovery. Current production execution still supplies one source until multi-frontier scheduling is enabled.
Verification
97 storage index tests passed after the registration/guard change. 11 maintenance runtime tests passed after lineage changes, including canonical ordering and second-input mutations. Source review identified and closed the registration lock gap; no concurrent interleaving test is claimed.
Limits
This does not yet enable multi-frontier DAG execution, shuffle completeness, continuous watermarks, or sliding scheduling. The initial publication guard holds the instance write lock through persistence; finer lock granularity is separate work.