Add a strict multi-fit ERP Figure 1 runner - #592
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Figure 1 needs to compare five independently executed methods without allowing dataset, candidate, or ERP-fit drift between arms. Fixed sketch parameters must not be relabeled as a Planner ERP selection.
The stack pins merged Planner #380 (
a9651cc) across every backend crate. It also adds a strict runner for AutoSketch-PerQuery, Planner Analytical, Planner ERP, ASAP-NoSharing, and Exact. A manifest pins the dataset SHA-256 and one candidate space, total-memory bound, accuracy contract, window workload, multi-family ERP observation, and complete fit/divergence/confidence selection policy from Planner #380. Each arm runs in a separate process and must echo the identical contract and actual selected candidates.The runner measures wall time, user/system CPU, and peak RSS. It verifies candidate membership, state size, and measured error; invalid/ambiguous fits, constraint drift, missing arms, and dataset changes fail closed. Failed processes remain in the artifact with resource counters and stderr.
Before this PR, sequential in-process prototypes could not isolate CPU/RSS and the fixed
--width/--depthconfiguration could be mislabeled as ERP. After this PR, an ERP result requires an actual selected-plan record under the same immutable inputs as all baselines.Verification:
python3 -m unittest tools/erp-figure1/test_run.py -v— 7 passedcargo check -p control_plane -p data_planeNo Figure 1 numbers are claimed here. Production arm adapters must implement the documented JSON boundary; publishing partial or relabeled smoke output remains disallowed.