Test workflow builder experiments through the product API - #1
Draft
deyton wants to merge 8 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Allow the existing Monarch competitor to test the workflow builder experiments through the product's authenticated HTTP API. Setting
builder_experimentselects the configuration and records durable authoring evidence; leaving it unset preserves the existing request and stream behavior.Shared product client
Poll persisted status, verify the resolved configuration, and export complete cursor history with raw structured failures. Preserve selection in configuration hashes and preserve the configured model identity. Hosted testing requires its source checkout to match the deployed revision.
Cancel authoring or execution when prototype polling fails or times out, retain cancellation responses and partial attempt evidence, and leave unsettled usage explicitly unknown.
Release the fixed synthetic HTTP port between attempts on POSIX while preserving Windows ownership checks and existing connection-closing behavior. Direct regressions cover restart after a request and refusal over an active listener.
Deferred evaluation scaffold
Validation
Full offline suite: 768 passed, 3 skipped. Earlier independent review also passed 175 focused keyless tests. Regression tests cover actual progress/lifecycle event shapes, repeated build keys after clarification, missing or duplicate receipts, cancellation confirmation, model identity and stored failure evidence. The regenerated manifest matches the implementation.
The original fixed-port failure reproduced on baseline locally and in Linux CI; the lifecycle correction passes the full local suite and refreshed Linux CI, including corpus validation. Graphify could not be refreshed because its module is unavailable. No paid model run, live product execution, or benchmark improvement is claimed. Private dogfood workflow content is excluded.
Product counterpart: https://github.com/TestBoxLab/monarch/pull/702