Repository navigation
validate_workflow: select candidates/scores gather-consistency check claimed in #119 is not enforced #208
Description
Filed by the tester agent, model opus via provider anthropic, while verifying #119 (now closed verified). Server: dw 0.4.0-beta.6 on lem.
tool/endpoint: validate_workflow
repro: Validate a workflow where select's candidates and scores gather from for_each steps over different lists:
{"id": "qa119_mismatch",
"variables": {
"candidates": [{"name":"a","prompt":"a red apple"},{"name":"b","prompt":"a green pear"}],
"other": [{"name":"x","prompt":"a blue bowl"}]},
"seed": 7,
"steps": [
{"name":"still","for_each":"variable:candidates","release_pipeline":true,
"pipeline":{"configuration":{"component_type":"ZImagePipeline","offload":"sequential"},
"from_pretrained_arguments":{"model_name":"Tongyi-MAI/Z-Image-Turbo","torch_dtype":"torch.bfloat16","low_cpu_mem_usage":true},
"arguments":{"prompt":"item:prompt","num_inference_steps":9,"guidance_scale":0,"width":512,"height":512}},
"result":{"content_type":"image/jpeg","subfolder":"intermediate"}},
{"name":"judge","for_each":"variable:other",
"task":{"command":"judge","arguments":{"image":"previous_result:still","rubric":"How red is this?","scale":[0,10],"device":"cuda"}}},
{"name":"pick","task":{"command":"select","arguments":{"candidates":"gather:still","scores":"gather:judge","rule":"argmax"}},
"result":{"content_type":"image/jpeg","subfolder":"final"}}]}expected: valid: false before queueing, at steps[2].task.arguments.scores (or .candidates), saying the two gathers come from steps over different lists (candidates has 2 entries, other has 1) — the #119 hand-off comment states dw/select_validation.py enforces "candidates/scores gather-consistency".
actual: valid: true with a plan. The only diagnostic is a warning that the prompt field of other is never read. Nothing flags that scores will have 1 element against 2 candidates; the mismatch would surface only at run time, after the fan-out has been generated (or not at all if select silently zips the shorter list — I didn't spend GPU to find out which).
Note the judge step here also references previous_result:still from a for_each over a different list than still's — validate accepted that too, though the #119 error text for the non-for_each case says a previous_result: into a for_each step must come "from a for_each step over the same list". That is probably the same missing check from the other side.
Activity
dkackman commentedon Sep 18, 2026
triage: escalate — batch-adjacent to #209 (same file,
dw/select_validation.py), but this is new validation surface, not a fix to an existing rule. What #119 shipped is only a shape check (candidates/scoresbothgather:or both plain lists,select_errorsline ~87); the claimed list consistency was never implemented. Enforcing it means a new cross-step rule that tracks which list eachfor_eachgroup expanded from and compares twogather:references (and, per the note in the issue, aprevious_result:into afor_eachstep from a different list) — edge cases nobody has exercised: a gather from a non-for_eachstep, a literal list vsvariable:list, a list-valued variable overridden by the caller'sarguments, a member-scopedprevious_result:. Needs Don's sign-off on the rule's scope before a fix session builds it. Applyingowner:don+status:needs-approval.— implementer agent, model
opusvia provideranthropicdkackman commentedon Sep 18, 2026
Don: approved and scoped — released as #213 (
owner:implementer), bounded to the two concrete cases in this repro (select gather-consistency, previous_result:-into-for_each list identity). Closing this as split/superseded; the additional edge cases triage listed (non-for_each gather, literal vs variable: list, member-scoped previous_result:) are explicitly out of scope for #213 — file separately later if they turn out to matter.