validate_workflow passes a workflow whose stored default asset does not exist (templates/ltx2/reference-sheet) #166

Activity

1. dkackman

   added

   owner:implementerImplementer's turn to actImplementer's turn to act

   on Sep 14, 2026
2. dkackman commented on Sep 15, 2026

   

   dkackman

   on Sep 15, 2026

   OwnerAuthor

   More actions

   triage: work

   Filed by the repo owner, not a duplicate, not already fixed. validate_workflow doesn't check stored-default media references (asset:/prompt:/output:) the way it checks caller-supplied arguments — that's the general gap; the specific template default (templates/ltx2/reference-sheet -> asset:reference_sheet.png) pointing at a file nobody has is a second, narrower question the fix session should also resolve (asset vs. required-argument-with-no-default).

   Model sonnet via provider anthropic.
3. dkackman

   added 2 commits that reference this issue on Sep 15, 2026

   fix(server): #166 - validate_workflow checks stored variable defaults…

   ...

   f23d5ad

   Merge fix for #166 - validate_workflow checks stored variable defaults

   a199d37
4. dkackman

   added

   owner:testerTester's turn to actTester's turn to act

   status:fixed-pending-verifyImplementer fixed, awaiting tester verificationImplementer fixed, awaiting tester verification

   and removed

   owner:implementerImplementer's turn to actImplementer's turn to act

   on Sep 15, 2026
5. dkackman commented on Sep 15, 2026

   

   dkackman

   on Sep 15, 2026

   OwnerAuthor

   More actions

   Fixed by model sonnet via provider anthropic.

   Root cause: _argument_reference_errors in dw/server/app.py only walked caller-supplied arguments for asset:/prompt:/output: reference existence, never the workflow's declared variables defaults. So validate_workflow(name="templates/ltx2/reference-sheet") with no arguments answered valid: true even though the stored default "reference_sheet": "asset:reference_sheet.png" names a file that doesn't exist, while the same call with that exact value passed explicitly as an argument correctly answered valid: false - one workflow, two verdicts depending on whether the caller happened to restate the default.

   Fix: _argument_reference_errors now merges every declared variables default (skipped when the caller overrides it) with the caller's arguments into one effective set before checking references, so a bare call and an explicit-default call agree. Reported at variables.<name> (vs arguments.<name>) so the error message still distinguishes "you wrote a bad reference" from "you didn't override a bad default." This is the general fix, not a template-specific one - the same placeholder-asset:-default pattern appears in several other catalog templates, and this closes the whole class rather than just reference-sheet.

   Also, while restructuring submit_job to reuse one Workflow candidate for both the argument-reference check and the acknowledged-cost path, found and fixed a related gap: constraint_errors (the variable_constraints.py bound checks, e.g. H3's num_frames grid) previously only ran inside the bound-acknowledgement branch of job submission, so a non-bound POST /api/jobs skipped it until run time. It now runs on every submission via the same unconditional candidate build.

   Behavior change to flag for verification: any catalog template shipping a placeholder asset: default (expected to be overridden by a real caller) will now report valid: false on a bare validate_workflow/job submission in a workspace that doesn't have that placeholder file - this is the intended fix, not a new bug, but the tester should expect it rather than treat it as a regression.

   Commit: a199d37 (merge of f23d5ad) on develop.
   Deploy: pulled to lem (~/diffusers-workflow, fast-forward b599cf3→a199d37), MCP/HTTP server process restarted in its screen session, confirmed healthy via GET /api/workflows (200) and by re-running the exact repro from this issue against the live server - POST /api/validate with workflow_path: templates/ltx2/reference-sheet and no arguments now answers valid: false at variables.reference_sheet, matching the explicit-argument call.

   Full test sweep: tests/ - 4865 passed, 14 skipped, no regressions. Added 3 new regression tests in tests/test_validate_arguments.py reproducing this exact scenario (bare call with bad stored default fails; explicit override of the same variable with a good value is judged on the override and passes; bare job submission with a bad stored default is refused before queuing).

   Proposed regression-suite case (for regression-suite-smoke.md, once verified over MCP):

   • Call validate_workflow(name="templates/ltx2/reference-sheet") with no arguments.
   • Expected: valid: false, an error at variables.reference_sheet naming the missing asset - not valid: true.
   • (Optionally pair with the same call passing reference_sheet pointing at a real asset in the workspace, expecting valid: true, to pin that overriding a bad default still works.)
6. dkackman commented on Sep 16, 2026

   

   dkackman

   on Sep 16, 2026

   OwnerAuthor

   More actions

   Verified over MCP. Tester agent, model opus via provider anthropic.

   Repro from the issue body, re-run against live lem:

   1. validate_workflow(name="templates/ltx2/reference-sheet"), no arguments →
      valid: false, one error at variables.reference_sheet: "Asset 'reference_sheet.png' not found in /home/don/diffusers-workspace/assets, /home/don/diffusers-workspace/common/assets, /home/don/diffusers-workflow/assets …". checked_arguments: []. This was valid: true when filed.
   2. list_assets() → still no reference_sheet.png in any root (the default is genuinely unresolvable, so step 1's new answer is the correct one, not a spurious failure).
   3. get_workflow("templates/ltx2/reference-sheet", variables_only=true) → "reference_sheet": "asset:reference_sheet.png" — the stored default is unchanged, so the fix is in the checker, not a template edit.
   4. validate_workflow(..., arguments={"reference_sheet": "asset:reference_sheet.png"}) → valid: false at arguments.reference_sheet, same message.

   Steps 1 and 4 now agree, which is what the issue asked for, and the path still distinguishes "you didn't override a bad default" (variables.) from "you wrote a bad reference" (arguments.).

   Adjacent cases (partial-fix checks):

   • Override the bad default with a real asset: validate_workflow(name="templates/ltx2/reference-sheet", arguments={"reference_sheet": "asset:qa-cast/hal-portrait.jpg"}) → valid: true, checked_arguments: ["reference_sheet"], plan of 2 steps with downloads_required: Lightricks/LTX-2.5-22b-IC-LoRA-Ingredients (1.3 GB). So the merged default+argument check doesn't over-report: a caller who supplies a good value is judged on the override, and the workflow's other stored defaults (prompt:ltx2/reference_sheet_workshop, the constant: negative prompt) resolve and are not falsely flagged.
   • Declared-bound check on the same call path: arguments={"reference_sheet": "asset:qa-cast/hal-portrait.jpg", "reference_frames": 100} → valid: false at arguments.reference_frames, "must be at least 121 … checked before anything loads". The constraint check the fix moved onto the unconditional candidate build still fires on this path.

   Behavior change acknowledged: per the hand-off comment, a bare validate_workflow on any catalog template shipping a placeholder asset: default now answers valid: false in a workspace without that file. Expected, not a regression — I will treat it as such rather than filing it.

   The narrower question in this issue's scope section (whether templates/ltx2/reference-sheet should ship a placeholder default at all, vs. a required argument with no default) is left as the implementer decided: the default stays and the bare call now reports it by name, which satisfies the expected: here.

   Adding the implementer's proposed case to regression-suite-smoke.md.
7. dkackman

   added

   status:verifiedTester confirmed the fix via a real MCP callTester confirmed the fix via a real MCP call

   and removed

   status:fixed-pending-verifyImplementer fixed, awaiting tester verificationImplementer fixed, awaiting tester verification

   on Sep 16, 2026
8. dkackman

   closed this as completedon Sep 16, 2026