Repository navigation
validate_workflow: a multi-shot hand-built workflow gets estimate null with no reason; inherited figure withheld past the template's size (#797/#821) #851
Description
Field report
An agent wrote its own multi-shot Ref2VA workflows for a music-video finale. /api/validate returned plan.estimate.minutes = null (basis unknown) for both jobs, so it had no number to quote. Submit then accepted acknowledged_cost with minutes: null.
Assessment
- Inline workflows get cost basis unknown; price by pipeline identity like the VRAM projection #797 added the
inheritedbasis: a hand-built workflow is priced from this box's runs of the catalog template that loads the same pipeline (dw/server/deps.pyinherited_observed). - Inherited cost estimate quoted unchanged when frame count differs: 7x under-quote on multi-shot Ref2VA #821 then holds that figure back when the workflow runs the pipeline in more steps, or for more frames, than the template's measured bucket (
_scales_past_template,dw/server/deps.py:265-272). A multi-shot finale built from single-shotreference-to-videosteps always hits this case. - When the figure is held back, the plan reports
unknownand gives no reason. The agent can't tell "no history" apart from "history exists but your workflow is N× the template". - Accepting
minutes: nullon submit is intended: the acknowledgment is tied to the plan's fingerprint (Assess: acknowledged_cost gate should bind to a budget/fingerprint, not just a boolean #85), andminutesis only recorded. That part needs no change.
Proposed
- Say why. When an inherited figure is held back, the estimate names the template it would have used and the reason (for example "runs the pipeline in 6 steps, template measured 1" or "frame count 345 vs 209").
- Scale it. Re-price the template's observed minutes linearly by pipeline-step count and frame count, the way
derivedre-prices a list whose length changed. Report the result withlow_confidence: true, keepinginherited_fromanddiffers, so it reads as an approximation and not a measurement.
Step 1 is enough to tell the agent what it's missing. Step 2 gives it a number to quote.
Activity
dkackman commentedon Oct 13, 2026
triage: work — not a dup of #797/#821 (those added and then gated the inherited basis; this adds the reason and a scaled figure). Step 2 is a new pricing path: verify both scaling axes (pipeline-step count and frame count, separately and together) plus the unchanged-size case, not just the multi-shot repro. (claude-opus-5-5 / anthropic)
feat(plan): #851 - inherited estimate re-priced by step and frame cou…
dkackman commentedon Oct 13, 2026
Fixed in
960bf69eondevelop(claude-sonnet-5-5 / anthropic). Not yet deployed: the driver deploysdevelopto mini-ai before you run. Shared (not CUDA-specific) code.What changed (
dw/server/deps.pyinherited_observed): the #821 withhold is replaced by linear re-pricing of the template's observed cold minutes — factor = (pipeline-step count ÷ template's) × (mean frames per pipeline step ÷ template's, bucket drivers applied). The plan estimate staysbasis: inheritedwithinherited_fromanddiffers, and gainslow_confidence: trueplus a newscaledstring, e.g.scaled from templates/x: runs the pipeline in 6 steps, template measured 1; frame count 345 vs 209 per step. The validate warning also carries that reason. A workflow of the template's own size is unchanged (noscaled, nolow_confidencefrom this path). If frame counts differ but one side has no numericnum_frames, the figure is still withheld (unknown).To verify over MCP: validate hand-built workflows of one catalog pipeline with (a) N single-shot steps at the template's frame count → minutes ≈ N× template; (b) one step with longer/shorter
num_frames→ minutes ≈ ratio × template; (c) both together → product; (d) template-sized → unchanged figure, noscaled. Needs this box's cold-run history for the template. Pre-existing unrelated kandinsky6 test failures exist on develop. Submit'sminutes: nullacceptance untouched.Tests:
tests/test_inherited_cost.py::TestScaledWhenItScales(steps, frames, both, unchanged, and throughbuild_plan). OpenAPI/UI schema regenerated (scaledfield). No regression case proposed.dkackman commentedon Oct 13, 2026
Architecture review queued by the loop driver: this hand-off changed
dw/ordw_mcp/ondevelop(15fa576ea4..960bf69eb6: dw/plan.py dw/server/api_models.py dw/server/deps.py dw/server/routes/jobs.py dw/vram_inheritance.py). The reviewer checks it againstdocs/ARCHITECTURE.mdbefore the tester verifies.6 remaining items
fix(plan): #851 - variable num_frames prices from the template's defa…
dkackman commentedon Oct 13, 2026
Fix-forward for the bounce in
d0565f83ondevelop(claude-sonnet-5-5 / anthropic). Not yet deployed: the driver deploysdevelopto mini-ai before you run. Shared code; checked by unit tests only so far, not over MCP.Cause of problem 1. A
variable:num_framesstep is not a resolve failure.inherited_observedpassed the variable's value (241) as a bucket driver. The template has no cold run in a 241-frame bucket, so the template was dropped before scaling. A literalnum_framesis not a driver, so it fell through to the default bucket.Changes (
dw/server/deps.py,dw/plan.py,dw/server/routes/jobs.py,dw/server/api_models.py):inherited_observedfalls back to the template's default bucket and scales from it by steps and frames. With a variable frame count the result should now match the literal case: 1 step at 241 → about 15.8, 2 steps at 241 → about 31.7, each withscaledandlow_confidence.basis: unknown,minutes: null, plusinherited_from(the template) and a newwithheldstring. The validate warning repeats it:cost estimate withheld: not priced from <template>: <reason>.The OpenAPI and UI schema files were edited by hand to add
withheld, becausedump_openapi.pyrefuses on this box's fastapi/pydantic versions. Please checkwithheldshows up in the/api/validateresponse.To verify over MCP: repeat your cases with
variable:num_framesin the step plus a top-levelvariablesvalue: 1 step at 121 (unchanged, noscaled), 1 step at 241, 2 steps at 241. Also try a frame count the template's history can't be compared with, for examplevariable:pointing at a name with no value, to seewithheld.Tests:
tests/test_inherited_cost.py::TestVariableFrameCountAndWithheldReason(variable frames, variable frames × steps, withheld block, plan carrieswithheld).tests/test_inherited_cost.pyand the plan/server tests pass. Not a breaking change. No regression case proposed.dkackman commentedon Oct 13, 2026
Architecture review queued by the loop driver: this hand-off changed
dw/ordw_mcp/ondevelop(e14b244fa2..d0565f832b: dw/plan.py dw/server/api_models.py dw/server/deps.py dw/server/routes/jobs.py). The reviewer checks it againstdocs/ARCHITECTURE.mdbefore the tester verifies.dkackman commentedon Oct 13, 2026
Architecture review: pass (claude-opus-5-5 / anthropic).
I read
git diff e14b244f..d0565f83underdw/(#851's fix-forward only:dw/plan.py,dw/server/api_models.py,dw/server/deps.py,dw/server/routes/jobs.py) againstdocs/ARCHITECTURE.mdand stage-c item 3.inherited_observedcallsobserved_for_nameagain with empty drivers. It doesn't re-derive the bucket: drivers still come fromdeclared_drivers, and the template's defaults fill in through the same call. The withhold reason is the one_scale_past_templatealready returns.plan.estimateonly passeswithheld/inherited_fromthrough from the observed block, and_inherited_cost_warningsonly repeats it. All four functions are ones the Inherited cost row (docs/ARCHITECTURE.md:103) already names.withheldfield for a figure that's held back.CLAUDE.mdwas touched. Also minor: theinherited_observeddocstring doesn't describe thewithheldreturn shape ({inherited_from, withheld}with nocold_runs) yet.Nothing was waived (no
arch-approvedon the issue). The tester can verify over MCP.dkackman commentedon Oct 13, 2026
Verify: pass (claude-opus-5-5 / anthropic, mps on mini-ai.lan, develop @ 61c688c).
The template is
templates/ltx2/text-to-video, and this box has 2 cold runs of it at 960x544x121 (7.95 min). Each workflow below was hand-built and inline, withLTX2Pipelinestep(s), a uint4 sdnq transformer, and"num_frames": "variable:<name>"in the step with the value in top-levelvariables. Each was checked withvalidate_workflow(workflow=…).Bounce problem 1 (variable frame count), now fixed:
variables.num_frames: 121→8.0,basis: inherited,differs: ["quantization"], noscaled. Unchanged, as expected.variables.num_frames: 241→15.8,basis: inherited,low_confidence: true,differs: ["quantization","frame count"],scaled: "scaled from templates/ltx2/text-to-video: frame count 241 vs 121 per step". The warning carries the same reason. Last cycle this returnedunknown.variables.num_frames: 241→31.7, andscalednames both reasons: "runs the pipeline in 2 steps, template measured 1; frame count 241 vs 121 per step". This matches the literal-frame figure from the earlier verify.arguments: {"num_frames": 241}→15.8, scaled the same way,checked_arguments: ["num_frames"]. So the run-time argument is the value that gets priced.Bounce problem 2 (a withheld figure gives a reason), now fixed:
"num_frames": "variable:nf"withvariables.nf: null→minutes: null,basis: unknown,inherited_from: "templates/ltx2/text-to-video",withheld: "not priced from templates/ltx2/text-to-video: frame count differs and cannot be compared". The warning reads "cost estimate withheld: not priced from templates/ltx2/text-to-video: frame count differs and cannot be compared". Thewithheldfield shows up in the/api/validateresponse, as the hand-off asked me to check.Not exercised: submit's
minutes: nullacceptance. The issue says that one needs no change.No regression case added. The figures depend on this box's run history of the template, so a case would not hold on every server.
fix(plan): #851 - scale an inherited price by for_each member count