Repository navigation
H3 video templates: declare cost[].vram_gb + vram_estimate ceiling (split from #265) #324
Description
Split out of #265, per that issue's own triage note ("declare both, calibrated from a real probe on lem, OR state in the hand-off that H3 is split to its own issue").
#265's original body named two repro shapes — LTX2 (fixed, both validate-time and now run-time enforced) and H3 (templates/minimax/video-with-audio-768p, 768p/345 frames, dies mid-denoise rather than at decode). The tester's verify pass (2026-09-22) confirmed the H3 half is still open: none of the shot-shape MiniMax H3 templates (video-with-audio-768p, video-with-audio, image-to-video, first-and-last-frame, reference-to-video, etc.) carry a cost array at all, so vram_estimate_errors has nothing to compare a projection against even once a vram_estimate block is added.
Why this is a separate issue rather than folded into #265's fix: the LTX2 ceiling was calibrated from a real, already-observed OOM figure (2.69 GiB needed for VAE decode conv3d at 960x544x345, 557 MiB free). No equivalent measured number exists for the H3 templates' failure mode — H3 OOMs mid-denoise, not at decode, so the mechanism (and therefore the formula's base_gb/bytes_per_voxel split) is not simply a copy of the LTX2 numbers. The closest existing real measurement in the repo is docs/WORKFLOW_GUIDE.md's on-demand-VAE table: image-to-video.json at 960x544/124 frames peaks at 18.03 GiB reserved. That is one data point for one template at one frame count — not enough to fit a two-parameter formula, and not at the 768p resolution or frame count the issue's own repro names. A trustworthy ceiling needs at least one more real data point near the failure boundary (e.g. a controlled run of video-with-audio-768p at increasing frame counts on lem, watching for the actual OOM), which is GPU time better spent as its own scoped session than folded into #265's.
Ask: on lem, run templates/minimax/video-with-audio-768p (and ideally image-to-video for a second resolution point) at a few frame counts approaching the OOM boundary, record the peak reserved VRAM at each, then declare cost ({device: "cuda", name: "RTX 3090", vram_gb: 24, minutes: <measured>}) and a vram_estimate (base_gb, bytes_per_voxel over width * height * num_frames, reason citing the calibration run) on each shot-shape H3 template that currently has neither. Wire-up (apply_vram_estimate at run time, vram_estimate_errors at validate time) already exists from #265/#266 — this issue is declaration-and-calibration only, no new engine code expected.
Depends on nothing further from #266 (already landed, offload profile stable).
Activity
fix(catalog): #324 - declare cost/vram_estimate ceiling for MiniMax H…
dkackman commentedon Sep 22, 2026
Fixed by declaring
cost/vram_estimateon the MiniMax H3 shot templates that have real measured minutes to satisfyworkflow_schema.json's "never derived" rule forcost.minutes.Calibration: two real data points fit a two-parameter linear ceiling over
width*height*num_frames:image-to-videoat 960x544x124 succeeded at 18.03 GiB peak reserved (docs/WORKFLOW_GUIDE.md, existing data point).video-with-audio-768pat 1344x768x345 (the family's ownnum_framesceiling) OOM'd mid-denoise onlem, needing >=25.82 GiB (20.25 GiB already allocated + a refused 5.57 GiB allocation) on a 23.56 GiB card (job282172105da1, 2026-09-22).base_gb=16.3,bytes_per_voxel=28.71,voxel_variables=["width","height","num_frames"].Declared on 5 templates (
workflows/templates/minimax/):video-with-audio-768p,video-with-audio,reference-to-video,enhance-prompt: added bothcost(minutes from this box's ownobserved_minutes— 9.87, 6.6, 7.88, 5.67 respectively) andvram_estimate.storyboard: already had a curatedcost(10.1 min); addedvram_estimateonly.Verified functionally (loaded
dw/vram_estimate.pydirectly and ranvram_estimate_errors()against all 5 edited files):video-with-audio-768patnum_frames=345— the issue's own OOM repro shape — now correctly refuses with"width*height*num_frames = 1344*768*345 projects to 25.82 GB VRAM, above the 24 GB declared...". No false positives on any of the 5 at their defaults or atnum_frames=345.Left undeclared (11 remaining minimax shot templates, none has a measured minutes figure — declaring
costfor them would mean fabricatingminutes, which the schema explicitly forbids):chain-matched-and-aligned,chain-matched-to-audio,chain-video-continuity,chained-segments,composable-references,enhance-prompt-with-image,first-and-last-frame,generated-subject-reference,image-to-video,last-frame-only,voice-timbre-reference. These need either real GPU calibration runs or maintainer-approved proxy minutes as a follow-up — happy to open a tracking issue if wanted, but didn't want to declare unverified numbers on this pass.Commit:
7b057b9onfix/324-h3-vram-estimate, merged todevelopatcd34174(8459606..cd34174), pushed.Deployed:
ssh lem '~/diffusers-workflow/scripts/deploy.sh develop'— systemd restart, healthy atcd34174, 2026-09-21 04:51 UTC.Worked as: model sonnet, provider anthropic.
Suggested regression case (for the tester to confirm and add, likely
regression-suite-model-specific.mdsince it's H3-specific):validate_workflowontemplates/minimax/video-with-audio-768pwitharguments={"num_frames": 345}should returnvalid: falsewith avram_estimateerror atarguments, citing ~25.82 GB projected against the 24 GB ceiling.dkackman commentedon Sep 22, 2026
Verified over MCP against lem
develop @ 8459606(deployed as cd34174 per hand-off), model opus, provider anthropic.Repro (the issue's OOM shape):
validate_workflow(name="templates/minimax/video-with-audio-768p", arguments={"num_frames": 345})→valid: false, one error at patharguments: "widthheightnum_frames = 1344768345 projects to 25.82 GB VRAM, above the 24 GB declared for RTX 3090 …" citing job282172105da1and #324 in the reason. Matches the expected shape exactly.Adjacent cases:
{}/{"num_frames": 124}→valid: true, estimate 9.9 min observed (5 runs). No false positive at the default.num_frames243, 260, 277 →valid: true; 294 →valid: falseat 24.41 GB. Cutoff lands between 277 and 294, consistent with the declared constants.templates/minimax/video-with-audioatnum_frames=345→valid: true(lower resolution stays under the ceiling; estimate 19.0 min observed).storyboarddefault →valid: true, 11.8 min.templates/minimax/image-to-videoat 345 →valid: truewith no vram error — expected, since it's one of the 11 left undeclared per the hand-off. Not a defect of this issue; the hand-off's proposed follow-up tracking issue would cover it.list_workflows(shape=shot, traits=has-audio)now showscost{cuda, RTX 3090, 24 GB}onvideo-with-audio-768p(9.87),video-with-audio(6.6),reference-to-video(7.88),enhance-prompt(5.67),storyboard(10.1).One anomaly unrelated to the fix: the very first no-argument
validate_workflowonvideo-with-audio-768p, fired in parallel with four other validate calls, came backHTTP 500: Internal Server Errorfrom/api/validate; the identical call retried sequentially (twice) returned valid. Filed separately rather than held against this issue.Regression case added to
regression-suite-model-specific.mdper the hand-off's suggestion.