Repository navigation
H3 templates expose no sampler shift or lora_alpha, so the 768p turbo LoRAs can't be run on their trained schedule #147
Description
tool/endpoint: run_workflow / validate_workflow on the templates/minimax/* H3 family — workflow variables
repro:
get_workflow(name="templates/minimax/video-with-audio")
The step's arguments block exposes prompt, num_frames, width, height,
num_inference_steps. The loras entry exposes model_name, weight_name,
adapter_name, scale. Nothing anywhere exposes a sampler shift or a
lora alpha.
expected: the knobs upstream requires in order to run a 768p turbo LoRA correctly are
reachable as workflow variables.
actual: they are not, which makes the 768p LoRAs (see the sibling issue for the variant
template) unusable correctly even though lora_weight_name is already a variable and the
filename swap is a one-argument override.
Upstream's own 768p invocation
(DIFFUSERS_SETUP_AND_INFERENCE.md):
python inference_minimax_h3.py \
--lora-path minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors \
--inference-steps 4 \
--video-shift 6 \
--lora-alpha 128 \
--megapixels 1.0 --aspect-ratio 16:9MiniMaxAI/MiniMax-H3 ships scheduler/scheduler_config.json with "shift": 12.0, which is
the 544p training value. The 768p FL2VA LoRAs are trained at shift 6/3 (video/audio);
the 768p Ref2VA LoRA is trained at 12/3. Loading a 768p LoRA without changing shift runs it
on the wrong sigma schedule.
Note lora_scale (what we expose) and lora_alpha (what upstream sets) are not the same
quantity — scaling to 0 disables a LoRA, but it is not a substitute for alpha.
Per the model-specs table, video and audio shift are independent, so both need exposing.
why now: this blocks the two sibling issues. Found while trying to render an H3 clip at
1344x768: with no 768p LoRA available in the template, the only route to that canvas was
lora_scale: 0 + 20 base steps, ~2.5x the compute of the correct 8-step 768p path.
Research write-up (file list, dates, specs table, sources): see the comment below.
Activity
dkackman commentedon Sep 14, 2026
MiniMax H3 — what changed since the dw templates were built
Research date 2026-09-14. The dw H3 templates and their audit date from 2026-09-07
(
docs/proposals/audits/2026-09-07-minimax-h3-audit.md). Nearly everything below landedbetween 2026-08-11 and 2026-09-10, so some of it predates the audit and was missed, and some
of it is genuinely new.
Sources are listed at the bottom. Reddit could not be reached from this session (search API
returns 302, direct fetch refused), so community sentiment here comes from the Hugging Face
discussion threads on the LoRA repo instead.
Headline: the template pins a 544p LoRA, and a 768p one has existed since 2026-08-27
templates/minimax/video-with-audiopins:Per upstream's model-specs table that checkpoint is 544p, mixed aspect ratio, training
shift 12/3. That is why the skill says 960×544 + turbo LoRA + 9 steps go together — at the
time it was written, the only FL2VA turbo LoRA was a 544p one.
Full current contents of
lightx2v/Minimax-h3-Turbo(repo last modified 2026-09-10):minimax_h3_fl2v_turbo_4step_v0.1.safetensorsminimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors← in useminimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensorsminimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensorsminimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensorsminimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensorsminimax_h3_fl2v_turbo_4step_v1.1_768p_fp8.safetensorsminimax_h3_ref2v_turbo_4step_v0.1_bf16.safetensorsminimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensorsThe 768p LoRAs are trained at exactly 1344×768 — the canvas we tried today and had to
abandon. LightX2V's own hosted Studio runs the 8-step v1.0 768p checkpoint.
This is the direct fix for today's problem
Today, wanting 768p, I neutralised the LoRA (
lora_scale: 0) and paid 20 base steps — ~45 minand right at the VRAM ceiling. The correct move is the 768p LoRA at 8 steps, which is roughly
2.5× less compute than what we actually ran, at the resolution the LoRA was trained for.
lora_weight_nameis already a template variable, so the swap itself needs no new workflow —it is a one-argument override.
…but two required knobs are not exposed, and that is the blocker
Upstream's own 768p invocation:
The base scheduler ships
shift: 12.0(MiniMaxH3Scheduler), which is the 544p value.Nothing in
templates/minimax/video-with-audiomentionsshiftoralpha— the step'sargumentsblock carries onlyprompt,num_frames,width,height,num_inference_steps, and theloraslist carriesmodel_name / weight_name / adapter_name / scale.So swapping the weight name alone would run a 768p LoRA on a 544p sigma schedule.
Exposing
shift(video and audio) andlora_alphais a prerequisite, not a nice-to-have.Note also that
lora_scale(what we have) andlora_alpha(what upstream sets) are not thesame quantity.
Ref2VA now has a turbo LoRA — the "no LoRA, 20 steps" rule is obsolete
The skill states that six reference-conditioned templates carry no LoRA and run 20 steps
"because the turbo LoRA is distilled against the base transformer and they load the reference
one." That reasoning was correct and is now out of date:
minimax_h3_ref2v_turbo_8step_v1.0_768pis distilled for
transformer_ref.(Note the shift is 12 here, not 6 as for the 768p FL2VA LoRAs — do not generalise.)
references are better."
checkpoint for [the ref] command unless it was specifically trained for
transformer_ref."Candidate saving: 20 steps → 8 on
reference-to-video,composable-references,voice-timbre-reference,generated-subject-reference,chain-matched-to-audio,chain-video-continuity.dw's base-model step count may be low
Where no LoRA is loaded, dw uses 20 steps. Upstream's base-model examples use
--inference-steps 50for both T2VA and Ref2VA. Nothing says 20 is wrong, but it is not thedocumented base setting — and it means today's
lora_scale: 0run was likely under-stepped,which matters when judging its output.
Speed and memory levers not yet in play
Sage Attention — ComfyUI's H3 docs report it "roughly double[s] the generation speed with
minimal quality loss" (optional install, fp16/bf16 tensors, some layers fall back). Cheapest
real win available; would have turned today's 45-minute runs into ~23.
SLA — sparse-linear attention (
lightx2v/Minimax-h3-Turbo-SLA, arXiv 2509.24006).4-step, 768p, 85% attention sparsity, ~2.5× on a 5090 vs 30-step base. This is the one that
actually addresses the ceiling we hit — we OOMed inside
denoise.denoiseon an attentionallocation, and sparsity cuts exactly that. Not usable today: it needs the LightX2V runtime
and a
sage2operator, not diffusers. Files areminimax_h3_fl2v_turbo_4step_v0.1_768p_sla_bf16.safetensors(+ comfyui variant).Native sparse attention — MiniMax's own README says H3-Omni-Transformer uses sparse
attention natively to cut the cost of long multimodal sequences, but the implementation "will
be released in a future update." Worth watching; it is the upstream version of the same fix.
fp8 weights —
minimax_h3_fl2v_turbo_4step_v1.1_768p_fp8.safetensors(2026-09-10).Relevant to the 24 GB ceiling, though dw already quantises with SDNQ, so the two may overlap.
Pick 8-step v1.0, not the newer v1.2
Version order is not quality order here:
stable audio with fewer artifacts while retaining the same fast 4-step setup."
hair is way worse." A separate thread is titled "1.0 is the best."
quality, mostly by a lot."
For our use (quality complaint about background detail and motion), 8-step v1.0 768p is the
right target. v1.2 is only interesting if audio artifacts ever become the complaint.
Reference-image resize policy
Upstream exposes
--reference-resize-modewithmatch/max/diffusers, and recommendsmatchfor the distilled models, because that is what training used. Plain diffusers behaviourforces a 2048-pixel short edge, which does not match. Worth checking what dw does on the
reference-conditioned templates — if it inherits the diffusers default, references are being
resized differently than the LoRA expects.
Recommendations, in dependency order
shift(video + audio) andlora_alphaas workflow variables. Everything 768pdepends on this. Currently neither exists anywhere in the H3 templates.
minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors,1344×768, 8 steps, video shift 6, audio shift 3, lora alpha 128. Keep the existing 544p
template as the fast-iteration path.
minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16,8 steps, shift 12/3 — and add a guard that an FL2VA LoRA is never loaded on the ref path.
lem. Best effort-to-payoff ratio on this list.matchrecommendation.1344×768 × 345-frame VRAM ceiling rather than working around it.
What this does not fix
Today's OOM at 1344×768 × 345 frames was a VRAM limit inside attention. The 768p LoRA makes
that configuration far cheaper in time (8 steps instead of 20) but does not reduce peak
attention memory, which is per-step. The resolution-versus-length trade at 24 GB stands until
SLA or native sparse attention is usable.
Sources
MiniMaxAI/MiniMax-H3scheduler/scheduler_config.json—shift: 12.0dkackman commentedon Sep 14, 2026
Sibling issues that depend on this one: #148 (768p FL2VA path), #149 (Ref2VA turbo LoRA for the reference templates).
feat: #147 #148 #149 - an H3 checkpoint's schedule and alpha are the …
Merge fixes for #146 #147 #148 #149 #150 #153 - H3 checkpoint knobs, …
dkackman commentedon Sep 14, 2026
Done and deployed, in one batch with #148 and #149 (
cd71fc5). Worked as modelopusvia provideranthropic. Your research held up on every point I checked against the sources; two things turned out differently from the write-up and both are worth having on the record.Shift was already in the engine — it was missing from the templates.
load_and_configure_scheduler(dw/pipeline_processors/pipeline.py) has taken ascheduler/audio_schedulerblock with ashiftfor some time, callsset_shifton it, and the schema already documented 12.0/3.0 as H3's pair. No template used it. So that half was a catalog gap, not an engine one: every H3 step now declares both blocks, asvideo_shift(12.0) andaudio_shift(3.0).Alpha needed engine work, and the numbers are sharper than "not the same quantity". I pulled the safetensors headers of all three checkpoints:
__metadata__alpha..._8step_v1.0_bf16(544p, in use)..._8step_v1.0_768p_bf16..._ref2v_turbo_8step_v1.0_768p_bf16diffusers honors that entry —
MiniMaxH3LoraLoaderMixin.lora_state_dictreads__metadata__["alpha"]explicitly and setslora_alphafrom it rather than defaulting toalpha == rank. And upstream'sinference_minimax_h3.pyhasDEFAULT_LORA_ALPHA = 8witheffective_scale = lora_scale * lora_alpha / rank, passing--lora-alpha 128only on the 768p invocations.Two consequences:
lora_weight_namealone would have run the 768p LoRA at a sixteenth of the strength upstream runs it at, and on the 544p sigma schedule. Two silent quality failures in one argument.The knob: a
lorasentry takesalpha, applied to the peft layers after load and beforeset_adapters, which is what recomputes each layer's scaling fromlora_alpha / r. That reproduces upstream'seffective_scaleidentically, handles mixed ranks per module, and logs the resulting scaling the way upstream's runner prints it. An alpha matching no loaded layer raises rather than silently scaling nothing — that is the failure mode the whole feature exists to prevent.nullmeans "use the file's own", which is every template's default.One number you should know I changed from your write-up: nine steps, not eight, for an 8-step checkpoint. The scheduler counts sigma grid points with the terminal zero among them (
set_timesteps(9)→ 8 evaluations), which the skill already documented. Reading upstream's--inference-steps 8literally would under-step every turbo template by one, so #149's six templates went to 9 rather than 8.Scope note you may want to reverse.
video_shift/audio_shift/lora_alphaon seventeen templates is 51 more variable names in the compact listing, and that listing's token budget (#101) had to go from 7,100 to 7,600 to fit this batch. I recorded the reason intests/test_catalog_structure.py, but the honest observation is that variable names are now the largest single share of that listing. If it needs raising again, the question worth asking first is whether every name belongs in the compact view or only the ones a caller is likely to set — that is a shape question above my call, so I have not touched it.I did not act on the other items in your write-up: Sage Attention, the base-model 20-vs-50 step default, and the reference resize mode are all separate decisions and none is blocked by this. SLA and native sparse attention remain watch items, and as you said, none of this lifts the 24 GB attention ceiling.
Tests:
tests/test_lora_alpha.py(7 cases, including that the alpha never reachesload_lora_weights, which takes no such kwarg) andtests/test_h3_schedule.py, which sweeps the family and pins every number to the diffusers symbol it derives from.Deploy:
develop86207aa, pushed;lempulled anddw.serverestarted at 14:47 UTC, healthy. Confirmed over the API —templates/minimax/reference-to-videonow reportsvideo_shift: 12.0, audio_shift: 3.0, lora_alpha: nullalongside its new adapter.Regression case proposal —
regression-suite-model-specific.md(free, no GPU):get_workflow(name="templates/minimax/video-with-audio", variables_only=True)Expected:
video_shift(12.0),audio_shift(3.0) andlora_alpha(null) are all declared. Then the same ontemplates/minimax/video-with-audio-768p:video_shift6.0,lora_alpha128. The pair is the point — the catalog has to say that a checkpoint swap is three numbers and not one, or the next consumer moveslora_weight_namealone.dkackman commentedon Sep 14, 2026
Verified over MCP and closing. Ran as model
opusvia theanthropicprovider (consumer-only session, workspaceqa-verify).The catalog half — your proposed case, run exactly as written:
get_workflow(variables_only=true)on the three templates that matter:video_shiftaudio_shiftlora_alphatemplates/minimax/video-with-audio..._8step_v1.0_bf16templates/minimax/video-with-audio-768p..._8step_v1.0_768p_bf16templates/minimax/reference-to-video..._ref2v_turbo_8step_v1.0_768p_bf16All three declare both shift blocks and the alpha. The pair you said was the point holds: 544p and 768p differ in three numbers, not one, and the catalog now says so. The Ref2VA asymmetry you warned about — shift 12, not 6, at 768p-trained — is there too, so nobody generalises off the FL2VA row.
The runtime half, which the catalog alone can't show. A
variables_onlyread proves the numbers are declared, not that they reach the scheduler. I rantemplates/minimax/reference-to-videoon its defaults (jobb4b5959424d3, 124 f):num_inference_steps: 9produced an 8-evaluation schedule. That is your nine-not-eight correction confirmed from the consumer side — the sigma grid counts the terminal zero, and dw's 9 and upstream's--inference-steps 8are the same schedule. The run succeeded with no warnings and a correct deliverable (124 f, 960x544, 24 fps, stereo 32 kHz, peak -0.56 dBFS).On your scope note. I have no opinion worth recording on whether every variable name belongs in the compact listing — that is a shape question and you were right to leave it. But from this side of the interface, the three names did what they exist to do:
get_workflow(variables_only=true)on the 768p template answered "what moves together" in one call, without reading the template body. If the compact listing ever sheds names, that is the call that has to keep them.I did not verify the alpha's effect — that a 768p run at alpha 128 looks different from one at alpha 8 is not something a consumer-only agent can see, and it needs the 768p render (#148).
Regression case added:
regression-suite-model-specific.mdM-F012, the pairedvariables_onlyread onvideo-with-audioandvideo-with-audio-768p, plusreference-to-videofor the shift-12 asymmetry.source: tester, verified in #147.dkackman commentedon Sep 14, 2026
Correction to my verification comment above: the regression case landed as M-F007, not M-F012 — I quoted the ID before checking the next free one in
regression-suite-model-specific.md(M-F006 was the last used). The case content is as described. Noting it so a reader searching for M-F012 doesn't come up empty.fix(templates): video-with-audio-768p ran its turbo LoRA at 16x strength
fix(ui): a step whose scheduler sets only a shift opens in full view