Repository navigation
#865 stage A: attention_backend variable on every H3 transformer template #883
Description
Stage A of #865. Builds the plan's Stage A section; the plan comment stays authoritative: #865 (comment) (plan v1, approved).
claude-opus-5-5 / anthropic
Builds
An attention_backend variable (default null, no constraint) wired to components.transformer.attention_backend at every H3 ModularPipeline step, copying the exact form the four templates that already have it use (e.g. video-with-audio.json). Templates:
first-and-last-frame,last-frame-only,chained-segmentscomposable-references,chain-matched-to-audio,chain-matched-and-aligned,chain-video-continuityenhance-prompt,enhance-prompt-with-image,voice-timbre-referencegenerated-subject-reference,storyboard- the
for_eachsteps ofdialogue-short,shots-batchandmusic-video
Not touched: Z-Image steps inside storyboard, generated-subject-reference, dialogue-short, music-video; music.json, music-video-cuts.json, compose-workflows.json. The four templates that already have it (video-with-audio, video-with-audio-768p, image-to-video, reference-to-video) stay unchanged. No engine change, no template default changes.
Owes
- a catalog test that every H3 transformer step wires
attention_backend(pins the rule for future templates); - raise
COMPACT_BUDGET(tests/test_catalog_structure.py) with a budget-log entry; docs/ACCELERATION.md(the list of four templates becomes "every H3 template"; line numbers shifted after docs: minimax-h3 skill + RECIPES_24GB/ACCELERATION - base-model (no LoRA, int8) quality path and measured speed levers #863);docs/WORKFLOW_GUIDE.mdwhere it lists them;workflows/templates/minimax/README.md;- the
minimax-h3skill: setsage_hubon CUDA (needskernels+DIFFUSERS_TRUST_REMOTE_KERNELS=true), leavenullelsewhere.
Deploy
Server: catalog files on lem and mini-ai. No engine change.
Passes when
Its repro works (on lem, dialogue-short with attention_backend: "sage_hub" completes and get_job_workflow shows components.transformer.attention_backend: "sage_hub" on the H3 step and none on the Z-Image step) and every pending: #<this issue> case the tester writes passes.
Blocked by: nothing.
Activity
style: #883 - ruff format
docs: #883 - attention_backend on every H3 template in ACCELERATION, …
Merge feat/865-A-h3-attention-backend: #883 stage A of #865
dkackman commentedon Oct 14, 2026
Stage A built: merged to
develop@ 54f0e9eclaude-opus-5-5 / anthropic
Shipped: branch
feat/865-A-h3-attention-backend, merged intodevelopas 54f0e9e. This is a catalog and docs change only, with no engine code. The driver deploysdevelopto mini-ai after this session. Nothing was deployed from here.Plan items covered:
attention_backendvariable (defaultnull, no constraint) on all 15 listed templates. It is wired as"attention_backend": "variable:attention_backend"at the H3 step's transformer component, in the same form asvideo-with-audio.json. The 15:first-and-last-frame,last-frame-only,chained-segments,composable-references,chain-matched-to-audio,chain-matched-and-aligned,chain-video-continuity,enhance-prompt,enhance-prompt-with-image,voice-timbre-reference,generated-subject-reference,storyboard, and theshotfor_each step ofdialogue-short,shots-batchandmusic-video.write_songstep inmusic-video,music.json,music-video-cuts.jsonandcompose-workflows.json;tests/test_catalog_structure.py:test_every_h3_transformer_step_wires_attention_backend, parametrized over every template with aMiniMaxAI/MiniMax-H3step (19 of them);test_only_h3_steps_wire_attention_backend;replace_variablesondialogue-short. Withsage_hub, only the H3shotstep gets the backend. With the default, the key is dropped.COMPACT_BUDGETwent from 10_700 to 10_800, with a budget-log entry. Measured at 10_758, so the 15 templates added about 79 tokens.docs/ACCELERATION.md: the list of four templates now reads "every MiniMax-H3 template".docs/WORKFLOW_GUIDE.md: the component-levelattention_backendbullet says so too. The guide had no list of templates; lines 1490-1499 cover the pipeline-level key.workflows/templates/minimax/README.md.minimax-h3skill:SKILL.mdgets the rule "CUDA:attention_backend=sage_hub(null elsewhere; mps fails at run)".references/quality-and-speed.mdgets the detail:kernelsandDIFFUSERS_TRUST_REMOTE_KERNELS=true.SKILL.mdwas 6 bytes under its 12 KiB cap. To make room I cut one non-rule sentence from the chain bullet ("The guide prefix is trimmed off again…").Deviation: tell the tester. The plan and the "passes when" line say
components.transformer.attention_backend. That holds for the t2va and fl2va templates. On ref2va templates the H3 transformer component istransformer_ref, and the backend is wired there, exactly as the existingreference-to-video.jsondoes. That coversdialogue-short,music-video,storyboard,composable-references, the threechain-*templates,generated-subject-referenceandvoice-timbre-reference. So ondialogue-short,get_job_workflowshowscomponents.transformer_ref.attention_backend: "sage_hub"on theshotstep.Hardware:
validate_workflowwithattention_backend: "sage_hub"is expected to be clean on mini-ai, because validation doesn't check the backend against the device. That gap is the plan's proposed validate: warn on acceleration settings that can't pay off or won't fit (cache at low steps, TaylorSeer VRAM, compile under offload) #868 follow-up.dialogue-shortsage_hub run in "passes when" can only be verified on CUDA (lem).Tests: the full
pytesthas the same 17 failures asdevelopbefore this branch (Kandinsky 6 and others). There are no new ones, andruff checkis clean.dkackman commentedon Oct 14, 2026
Architecture review queued by the loop driver: this hand-off changed
dw/ordw_mcp/ondevelop(f8fda594ea..dfc7c49c4a: dw/references.py dw/security.py). The reviewer checks it againstdocs/ARCHITECTURE.mdbefore the tester verifies.dkackman commentedon Oct 14, 2026
Architecture review: pass
claude-opus-5-5 / anthropic
What I read:
git diff f8fda594..dfc7c49c -- dw/ dw_mcp/(dw/references.py,dw/security.py) andgit log f8fda594..dfc7c49c -- dw/ dw_mcp/.Result: none of this issue's commits (4515906, 0ec85cb, 5e7b0cc, merge 54f0e9e) touch
dw/ordw_mcp/. The only engine commit in the range is c467a54, which is #873's (one owner of the variable-name characters), so this review doesn't cover it. #883 has no engine code to check againstdocs/ARCHITECTURE.mdor stage-c item 3, so there are no findings and nothing waived.The tester verifies next, including the
transformer_refdeviation on the ref2va templates that the build comment flagged.dkackman commentedon Oct 14, 2026
Verify pass on mps on mini-ai.lan (develop @ dfc7c49). Tester: claude-opus-5-5 via anthropic.
The stage's
pending: #883cases, run as written:list_workflowslistsattention_backendon all 19 H3 templates.get_workflowshows each one declares it with defaultnull, and wires"attention_backend": "variable:attention_backend"into exactly the H3 step's transformer. The ref2va templates wire it atcomponents.transformer_ref, the t2va/fl2va ones atcomponents.transformer, as the hand-off says. No Z-Image,write_song,generate_speechorprompt_enhancerstep references it.validate_workflowreturnedvalid: trueon all 19 templates, both with<base>alone and with<base>+{"attention_backend": "sage_hub"}.<base>used the priya qa-cast portrait/voice assets for each template's required media inputs. No error mentionedattention_backend; mps gave no CUDA warning, and the case doesn't require one. Step 3,dialogue-short+{"attention_backend_x": "sage_hub"}: refused atarguments.attention_backend_x, "Unknown variable 'attention_backend_x'; declared variables: attention_backend, ...".run_workflow(templates/minimax/first-and-last-frame, {first/last image = asset:qa-cast/priya-portrait.jpg, width 512, height 288, num_frames 124, seed 42})started as joba1264f45fe71. It sat in phaseloading/pipeline: MiniMaxAI/MiniMax-H3for over 18 minutes, emitting onlyphase_stallnotices.validate_workflowreportsdownloads_requiredfor this server:MiniMaxAI/MiniMax-H3464.2 GB andlightx2v/Minimax-h3-Turbo54.7 GB. That model doesn't fit mini-ai's 64 GB of unified memory. I calledcancel_job. It hadn't taken effect by the end of the session, because loading isn't a cancel boundary. The job and any partial run dir20261014-031505-9a0fa304may still be on the server.The issue's own repro (
dialogue-shortwithsage_hubon CUDA) and C-F422/C-F423 need lem. Handing to lem's loop. owner:tester and status:fixed-pending-verify are unchanged, and thepending: #883lines stay until C-F422/C-F423 pass there.Note for lem's tester: the C-F420/C-F422/C-F423 text says
components.transformer. On the ref2va templates the wiring is atcomponents.transformer_ref(dialogue-short is one of them), so read C-F423's "H3 step" check againsttransformer_ref. I'm not editing the cases from here.dkackman commentedon Oct 14, 2026
Verified on lem (CUDA): pass
Tester: claude-opus-5-5 via anthropic. lem on develop @ 7e1bfdf,
get_server_infodevicecuda.This completes the mini-ai pass above, which covered C-F420 and C-F421. It runs the CUDA half, workspace
regression-complete.C-F422, the default run.
run_workflow(templates/minimax/first-and-last-frame, {image, last_image = asset:qa-cast/priya-portrait.jpg, width 512, height 288, num_frames 124, seed 42})gave job6211d8075e4e, which succeeded on cuda:0 in about 3 minutes.get_job_workflowreturnsvariables.attention_backend: null. The H3 step'scomponents.transformerholds"attention_backend": "variable:attention_backend", which is the template's reference and not a substituted value.get_job_workflowrealizes a run by folding the arguments intovariablesand leaves step references unsubstituted, so the case's literal check ("noattention_backendkey at all") can't be observed from this tool. What I could observe: the null default doesn't break the run.C-F423, the issue's own repro.
run_workflow(templates/minimax/dialogue-short, {character_a_voice/character_b_voice = asset:qa-cast/priya-voice.wav, attention_backend: "sage_hub", seed 42})gave job4637d362563a, which succeeded on cuda:1. It took 1687 s (about 28 min), against the 38.6 min the observed estimate gave. That is consistent with a faster attention path, but it doesn't prove it. Inget_job_workflow:variables.attention_backendis"sage_hub".shotfor_each step (MiniMax-H3, ref2va) wires it atcomponents.transformer_ref.attention_backend: "variable:attention_backend". That istransformer_ref, as the build comment's deviation note says.draw_character_aanddraw_character_b(Z-Image) carry noattention_backend.episodetask doesn't either.Neither job's
log/warningevents name the attention backend, so there is no direct runtime confirmation thatsage_hubwas applied, only that the run accepted it and completed. I'm filing that observability gap separately; it doesn't block this stage.Both runs were removed with
delete_output(job_id=…).Result: the stage's repro and all four
pending: #883cases pass on intent. I removed theirpending:lines. The case text for C-F420, C-F422 and C-F423 sayscomponents.transformerand expects a substituted key inget_job_workflow. I'm sending that wording through a suite amendment on the harness repo rather than editing the cases. Closing as verified.