feature: H3 templates - expose attention_backend + cache everywhere, and a quality tier preset that sets the coupled knobs together

Problem

To run the cinema path on dialogue-short the matrix had to rebuild the workflow (tools/ladder.py) because:

templates/minimax/dialogue-short has no attention_backend variable (its variables: width, height, num_inference_steps, video_shift, audio_shift, weights_dtype, lora_*, seed, ...). reference-to-video has one. sage_hub is lossless and 23-33% faster on H3; every H3 template that runs a transformer should expose it.
No H3 template exposes cache. first_block 0.05 is -22% at 30 base steps.
A tier is 6-8 coupled settings: canvas, weights_dtype, lora_model_name/weight_name, num_inference_steps, video_shift, audio_shift, cache, attention_backend. Getting one wrong silently degrades or re-stages (e.g. dropping the LoRA without raising steps and shift - validate warns about steps, but the agent still has to know 12/3 shift).

Proposal

Add attention_backend and cache variables to every H3 template that has a transformer step (dialogue-short, shots-batch, music-video, chains, storyboard...). Default attention could stay null for portability, but the skill should say to set sage_hub on CUDA.
Add a quality (or tier) variable whose values expand to a coherent bundle:

tier	canvas	weights	LoRA	steps	shift v/a	cache	measured min/shot
draft	960x544	int4	ref2v 4-step v0.1	5	LoRA's	none	3.5
preview	1344x768	int8	ref2v 4-step v0.1	5	LoRA's	none	7.2
cinema-draft	1344x768	int8	none	12	12/3	none	12.9
cinema	1344x768	int8	none	30	12/3	first_block 0.05	23.1

Explicit variables still override. This needs a design decision: dw variables don't have preset bundles today - options are (a) a new presets block on a template, (b) separate catalog templates per tier (dialogue-short-cinema), (c) leave it to the skill. (a) keeps one template and makes the tier visible in get_workflow.

Open questions

Should the cinema tier reduce num_frames/refs automatically when the 1344x768 ceiling bites (124 frames max with 4 refs)?
Is int8 the right default for the cinema tier on a 64 GB host? int8 H3 holds ~63 GB RSS; mini-ai/MPS can't run SDNQ anyway.

Source: H3 quality matrix on lem (2x RTX 3090 24 GB, 125 GB host RAM), workspace h3-quality, Known Issues ep 6 shots, seed 42. Full data and per-run assessments: ~/testing/opus/known-issues/quality/matrix.json, report quality/REPORT.md; every run in h3-quality carries a note with its config, minutes and score. Related: #861 (follow-up experiments).

Plan v1: H3 templates expose `attention_backend` and `cache`; no preset mechanism

claude-opus-5-5 / anthropic

Verdict: build smaller

Build part 1 of the proposal: attention_backend and cache become variables on every H3 template. Cut part 2, the quality/tier preset mechanism. Once part 1 lands, every tier in the issue's table can be reached with plain arguments. The tier table then belongs in the minimax-h3 skill as ready-to-pass argument sets, which #862's plan already proposes.

Value. The gain is Don's. The h3-quality matrix had to rebuild workflows with tools/ladder.py, because today the cinema path (sage_hub, first_block 0.05) can't be reached by arguments on dialogue-short, shots-batch, music-video, storyboard or the chains. The two knobs are measured wins on Don's own work: sage_hub is 23–33% faster and lossless, and first_block 0.05 saves 22% at 30 steps. The same matrix produced H3 quality ramp: follow-up experiments (seed stability, cross-family composition lock, int4 base draft) #861–plan.estimate prices every H3 variant at the catalog figure - measured range is 3.5-44 min #866, so every H3 episode from now on hits these knobs.
Build cost. Two stages, about $3–4 plus $4–6, so roughly $7–10 in total.
Inertia.
- Part 1: two variables on about 19 templates (compact catalog budget, below). It also widens the schema in one place: cache will accept a variable: string, the same pattern the schema already uses for other fields (dw/workflow_schema.json:104,202,212,330,553,1247). It adds no new concept, no MCP/REST surface and no security change.
- Part 2, the cut: option (a) is a new concept in the engine: a presets block in the schema (top level is additionalProperties: false, :119), expansion in _fold (dw/workflow.py:320-370), a new field on every discovery surface (dw/server/catalog.py, catalog_shape.py, routes/library.py:552-620, dw_mcp/catalog.py), the UI variable form, and tests for all of it.
- The values a preset would hold are a 3090 measurement of a fast-moving configuration. LoRAs were swapped in Add a 768p H3 path: the FL2VA turbo LoRA pinned in the templates is the 544p one, and an 8-step 768p checkpoint has existed since 2026-08-27 #148, Reference-conditioned H3 templates can drop 20 steps to 8: a Ref2VA turbo LoRA trained for transformer_ref now exists #149 and Test lightx2v's 4-step Ref2VA turbo LoRA (v0.1) for music-video renders #824. Values frozen into engine-checked template JSON go stale, and stale presets are harder to retire than stale prose.
Reversibility.
- Part 1: the new variables default to null, so behaviour is unchanged. Removing a variable later would break callers that pass it, so treat them as permanent.
- Part 2: a presets block, once tier names are depended on, could only be removed with a breaking-change.
Cheaper alternatives for the tier part:
- (c) The skill holds the tiers. This plan's choice. Each tier is an arguments object the agent passes as it stands. It gives up a tier you can see in get_workflow. The risk of mis-coupling the settings is covered by the existing 12-step warning (dw/argument_warnings.py:167) and by validate: warn on acceleration settings that can't pay off or won't fit (cache at low steps, TaylorSeer VRAM, compile under offload) #868's planned acceleration warnings.
- (b) One template per tier (dialogue-short-cinema, …): 4× the H3 catalog entries against a compact budget that has about 24 tokens of headroom (tests/test_catalog_structure.py:474-477). Rejected.
- What would bring presets (a) back: a field-report showing a session mis-coupled a tier even with the skill's table, or a second family with a measured ladder that wants the same mechanism.
Overlap.
- idea: quality-ramp skill - draft -> preview -> final tiers with measured cost/quality trade-offs (H3 first) #862 (plan-review): the quality ramp as a minimax-h3 reference. Once this feature lands, that reference can state each tier as arguments. That rewrite is idea: quality-ramp skill - draft -> preview -> final tiers with measured cost/quality trade-offs (H3 first) #862's work (or a follow-up fix), not this plan's. idea: quality-ramp skill - draft -> preview -> final tiers with measured cost/quality trade-offs (H3 first) #862 v1 already says "P1 only by save_workflow until feature: H3 templates - expose attention_backend + cache everywhere, and a quality tier preset that sets the coupled knobs together #865 lands".
- validate: warn on acceleration settings that can't pay off or won't fit (cache at low steps, TaylorSeer VRAM, compile under offload) #868: validate warnings on acceleration settings. See the follow-up below.
- docs: minimax-h3 skill + RECIPES_24GB/ACCELERATION - base-model (no LoRA, int8) quality path and measured speed levers #863: puts the matrix into minimax-h3 and RECIPES_24GB. No conflict.
- plan.estimate prices every H3 variant at the catalog figure - measured range is 3.5-44 min #866: estimate per configuration. No conflict.

Problem and non-goals

Problem. On most H3 templates you can't set the attention backend or the block cache through arguments. Reaching the measured fast and cinema configurations means editing the JSON.

Non-goals:

tier presets (cut; see the verdict);
changing any template's defaults (int4 + turbo LoRA stays the default);
automatically reducing num_frames or refs at 1344×768. The issue asks this as an open question; my answer is no, because the VRAM ceiling warning already reports it and silently changing the shot is worse;
per-device resolution of attention_backend.

The design, with corrections to the proposal

The sweep checked the proposal against develop @ 61838a9. It found these corrections:

cache can't be a variable today. The raw workflow is checked against the schema before substitution (dw/validation.py:746), and the schema requires cache to be an object with type (dw/workflow_schema.json:759-823). Only threshold takes variable:. So the proposal needs a syntax change:
- cache also accepts a ^variable: string.
- Substitution already swaps in whole objects, and a null result drops the key, which means no cache (dw/variables.py:62-132, :116-117).
- The substituted value must still be checked against the cache object schema at validate time. Otherwise {"type":"bogus"} would pass validation and fail at load, the very failure the {} rule in CLAUDE.md exists to prevent.
The engine already supports H3 caching. MiniMaxH3TransformerBlock is registered in dw/cache_blocks.json:4, and first_block defaults to 0.05 (config_objects.py:217-285). Only the wiring and the schema change.
attention_backend already exists on four templates: video-with-audio, video-with-audio-768p, image-to-video and reference-to-video. On each it defaults to null, has no constraint, and is wired to components.transformer.attention_backend (e.g. video-with-audio.json:45,135). The new ones copy that exact form.
Scope is the H3 transformer steps only:
- Z-Image still steps inside storyboard, generated-subject-reference, dialogue-short and music-video don't get the wiring.
- music.json (Music3) and music-video-cuts.json (tasks only) are out.
- compose-workflows.json passes arguments through to image-to-video and gets nothing new.
sage_hub is CUDA-only, and nothing checks it. It needs kernels plus DIFFUSERS_TRUST_REMOTE_KERNELS=true (.env.example:96-108). Validation doesn't compare the backend to the device, so a sage_hub run on mps passes validate and fails at run time. The default stays null, and the skill says "sage_hub on CUDA".

Surfaces touched:

syntax: the cache schema widening (decided here);
template variables: the two new variables;
engine: validate-time check of a substituted cache;
docs and skill.

No MCP tool, REST route or security change. Discovery already lists template variables, so the new ones show up in get_workflow with no surface change.

Stages

Stage A: `attention_backend` on every H3 transformer template

Builds: an attention_backend variable (default null) wired to components.transformer.attention_backend at every H3 ModularPipeline step of:
- first-and-last-frame, last-frame-only, chained-segments
- composable-references, chain-matched-to-audio, chain-matched-and-aligned, chain-video-continuity
- enhance-prompt, enhance-prompt-with-image, voice-timbre-reference
- generated-subject-reference, storyboard
- the for_each steps of dialogue-short, shots-batch and music-video
It touches no Z-Image steps.
Tests and docs:
- a catalog test that every H3 transformer step wires attention_backend (it pins the rule for future templates);
- raise COMPACT_BUDGET with a budget-log entry;
- docs/ACCELERATION.md:137-149 (the list of four templates becomes "every H3 template");
- docs/WORKFLOW_GUIDE.md:1490-1499 where it lists them;
- workflows/templates/minimax/README.md;
- the minimax-h3 skill line saying to set sage_hub on CUDA.
Deploy: server (catalog files on lem and mini-ai). No engine change.
Estimate: $3–4.
Acceptance intent (over MCP):
- get_workflow on each listed template shows attention_backend with default null.
- validate_workflow with no new argument, and with attention_backend: "sage_hub", is clean on each.
- A default run's get_job_workflow has no attention_backend key on the transformer.
- On lem (CUDA), a dialogue-short run with attention_backend: "sage_hub" completes. Its get_job_workflow shows components.transformer.attention_backend: "sage_hub" on the H3 step and none on the Z-Image step.
- The four templates that already had it are unchanged.

Stage B: `cache` as a variable

Builds:
- The schema accepts cache as an object (as today) or a ^variable: string.
- Validation checks the substituted value against the cache object schema. A substituted null means no cache.
- A cache variable (default null) on every template from stage A plus the four older ones, wired to the H3 transformer step's cache only.
Tests and docs:
- unit tests for: a variable: cache resolving to a valid object; resolving to null (key dropped); and resolving to an invalid object (refused at validate, naming the field);
- tests/test_catalog_speed_defaults.py:37-49 (test_turbo_schedules_carry_no_block_cache) changes to check the effective default instead of the presence of a cache key, since a variable:cache defaulting to null is still "no cache";
- catalog test: every H3 transformer step wires cache;
- raise COMPACT_BUDGET;
- docs/WORKFLOW_GUIDE.md cache section (a whole-object variable, null = off);
- docs/RECIPES_24GB.md and docs/ACCELERATION.md: replace "edit the JSON" with the argument;
- minimax-h3 skill: cache only at ≥ ~20 base steps, never with the turbo LoRAs.
Deploy: server, which needs a restart for the validation code.
Estimate: $4–6.
Acceptance intent (over MCP):
- get_workflow shows cache with default null on every H3 template.
- validate_workflow on dialogue-short:
  - with cache: {"type":"first_block","threshold":0.05}: clean;
  - with cache: {"type":"bogus"}: refused at validate, naming cache;
  - with cache: "first_block" (a string, not an object): refused at validate.
- A run with no cache argument behaves as today: no cache in get_job_workflow.
- On lem, a 30-step no-LoRA dialogue-short run with first_block 0.05 completes, and get_job_workflow shows the cache on the H3 step only.
- A hand-written workflow with a literal cache object still validates and runs as before.

Order: A, then B. B depends on A only because they edit the same files. Neither waits on another issue.

Risks

Compact budget. Two variables × ~19 templates will overshoot the budget. Each stage raises it with a log entry, and that cost is part of the inertia accepted above.
sage_hub on mps fails at run time, not at validate. The default stays null and the skill states the rule. A validate warning would be new code; see the follow-up below.
The validate-time check in stage B has to run on the substituted value in every path that validates: MCP validate_workflow, the run-time validate, and for_each members. The stage tests must cover a for_each template (dialogue-short).

Proposed follow-up (Don decides)

Warn at validate when attention_backend names a CUDA-only kernel (sage_hub, other *_hub backends) on a non-CUDA server. This fits #868's scope ("acceleration settings that can't pay off"), so I suggest adding it there rather than in a new issue.

Questions

Q1. Cut the tier presets? Default: yes, cut; the tiers live in the skill as argument sets (idea: quality-ramp skill - draft -> preview -> final tiers with measured cost/quality trade-offs (H3 first) #862). If you want (a), I add stage C: a presets schema block, expansion in _fold with explicit arguments overriding, a field on the catalog/get_workflow//variables routes and the UI form, about $8–12. That makes it a new engine concept, and you'd be approving the tier names as permanent surface.
Q2. Shape of the cache variable. Default: a whole object ({"type":…,"threshold":…} or null). This allows turning the cache off and choosing other cache types. The alternative is a cache_threshold number inside a fixed first_block object. That needs no schema change and so drops stage B's syntax work (~$2 cheaper), but it can't turn the cache off unless the engine treats 0 as off, and it pins first_block forever.
Q3. Template defaults. Default: unchanged (int4 + turbo LoRA; attention_backend and cache null). This also answers the issue's int8 question: int8 is a tier choice the skill states, not a template default, because of host RAM (bug: two int8 H3 jobs on two cards OOM-kill the host - dispatch is VRAM-aware only, and the earlier job was the one killed #869) and mps. If you'd rather default attention_backend to sage_hub, every mps run of an H3 template would fail until the per-device follow-up exists.

Reply in comments and hand back with owner:lead. Approve by adding status:plan-approved yourself. No agent may set it.

Review of plan v1: approve as written. The code facts were checked against current develop and hold:

The 19 templates match. There are 19 H3 templates with a transformer step, and only the four named already have attention_backend. music.json and music-video-cuts.json have no H3 step, and no template sets cache.
cache can't be a variable today. The raw definition is checked against the schema before substitution (dw/validation.py:746), and the schema requires cache to be an object.
A null substitution drops the key (dw/variables.py:116).
MiniMaxH3TransformerBlock is already registered in dw/cache_blocks.json.
test_turbo_schedules_carry_no_block_cache needs the planned change. It checks whether a cache key is present, so it has to check the effective default instead.

Questions

Q1: cut the presets. The tiers live in the skill as argument sets (see the idea: quality-ramp skill - draft -> preview -> final tiers with measured cost/quality trade-offs (H3 first) #862 review).
Q2: a whole-object cache variable. feature: per-phase timing and cache skip stats in get_job / job events #867's P2 adds a reason: first_block at 12 steps skipped nothing yet changed the output (PSNR 25.6 dB, where a plain rerun scores 44 dB). Just having the hook installed changes the numbers, so "threshold 0 means off" wouldn't really be off. Only null dropping the key truly turns the cache off.
Q3: keep the template defaults unchanged.

Implementation notes

Stage B's validate check should go in the existing list of checks over the expanded definition (dw/validation.py:291 onward). That list feeds MCP validate_workflow, run-time validation and for_each members alike, which covers the plan's main risk without new plumbing.
docs: minimax-h3 skill + RECIPES_24GB/ACCELERATION - base-model (no LoRA, int8) quality path and measured speed levers #863 (21a6f602) added text that stage B must update. The H3 first_block row in docs/RECIPES_24GB.md says "no template sets it today, so add it to the workflow", and plugins/dw/skills/minimax-h3/references/quality-and-speed.md describes the same JSON-edit route. Both should name the argument once stage B lands. The plan's docs/ACCELERATION.md line numbers are from before docs: minimax-h3 skill + RECIPES_24GB/ACCELERATION - base-model (no LoRA, int8) quality path and measured speed levers #863 and have shifted slightly.
Agreed: the sage_hub-on-a-non-CUDA-server validate warning belongs in validate: warn on acceleration settings that can't pay off or won't fit (cache at low steps, TaylorSeer VRAM, compile under offload) #868.

claude-opus-5-5 / anthropic, for Don

Merge feat/865-A-h3-attention-backend: #883 stage A of #865

Moved to lem's loop (Don's call): this feature's spec and verification run MiniMax H3, which can't load on mini-ai (64 GB unified memory, no SDNQ on mps). Claim swapped target:mini-ai → target:lem and backend:cuda added so the Mac loop doesn't claim it back. Stages read the parent's claim and backend. The existing spec cases stand.

docs(proposals): #865 - design record for H3 template attention_backe…

Close-out: design record filed, handing to the tester for the final check

claude-opus-5-5 / anthropic

Closed: #883 stage A (attention_backend on all 19 H3 templates, 0 bounces) and #884 stage B (cache as a whole-object variable, with validate-time cache_errors; 2 bounces: the architecture map row, and a string threshold that passed validate). Both were verified on lem.

Design record: docs/proposals/complete/h3-template-acceleration-complete.md, with a todo.md entry under Shipped since the ranking. Docs-only commit, merged to develop @ b4916c7c. Nothing deployed. The stages already updated the user-facing docs and skill.

Owed by this verify (from the spec comment): C-F417 (pending #882) expects attention_backend and cache to be refused on dialogue-short. Once #865 lands that no longer holds, so file a suite request to amend it.

verify: run every case whose source: names one of this feature's stages, and close the parent if they all pass.

Verified on lem (develop @ b4916c7): claude-opus-5-5 via anthropic, feature-parent verify.

I ran every case whose source: names #883 or #884 (C-F420 to C-F429) over MCP this session. All pass on intent.

C-F420 / C-F424 (wiring):
- All 17 shape=shot minimax entries declare attention_backend and cache.
- dialogue-short wires components.transformer_ref.attention_backend and configuration.cache on the shot for_each step only. The Z-Image steps and the episode task carry neither.
- video-with-audio and first-and-last-frame wire components.transformer.
C-F421: attention_backend_x on dialogue-short is refused at arguments.attention_backend_x (unknown variable). sage_hub with cache null validates on video-with-audio.
C-F425 / C-F426 (cache validation, with member names on dialogue-short):
- The cache shapes are refused at ...configuration.cache(.type|.threshold): {"type":"bogus"}, the bare string "first_block", {}, {"threshold":0.05}, and threshold "0.05".
- first_block 0.05 is valid, with a cache_short_schedule warning on 9 steps.
- The same refusals hold on video-with-audio and first-and-last-frame.
C-F422 / C-F427: job f24f76d9dcdb, first-and-last-frame, defaults. Succeeded; realized attention_backend: null and cache: null.
C-F423: job 0b6252688241, dialogue-short with attention_backend: sage_hub. All 5 shots and the episode succeeded on cuda:1. Realized attention_backend: "sage_hub".
C-F428: job c528260fa756, dialogue-short with cache: {"type":"first_block","threshold":0.05}, 30 steps, lora null. Succeeded. Every shot's manifest timing reports cache: first_block 0.05, 29 forwards, 10–11 skipped (hit_rate 0.345–0.379).
C-F429: job fe28835ace0a, inline video-with-audio with a literal cache and the variable removed. Succeeded with the cache_short_schedule warning; manifest cache: first_block 0.05. A literal {"type":"bogus"} is refused at steps[0].pipeline.configuration.cache.type.

Two caveats:

Wording, same as #865 stage A: attention_backend variable on every H3 transformer template #883's verify. get_job_workflow folds arguments into variables but leaves step references as "variable:attention_backend" / "variable:cache". So the literal "no key" / "value on the step" checks in C-F422/C-F423/C-F427/C-F428 are judged from realized variables and the manifest's timing.cache. That evidence is stronger than the step dict. I'm taking the wording to a suite amendment rather than counting it as a failure.
C-F428 as written (1344x768) was refused at queue time by the VRAM gate: "needs 23.9 GB … largest is … RTX 3090 (23.6 GB usable)". The refusal is the same without cache and with the default LoRA, so it is the canvas, not feature: H3 templates - expose attention_backend + cache everywhere, and a quality tier preset that sets the coupled knobs together #865. validate_workflow on the same args said valid with no VRAM warning. I'm filing that separately. C-F428 was judged on the default canvas (960x544), above.

All outputs were deleted (f24f76d9dcdb, 0b6252688241, c528260fa756, fe28835ace0a).

Navigation Menu

Repository navigation

feature: H3 templates - expose attention_backend + cache everywhere, and a quality tier preset that sets the coupled knobs together #865

Problem

Proposal

Open questions

Plan v1: H3 templates expose `attention_backend` and `cache`; no preset mechanism

Verdict: build smaller

Problem and non-goals

The design, with corrections to the proposal

Stages

Stage A: `attention_backend` on every H3 transformer template

Stage B: `cache` as a variable

Risks

Proposed follow-up (Don decides)

Questions

9 remaining items

Close-out: design record filed, handing to the tester for the final check

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

feature: H3 templates - expose attention_backend + cache everywhere, and a quality tier preset that sets the coupled knobs together #865

Description

Problem

Proposal

Open questions

Activity

dkackman commented on Oct 14, 2026

Plan v1: H3 templates expose attention_backend and cache; no preset mechanism

Verdict: build smaller

Problem and non-goals

The design, with corrections to the proposal

Stages

Stage A: attention_backend on every H3 transformer template

Stage B: cache as a variable

Risks

Proposed follow-up (Don decides)

Questions

dkackman commented on Oct 14, 2026

9 remaining items

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

Close-out: design record filed, handing to the tester for the final check

dkackman commented on Oct 14, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Plan v1: H3 templates expose `attention_backend` and `cache`; no preset mechanism

Stage A: `attention_backend` on every H3 transformer template

Stage B: `cache` as a variable