#865 stage B: cache as a whole-object variable on every H3 template

Stage B of #865. Builds the plan's Stage B section; the plan comment stays authoritative: #865 (comment) (plan v1, approved).

claude-opus-5-5 / anthropic

Builds

The schema (dw/workflow_schema.json, cache definition) accepts cache as an object (as today) or a ^variable: string.
Validation checks the substituted value against the cache object schema; a substituted null drops the key (no cache). Per Don's review, put this check in the existing list of checks over the expanded definition (dw/validation.py:291 onward), which feeds MCP validate_workflow, run-time validation and for_each members alike.
A whole-object cache variable (default null) on every template from stage A plus the four older ones (video-with-audio, video-with-audio-768p, image-to-video, reference-to-video), wired to the H3 transformer step's cache only. Template defaults unchanged.

Decided in review (Q2): whole object, not a cache_threshold number. null is the only true "off", since an installed first_block hook changes output even when it skips nothing (#867 P2).

Owes

unit tests: a variable: cache resolving to a valid object; resolving to null (key dropped); resolving to an invalid object (refused at validate, naming the field); cover a for_each template (dialogue-short);
tests/test_catalog_speed_defaults.py test_turbo_schedules_carry_no_block_cache: check the effective default, not the presence of a cache key;
catalog test: every H3 transformer step wires cache;
raise COMPACT_BUDGET with a budget-log entry;
docs/WORKFLOW_GUIDE.md cache section (whole-object variable, null = off);
docs/RECIPES_24GB.md H3 first_block row and docs/ACCELERATION.md: replace "no template sets it today, add it to the workflow" with the argument;
plugins/dw/skills/minimax-h3/references/quality-and-speed.md: name the argument instead of the JSON-edit route (docs: minimax-h3 skill + RECIPES_24GB/ACCELERATION - base-model (no LoRA, int8) quality path and measured speed levers #863 text); cache only at ≥ ~20 base steps, never with the turbo LoRAs.

Deploy

Server, restart needed (validation code).

Passes when

Its repro works (on lem, a 30-step no-LoRA dialogue-short run with cache: {"type":"first_block","threshold":0.05} completes and get_job_workflow shows the cache on the H3 step only; cache: {"type":"bogus"} and cache: "first_block" are refused at validate) and every pending: #<this issue> case the tester writes passes.

Blocked by: stage A (same files).

feat(validation): #884 - cache variable tests, H3 wiring catalog test…

Built and deployed: stage B (cache as a whole-object variable on every H3 template)

Shipped: develop @ b692c402, which merges feat/865-B-h3-cache-variable (commits 4045d2b and the follow-up tests/docs commit). Deploy line: deployed develop @ b692c402, healthy after 5s (server restarted, so the new validation is live).

What the plan asks for and where it is:

The schema takes cache as an object or a ^variable: string. In dw/workflow_schema.json, pipeline_configuration.cache is now "type": ["object","string"] with "pattern": "^variable:". The object properties and required: ["type"] are unchanged, so a literal string such as "first_block" still fails the raw schema.
The substituted value is checked against the cache object schema in ERROR_CHECKS.
- cache_errors is in dw/step_value_checks.py. It is registered as Check("cache", …) after fps in dw/validation.py and runs over the substituted, for_each-expanded definition.
- A non-object value is refused at steps.<step>.pipeline.configuration.cache, with an example of the object it expects. An invalid object is refused at the field inside it, e.g. .cache.type.
- A substituted null drops the key, via the existing replace_variables behavior, so no cache is built.
All 19 H3 templates wired. Each gets a cache variable defaulting to null, and "cache": "variable:cache" in the H3 step's pipeline.configuration only. Template defaults are unchanged.
Unit tests:
- tests/test_validation.py::TestCacheVariable covers: a valid object, null dropping the key, a bogus type at .cache.type, a bare string, a missing type, an invalid default, a literal string failing the schema, and a literal object validating.
- TestCacheVariableOnAForEachTemplate runs on dialogue-short: valid is clean, and a bogus value is reported once and names the member.
The turbo test checks the effective default. test_catalog_speed_defaults.py::test_turbo_schedules_carry_no_block_cache now resolves variable: to the variable's default.
Catalog tests. test_catalog_structure.py checks that every H3 transformer step wires cache, that only H3 steps do, that cache reaches only the H3 shot, and that the default drops the key.
COMPACT_BUDGET went from 10_800 to 10_850, with a log entry. The measured cost is about 42 tokens.
Docs:
- WORKFLOW_GUIDE (the cache section), RECIPES_24GB (the H3 row), ACCELERATION, and plugins/dw/skills/minimax-h3/references/quality-and-speed.md. The skill reference says to use the cache only at about 20 or more base steps and never with the turbo LoRAs, and that null is off.
- The minimax templates README also gets a sentence on cache.

Deviations, so none of these reads as a bug:

One error per for_each step, not per member. A for_each step's members share one written path, so an invalid cache on a for_each step is reported once, with member '<name>@…' in the message.
One extra entry in LEGITIMATE_MENTIONS. The description of workflows/templates/step-caching.json mentions "cache", which is now a catalog variable name, so it is listed there.
No breaking-change label. The schema only widens, and every template default is unchanged.

Local test run: the full suite has 17 failures, all in test_examples, test_component_type_errors and test_plugin_skills (Kandinsky6). The same tests fail with this stage's changes stashed, so they predate it. They are not from this stage.

claude-opus-5-5 / anthropic

Architecture review queued by the loop driver: this hand-off changed dw/ or dw_mcp/ on develop (31e4501304..b692c402b9: dw/step_value_checks.py dw/validation.py dw/workflow_schema.json). The reviewer checks it against docs/ARCHITECTURE.md before the tester verifies.

Follow-up: the stage repro run passed (C-F428 at 960x544). Tester: claude-opus-5-5 / anthropic.

Job a104407184ad: dialogue-short with lora_model_name: null, 30 steps, video_shift 12, audio_shift 3, 960x544, cache: {"type":"first_block","threshold":0.05}, seed 42.

It succeeded in about 67 min.
All 5 shots ran passes: 29.
Steady-state pass_s was 27.3 s once the model was resident.
From get_job_workflow (realized: true):
- the shot for_each step's pipeline.configuration.cache is "variable:cache", and the variable is pinned to {"type":"first_block","threshold":0.05};
- draw_character_a and draw_character_b (Z-Image) have no cache key.
I deleted the output with delete_output(job_id="a104407184ad").

This doesn't change the bounce. The only failure is still C-F425 and C-F426 value 7: a string threshold is accepted at validate.

Bounce resolved: a string cache threshold is refused at validate (stage B, third hand-off) (claude-opus-5-5 / anthropic)

Shipped: develop @ 340e31d1. That merges feat/865-B-cache-threshold-type (commit e07fbf38) and the other work already on origin/develop. Deploy line: deployed develop @ 340e31d1, healthy after 7s. The server restarted, so the check is live.

The bounce (C-F425 / C-F426 value 7). The tester's diagnosis was right. The whole-object check reused the schema's threshold type, ["number","string"], and the string branch exists only so a raw document can say "threshold": "variable:…".

By the time this check runs, every reference has been substituted. _cache_object_schema() in dw/step_value_checks.py now narrows any cache field typed number-or-string to its numeric type only. Today that field is threshold.
{"type":"first_block","threshold":"0.05"} and {"type":"first_block","threshold":"abc"} are now refused at steps.<step>.pipeline.configuration.cache.threshold. On a for_each step it is reported once and names the member.
The raw schema is unchanged, so a literal cache object with "threshold": "variable:t" still validates.

Tests. In tests/test_validation.py:

TestCacheVariable::test_a_string_threshold_is_refused_at_the_field covers "0.05" and "abc".
test_a_threshold_variable_in_a_literal_object_still_validates covers a literal object whose threshold is a variable.
TestCacheVariableOnAForEachTemplate::test_a_string_threshold_is_refused_once runs on dialogue-short.

Gate. ruff is clean. The full pytest has 17 failures, and the same 17 fail on 92c3952f without this change: Kandinsky6 / component-type tests that depend on the local diffusers install. None of them is in the cache tests.

Nothing else in the stage changed. The first hand-off comment still describes the rest of the stage, including its deviations.

Architecture review queued by the loop driver: this hand-off changed dw/ or dw_mcp/ on develop (e37f2fbdc8..340e31d1a3: dw/step_value_checks.py). The reviewer checks it against docs/ARCHITECTURE.md before the tester verifies.

Architecture review: passed, no findings (claude-opus-5-5 / anthropic)

What I read: git diff e37f2fbd..340e31d1 -- dw/ dw_mcp/. The only file in the range is dw/step_value_checks.py, changed by commit e07fbf38 (#884): eight lines in _cache_object_schema(). I checked them against docs/ARCHITECTURE.md and item 3 of docs/stabilization/harness/stage-c-guardrails.md.

Second owner: none. The change narrows a number-or-string cache field to its numeric type inside the derived copy of the schema's cache shape. It still reads that shape from load_schema and doesn't restate it. It stays inside cache_errors, which the Step-value checks row already assigns to this module. It doesn't re-derive the variable: prefix, and substitution stays with dw/variables.py.
Build vs. buy: none. Validation still goes through the existing jsonschema wrapper.
Map: no update owed. No owner was added and no rule moved. The Step-value checks row already names cache_errors, since the earlier bounce was fixed.
Context: fine. The new comment explains the narrowing, and no CLAUDE.md changed.

The 340e31d1 merge brings in no other dw/ or dw_mcp/ change. The hand-off can go to verify.

Verified: stage #884 against develop @ 340e31d. Every pending: #884 case passes. Tester: claude-opus-5-5 / anthropic, MCP calls only, workspace regression-complete.

The bounce (C-F425 / C-F426, value 7): fixed

dialogue-short. validate_workflow(name="templates/minimax/dialogue-short", arguments={"character_a_voice":"asset:qa-cast/priya-voice.wav","character_b_voice":"asset:qa-cast/priya-voice.wav","cache":{"type":"first_block","threshold":"0.05"}}) returned valid: false, refused at steps[2].pipeline.configuration.cache.threshold: "Invalid cache: '0.05' is not of type 'number' in member 'shot@cold_open'". There is one error, not one per member.
video-with-audio. {"type":"first_block","threshold":"0.05"} is refused at steps[0].pipeline.configuration.cache.threshold.
first-and-last-frame. Both "0.05" and "abc" are refused at the same path.
Values that should still pass do. On dialogue-short and video-with-audio, {"type":"first_block","threshold":0.05} validates, with only the existing cache_short_schedule warning. cache: null validates on video-with-audio.

The cases not run in the last verify

C-F427 passes. Job 8fdddc394eed ran templates/minimax/first-and-last-frame at 512x288, 124 frames, seed 42, with both images set to asset:qa-cast/priya-portrait.jpg.
- It succeeded.
- In get_job_workflow (realized: true), the step keeps "cache": "variable:cache", with the variable pinned to null. This is the same realized form C-F422 accepted for attention_backend.
- The manifest's timing has no cache block. C-F429's literal-cache run does have one, so no cache was installed on this run.
C-F429 passes. I built an inline copy of video-with-audio with the cache variable declaration dropped and the step's cache set to a literal {"type":"first_block","threshold":0.05}.
- Validate returned valid: true.
- The same workflow with {"type":"bogus"} was refused at steps[0].pipeline.configuration.cache.type: "'bogus' is not one of ['first_block', 'faster', 'mag', 'taylorseer', 'text_kv']".
- The run at 512x288, 124 frames, seed 42 (job 223b74fd4d0b) succeeded. The realized step carries the literal cache as written, and timing.cache shows first_block / 0.05, with 4 forwards, 0 skipped.

Carried from the earlier verify against `92c3952`

C-F424 and C-F425/C-F426 values 1–6 passed there. The only change since is the narrowing of threshold, and I re-checked values 1 and 2 above.
C-F428 passed at 960x544 (job a104407184ad, comment above). As written, at 1344x768, run_workflow's VRAM gate refuses it on lem's 3090. That is a sizing problem in the case, not a failure of the stage. The resize is requested in C-F428: resize to 960x544 - 1344x768 is refused for VRAM on lem's 3090 harnest#114.

I deleted both outputs with delete_output(job_id=...). I removed the pending: #884 line from C-F424 through C-F429 in regression-suite-complete.md, so they are now ordinary cases.

Navigation Menu

Repository navigation

#865 stage B: cache as a whole-object variable on every H3 template #884

Builds

Owes

Deploy

Passes when

9 remaining items

The bounce (C-F425 / C-F426, value 7): fixed

The cases not run in the last verify

Carried from the earlier verify against `92c3952`

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

#865 stage B: cache as a whole-object variable on every H3 template #884

Description

Builds

Owes

Deploy

Passes when

Activity

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

9 remaining items

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

The bounce (C-F425 / C-F426, value 7): fixed

The cases not run in the last verify

Carried from the earlier verify against 92c3952

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Carried from the earlier verify against `92c3952`