plan.estimate prices every H3 variant at the catalog figure - measured range is 3.5-44 min

Split out of #861.

What happened

POST /api/validate quoted ~42 min for the ep 6 dialogue-short regardless of what changed. The matrix measured the same shot from 3.5 to 44.2 min depending on canvas, weights, LoRA, steps, attention and cache. Agents quote plan.estimate to the user before a run, so the quote is wrong by up to 12x in both directions.

Measured (one H3 Ref2VA shot, 124 frames, 4 refs, 3090)

Rung	Canvas	Weights	LoRA	Steps	Attn	Extra	Min (3090)	Face score /5	Staging family
L0	960x544	int4	ref2v 4-step v0.1	5	sage_hub		3.5	2	turbo-4
L1	1344x768	int4	ref2v 4-step v0.1	5	sage_hub		5.4	2.5	turbo-4
L7	1344x768	int8	ref2v 4-step v0.1	5	sage_hub		7.2	3	turbo-4
L3	1344x768	int8	ref2v 8-step v1.0 768p	9	sage_hub		11.2	3	turbo-8
L4	1344x768	int8	ref2v 8-step v1.0 768p	9	native		14.5	3	turbo-8
B4	960x544	int4	none	12	sage_hub		6.6	3	base-int4
B1	960x544	int4	none	30	sage_hub		15.1	3	base-int4
B5	1344x768	int4	none	12	sage_hub		11.8	3.5	base-int4
B6	1344x768	int4	none	8	sage_hub		8.5	1.5 (duplicated a character)	base-int4
B3	1344x768	int8	none	12	sage_hub		12.9	3.5	base-int8
L5	1344x768	int8	none	30	sage_hub		29.5	4	base-int8
L6	1344x768	int8	none	30	native		44.2	4	base-int8
P1	1344x768	int8	none	30	sage_hub	first_block 0.05	23.1	4	base-int8
C1 / C2 (Pat close-up)	1344x768	int8	none	12 / 30	sage_hub		13.9 / 22.8	4 / 4	base-int8

Base model = lora_model_name: null, shift video 12 / audio 3. Turbo = shift per the LoRA. All Ref2VA with 4 references (2 portraits, 2 voice clips), 124 frames. Scores are face realism at native-resolution crops, judged by Claude from frames (5 = cinema-grade photographic).

Per-pass cost at 1344x768 int8 is ~56 s (sage), ~81 s native-equivalent; total ~= load 0.7 + encode 0.9 + passes x s/pass + decode 0.9 min.

Expected

The estimate should scale with the things that move it:

passes = num_inference_steps - 1 (scheduler counts the terminal zero);
s/pass by canvas x frames x weights x attention backend (a small table per pipeline, or learned from observed runs keyed by those fields - observed_cost already records runs, it just keys too coarsely);
cache: first_block at >=30 steps ~0.75x passes; at <20 steps 1.0x;
compile: + first-pass compile time per new shape.

_tempered blends toward the curated figure; with the curated figure being the 8-step template's, an observed 30-step run gets pulled down toward it.

Questions

Key observed cost by (pipeline, canvas, frames, weights_dtype, lora present, steps, attention, cache)? That's a lot of buckets; a per-pass model (fixed + passes x s/pass) generalizes better from few runs.
Should validate say basis: "derived" when it scales a measured s/pass to a new step count?

Source: H3 quality matrix on lem (2x RTX 3090 24 GB, 125 GB host RAM), workspace h3-quality, Known Issues ep 6 shots, seed 42. Full data and per-run assessments: ~/testing/opus/known-issues/quality/matrix.json, report quality/REPORT.md; every run in h3-quality carries a note with its config, minutes and score. Related: #861 (follow-up experiments).

triage: escalate — replaces the per-workflow estimate with a per-pass cost model keyed by canvas × frames × weights × LoRA × steps × attention × cache × compile, and a possible new basis: "derived". That's a wide argument matrix plus new estimate semantics, and the issue's own design questions (bucket keys vs fixed + passes × s/pass; the basis value) need a decision before a fix session builds it. A repro-only check on one H3 shot would pass with most of the matrix still untested. (claude-opus-5-5, anthropic)

Review: approved for planning, smaller than the full per-pass model.

What the code does today (dw/server/observed_cost.py, dw/plan.py):

Observed cost is already bucketed by the workflow's cost_drivers. dialogue-short declares shots, num_inference_steps, width and height.
Two failure modes produce the flat ~42 min:
1. An empty bucket falls back to the curated figure, unscaled. Most matrix rungs are new combinations, and many ran as ladder.py's custom workflows, which have no history.
2. _tempered (plan.py:260) blends toward the curated figure even when the requested bucket isn't the default one, so a thin 30-step observation gets pulled toward the 8-step number.
Weights dtype, LoRA, attention and cache are not drivers. Runs that differ only in those share a bucket.
basis: "derived" already exists (plan.py:249, used by per_entry scaling at :873).

Plan scope:

Scale, don't fall back. When the requested bucket is empty, scale the curated figure by passes (steps - 1) and by pixels × frames, and report it as basis: "derived". Temper toward the curated figure only when the bucket is the default one; otherwise temper toward the scaled figure. This fixes both failure modes.
Drivers that can't be scaled. For weights dtype, LoRA presence, attention and cache, where a template exposes them as variables (feature: H3 templates - expose attention_backend + cache everywhere, and a quality tier preset that sets the coupled knobs together #865 adds some): add them to cost_drivers. When the request differs from the curated run's value on one of them, flag low_confidence rather than quote a scaled number as if it were exact.
Deferred: a learned fixed + passes × s/pass fit over observed rows. It comes back only if stage 1's quotes are still off by more than ~30% against the h3-quality matrix, which is the acceptance test set.

Q1: keep cost_drivers bucketing and add the scaling, rather than a seven-key bucket.
Q2: yes. Reuse the existing derived value; no new basis.

claude-opus-5-5 / anthropic, for Don

Spec for plan v4. Tester: claude-opus-5-5 via anthropic.

All cases are in regression-suite-complete.md and carry pending: #<stage>. v4 reshaped stages 0 and A, so those two stages get new cases. The v2 cases they replace are up for retirement in dkackman/harnest#123.

Stage	Cases (v4)	What they cover
#905 (stage 0)	C-F497, C-F498, C-F499	C-F497 (SD1.5, inline): metadata, seed 42/43 (as a variable and as an argument) and `25.0` vs `25` all keep the fingerprint. Prompt, steps, dtype and image count move it. C-F498 (H3): inline and by-name match (the integral-float fix), and seed 43, stripped metadata and a saved copy with other `cost_drivers` all keep the fingerprint. Attention (two values), cache, int8 and LoRA move it. C-F499: one saved workflow edited in place. A metadata edit keeps `cached_steps: 1`, and a re-run is served from the cache (about a minute of GPU). Seed, dtype, `subfolder`, `file_base_name` and attention give `cached_steps: 0`.
#906 (stage A)	C-F500, C-F501, C-F502, plus C-F477 and C-F480 (kept from v2)	C-F500: a freshly saved reference-to-video copy is `catalog` at its defaults, then `derived` scaling by steps, canvas, frames, scaling down, the 130→141 snap edge, the top of the range, and the defaults passed explicitly. C-F501: a saved dialogue-short copy is ≈21 and scales by summed frames. C-F502: the served ratios (2.5 for steps, 1.976 for canvas) reproduce B4→B1, B4→B5 and B3→L5 inside the 30% bar. C-F477: refusals and the non-numeric `unknown`. C-F480: tempering toward the scaled figure.
#907 (stage B)	C-F481, C-F482, C-F483	Unchanged in v4. These stand.
#908 (stage C)	C-F484, C-F485	Unchanged in v4. These stand.

Superseded, retirement requested in harnest#123: C-F473, C-F474, C-F475, C-F476, C-F478 and C-F479.

Notes (none blocks the spec):

A shot dropped from dialogue-short. The plan says list-length scaling (_repriced) "multiplies with" the summed-frames ratio. When a shot is removed, applying both counts the dropped frames twice: D×4/5×496/637 ≈ 0.62·D, against ≈ 0.78·D from summed frames alone. C-F501 call 3 only records which one the server gives. The plan should say which is intended.
Price precedence. C-F500 relies on the plan's "as today": a saved copy at its defaults is priced catalog. If it comes back inherited, the case calls that a finding against the plan's premise rather than a pass. C-F502 judges by ratios, so it doesn't depend on which price wins, unless the copy is priced inherited, which the stage A path doesn't scale.
Where the 30% bar is tested. The bar itself is a unit test the tester can't see. The #866 stage A: scale the catalog estimate along steps, canvas and frames #906 hand-off should name it so a reviewer can check it.
Attention on SD1.5. C-F499's attention edit borrows the H3 step's key. If SD1.5 refuses that key, the case skips the edit and C-F498 covers attention.

Merge docs/866-closeout (#866)

Close-out: all four stages closed and verified. claude-opus-5-5 / anthropic, lead

Closed: #866 stage 0: key the plan fingerprint and step cache only by what changes the output #905 (stage 0, the fingerprint and step cache keyed by output only), #866 stage A: scale the catalog estimate along steps, canvas and frames #906 (stage A, catalog scaling), #866 stage B: H3 weights, LoRA, attention and cache as unscalable cost drivers #907 (stage B, unscalable drivers) and #866 stage C: the inherited estimate scales by steps and canvas too #908 (stage C, inherited scaling).
Design record: docs/proposals/complete/estimate-scaling-complete.md, with a todo.md entry. Merged to develop @ 828cccb (commit edfd00d). It covers what was built, what was cut (compile as a driver, the learned per-pass fit), the follow-ups bug: validate's cached_steps probe ignores the card's warm workflow identity #911 and idea: step cache across workflow identities (renamed and inline copies hit) #912, and the bounces per stage. Stage costs are left out because no stage comment records usage:.
verify: run every case whose source: names one of this feature's stages (#866 stage 0: key the plan fingerprint and step cache only by what changes the output #905, #866 stage A: scale the catalog estimate along steps, canvas and frames #906, #866 stage B: H3 weights, LoRA, attention and cache as unscalable cost drivers #907, #866 stage C: the inherited estimate scales by steps and canvas too #908), and close the parent if they all pass.

tester verify of #866 (feature parent), progress record. Model: claude-opus-5-5 via anthropic. lem on develop @ 828cccb.

The session budget ran out before every stage case could be run, so labels are left unchanged: still status:fixed-pending-verify. No failures so far.

Passed (all over MCP this session, workspace regression-complete):

C-F500 (#866 stage A: scale the catalog estimate along steps, canvas and frames #906 v4), all 10 calls pass:
- 0: 7.9 catalog, fingerprint 3c07adc7
- 1: 26.3 derived, steps 30 vs 9
- 2: 15.6 derived, canvas
- 3: 51.9 derived, steps and canvas
- 4: 12.2 derived, frames 192
- 5: 4.4 derived, steps 5
- 6: 11.0 derived, canvas 1344x544
- 7: 9.0 derived, frames 141
- 8: 21.9 derived, frames 345
- 9: 7.9 catalog
C-F477 (#866 stage A: scale the catalog estimate along steps, canvas and frames #906 v2): width 1000, "thirty" and frames 400 are refused. Width 0 gives basis unknown.
C-F481 (#866 stage B: H3 weights, LoRA, attention and cache as unscalable cost drivers #907): int8, sage_hub, cache, lora-null+30 steps and int8+30 steps each give low_confidence, with the moved variable in differs. Defaults give 7.8 observed with no flags.
C-F502 (#866 stage A: scale the catalog estimate along steps, canvas and frames #906 v4): M0 7.9, M1 19.7, M2 15.6, M3 15.6, M4 38.9. The ratios r1 2.494, r2 1.975 and r3 2.494 all fall inside the bands.
C-F498 (#866 stage 0: key the plan fingerprint and step cache only by what changes the output #905 v4): F0 = sha256:3c07adc7…
- The seed-43, inline, inline-without-cost_drivers/vram and saved-with-narrowed-cost_drivers variants all reproduce F0.
- The controls all give distinct fingerprints: sage_hub 5fbc267b, native af01396c, cache ceef4523, int8 10d09db1, lora-null 593e0d99.
C-F484 (#866 stage C: the inherited estimate scales by steps and canvas too #908):
- Call 1: inherited 7.8 from templates/minimax/reference-to-video, differs [], no scaled.
- Call 2 (steps 30, 1344x768): 51.1 inherited, with scaled "steps 30 vs 9; canvas 1344x768 vs 960x544". That is within ±5% of M0×(30/9)×1.9765 ≈ 51.4 and within 10% of T = 51.9.
- Call 4 (num_frames 192): 12.0 inherited, scaled "frames 192 vs 124", inherited_from kept. Expected ≈ 12.1.
C-F485 (#866 stage C: the inherited estimate scales by steps and canvas too #908): every variant keeps basis inherited with a number, sets low_confidence true and names the variable in differs.
- Call 1: differs [] with no low_confidence.
- native: 7.1, differs ["attention backend"].
- cache first_block: 7.8, differs ["cache"].
- lora nulls: 7.8, differs ["LoRA"].
C-F482 / C-F483 (#866 stage B: H3 weights, LoRA, attention and cache as unscalable cost drivers #907): passed in #866 stage B: H3 weights, LoRA, attention and cache as unscalable cost drivers #907's verify (job 081661f70d65). Their pending: #907 line has now been removed from C-F482.

Observations (not failures):

C-F484 call 2 sets low_confidence: true with an empty differs. That is presumably the scaling itself; the case doesn't forbid it.
C-F485 native: inherited_from switched to templates/minimax/voice-timbre-reference (runs 3) instead of reference-to-video. That template presumably has a native-attention bucket. The case only asks for basis inherited with a number.

Deviations from the cases as written:

C-F502 reused c-f500-ref2v patched to steps 12, rather than separate copies. It passed the LoRA nulls as validate arguments, because a merge patch can't set a variable to null.
C-F498's inline calls passed seed 42 as an argument.
The inline W in C-F484/C-F485 drops vram_estimate and the variable_constraints reason strings, and sets seed 42 in variables.
Cleanup: c-f500-ref2v and c-f498-ref2v are deleted.

Not yet run (the next session must finish these before closing):

C-F497 (#866 stage 0: key the plan fingerprint and step cache only by what changes the output #905 v4, SD1.5 inline fingerprint)
C-F501 (#866 stage A: scale the catalog estimate along steps, canvas and frames #906 v4, dialogue-short)
C-F499 (#866 stage 0: key the plan fingerprint and step cache only by what changes the output #905 v4, GPU)
C-F480 (#866 stage A: scale the catalog estimate along steps, canvas and frames #906 v2, GPU)

Verified: feature parent close-out. Tester claude-opus-5-5 via anthropic, against lem develop @ 8605824, workspace regression-complete.

I ran every case whose source: names #905, #906, #907 or #908. All of them pass. No pending: #905–#908 lines remain in the suites.

Carried from the previous verify session on 828cccb (also claude-opus-5-5): C-F477, C-F481, C-F482/C-F483, C-F484, C-F485, C-F498 and C-F500 passed. C-F502 also passed there, with one variance noted on the issue.

Run this session on 8605824:

C-F497, fingerprint keyed on output only.
- F0 is sha256:70cf5bf2…, and a repeat call returned the same value.
- Each of these variants returned F0:
  - (a) cost_drivers removed
  - (b) cost_drivers extended
  - (c) cost minutes 9.9
  - (d) description changed
  - (e) subfolder changed
  - (f) file_base_name changed
  - (g) vram_estimate added
  - (h) seed 43 in variables
  - (i) arguments seed 43
- The controls all returned distinct fingerprints: (k) prompt, (l) 26 steps, (m) float16, (n) 2 images.
- (j), the num_inference_steps: 25.0 literal: the first batch dropped it, so I ran it separately on my reconstructed inline copy, which hashes to 229cc5…, not F0. 25.0 and 25 both gave sha256:229cc5bd…, so integral floats are canonicalized.
C-F501, dialogue-short re-priced and frame-scaled.
- D = 21.2, so it is no longer ≈42. The vram_estimate reason now says the 42 min "was that field run at 1344x768; re-priced to these 960x544 defaults it is … 21.2".
- Call 2 (frames 705): derived 23.5. D×705/637 = 23.46. scaled reads "frames 705 vs 637 over shots".
- Call 3 (four shots, 496 frames): derived 16.5. That equals 0.78·D = 16.5, not 0.62·D = 13.2.
  - The scaled string reads "scaled from the curated 17 min: frames 496 vs 637". 17 is D×4/5, yet 17×496/637 would be 13.2. So the label names a different base than the number used.
  - This is cosmetic and doesn't fail the case. Noting it for the lead.
- Deviation: I validated templates/minimax/dialogue-short by name instead of saving an observed-stripped copy.
  - Because of that, call 1 returned observed 38.6 (5 runs), not catalog.
  - Calls 2 and 3 both report scaling "from the curated 21.2 min", so the catalog D is what gets priced.
  - Calls 2 and 3 used shortened shot prompts and references. Only the num_frames values and the entry count matter to the estimate.
C-F499, in-place metadata edit served from the step cache.
- Saved c-f499-t2i (W0, seed 42). J1 d4286bd05039 succeeded in 12.5 s, and validate returned cached_steps: 1.
- Saved W1 as a merge patch with: description "c-f499 meta", cost_drivers null, cost minutes 5, and vram_estimate added. The fingerprint was unchanged and validate returned cached_steps: 1.
- J2 2d4851f99650 succeeded in 0.4 s. Its manifest marks main as reused: true, serving J1's own file …/20261015-122442-475c5693/final/test_image-0.0.jpg.
  - I did not call get_job_events(J2), since the manifest already shows reused on the same card.
  - I did not compare the two images, since J2 lists J1's own file as its output.
- Step 4: each edit returned cached_steps: 0:
  - (a) seed 43. The fingerprint was unchanged, as the Assess: acknowledged_cost gate should bind to a budget/fingerprint, not just a boolean #85 contract requires, but the cache still missed.
  - (b) float16
  - (c) subfolder "other"
  - (d) file_base_name "renamed"
  - (e) configuration.components.unet.attention_backend: "native". Validate accepted it, giving a new fingerprint.
- After (a) and after (e), I saved W1 back and validate returned 1. I applied (b)–(e) as full-steps patches onto W1 without a separate restore between each. Every one was a single change from W1.
- Cleanup is done: both outputs and the workflow are deleted.
C-F480, observed tempering against a scaled curated estimate.
- Call 1: derived 0.8, with scaled "steps 100 vs 25".
- J1 8ab7b8cd1e9e ran 100 steps in T = 20.5 s (0.34 min).
- Call 3: observed 0.6, runs 1, curated_minutes 0.8 (not 0.2), observed_minutes 0.3, tempered: true. 0.6 lies within [T, curated].
- Cleanup is done.

Closing as completed.

fix(plan): #851 - scale an inherited price by for_each member count

Navigation Menu

Repository navigation

plan.estimate prices every H3 variant at the catalog figure - measured range is 3.5-44 min #866

What happened

Measured (one H3 Ref2VA shot, 124 frames, 4 refs, 3090)

Expected

Questions

44 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

plan.estimate prices every H3 variant at the catalog figure - measured range is 3.5-44 min #866

Description

What happened

Measured (one H3 Ref2VA shot, 124 frames, 4 refs, 3090)

Expected

Questions

Activity

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

44 remaining items

dkackman commented on Oct 15, 2026

dkackman commented on Oct 15, 2026

dkackman commented on Oct 15, 2026

dkackman commented on Oct 15, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions