idea: musical join task - spoken scene into song with the song entering under the last line, as a run output with provenance

Idea: a musical join task (a spoken scene breaking into a song) whose output is a normal run output with provenance.

Source: field report from an Opus 5.5 session on lem, 2026-09-25/26 (La Vela del Pueblo: 6 dialogue shots breaking into 18 lip-synced song shots).

The shape assemble-and-score can't make:

Dialogue shots keep their own generated audio, each loudness-matched.
The song enters at an arbitrary offset under the last spoken line (its intro vamp plays under it), so a given song frame lands exactly on the first song shot's first frame.
The outgoing dialogue audio is ducked after a delay.
Song shots play over the unbroken song (their own generated audio discarded), so lip sync generated against frame-exact slice_audio slices is preserved. In the final cut the first sung line landed at 41.10 s against a predicted 41.08 s.

assemble-and-score lays one score under everything. The session did the join locally in ffmpeg (la-vela/tools/assemble.py, ~130 lines).

The second half of the gap is provenance. The finished film exists only on the client: not in the workspace gallery, not reachable by export_job, and with no record of how it was built from 24 shots and a song asset. Uploading the file back is already possible (POST /api/uploads, see #481). But an upload doesn't record its inputs. A dw-side join closes both halves. Failing that, a "register external output" path that attaches a file together with its input references would.

Implementation notes from the local version:

ffmpeg loudnorm inside a filter_complex broke stream timing (half-length output, adelay ignored). Measuring then applying a static gain was robust. dw already measures with pyloudnorm.
alimiter auto-levels by default; after an AAC encode that clipped. Use level=disabled (see Limiter for normalize_audio: reach target_lufs past the peak ceiling (split from #467) #474).

Not the plan. Don chose the other plan v1 (comment of 23:54) on 2026-09-27, so this comment's plan marker was removed. It is kept as a record of the alternative. Lead agent, model claude-opus-5-5 via anthropic, decompose session.

(Alternative, not adopted) `join_into_song`, a musical join task, plus a `scene-into-song` template

Lead agent: claude-opus-5-5 via anthropic. Design session, 2026-09-26, at develop 0540972.

Verdict: build smaller

Value. The only user who gains is a driving session making a musical (Don's own work), and only when a piece breaks from dialogue into song. The evidence is one field report (this issue, La Vela del Pueblo on lem, 6 spoken + 18 sung shots). That session had to drop out of dw for the last step: ~130 lines of local ffmpeg, a film that isn't in the gallery or export_job, and no record of how it was built. The same class keeps coming back to assemble-and-score (#180, #215, #339, #342, #402, #454, #455, #467), and ducking under voice has already needed skill text (#340). None of those fixes can make this shape, because the score starts at sample 0 of the cut. Real use: the la-vela workspace lives on lem, not on this server. Several of Don's workspaces here (encanto-uprising, shedding-skin-quartet, iron-bloom) are music-led pieces, but I didn't count in them how many would have needed a join. Demand is real but thin: one report, one strong use.

Build cost. Two stages, about $4–6 and $2–4. Roughly $6–10 in total.

Inertia.

One new task command. It shows up in list_tasks, which is runtime data outside SURFACE_BUDGET, and no MCP tool docstring changes.
One new template, which costs about 100 chars of COMPACT_BUDGET in the compact catalog.
A docs/TASKS.md section, domain entries, one new AudioVideo constructor site in tests/test_shots.py, and a paragraph in two skills.
No engine, syntax, REST or security change, and no new concept: it is one more audio task beside mix_audio and pair_audio.

Reversibility. High. It is an unreferenced task and a template. Removing both later is a breaking-change only if someone's workflow names them.

Cheaper alternatives.

Add offsets to mix_audio and document a recipe. This fails. The offset the song needs is (length of the spoken cut − song position), and a workflow cannot do that arithmetic: pair_audio's docstring already records that nothing can multiply a list's length by a frame count (music-video template: soundtrack slice is hardcoded to 496 frames, so any shots list other than 4 gives a deliverable whose audio is longer than its video — succeeded, warnings: [] #142). A caller would have to measure the cut and hard-code the number. It is also brittle, since the length moves every time a spoken shot is regenerated.
Anchor on a named shot in a single concat_videos output (score_from_shot(video, song, shot=...)). This is more general, but fragile. The sweep found that shots are called video N or the file basename until the join step saves (dw/workflow.py:1598, dw/result.py:723). An anchor name would therefore depend on how the step was written, and a template cannot compute "the index after the last spoken shot" either.
Register an external output with its inputs (the issue's fallback). This closes the provenance half but not the join. It also adds a new write route and a sidecar format. See non-goals.
Do nothing. The local ffmpeg route works, as the report shows (41.10 s against a predicted 41.08 s). What is lost is provenance, export_job, the assessment probes and the frame-exact alignment, which every new session would have to re-derive.

Cut, and what would bring it back:

Register an external output with provenance is cut, because once the join runs in dw, the film is a normal run output with a manifest and a realized workflow. It comes back with a second report of a finished deliverable that had to be built off-box for a reason no task covers.
LUFS matching of the spoken shots is cut. concat_videos(match_levels="rms") already evens them out. It comes back if rms matching is reported to fail audibly for dialogue (Q2).

Overlap.

Limiter for normalize_audio: reach target_lufs past the peak ceiling (split from #467) #474 (a limiter for normalize_audio, in plan review) is not a dependency. This plan normalizes to a −3 dBFS peak the way every mux template does, and Limiter for normalize_audio: reach target_lufs past the peak ceiling (split from #467) #474's level=disabled note is about the local ffmpeg alimiter, which this design doesn't use.
idea: attribute_voices - which reference singer sings each lyric line, by timbre (pitch mislabelled 6 lip-sync shots) #485 and idea: lip-sync target check - is the singer's mouth the one moving (attribute_voices + get_output_frames) #488 are separate.

Problem and non-goals

Problem. A spoken scene needs to break into a song:

The spoken shots keep their own audio, level-matched.
The song enters under the last spoken line, so that a chosen song position (the frame the first sung shot was sliced from) lands exactly on the first sung shot's first frame.
The spoken audio ducks a little after the song enters.
From the cut on, the sung shots play over the song, unbroken. Their own generated audio is discarded, so lip sync generated against slice_audio slices holds.

The result must be an ordinary run output.

Non-goals.

A general timeline or offset mixer.
Crossfading picture at the join. The picture join is a hard cut; dissolve_videos exists.
More than one spoken-to-song transition per step. A song back into dialogue is a second call.
Registering external files (see above).
LUFS matching.

Design

The proposal said "a musical join task". It is right about the shape. It doesn't choose between a mixer offset and an anchor, and both are wrong for the reasons in Cheaper alternatives 1 and 2. The design takes two already-joined videos instead, so the anchor is simply "where spoken ends". No names or arithmetic are needed.

New task `join_into_song` (in `dw/tasks/audio_utils.py` or its own module, registered in `dw/tasks/task.py`)

join_into_song(spoken, sung, song,
               song_start_frame=None, song_start_seconds=None, fps=None,
               duck_db=-12, duck_delay_ms=500, duck_ramp_ms=250,
               song_gain_db=0, sample_rate=None)

Inputs

spoken and sung are one video each: an AudioVideo from concat_videos, or a file path, which brings shots from shots_beside.
song is an audio track or file.

Picture. The frames are spoken followed by sung, as a hard cut. Both must share frame size and fps (check_same_frame_size). A mismatch is refused.

Where the song enters

T is the length of spoken, in samples at the output rate.
song_start_* (frames with fps, else seconds) is S, the song position that must land on sung frame 0.
The song is placed so that sample S sits at T, which puts its entry at T − S.
If S > T, the song's head is cut to fit and the step warns song_head_trimmed.
Exactly one of song_start_frame and song_start_seconds is required. Neither, or both, is an error.

The track

spoken's own audio runs until T.
From entry + duck_delay_ms it ramps down by duck_db over duck_ramp_ms. The ramp is equal-power, and the gain is in dB, not a multiplier (the sweep flagged mix_audio's mix_audio_gain_not_db trap).
If the duck point falls at or after T, there is no duck and the step warns duck_after_cut.
sung's own audio is discarded.
The song is mixed in at song_gain_db.
The track is cut to the picture's length. A song that ends before the picture ends is padded with silence and warns song_short_of_picture, which is the same reading as slice_past_end.

Sample rates. They follow mix_audio's rule: the highest rate wins and the rest are resampled up, with a warning. sample_rate pins the rate.

Returns an AudioVideo.

Its shots are spoken's shots, then sung's shifted by spoken's frame count, with their samples remeasured against the new track (remeasured_shots).
This adds a new constructor site ("remeasures", 1) to tests/test_shots.py.
The step's result logs the entry sample, the duck window and the song offset, so a caller can check the join without decoding.

No normalization inside the task. Levels are the workflow's decision (the mix_audio rule). The template follows it with normalize_audio(peak_dbfs=-3) and pair_audio(fit="video"), the shape assemble-and-score uses. The headroom and clip warnings in result.py then apply unchanged.

Domains (dw/task_domains.py, checked with check_arguments):

song_start_frame and song_start_seconds: ≥ 0.
duck_delay_ms and duck_ramp_ms: ≥ 0.
fps and sample_rate: > 0.
duck_db stays undeclared: a positive value is odd but legal.

Surfaces touched

Task registry (list_tasks/get_task, derived from the signature).
docs/TASKS.md.
One template.
Skill text.

No engine, syntax, REST, MCP-tool or schema change. Nothing here needs status:needs-approval beyond this plan.

New template `templates/minimax/scene-into-song`

Variables

spoken_shots: a list of video references.
sung_shots: a list of video references.
song: the song asset.
song_start_frame: the frame the first sung shot was sliced from.
fps.
match_levels: default "rms".
duck_db, duck_delay_ms.

The shots go in as output:/asset: references, which lets it cut the output of shots-batch/dialogue-short and music-video runs.

Steps

spoken = concat_videos(spoken_shots, match_levels), marked intermediate.
sung = concat_videos(sung_shots), intermediate.
joined = join_into_song.
normalize_audio(peak_dbfs=-3).
pair_audio(joined, normalized, fit="video"), marked final.

Catalog. Shape sequence, traits has-audio, needs-input-media, and a curated cost (it is CPU work, minutes). This touches the entries in tests/test_catalog_structure.py (EXPECTED_SHAPES, COSTED, COMPACT_BUDGET) and the −3 dBFS pin in tests/test_result.py.

Stages

Stage 1: the `join_into_song` task (server)

Builds the task, its registration, its domain entries, the tests/test_shots.py site and a docs/TASKS.md section.
Unit tests
- Placement: song sample S at output sample T, exact to the sample, for both frames and seconds.
- S > T trims the head and warns.
- Duck window position and depth.
- Duck at or after the cut warns and doesn't duck.
- The sung audio really is gone: a tone in sung is absent from the output.
- Frame-size and fps mismatch is refused.
- Neither or both start arguments are refused.
- Rate mismatch resamples and warns.
- Shots are concatenated and remeasured.
- Domain violations are refused.
Deploy: server.
Estimate: $4–6.
Acceptance intent (over MCP, with the tester's own short clips as assets):
- list_tasks shows join_into_song with the arguments above.
- A validated workflow of two concat_videos steps plus join_into_song plus a saving pair_audio runs. get_gallery_metadata then shows:
  - duration = spoken + sung;
  - media.shots covering every input shot in order, with sample positions matching frames.
- Take a song asset with an audible marker (a click) at a known time S. The click is heard (get_output_audio) and measured at spoken length ± one frame.
- assess_output reports no sync_drift or sync_length finding. A seam_level_step at the join is expected, not a failure.
- Level at the duck:
  - it measures lower after entry + delay than before it, by about duck_db;
  - with duck_delay_ms pushing past the cut, the job's warnings carry duck_after_cut.
- With song_start_frame beyond the spoken length, the job succeeds with song_head_trimmed.
- validate_workflow refuses:
  - duck_ramp_ms: -1;
  - both start arguments given;
  - neither start argument given (at run time if it came from a variable).
- A sung clip at a different frame size fails with a message naming both sizes.

Stage 2: `templates/minimax/scene-into-song` and skill text (server, for the template, plus plugin)

Builds
- The template.
- Its catalog entries and cost.
- A paragraph in plugins/dw/skills/minimax-h3/SKILL.md and minimax-music3/SKILL.md: "a dialogue scene that breaks into song: generate the sung shots against slice_audio slices, then scene-into-song with song_start_frame = the first sung slice's start". These sit beside the existing gain_audio ducking text, under the SKILL_SIZE_LIMIT.
- A pointer from the assemble-and-score description ("one score under everything. For a song entering mid-scene, use scene-into-song").
Unit tests: the catalog-structure entries, the template validating with stock arguments and no warnings, and skills naming a real template (tests/test_plugin_skills.py:101).
Deploy: server (template) and plugin (skills, plugin.json version goes with release).
Estimate: $2–4.
Acceptance intent
- list_workflows(shape="sequence") lists scene-into-song with its variables and the lists block.
- validate_workflow with the stock arguments is clean and carries a plan.estimate.
- A run with two spoken and two sung tester clips and a marked song gives a final/ deliverable. It has:
  - four shots in media.shots;
  - the marker on the first sung frame;
  - a peak ≤ −3 dBFS plus the encoder's margin, and no audio_clipped.
- export_job bundles it, and get_job_workflow shows the realized inputs. That is the provenance half, closed.
- Changing duck_db in rerun_job changes only the level before the cut.

Order. Stage 2 needs stage 1. There are no external blockers.

Risks

The song position a caller passes can be wrong. The task can only honour S, and a caller who sliced the sung shots from a different start gets drift. The mitigations are the skill text, which names the relation, and the logged entry and offset.
seam_level_step will fire at the join by design. A later reader may take it for a defect. The skill text says so.
A spoken input made from a file whose shots came from no manifest gives one whole-file shot. That is harmless: placement uses only its length.
The tester needs a marked song and short clips. These are synthetic assets, which the tester already builds for assemble-and-score cases.

Questions

Q1. Duck defaults. My default is duck_db=-12, duck_delay_ms=500, duck_ramp_ms=250, chosen as a speech-under-intro duck. The report didn't state its numbers. If you know La Vela's values, give them and they become the defaults. Nothing else changes.
Q2. LUFS matching for the spoken shots. My default is no: match_levels="rms" is already on in the template. If yes, concat_videos gains match_levels="lufs" via dw/loudness.py. That is a third stage (about $3) touching a task every join uses.
Q3. Name. My default is join_into_song / scene-into-song. If you'd rather have a name that covers a song entering under any picture (for example enter_score), say so. Only names change.

Plan v1: #486 musical join (a spoken scene breaking into a song)

Written by the lead agent, model claude-opus-5-5 via anthropic. Design session, read-only against develop @ 3e8bf8d.

Disposition of the idea: this is a feature, not a single fix. It adds a task, which is new syntax surface that Don has to approve. It is not a duplicate. Nearest neighbours: #187 and #199 (region gain and bleed ducking, both shipped), #340 (the skill documents score ducking as a chain of gain_audio steps), #474 (a limiter for normalize_audio, in plan review) and #481 (the upload route; only the text changes). None of them join two audio regimes.

Verdict: build smaller

Build one task, join_into_song, that owns the timing arithmetic. Don't build the "register external output" path. Don't build a template yet.

Value. The user who gains is Don's driving sessions making musicals. There is one field report so far (La Vela del Pueblo, 6 dialogue shots + 18 song shots). The session wrote ~130 lines of local ffmpeg and hit two ffmpeg traps (loudnorm in a filter_complex, and alimiter auto-levelling). The result sat outside the gallery, export_job couldn't reach it, and nothing recorded how it was built. H3 and Music3 make lip-synced song shots routine, and "dialogue, then the song" is the standard shape of a musical number, so this should recur. It is still one report, though, so I'm pricing it as one task and not a subsystem. The only server I can query is the local MPS box, which has no workspaces of Don's (only default, plus regression-* at 0 files), so there is no usage signal either way.
Build cost. Stage A is ~$5 and stage B is $2, so **$7** in total.
Inertia.
- One task in list_tasks. That costs no MCP tool-description budget, because tasks are fetched at run time and are not in SURFACE_BUDGET.
- One EXPECTED_SITES entry in tests/test_shots.py.
- One docs/TASKS.md section and one skill recipe.
- No new MCP tool, REST route or engine concept, and no security boundary is loosened. Inputs arrive as ordinary resolved references (previous_result:, output:, asset:).
Reversibility. It is a leaf task. Removing it later breaks only workflows that name it, which is a breaking-change but a small one.
Cheaper alternatives:
- Do nothing and keep assembling locally. This keeps both halves of the gap, and the provenance half is the one the report stresses.
- Compose it from existing tasks. The sweep found this is possible but not viable:
  - mix_audio has no per-track offset (dw/tasks/audio_utils.py:1068-1075: every track starts at 0), so placing the song needs a hack. You slice past the end of a track to get silence, then butt-join it with crossfade_audio(crossfade_ms=0). That warns slice_past_end every time.
  - gain_audio has no ramp (:690-695), so the duck is a hard step.
  - The song offset is dialogue length − cue and has to be a hand-computed literal, because a workflow can't do arithmetic. It goes stale as soon as a dialogue shot is regenerated at another length.
  - That third point decides it: the report's frame-exact landing (41.10 s against 41.08 s predicted) only holds if something measures the actual joined dialogue length at run time.
- Generic primitives instead: mix_audio.offsets + a gain_audio ramp + a silence task. That is three surface additions and still no arithmetic. See Q2.
Overlap.
- Limiter for normalize_audio: reach target_lufs past the peak ceiling (split from #467) #474 (the limiter): this task normalizes with a static gain, as the report recommends, and adds no limiter of its own. If Limiter for normalize_audio: reach target_lufs past the peak ceiling (split from #467) #474 lands, the task can reuse it later.
- upload_asset's refusal and the guide never name POST /api/uploads: a remote client's reference images never reached the server #481 is the text-only fix for uploads. It doesn't overlap with this.

Cut, and what would bring it back:

Register external output (attach a client-made file to a record of its inputs): no. Bring it back if a second deliverable gets assembled outside dw for a reason this task doesn't cover.
A template (templates/scene-into-song): no. The inputs are finished shots from earlier runs, and the task is one step that a caller writes inline. Bring it back if a second musical field report asks for it by shape.

Problem and non-goals

Problem. A spoken scene (shots that carry their own dialogue audio) has to break into a song (shots lip-synced to one song track) with:

each dialogue shot loudness-matched;
the song entering under the last spoken line, so that song time cue_seconds lands exactly on the first song shot's frame 0;
the dialogue ducked after a delay;
the song shots playing over the unbroken song, with their own audio discarded.

The output has to be an ordinary run output, which gives it gallery, export_job, a realized workflow.json, a manifest and a shot map.

Non-goals:

Song → dialogue (the song ending into speech).
More than one song.
Dissolves at the seam.
A limiter.
Changing assemble-and-score.

Design

Correction to the proposal. The issue treats the provenance half as needing a new path. It doesn't. A task's output already gets the manifest, the realized workflow (which records every input reference) and export_job. Doing the join in dw closes both halves, so the external-output path is cut (above).

One new task, join_into_song (dw/tasks/join_into_song.py, registered in dw/tasks/task.py). It reuses concat_videos' frame and shot machinery and normalize_audio's pyloudnorm measurement.

argument	type	meaning
`dialogue`	list of videos	Spoken shots, in order; each keeps its own audio.
`song_shots`	list of videos	Sung shots, in order; their audio is discarded.
`song`	audio	The unbroken track the song shots were sliced from.
`cue_seconds`	number ≥ 0, default 0	Song time that lands on `song_shots[0]` frame 0. That is the slice start the first song shot was generated against.
`dialogue_target_lufs`	number, optional	Static gain per dialogue shot to this integrated loudness (measure, then apply). Omitted means no per-shot matching.
`duck_delay_ms`	≥ 0, default 0	How long after the song enters the dialogue starts to duck.
`duck_db`	≤ 0, default −12	The duck depth.
`duck_ramp_ms`	≥ 0, default 250	A linear ramp into the duck. It is not a step.

Behaviour:

The picture is dialogue + song_shots at the dialogue's fps. A mismatched fps or size is refused, as concat_videos refuses it.
Let D = the measured length of the joined dialogue (in samples, at the song's rate after resampling). The song is placed at D − cue_seconds·sr, and the placed song runs to the end of the picture.
cue_seconds longer than the dialogue is refused with a directed error. The song would have to start before the film does.
The song shorter than what the song shots need: pad with silence and warn song_short, as pair_audio(fit="video") does.
Mix: dialogue (loudness-matched, then ducked from song_entry + duck_delay) plus the placed song. After song_entry + duck_delay + duck_ramp the dialogue is silent past its own end, because the song shots contribute nothing.
Then the existing headroom and written-peak warnings apply (warn_without_headroom, warn_if_written_above_full_scale). This task does no final normalization, because the output level is the workflow's to decide: follow it with normalize_audio and then pair_audio, as music-video does. Q3 covers this.
Shots: one record per input, named shot@<key> from the step's references the way concat_videos names them. Samples are measured off the built waveform. The EXPECTED_SITES entry is "populates".
Result: {"content_type": "video/mp4"}, muxed by the existing path.
Validation: the numeric domains above go into dw/task_domains.py (cue_seconds ≥ 0, duck_delay_ms ≥ 0, duck_ramp_ms ≥ 0, duck_db ≤ 0). An empty dialogue or song_shots list is refused.

Surfaces:

Engine: one task.
Syntax: the task's arguments.
MCP: none. It appears through list_tasks/get_task automatically.
REST: none.
UI: none.

Found by the sweep and not part of this plan (Don decides whether each becomes its own issue):

concat_videos desyncs when inputs mix silent and sounded clips. An input without audio is skipped (dw/tasks/concat_videos.py:244-245) and not filled with silence, so every later shot's audio lands early. This is a latent bug. join_into_song will not inherit it, because it fills silence itself.
pair_audio doesn't load a string video (dw/tasks/pair_audio.py:252), while it does load a string audio. An output: video can't be re-scored without a previous step.
The normalize_audio arg table in docs/TASKS.md:646-650 omits target_lufs. This is a docs fix.

Stages

Stage A: the join_into_song task. Server. Est. $5.

Builds: the task, its domains and its EXPECTED_SITES entry.
Unit tests on synthetic clips:
- the song landing sample equals D − cue·sr exactly;
- the duck starts at the right sample and ramps linearly;
- song-shot audio is absent from the mix;
- per-shot LUFS lands within 0.5 LU;
- the shot map is partitioned and measured;
- the refusals.
Docs: a docs/TASKS.md section.
Acceptance intent (over MCP):
- A workflow that joins 2 dialogue clips and 2 song clips with a song asset, cue_seconds > 0, validates clean and runs. The output appears in list_gallery and export_job bundles it.
- get_job_workflow shows every input reference.
- get_gallery_metadata media.shots lists 4 shots named after their inputs.
- get_output_audio over the seam shows the song audible before the first song shot's first frame. The song's cue_seconds point, identifiable as an onset in a test song, falls on that frame within one frame.
- The dialogue level drops by ≈ duck_db after the delay and is gone once the dialogue ends.
- assess_output analyze_sync_drift shows no drift on the song shots.
- Refusals:
  - cue_seconds longer than the dialogue gets a directed error;
  - negative duck_delay_ms and positive duck_db are validation errors at their paths;
  - an empty song_shots is refused.
- Edge cases:
  - cue_seconds: 0 (the song starts exactly at the cut, with nothing under the last line);
  - a song too short for the song shots succeeds with a song_short warning.

Stage B: teach it. Plugin + docs only, no server deploy. Est. $2.

Builds:
- A recipe in plugins/dw/skills/minimax-music3/SKILL.md or minimax-h3/SKILL.md for dialogue into a song: generate the song shots against slice_audio slices starting at cue_seconds, then join_into_song, then normalize_audio, then pair_audio.
- It names cue_seconds as the slice start of the first song shot.
- A docs/WORKFLOW_GUIDE.md cross-reference.
- tests/test_plugin_skills.py stays green.
Acceptance intent: the tester reads get_guide and the skill text and follows the recipe verbatim on the stage A fixture, and it produces the same result. Refusals don't apply.

Order: A, then B. There are no cross-feature blockers.

Risks

The dialogue fps must match the song shots' fps. H3 is fixed-rate, so this should hold, but it is refused rather than resampled.
"Measure then static gain" on very short dialogue shots. pyloudnorm needs ≥ 400 ms of audio. A shorter shot is left unmatched with a warning rather than failing.
Encoder overshoot. The mux can add ~1.9 dB on AAC (per CLAUDE.md). This is covered by the existing written-peak warning, and by the recipe normalizing to −3 dBFS.

Questions

Q1. The task's name. Default: join_into_song. Say if you want another name. Only the name changes.
Q2. A bespoke task, or generic primitives? Default: the bespoke task, because only a task can measure D at run time. The alternative is mix_audio.offsets + a gain_audio ramp + a silence task. That reuses better, but costs three surface changes and still needs a hand-computed offset. If you pick it, stage A becomes three smaller changes and the recipe gains the arithmetic.
Q3. Should the task normalize its own output? Default: no. normalize_audio downstream decides the level, which matches music-video. If yes, the task adds target_lufs/peak_dbfs and the recipe loses a step.
Q4. The three sweep findings. Default: file (1) as a bug for the implementer, and fold (2) and (3) into it or into their own small fixes. Say "no" to drop them.

Reply in comments and hand back with owner:lead. Approve by adding status:plan-approved yourself. No agent may set it.

approved build smaller

Spec for plan v1, written by the tester (claude-opus-5-5 via anthropic). All cases are in regression-suite-complete.md and carry pending: lines.

#513, stage A (join_into_song task):

C-F151: the schema and guide can be discovered: params, defaults and domains.
C-F152: placement and provenance for 2 dialogue shots and 2 song shots.
- A −60 dB marker hole at song time cue_seconds must land on the seam.
- The song must be audible before the seam, and song-shot audio must be absent.
- Also checks: 4 media.shots named after their inputs, the headroom warning, no sync drift, get_job_workflow, list_gallery and export_job.
C-F153: ducking. Delay, depth −12 ±1 dB, a ramp rather than a step (duck_ramp_ms 2000), and the duck_db: 0 / duck_delay_ms: 0 boundaries.
C-F154: dialogue_target_lufs.
- Two shots 11 LU apart must each land within 1.5 LU of −23 and within 1 LU of each other.
- Omitting the target changes nothing.
- An 8-frame (<400 ms) shot is warned about, not failed.
C-F155: a dialogue input with no audio becomes silence of its own length, with no shift of what follows.
C-F156: refusals.
- Four domain errors at their paths.
- The legal zeros must validate.
- Empty song_shots and empty dialogue are refused.
- A cue of D + 1 frame is refused and names cue_seconds and the dialogue length. D − 1 frame succeeds.
- 24 vs 12 fps is refused.
C-F157: edges. cue_seconds: 0 starts the song on the seam. A short 16 kHz mono song is padded, resampled and warned song_short.

#514, stage B (recipe in skill and guide):

C-F158: the skill states the order slice_audio → join_into_song → normalize_audio → pair_audio, and that cue_seconds is the slice start. The guide cross-reference resolves. Followed verbatim on C-F152's fixture, the recipe reproduces the placement and shot records, and the headroom warnings clear.

Notes. None of these blocks a stage.

Size mismatch isn't covered because no CPU fixture differs in size. The fps mismatch is covered.
The output sample rate is assumed to be the song's. A different rate is a note in C-F152, not a finding, as long as the timing holds.
An empty list or a too-long cue may be refused at validate or at run time; either satisfies the cases. A succeeded job is the failure.
C-F154(c) and C-F155 build their short and silent inputs with loop_frames, slice_audio and pair_audio. If the join refuses those inputs for an unrelated reason (a rate that isn't carried through), the case says to treat that as a fixture problem, not a finding.

Close-out: every stage is closed and the design record is written. Lead agent, claude-opus-5-5 via anthropic.

Closed stages: #486 stage A: the join_into_song task #513 (stage A, the join_into_song task; one bounce, C-F156 arm 5, fixed in Result.save) and #486 stage B: teach the dialogue-into-song recipe #514 (stage B, the H3 skill recipe plus the WORKFLOW_GUIDE section; no bounces).
Design record: docs/proposals/complete/join-into-song-complete.md, with its entry in docs/proposals/todo.md. Commit 52706b1, merged to develop @ 207dc3e. It is docs only, so nothing was deployed.
Q4 sweep findings: filed as concat_videos shifts later audio early after a silent input; pair_audio doesn't load a string video (#486 sweep) #553 for the implementer: the concat_videos silent-input desync, and pair_audio not loading a string video. The third finding, target_lufs missing from TASKS.md, was already fixed by Limiter for normalize_audio: reach target_lufs past the peak ceiling (split from #467) #474.

verify: run every case whose source: names one of this feature's stages, and close the parent if they all pass.

Feature parent verified: all 8 cases pass (tester, claude-opus-5-5 via anthropic). Run over MCP against lem, develop @ 207dc3e, in workspace regression-complete.

I ran every case whose source: names stage #513 (C-F151–C-F157) or #514 (C-F158):

C-F151 PASS:
- list_tasks includes join_into_song.
- get_guide("tasks", section="join_into_song") documents the arguments and the defaults: cue 0, duck_delay 0, duck_db −12, duck_ramp 250.
- The schema's extra fps parameter is already covered by the pending harnest suite amendment.
C-F152 PASS (job f98e0a4a7c95):
- Windows: A −14.44 vs Ref −33.20 dBFS; H −74.98; R −17.28.
- The film is 496 frames with 4 shots. Max sync drift is −0.66 ms, no findings.
- get_job_workflow shows 5 asset refs, list_gallery lists the final film, and export_job bundled the assets and the film.
C-F153 PASS (job 2d6dfd2ded64):
- (a) d = 0.0, 0.0 and −12.0.
- (c) d = 0.0.
- (b) d is −0.75, −3.25, −4.13 and −9.70: monotonic, the first ≥ −6.5, the last within [−13, −8].
C-F154 PASS (job f0621448dd56):
- (a) W1 −23.00 LUFS, W2 −22.97.
- (b) W1 −16.42 vs ref −16.42; W2 −27.38 vs ref −27.39.
- (c) 504 frames, 5 shots. The short shot is left at its own level, with the warning "video 2 … 0.33 s, under the 400 ms loudness measurement needs".
C-F155 PASS (same job):
- The silent dialogue shot gives 4 shots at 0/124/248/372 (496 frames).
- e1 is −120 dBFS; e2 is −31.008 vs ref −31.008.
- Sync drift findings are empty.
C-F156 PASS:
- Out-of-range values: validate_workflow with cue −1, duck_delay −1, duck_ramp −1 and duck_db 3 returns valid:false with 4 errors, each at steps[0].task.arguments.<name>.
- Zeros: all four at 0 validate clean.
- Empty lists: song_shots: [] failed with "needs a non-empty list of videos for 'song_shots'", and dialogue: [] with the same message for 'dialogue'. Both failed at run time with an empty manifest.
- Cue past the dialogue: cue 10.375 failed with "cue_seconds=10.375 is longer than the 10.33 s of dialogue", and wrote no film. Cue 10.29 succeeded with a 496-frame film of 4 shots.
- 12 fps song shot: a 12 fps pair_audio copy in song_shots failed with "needs every video at one frame rate, got … 24 fps … video 3: 12 fps". The join wrote no film.
C-F157 PASS (job f0621448dd56):
- (a) 4 shots, 496 frames. g1 −19.953 vs −19.953; g2 −31.053 vs −31.053; g3 −17.27.
- (b) The short song is padded, with the warning "song is 4.96 s … padded with 6.37 s of silence". h1 −120; h2 −50.02.
C-F158 PASS (job 602e89ac70ea):
- Skill: the dw:minimax-h3 skill ("Dialogue into a song") states the order slice_audio (the first at cue_seconds) → join_into_song (same cue_seconds) → normalize_audio −3 dBFS → pair_audio.
- Guide: the skill's cross-reference, get_guide("workflows", section="A spoken scene breaking into a song"), exists. The guide says "The first slice starts at cue_seconds" and gives the same four steps.
- Recipe run: I followed the recipe verbatim on C-F152's marked song, with cue 3.0 and the fixture shots in place of the generated ones.
  - The first 10 frames of slice1 read −75.04 dBFS.
  - The final film is 496 frames, with 4 shots named as in C-F152.
  - The windows read H −78.65, and A −18.16 vs Ref −33.20 (+15 dB).
  - The number step raised no audio_clipped or audio_no_headroom warning; the only headroom warnings were on the unnormalized intermediate wavs.
  - analyze_sync_drift gives a max of −0.66 ms, no findings.

Every output these cases wrote was removed with delete_output(job_id=…), failed runs included.

docs(proposals): dkackman#486 - join_into_song design record

Navigation Menu

Repository navigation

idea: musical join task - spoken scene into song with the song entering under the last line, as a run output with provenance #486

(Alternative, not adopted) `join_into_song`, a musical join task, plus a `scene-into-song` template

Verdict: build smaller

Problem and non-goals

Design

New task `join_into_song` (in `dw/tasks/audio_utils.py` or its own module, registered in `dw/tasks/task.py`)

New template `templates/minimax/scene-into-song`

Stages

Stage 1: the `join_into_song` task (server)

Stage 2: `templates/minimax/scene-into-song` and skill text (server, for the template, plus plugin)

Risks

Questions

Plan v1: #486 musical join (a spoken scene breaking into a song)

Verdict: build smaller

Problem and non-goals

Design

Stages

Risks

Questions

14 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

idea: musical join task - spoken scene into song with the song entering under the last line, as a run output with provenance #486

Description

Activity

dkackman commented on Sep 26, 2026

(Alternative, not adopted) join_into_song, a musical join task, plus a scene-into-song template

Verdict: build smaller

Problem and non-goals

Design

New task join_into_song (in dw/tasks/audio_utils.py or its own module, registered in dw/tasks/task.py)

New template templates/minimax/scene-into-song

Stages

Stage 1: the join_into_song task (server)

Stage 2: templates/minimax/scene-into-song and skill text (server, for the template, plus plugin)

Risks

Questions

dkackman commented on Sep 26, 2026

Plan v1: #486 musical join (a spoken scene breaking into a song)

Verdict: build smaller

Problem and non-goals

Design

Stages

Risks

Questions

dkackman commented on Sep 27, 2026

14 remaining items

dkackman commented on Sep 27, 2026

dkackman commented on Sep 28, 2026

dkackman commented on Sep 28, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

(Alternative, not adopted) `join_into_song`, a musical join task, plus a `scene-into-song` template

New task `join_into_song` (in `dw/tasks/audio_utils.py` or its own module, registered in `dw/tasks/task.py`)

New template `templates/minimax/scene-into-song`

Stage 1: the `join_into_song` task (server)

Stage 2: `templates/minimax/scene-into-song` and skill text (server, for the template, plus plugin)