Repository navigation
concat_videos shifts later audio early after a silent input; pair_audio doesn't load a string video (#486 sweep) #553
Description
Filed by the lead agent, model claude-opus-5-5 via anthropic, at #486's close-out. These are the plan's Q4 sweep findings, which Don approved at their default: file them for the implementer. Rechecked against develop @ d9551b7.
1. concat_videos desyncs audio when a silent input sits among inputs with sound
dw/tasks/concat_videos.py:260-261: an input whose waveforms[index] is None is skipped (continue). No silence of its own length is added. Its frames are still added to the picture, so every later shot's audio starts early by the silent shot's length. The shot records inherit the same drift: start_sample is read from audio.shape[1].
Expected: a silent input contributes silence of frames_to_samples(len(frames), fps, sample_rate). That is what join_into_song does (#513, C-F155). When no input has audio, the output audio stays None, as it does today.
Test: 3 inputs of the form [sound, silent, sound]. The third shot's audio must start at the sample of its first frame, and media.shots must stay contiguous.
2. pair_audio loads a string audio but not a string video
dw/tasks/pair_audio.py:265 loads audio when it is a path string. video at :288 expects frames or an AudioVideo, so re-scoring an output:/asset: video needs an extra loading step first. The fix is to load a string video the same way (confined, as the audio path is).
(The sweep's third finding, docs/TASKS.md missing normalize_audio's target_lufs, was already fixed by #474.)
Activity
dkackman commentedon Sep 28, 2026
triage: work — two independent engine fixes, neither on develop (checked 207dc3e). (1) dw/tasks/concat_videos.py ~:260: a silent input (
waveforms[index] is None) must append silence offrames_to_samples(len(frames), fps, sample_rate)(as join_into_song does, #513) once any input has audio — including a silent input before the first sounded one; output audio stays None when no input has any. Test [sound, silent, sound]: shot 3's audio starts at its first frame's sample, shots contiguous. (2) dw/tasks/pair_audio.py: load a stringvideo(asset:/output:/path) confined the same way the stringaudiopath is. Don approved both at #486's close-out, so no escalation. Note #551's batch also edits concat_videos.py'svideosdocstring — rebase over it if that lands first.Triaged by the implementer agent, model
claude-opus-5-5viaanthropic.fix(mcp): #553 - fill silent concat_videos inputs, load a string pair…
fix(mcp): #553 - satisfy ruff format on the concat_videos/pair_audio fix
dkackman commentedon Sep 28, 2026
Fixed and deployed (model: claude-sonnet-5, Anthropic).
concat_videos (
dw/tasks/concat_videos.py): a video contributing no soundtrack among inputs where at least one other does now fills silence of its own length (frames_to_samples(len(frames), fps, sample_rate)) instead of being skipped viacontinue- matching the #513join_into_songprecedent - so a silent shot no longer shifts every later shot's audio early, whether the silent shot is first, middle, or last. Emitsconcat_videos: <name> carries no audio - filled with silenceviaemit_logfor caller visibility, same style asjoin_into_song. When no input carries audio, output audio still staysNone(non-regression, covered by test). Note:fpsis now required whenever any input in the list is silent (needed to compute the fill length) - all three real catalog templates that callconcat_videos(music-video,dialogue-short,assemble-and-score) already passfps, so no template needed a change.pair_audio (
dw/tasks/pair_audio.py):videonow loads a string path or{"location": ...}dict the same wayconcat_videosandget_last_framedo (viais_video_location/load_audio_video), matching the loadingaudioalready had. A workflow naming its video by anasset:/output:reference (a plain string by the time the task runs) now works instead of failing.Shipped as:
3ca4de3- fix(mcp): concat_videos shifts later audio early after a silent input; pair_audio doesn't load a string video (#486 sweep) #553 - fill silent concat_videos inputs, load a string pair_audio video (substantive fix + tests)2b5610a- fix(mcp): concat_videos shifts later audio early after a silent input; pair_audio doesn't load a string video (#486 sweep) #553 - satisfy ruff format on the concat_videos/pair_audio fix (formatting fix-forward)Merged to
develop, pushed, and deployed to lem (ssh lem 'scripts/deploy.sh develop') - confirmed healthy at2b5610a. Full test suite green (7016 passed, 14 skipped, 1 xfailed) before hand-off.Possible regression case for
regression-suite-complete.md(concat_videos is used by every multi-shot template, so a silence-shift regression here is easy to miss and only shows up as subtly-wrong sync): callconcat_videoswith avideoslist where the first entry has no audio and a later entry does, passingfps; expect the total audio duration to equal the sum of all shots' durations (not just the sounded ones), and the sounded shot's audio to start at its own shot's frame boundary rather than earlier. Leaving it to the tester to add once verified live, per the role.dkackman commentedon Sep 28, 2026
Verified over MCP against lem
develop@ 1175570 by the tester agent, modelclaude-opus-5-5viaanthropic.Inputs (workspace
qa-ep80):asset:qa-cast/ep4-shot1-amnesty.mp4andasset:qa-cast/ep4-shot2-desk.mp4. Each is 124 frames, 24 fps, 32 kHz stereo. The silent input isvideo_framesof shot1, which has frames and no audio.1. concat_videos [sound, silent, sound], fps=24 (job
a2a740f77037): passes.media.shotsis contiguous: starts 0 / 165333 / 330666, lengths 165333 / 165333 / 165334.Adjacent cases:
ba5c00de7131):start_sample/num_samplesare null. The output audio staysNone, as it did before the fix.2. pair_audio with a string video: passes.
video: "asset:qa-cast/ep4-shot2-desk.mp4",audio: asset:qa-cast/hal-voice.wav,fit: "video"succeeded. The output is 124 frames, 5.167 s, at 24 kHz.video: "output:<run>/final/first-0.0.mp4"(an earlier concat's output) also succeeded. It loaded all 248 frames, and fit padded the track to the 10.33 s cut. The shot records came through with the input.Both test runs were deleted afterwards. The proposed regression case goes in
regression-suite-complete.md.fix(mcp): dkackman#562 - conform each concat_videos input's audio to …