Repository navigation
[Bug]: Enabling B-frames in FFMPEG hangs up with HQVBR #586
Description
Activity
- Could you please share detailed FFmpeg log?
- Could you please try AMF decoder? -hwaccel amf -hwaccel_output_format amf
- Please try to increase pool size: -hwaccel d3d11va -extra_hw_frames 30
Reacted by Pyotr BaykalovThank you!
I should have said that hwaccel has zero impact on the problem, even dropping it entirely does not fix the issue.ffmpeg -i .\out.mp4 -c:v h264_amf -usage transcoding -rc hqvbr -b 8M -preanalysis true -bf 3 temp.mp4 ffmpeg version 8.0.1-full_build-www.gyan.dev Copyright (c) 2000-2025 the FFmpeg developers built with gcc 15.2.0 (Rev8, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-lcms2 --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-libdvdnav --enable-libdvdread --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libopenjpeg --enable-libquirc --enable-libuavs3d --enable-libxevd --enable-libzvbi --enable-liboapv --enable-libqrencode --enable-librav1e --enable-libsvtav1 --enable-libvvenc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxeve --enable-libxvid --enable-libaom --enable-libjxl --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-openal --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-liblc3 --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint --enable-whisper libavutil 60. 8.100 / 60. 8.100 libavcodec 62. 11.100 / 62. 11.100 libavformat 62. 3.100 / 62. 3.100 libavdevice 62. 1.100 / 62. 1.100 libavfilter 11. 4.100 / 11. 4.100 libswscale 9. 1.100 / 9. 1.100 libswresample 6. 1.100 / 6. 1.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\out.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf62.3.100 Duration: 00:04:43.57, start: 0.000000, bitrate: 6181 kb/s Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 6002 kb/s, 59.99 fps, 60 tbr, 90k tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 164 kb/s, start 0.018000 (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] Please use -b:a or -b:v, -b is ambiguous Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_amf)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) Press [q] to stop, [?] for help 2026-01-27 21:01:01.6748092 [V:onnxruntime:Default, env.cc:885 onnxruntime::WindowsEnv::InitializeCpuInfo] Found total 8 core(s) from windows system: 2026-01-27 21:01:01.6770067 [V:onnxruntime:Default, env.cc:886 onnxruntime::WindowsEnv::InitializeCpuInfo] core 1 consist of logical processors: 1 2 core 2 consist of logical processors: 3 4 core 3 consist of logical processors: 5 6 core 4 consist of logical processors: 7 8 core 5 consist of logical processors: 9 10 core 6 consist of logical processors: 11 12 core 7 consist of logical processors: 13 14 core 8 consist of logical processors: 15 16 Output #0, mp4, to 'temp.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf62.3.100 Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 8000 kb/s, 60 fps, 15360 tbn (default) Metadata: encoder : Lavc62.11.100 h264_amf handler_name : VideoHandler vendor_id : [0][0][0][0] Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Metadata: encoder : Lavc62.11.100 aac handler_name : SoundHandler vendor_id : [0][0][0][0] frame= 1 fps=0.1 q=-0.0 size= 0KiB time=-00:00:00.01 bitrate=N/A speed=N/A elapsed=0:00:11.33Maybe it will be also useful to mention that there is zero codec utilization.
-loglevel verbose
-extra_hw_frames 30
-hwaccel amf -hwaccel_output_format amfReacted by Pyotr BaykalovAny update?
Reacted by Pyotr BaykalovSame-ish behavior without
hwaccelwhich proves that it's specific encoding parameter combo which is the issue IMO:ffmpeg -loglevel verbose -i out6.mp4 -c:v h264_amf -usage transcoding -rc hqvbr -b 8M -preanalysis true -bf 3 temp.mp4 ffmpeg version 8.1-full_build-www.gyan.dev Copyright (c) 2000-2026 the FFmpeg developers built with gcc 15.2.0 (Rev11, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-cairo --enable-fontconfig --enable-iconv --enable-gnutls --enable-lcms2 --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-libdvdnav --enable-libdvdread --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libopenjpeg --enable-libquirc --enable-libuavs3d --enable-libxevd --enable-libzvbi --enable-liboapv --enable-libqrencode --enable-librav1e --enable-libsvtav1 --enable-libvvenc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxeve --enable-libxvid --enable-libaom --enable-libjxl --enable-libsvtjpegxs --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-openal --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-liblc3 --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint --enable-whisper libavutil 60. 26.100 / 60. 26.100 libavcodec 62. 28.100 / 62. 28.100 libavformat 62. 12.100 / 62. 12.100 libavdevice 62. 3.100 / 62. 3.100 libavfilter 11. 14.100 / 11. 14.100 libswscale 9. 5.100 / 9. 5.100 libswresample 6. 3.100 / 6. 3.100 [in#0 @ 00000226956b4900] UDTA parsing failed retrying raw [h264 @ 00000226956b5cc0] Reinit context to 3840x2160, pix_fmt: yuvj420p Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out6.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf62.12.100 Duration: 00:00:12.38, start: 0.000000, bitrate: 8118 kb/s Stream #0:0[0x1](und): Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive, left), 3840x2160 [SAR 1:1 DAR 16:9], 7979 kb/s, 60 fps, 60 tbr, 15360 tbn (default) Metadata: handler_name : VideoHandler encoder : Lavc62.28.100 h264_amf Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Metadata: handler_name : SoundHandler name : desktop Please use -b:a or -b:v, -b is ambiguous [out#0/mp4 @ 00000226956cd600] No explicit maps, mapping streams automatically... [vost#0:0/h264_amf @ 00000226956bd200] Created video stream from input stream 0:0 [aost#0:1/aac @ 0000022695cefc80] Created audio stream from input stream 0:1 File 'temp.mp4' already exists. Overwrite? [y/N] y Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_amf)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) [vost#0:0/h264_amf @ 00000226956bd200] Starting thread... [aost#0:1/aac @ 0000022695cefc80] Starting thread... [vf#0:0 @ 00000226961d51c0] Starting thread... [af#0:1 @ 000002269601b840] Starting thread... [vist#0:0/h264 @ 00000226956bcdc0] [dec:h264 @ 00000226961d5bc0] Starting thread... [aist#0:1/aac @ 00000226956cd440] [dec:aac @ 0000022695cf2440] Starting thread... [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 00000226956b46c0] Starting thread... Press [q] to stop, [?] for help [h264 @ 0000022695d01bc0] Reinit context to 3840x2160, pix_fmt: yuvj420p [graph_-1_in_0:1 @ 00000226956ca9c0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:stereo [graph -1 input from stream 0:0 @ 00000226956cb340] w:3840 h:2160 pixfmt:yuvj420p tb:1/15360 fr:60/1 sar:1/1 csp:bt709 range:pc alpha:unspecified [auto_scale_0 @ 00000226961b6280] w:iw h:ih flags:'' interl:0 [format @ 00000226961b6900] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format' [auto_scale_0 @ 00000226961b6280] w:3840 h:2160 fmt:yuvj420p csp:bt709 range:pc sar:1/1 -> w:3840 h:2160 fmt:yuv420p csp:bt709 range:pc sar:1/1 flags:0x00000004 [auto_scale_0 @ 00000226961b6280] [framesync @ 0000022695d0c850] Selected 1/15360 time base [auto_scale_0 @ 00000226961b6280] [framesync @ 0000022695d0c850] Sync level 1 [swscaler @ 000002269685e300] deprecated pixel format used, make sure you did set range correctly [AMF @ 0000022695cf0600] AMF initialisation succeeded via D3D11. 2026-07-20 22:31:00.2939734 [V:onnxruntime:Default, env.cc:885 onnxruntime::WindowsEnv::InitializeCpuInfo] Found total 8 core(s) from windows system: 2026-07-20 22:31:00.2959077 [V:onnxruntime:Default, env.cc:886 onnxruntime::WindowsEnv::InitializeCpuInfo] core 1 consist of logical processors: 1 2 core 2 consist of logical processors: 3 4 core 3 consist of logical processors: 5 6 core 4 consist of logical processors: 7 8 core 5 consist of logical processors: 9 10 core 6 consist of logical processors: 11 12 core 7 consist of logical processors: 13 14 core 8 consist of logical processors: 15 16 Output #0, mp4, to 'temp.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf62.12.100 Stream #0:0(und): Video: h264, 1 reference frame (avc1 / 0x31637661), yuv420p(pc, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 8000 kb/s, 60 fps, 15360 tbn (default) Metadata: encoder : Lavc62.28.100 h264_amf handler_name : VideoHandler Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default) Metadata: encoder : Lavc62.28.100 aac handler_name : SoundHandler name : desktop [out#0/mp4 @ 00000226956cd600] Starting thread... frame= 145 fps=1.5 q=-0.0 size= 2048KiB time=00:00:02.21 bitrate=7568.8kbits/s speed=0.0232x elapsed=0:01:35.38Thanks for the update. I confirm that the bug reproduced with the latest public driver and fixed internally. The fix will appear in the next major driver release. Current public driver is 26.10.21.05 (internal versioning can be found in readme for driver). Watch for 26.20.x.x.
Reacted by Pyotr BaykalovWorks now!
@MikhailAMD
Actually, there's different command which also hangs FFMPEG, even with 26.9.2 driver:ffmpeg -hwaccel amf -hwaccel_output_format amf -i .\av1.mp4 -c:v h264_amf -usage transcoding -rc hqvbr -b 8M -preanalysis true -bf 1 -pa_lookahead_buffer_depth 41 temp.mp4But these work:
ffmpeg -i .\av1.mp4 -c:v h264_amf -usage transcoding -rc hqvbr -b 8M -preanalysis true -bf 1 -pa_lookahead_buffer_depth 41 temp.mp4
ffmpeg -hwaccel amf -hwaccel_output_format amf -i .\av1.mp4 -c:v h264_amf -usage transcoding -rc hqvbr -b 8M -preanalysis true -bf 1 temp.mp4Please try -extra_hw_frames 50. It is usually needed for PA and look ahead.
I don't have access to PC right now but will ask FFmpeg folks to check.@pbaykalov
I'm investigating this issue.The problem is reproduced when using HQVBR + B-frames with a high pa_lookahead_buffer_depth.
As a temporary workaround (not a fix), I recommend avoiding explicitly setting pa_lookahead_buffer_depth.Reacted by Pyotr BaykalovHi @pbaykalov
I investigated this further.
The hang appears to be caused by HW surface starvation in the zero-copy AMF decode → encode pipeline.
With HQVBR + PreAnalysis + B-frames and a large pa_lookahead_buffer_depth, the encoder retains decoder-owned HW surfaces for lookahead buffering.
Eventually the decoder runs out of free surfaces and ff_encode_get_frame() returns EAGAIN, while the encoder still returns AMF_REPEAT from QueryOutput().
At that point the pipeline stops making progress.
The AMF decoder currently uses a default surface_pool_size of 36, and the corresponding AVHWFramesContext is initialized with surface_pool_size + 8.I found two workarounds:
- Increase -extra_hw_frames on the decoder side.
For example, with pa_lookahead_buffer_depth=41, values around 14+ avoided the stall in my tests. - Use -copy_output 1.
This also avoids the issue, but introduces an additional surface copy and therefore breaks strict zero-copy transcoding.
So the earlier recommendation to avoid explicitly setting a high pa_lookahead_buffer_depth is still a valid temporary workaround, but the underlying issue seems to be insufficient decoder surface headroom when the encoder retains many frames.
Reacted by Pyotr Baykalov- Increase -extra_hw_frames on the decoder side.
Describe the bug
Enabling B-frames in FFMPEG hangs up with HQVBR on frame 1.
ffmpeg -hwaccel d3d11va -i in.mp4 -c:v h264_amf -usage transcoding -rc hqvbr -b 8M -preanalysis true -bf 3 out.mp4- hangs upffmpeg -hwaccel d3d11va -i in.mp4 -c:v h264_amf -usage transcoding -rc cbr -b 8M -preanalysis true -bf 3 out.mp4- works fineffmpeg -hwaccel d3d11va -i in.mp4 -c:v h264_amf -usage transcoding -rc hqvbr -b 8M -preanalysis true out.mp4- works fineTo Reproduce
Steps to reproduce the behavior:
ffmpeg -hwaccel d3d11va -i in.mp4 -c:v h264_amf -usage transcoding -rc cbr -b 8M -preanalysis true -bf 3 out.mp4Setup (please complete the following information):
Debug Log (please upload or paste):
N/A
Expected behavior
Can't see why B-frames should not work with HQVBR specifically.
Screenshots
N/A
Additional context
I've set up HQVBR in OBS 32 successfully with B-frames enabled.