Skip to content

Fix quadratic buffer growth in lossy WebP Vp8BitWriter - #3207

Merged
JimBobSquarePants merged 2 commits into
SixLabors:mainfrom
gianlucaratta:fix/webp-vp8-bitwriter-resize
Oct 9, 2026
Merged

JimBobSquarePants merged 2 commits into
SixLabors:mainfrom
gianlucaratta:fix/webp-vp8-bitwriter-resize

Conversation

@gianlucaratta

@gianlucaratta gianlucaratta commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
  • I have provided test coverage for my change (where applicable)

Description

Fixes #3206

Vp8BitWriter.BitWriterResize compared the needed size against private readonly int maxPos, which is set to 0 in the constructor and never updated. The early return therefore never fired and ResizeBuffer(0, neededSize) computed the new size from a base of 0, i.e. neededSize rounded up to the next KiB. As a result the partition buffer was reallocated and copied every time the encoded output crossed a 1 KiB boundary (the very first flush even shrank the expected-size buffer down to 1 KiB), so allocations were quadratic in the encoded size.

libwebp updates bw->max_pos = new_size after resizing (src/utils/bit_writer_utils.c), and Vp8LBitWriter.BitWriterResize already grows from this.Buffer.Length. This PR removes the dead maxPos field and compares against / grows from this.Buffer.Length, so the buffer only reallocates when it is actually full and grows geometrically (1.5x, rounded up to the next KiB).

Flush is the only caller: it requests run + 1 bytes and then writes exactly run + 1 bytes starting at pos, so checking pos + extraSize <= Buffer.Length guarantees capacity for every write. NumBytes (pos) is unchanged, so WriteToStream / WriteToBuffer emit the same bytes.

Tests

  • New Vp8BitWriterTests:
    • BitWriterResize_WithinCapacity_DoesNotReallocate: a resize request that fits the current buffer keeps the same array instance.
    • BitWriterResize_GrowsBufferGeometrically: writes 256 KiB through the writer and counts buffer replacements (12 with the fix, 256 before), and checks Buffer.Length >= NumBytes.
    • Both fail on main and pass with the fix.
  • All WebP tests pass (dotnet test -c Release --filter "FullyQualifiedName~Webp": 444 passed, net10.0).
  • Encoded output is byte-identical before/after: 42 files compared (the Lossy WebP encoder: Vp8BitWriter.maxPos is never updated, buffer is reallocated every 1 KiB (quadratic allocations) #3206 repro and a heavier noise image at 3 sizes, plus Calliphora.jpg, Bike.png, rgb-48bpp.png and alpha-blend-2.webp at Quality 10/75/100 x Method Fastest/Default/BestQuality).

Before / after (lossy, Quality = 90, default method, .NET 10, Release, Windows 11 x64; GC.GetTotalAllocatedBytes(true) around a single Save):

Repro from #3206:

Size Output Alloc. before Alloc. after Gen2 GCs Time
1000x750 283 KiB 36.7 MiB 2.5 MiB 0 → 0 0.33 → 0.33 s
2000x1500 1,132 KiB 558.0 MiB 5.5 MiB 8 → 0 0.83 → 0.71 s
4000x3000 4,768 KiB 9,897.0 MiB 17.4 MiB 282 → 0 4.7 → 3.1 s

Noisier image (gradient + ±64 per-channel noise, larger output):

Size Output Alloc. before Alloc. after Gen2 GCs Time
1000x750 465 KiB 99.9 MiB 3.0 MiB 2 → 0 0.41 → 0.35 s
2000x1500 1,848 KiB 1,548.5 MiB 7.4 MiB 34 → 0 1.10 → 0.77 s
4000x3000 7,401 KiB 24,797.9 MiB 24.9 MiB 686 → 0 7.6 → 3.0 s

Vp8BitWriter.BitWriterResize compared the needed size against a
readonly maxPos field that was initialized to 0 and never updated, so
the early return never fired and ResizeBuffer always grew from a base
of 0. The partition buffer was therefore reallocated and copied every
time the output crossed a 1 KiB boundary (the first call even shrank
the expected-size buffer to 1 KiB), making allocations quadratic in the
encoded size: a 4000x3000 lossy Quality=90 encode allocated ~24 GiB.

Compare against and grow from the current buffer length instead, which
mirrors libwebp updating max_pos after each resize and matches
Vp8LBitWriter. Encoded output is unchanged.
@CLAassistant

CLAassistant commented Oct 9, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@JimBobSquarePants
JimBobSquarePants merged commit ec0c47e into SixLabors:main Oct 9, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lossy WebP encoder: Vp8BitWriter.maxPos is never updated, buffer is reallocated every 1 KiB (quadratic allocations)

3 participants