Skip to content

AArch64: Add SVE2 implementation of histogram computation - #4411

Merged
Cyan4973 merged 2 commits into
meta:devfrom
arpadpanyik-arm:hist_sve2
Jun 18, 2025
Merged

Cyan4973 merged 2 commits into
meta:devfrom
arpadpanyik-arm:hist_sve2

Conversation

@arpadpanyik-arm

Copy link
Copy Markdown
Contributor

The existing scalar implementation uses a 4-way pipelined histogram calculation which is very efficient on current CPUs. However, this can be further accelerated using the SVE2 HISTSEG instruction. Details are in the commit messages.

We also added some unit tests to the fuzzer.c to cover the existing and new SVE2 histogram functions, along with new SVE2-enabled QEMU runs in the CI (had to increase the timeout of the test-cli-tests from 200s to 800s).

Compression uplifts on a Neoverse V2 system, using Zstd-1.5.8 (e26dde3) as a baseline, compiled with
-O3 -march=armv8.2-a+sve2:

                Clang-20    GCC-14
1#silesia.tar:   +6.173%   +5.987%
2#silesia.tar:   +5.200%   +5.011%
3#silesia.tar:   +4.332%   +5.031%
4#silesia.tar:   +2.789%   +3.064%
5#silesia.tar:   +2.028%   +1.838%
6#silesia.tar:   +1.562%   +1.340%
7#silesia.tar:   +1.160%   +0.959%

The uplifts are much smaller at higher compression levels!

The existing scalar implementation uses a 4-way pipelined histogram
calculation which is very efficient on out-of-order CPUs. However,
this can be further accelerated using the SVE2 HISTSEG instructions -
which compute a histogram for 16 byte chunks in a vector register.

On a system with 128-bit vectors (VL128) we need 16 HISTSEG executions
to compute the histogram for the whole symbol space (0..255) of 16
bytes input. However we can only accumulate 15 of such 16 byte strips
before possible overflow. So we need to extend and save the 8-bit
histogram accumulators to 16-bit after every 240 byte chunks of input.
To store all in registers we would need 32 128-bit registers. Longer
SVE2 vectors could help here, if such machines become available.

The maximum input block size in Zstd is 128 KiB, so 16-bit accumulators
would not be enough. However an LZ pass will prepend the histogram
calculation, so it is impossible (my assumption) to overflow the 16-bit
accumulators.

The symbol distribution is also not uniform, the lower values are more
common, so we used a 3 pass algorithm to prevent stack spilling. In the
first pass we only compute histograms for 64 symbols (4-way SIMD) while
also computing the maximum symbol value. If we have symbol values
larger than 64 we start the second pass to compute the next 96 elements
of the histogram. The final pass calculates the remaining part of the
histogram (256 symbols in total) if needed. This split of histogram
generation gave the best overall results for performance.

This implementation is the best performing of a number of different
cache blocking schemes tested.

Compression uplifts on a Neoverse V2 system, using Zstd-1.5.8
(e26dde3) as a baseline, compiled with "-O3 -march=armv8.2-a+sve2":

                 Clang-20    GCC-14
 1#silesia.tar:   +6.173%   +5.987%
 2#silesia.tar:   +5.200%   +5.011%
 3#silesia.tar:   +4.332%   +5.031%
 4#silesia.tar:   +2.789%   +3.064%
 5#silesia.tar:   +2.028%   +1.838%
 6#silesia.tar:   +1.562%   +1.340%
 7#silesia.tar:   +1.160%   +0.959%
The following tests are included:
- Empty input scenario test.
- Workspace size and alignment tests.
- Symbol out-of-range tests.
- Cover multiple input sizes, vary permitted maximum symbol
  values, and include diverse symbol distributions.

These tests verifies count table correctness, maxSymbolValuePtr
updates, and error-handling paths. It enables automated regression
of core histogram logic as well.

@Cyan4973 Cyan4973 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a well located change, easy to follow, complete with published benchmark and a test set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants