Skip to content

perf: float32 sum blocks on SSE2 and NEON; integer widening folds without bounds checks - #54

Merged
tannevaled merged 5 commits into
mainfrom
float32-block-kernels
Oct 18, 2026
Merged

tannevaled merged 5 commits into
mainfrom
float32-block-kernels

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

float32 sums. NumPy's pairwise sum keeps eight accumulators so that the inner loop can be vectorised "without changing summation ordering".

  • kernels.AddBlocks8F32 is that loop: blocks of eight float32 added lane by lane, one IEEE float32 addition per lane per block, in order.
  • amd64 uses SSE2 (ADDPS on two XMM registers), arm64 uses NEON (VFADD .S4 on two V registers), and the other architectures keep the scalar loop.
  • pairwiseSum calls it, so float32 sums are unchanged bit for bit.

Tests:

  • The kernel matches the scalar loop bit for bit over 1–40 blocks, with NaN, the infinities, −0, subnormals and overflow.
  • Refusals: an incomplete block panics, and an empty slice is a no-op.
  • A PROT_NONE fence guards both operands.
  • Both tests can fail: a kernel adding from the wrong register fails the match test, and one reading a block too many faults in the fence.
  • Run natively on arm64 and on linux/amd64 (docker). Coverage 100%, and the kernels package compiles for all seven targets.

Measured with the new bench workflow on the hosted runners: base main and head alternated 10 times, one thread, benchstat, p ≤ 0.002.

vs main amd64 (EPYC 9V45) arm64
float32 Sum, 2¹⁴ −50% −33%
float32 Sum, 2²⁰ −49% −43%
int32 Sum, 2¹⁴ −12% −12%
int32 Sum, 2²⁰ −15% −13%
float64 / int64 (controls) ~ ~

A defect the first A/B exposed. With only the kernel added, int32 Sum was 48–56% slower on amd64 (reproduced with 10 runs), even though its code had not changed.

  • Cause: the disassembly showed the same loop byte for byte, moved 224 bytes so that it straddled a 64-byte line. The loop also reloaded the operation from the stack on every element.
  • Fix: one loop per operation and four accumulators. This is exact, because integer addition and multiplication wrap modulo 2⁶⁴, so the order changes nothing.
  • And a second one: the first unrolled version still had four bounds checks per iteration (amd64 disassembly). It now consumes the slice rather than indexing i+1…i+3, and the loop has none.

Left: ppc64le, s390x, loong64 and riscv64 keep the scalar block loop. ppc64le needs xvaddsp, which Go's assembler lacks; go-asmgen/asmgen#27 (draft) adds the VSX single-precision family to its registry, with references from GNU as and llvm-mc.

🤖 Generated with Claude Code

tannevaled and others added 5 commits October 18, 2026 10:01
NumPy's pairwise sum keeps eight accumulators so that the inner loop can
be vectorised "without changing summation ordering". AddBlocks8F32 is
that loop: blocks of eight float32 added lane by lane into r, one IEEE
float32 addition per lane per block, in order; SSE2 on amd64 (ADDPS on
two XMM), NEON on arm64 (VFADD .S4 on two V), the scalar loop elsewhere.
pairwiseSum calls it, so float32 sums are unchanged bit for bit.

Tests: kernel vs scalar bit for bit over 1-40 blocks with NaN, the
infinities, -0, subnormals and overflow; refusals; a PROT_NONE fence on
both operands. Proven to bite: a wrong-register kernel fails the match
test, a kernel reading one block too many faults in the fence. Run on
arm64 natively and on linux/amd64 (docker). Coverage 100%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ration

On amd64 the single loop reloaded the operation from the stack every
element, and the A/B on the hosted EPYC showed int32 Sum 48-56% slower
on this branch with byte-identical loop code: the new kernel code moved
the loop across a 64-byte line. Four independent accumulators (exact:
integer + and * wrap modulo 2^64, so order changes nothing) and one loop
per operation make it faster and far less sensitive to where it lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Indexing s[i+1..i+3] left four bounds checks an iteration in the amd64
code (the compiler proved none of them); consuming the slice under
len(s) >= 4 proves all four. Disassembly: the unrolled int32 loop is now
four MOVSXD loads and no check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the first A/B exposed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d a Zen 3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit c44fc7b into main Oct 18, 2026
12 checks passed
@tannevaled
tannevaled deleted the float32-block-kernels branch October 18, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant