Skip to content

perf: float32 Add/Sub/Mul/Div kernels on six targets; *Into writes typed results in place - #59

Merged
tannevaled merged 3 commits into
mainfrom
float32-elementwise
Oct 19, 2026
Merged

tannevaled merged 3 commits into
mainfrom
float32-elementwise

Conversation

@tannevaled

@tannevaled tannevaled commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

float32 elementwise arithmetic was the slowest row against NumPy. On one POWER9 core with NumPy 2.5.3, float32 Add was 7.1× slower at 2¹⁴ (33.8 µs against 4.74 µs). A profile put 57% of the time in binNum[float32], a scalar add per element with two bounds checks.

Changes

  • Bounds checks (typed.go): the elementwise loops reslice their operands to len(d) once, so the compiler drops the per-element checks (verified with -d=ssa/check_bce). float32 and int32 Add gain 7–12% on POWER9.

  • kernels.Bin32: Add/Sub/Mul/Div on 16 floats per iteration. Each lane is one IEEE operation, so the result is bit-identical to the scalar loop.

    target kernel gate
    amd64 SSE2 ADDPS… baseline
    arm64 NEON .S4 baseline
    ppc64le VSX xvaddsp… (go-asmgen v0.19.0) baseline
    s390x vfasb… (go-asmgen v0.19.0) useVXE (z14+)
    loong64 LSX vfadd.s… useLASX
    riscv64 RVV E32, strip-mined useRVV

    binFloat becomes the non-generic binFloat32, since only float32 reaches it. The scalar fallback lives in the per-arch files, so the coverage gate has no unreachable line.

  • *Into (arith.go): when out already has the result dtype, the operation writes into it directly. Before, it computed a new array and then copied it into out.

  • asmgen/regen: one Go command regenerates all six .s files, which go generate cannot do from one host.

  • BenchmarkDTypeAddInto: the kernel without the allocation.

Measured on POWER9, one core. Base and head ran sequentially, not alternated, on an idle host; the changes are far outside the noise.

float32 before after NumPy 2.5.3
Add, 2¹⁴ 33.6 µs 15.2 µs (−55%) 4.74 µs
Add, 2²⁰ 1.70 ms 0.537 ms (−68%) 0.342 ms
AddInto, 2¹⁴ — 4.35 µs 4.23 µs (out=)
AddInto, 2²⁰ — 294 µs 366 µs (out=)

The controls, float32 and float64 Sum, are flat. float64 Add at 2¹⁴ came out +4.4% in this sequential run, though its code path is unchanged.

Tests on real hardware. The kernel tests (special values, every length to 70, in place) and the PROT_NONE fence pass on Zen 3, Apple M, POWER9, IBM Z, SpacemiT X60 (V) and Loongson 3C5000L (LASX). A sabotaged NEON kernel (one VFSUB in add32NEON) fails TestBin32MatchesScalar.

Still open (written in docs/perf.md): the allocating Add is still 1.5–3.2× slower than NumPy's a+b, float64 included (21.9 µs against 8.48 µs at 2¹⁴). The allocation and the GC account for that, not the kernel. Until it is addressed, hot loops should use *Into.

Audit.

  • Bin32 reslices a and b to len(dst) before calling a kernel, so a short operand panics in Go and is never read past its end.
  • A partial overlap between out and an operand cannot reach the in-place path. A probe confirmed that out views with an offset are refused as non-contiguous, in both overlap directions.
  • Coverage gate (second commit): the gate read the rounded total of go tool cover -func. It printed 100.0% with two statements never run: fill32's error return and the complex Log2. Both are now tested, and the gate counts uncovered blocks with fleettools' covblocks (pinned v0.27.0), which reports 0.

🤖 Generated with Claude Code

tannevaled and others added 3 commits October 18, 2026 22:57
…ped results in place

Against NumPy 2.5.3 on one POWER9 core, float32 Add was 7.1x slower at
2^14 (33.8 us against 4.74). A profile put 57% in binNum[float32]: a
scalar add per element with two bounds checks.

- typed.go: the elementwise loops reslice their operands to len(d), so
  the compiler drops the per-element bounds checks (checked with
  -d=ssa/check_bce): float32/int32 Add -7..-12% on POWER9.
- kernels.Bin32: 16 floats per iteration with SSE2 (amd64), NEON .S4
  (arm64), VSX xvaddsp... (ppc64le), z14 vfasb... (s390x, behind
  useVXE), LSX vfadd.s... (loong64, behind useLASX) and RVV E32
  (riscv64, behind useRVV). One IEEE operation per lane: bit-identical to
  the scalar loop. binFloat becomes binFloat32 (only float32 reaches it).
- arithInto: when out has the result dtype, write into it directly
  instead of computing a new array and copying it.
- asmgen/regen: one Go command regenerates all six .s files.
- benchmarks: BenchmarkDTypeAddInto.

POWER9: float32 Add -55% (2^14) / -68% (2^20). float32 AddInto 4.35 us
(NumPy out=: 4.23) and 294 us (NumPy: 366). The kernel tests and the
PROT_NONE fence pass on Zen 3, Apple M, POWER9, IBM Z, SpacemiT X60 and
Loongson 3C5000L; a sabotaged NEON kernel fails them.

Still open, written in docs/perf.md: the allocating Add is 1.5-3.2x
slower than NumPy's a+b, float64 included. The allocation and the GC
account for the difference, not the kernel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gate read the total of `go tool cover -func`, which rounds to one
decimal: it printed 100.0% with two statements never run
(random/generator.go fill32's error return, and the complex Log2). Both
are now tested, and the gate counts uncovered blocks with fleettools'
covblocks (pinned v0.27.0), which fails on any.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The block-count gate found linalg/eig.go:718 never run on amd64. The
branch rescales h(m+2, m+1) when a QR sweep starts at m > l with rows
below it. On arm64 TestEigBranches reached it by rounding alone; on
amd64 nothing did, and the rounded percentage had hidden that.

A new case plants two consecutive 1e-9 subdiagonals in an upper
Hessenberg matrix: each is far above the deflation threshold and their
product far below the two-small-subdiagonals test, so the sweep starts
at m = 3 on any arch. Checked: the block is run on arm64 and on amd64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 8979001 into main Oct 19, 2026
12 checks passed
@tannevaled
tannevaled deleted the float32-elementwise branch October 19, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant