adler32

README

adler32

A drop-in fast path for the Adler-32 checksum, bit-for-bit identical to the standard library's hash/adler32. The bulk of the input is summed by a SIMD kernel generated by go-asmgen; the short tail reuses a hash/adler32-equivalent scalar loop, so the result always equals adler32.Checksum. Pure Go, CGO_ENABLED=0, stable Go, no GOEXPERIMENT.

sum := adler32.Checksum(data)   // == hash/adler32.Checksum(data)

h := adler32.New()              // hash.Hash32, == hash/adler32.New()
h.Write(data)
sum = h.Sum32()

API mirrors hash/adler32: Checksum and New (a hash.Hash32).

Status per architecture

arch	kernel	notes
amd64	SSE3/SSSE3 (2× unroll) + AVX2 (4× unroll, paired deferred-widen, runtime dispatch via x/sys/cpu)	PMADDUBSW weighted sum + PSADBW byte sum, deferred s1 carry
riscv64	RVV (runtime dispatch via x/sys/cpu HasV)	length-agnostic VWMULU weighted sum + VWREDSUMU; scalar fallback without V
arm64	NEON on Go 1.27+, scalar on stable	needs the integer VUMULL, upstreamed in Go 1.27 (see below)
ppc64le	VSX / AltiVec	VMULEUB/VMULOUB widening byte multiplies for the weighted sum, word-lane accumulation; qemu-validated (power9), native perf pending
s390x	vector facility (big-endian; runtime dispatch via x/sys/cpu HasVX)	VSUMB byte sum + VMLEB/VMLOB weighted sum + VSUMQF reduce; scalar fallback without VX; measured on real IBM z15 (VXE2), 2026-07-03: ~5.4× vs scalar (-count=6)
loong64 / others	scalar (hash/adler32-equivalent)	LSX kernel not yet shipped — could not be validated in CI here

How it works

Adler-32 over data d is s2<<16 | s1 where s1 = 1 + Σd[i] (mod 65521) and s2 = Σ running-s1 (mod 65521). hash/adler32 is scalar, byte-by-byte (see golang/go#68188, the open proposal to add SIMD to the standard library).

Following the classic zlib/Chromium SIMD Adler-32, the input is processed in chunks of at most nmax = 5552 bytes, so the 16-bit lane sums cannot overflow before the modular reduction. Per chunk:

• s1 += Σbytes — PSADBW (amd64) / VWREDSUMU (riscv64) / VUADDLV (arm64) / VSUMB (s390x) / widening multiply-by-1 (ppc64le).
• s2 += chunkLen * s1_before + Σ weight_i·byte_i with weights chunkLen..1 — the weighted sum is the SIMD core: PMADDUBSW (amd64, bytes × weight bytes → 16-bit pairwise sums), VWMULU (riscv64, widening multiply), VUMULL (arm64 NEON), VMULEUB/VMULOUB (ppc64le, even/odd widening byte multiplies, Go's ppc64 assembler exposing neither vmsumubm nor vsum4ubs), or VMLEB/VMLOB (s390x). The per-block chunkLen·s1 carry is folded as a vector shift-add on the running s1 accumulator and reduced at the chunk end.

On s390x the platform is big-endian: VL loads byte i of memory into vector element i, and the descending weight vector is emitted as the bytes 16,15,…,1, so weight element i (= 16-i) lines up with source element i under the same load — the positional weighting is correct without any byte swap. The byte sum and the word reductions are position-independent (endian-neutral); the ascending-byte differential test is the positional check that proves it.

The accumulators are reduced to scalars at each chunk boundary, where the two mod 65521 reductions land at exactly the same points as hash/adler32, so the result is identical.

The arm64 / Go 1.27 VUMULL path

The weighted sum needs an integer vector multiply. Go's arm64 assembler historically exposed only the polynomial VPMULL; the integer VMUL / VUMULL / VUMLAL mnemonics were upstreamed in Go 1.27. So:

• stable Go (≤ 1.26) — arm64 falls back to the scalar hash/adler32 loop.
• Go 1.27+ — a //go:build go1.27 NEON kernel (adler32_arm64.s) uses the now-available VUMULL widening multiply for the weighted sum. This is a concrete demonstration of the new Go 1.27 integer-NEON multiply, validated on native arm64 with the gotip (1.27-devel) toolchain.

Performance

Throughput on a 1 MiB random buffer, native amd64 (GitHub Actions), -count=6, median MB/s — see .github/workflows/bench.yml. The dev box is arm64, where the amd64 kernel only runs under Rosetta (which has no AVX2), so the authoritative numbers come from native CI. Absolute MB/s varies with the runner's CPU, so the meaningful figure is the per-run ratio. The result is strongly microarchitecture-dependent, so it is reported split by CPU (each value is the median of -count=6, and was reproduced across multiple runs):

CPU (CI runner)	this package (AVX2) vs stdlib	vs mhr3
Intel Ice Lake (Xeon 8370C)	~14.7×	1.17× (we are ~17% faster)
AMD Zen4 (EPYC 9V74)	~15×	0.99× (near-parity)
AMD Zen3 (EPYC 7763)	~15×	0.95× (mhr3 ~5% faster)

These native-CI ratios remain authoritative (verdict re-confirmed as of 2026-06-14): beat mhr3 on Intel, near-parity on Zen. A -count=6 re-bench on a QEMU x86_64 lima VM (TCG, no out-of-order modelling) put mhr3 ahead of our AVX2 path (~2135 vs ~1433 MB/s) — this does not contradict the table; it is the same caveat the table already carries (Rosetta/QEMU mis-model the AVX2 vs SSE3 instruction mix, which is exactly why only native silicon is quoted). Both beat stdlib comfortably even under TCG (~1433 / ~2135 vs ~728 MB/s).

mhr3 is mhr3/adler32-simd, a pure-Go SIMD Adler-32 (AVX2 + SSE3 + NEON) transpiled from Chromium/zlib via gocc; it is also bit-identical to hash/adler32, so the comparison is like for like.

arm64 note (re-bench 2026-06-14): on stable Go (≤ 1.26) our arm64 NEON kernel is not compiled — it is gated //go:build arm64 && go1.27 (it needs the Go 1.27 integer-NEON multiply) — so on a stable toolchain Checksum falls back to the scalar path and mhr3 (which ships hand-NEON) is ~4× faster (~3.5 vs ~14 GB/s, native M-series). This is the documented build-tag situation, not a regression: under gotip / go1.27 the NEON kernel engages (CI benches it), which is where the arm64-vs-mhr3 comparison is meaningful. On stable Go, prefer mhr3 on arm64; on go1.27+ the gap closes.

Honest notes:

• The AVX2 kernel is unrolled 4× (128 bytes/iteration) with a deferred running-s1 carry (no per-block vs2 += vs1<<5) and a paired deferred VPMADDWD widen: the two VPMADDUBSW weighted sums of a block pair are added in 16-bit (each lane ≤ 16065, a pair ≤ 32130 < 2¹⁵, no overflow) and widened to 32-bit with a single VPMADDWD instead of one per block. That halves the per-iteration VPMADDWD count (4 → 2). VPMADDWD, VPMADDUBSW and VPSADBW are all on the same FP issue ports, so the kernel is FP-port-bound, and removing two of them per iteration directly buys throughput. llvm-mca: 24.6 → 26.7 bytes/cycle on icelake-server, 24.6 → 25.6 on znver3.
• On Intel (Ice Lake), this flips the result against mhr3: the earlier single-VPMADDWD-per-block kernel was ~6% slower than mhr3 on the same Xeon 8370C (ours/mhr3 ≈ 0.94); the paired-widen kernel is ~17% faster (ours/mhr3 ≈ 1.17). On Intel, VPMADDWD is restricted to ports 0/1, so the port relief is large — and it is exactly where the static model said it would land.
• On AMD Zen4 (EPYC 9V74) the two are at near-parity (ours/mhr3 ≈ 0.99, up from ≈ 0.94 for the earlier kernel).
• On AMD Zen3 (EPYC 7763), mhr3 stays ~5% ahead (ours/mhr3 ≈ 0.95, up from ≈ 0.94). Zen3 spreads the mul-class ops across four FP ports, so the saved VPMADDWDs relieve less pressure, and mhr3's tighter NMAX-chunked, software-pipelined 2× loop keeps a small edge that the static port model idealizes away. So on Zen the result is near-parity, slightly behind; on Intel it is a clear win. Both remain bit-identical to hash/adler32.
• Go 1.26 added simd/archsimd, but it is amd64-only; this package differentiates by being multi-arch (amd64 + riscv64 + arm64-on-1.27 + ppc64le + s390x) and Go 1.20+ compatible for the amd64 fast path.

s390x — measured on real IBM z15; ppc64le — llvm-mca cycle-model estimate

s390x — measured on real IBM z15 (VXE2), native execution, -count=6, 2026-07-03: the vector-facility kernel (VSUMB/VSUMQF byte sum + VMLEB/VMLOB weighted sum) runs at ~5.4× the scalar hash/adler32 baseline on a 1 MiB buffer. This supersedes the z14 llvm-mca estimate for s390x below (which projected ~6.4×); the ppc64le row remains a cycle-model estimate — no POWER runner is available here.

For ppc64le this is static analysis, NOT a hardware measurement; native POWER perf pending real silicon. QEMU is not cycle-accurate, so the defensible perf signal is a cycle-model estimate. The committed 16-byte inner loops were extracted from adler32_ppc64le.s / adler32_s390x.s and run through llvm-mca (LLVM 22; production PowerPC + SystemZ backends):

llvm-mca -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr9 <loop.s>
llvm-mca -mtriple=s390x-unknown-linux-gnu -mcpu=z14  <loop.s>

The ×scalar baseline is the classic s1 += p[i]; s2 += s1 byte loop. Because that loop carries s2 across iterations, I report the asymptotic per-iteration cost (-iterations=2000 total-cycles, which captures the loop-carried dependency that Block RThroughput alone ignores), not just RThroughput:

arch (cpu)	SIMD (16 B/iter)	est. SIMD bytes/cycle	scalar (1 B/iter)	est. scalar bytes/cycle	est. ×scalar
ppc64le (pwr9)	~14.8 cyc/iter (RThroughput ~8.3)	~1.1	~2.0 cyc/byte	~0.5	~2.2×
s390x (z14, est.)	~5.0 cyc/iter (RThroughput ~5.0)	~3.2	~2.0 cyc/byte	~0.5	~6.4× (measured z15: ~5.4×)

Honest read: on POWER9 the VSX kernel is loop-carried-latency-bound, not throughput-bound — its 14.8 cyc/iter is well above the 8.3 throughput ceiling because the V10/V11/V12 accumulators form a serial add chain across iterations (and VMULEUB/VMULOUB even/odd widening multiplies feed it). It still beats scalar ~2.2×, but POWER's per-doubleword extract/accumulate caps the win. On z14 the kernel runs at its throughput ceiling (5.0 cyc/iter == RThroughput): VSUMB/VSUMQF and VMLEB/VMLOB schedule cleanly with no cross-iteration stall, for ~6.4× scalar. Caveats: no cache/front-end/branch modelling; the scalar baseline is an idealised dependent-add loop (real Go scalar with periodic NMAX modulo would be a touch slower → ×scalar is a conservative lower bound). All instructions in both loops are modelled by llvm-mca (no unmodelable op). Ballpark ordering only; the s390x estimate has now been replaced by the native z15 measurement above (~5.4×), and the ppc64le row awaits native bytes/cycle on real POWER9.

Regenerating the assembly

adler32_amd64.s, adler32_arm64.s, adler32_riscv64.s, adler32_ppc64le.s and adler32_s390x.s are committed. go-asmgen is a build-time tool, not a runtime dependency:

go get github.com/go-asmgen/asmgen@v0.5.0
go run adler32_gen.go           # amd64 (SSE3 + AVX2)
go run adler32_arm64_gen.go     # arm64 NEON (Go 1.27 VUMULL)
go run adler32_riscv64_gen.go   # riscv64 RVV
go run adler32_ppc64le_gen.go   # ppc64le VSX/AltiVec
go run adler32_s390x_gen.go     # s390x vector facility
go mod edit -droprequire github.com/go-asmgen/asmgen
go mod tidy

Coverage

The CI gate enforces 100% coverage of the Go code on every arch job: native amd64 (AVX2 + SSE — the Force test drives both kernels, and the dispatcher hits both branches across the size table), native arm64 on the stable toolchain (the generic scalar fallback), native arm64 on gotip / go1.27 (the NEON kernel, which compiles only under //go:build go1.27), riscv64 under QEMU (the RVV kernel, with the Force test toggling hasV to also cover the no-V scalar fallback), ppc64le under QEMU (power9, the VSX kernel, Force test toggling hasVSX), s390x under QEMU (the big-endian vector kernel, Force test toggling hasVX), and loong64 under QEMU (the generic fallback). Coverage is of the Go statements only: the generated .s SIMD kernels are not measured by go test -cover — they are validated by differential tests against hash/adler32 plus fuzzing (on real AVX2, RVV/VSX/vector under QEMU, and NEON under gotip).

License

BSD-3-Clause.

Documentation

Overview

Package adler32 is a drop-in fast path for the Adler-32 checksum, bit-for-bit identical to the standard library's hash/adler32. The bulk of the input is summed by a SIMD kernel generated by go-asmgen (amd64 SSE3/SSSE3, with an AVX2 path; arm64 NEON on Go 1.27+; riscv64 RVV uses its integer vector multiply; ppc64le VSX/AltiVec; s390x vector facility); architectures without a kernel (loong64, arm64 on stable Go, and others) fall back to a portable scalar loop that is identical to hash/adler32. The result always equals adler32.Checksum.

Adler-32 over data d is s2<<16 | s1 where s1 = 1 + Σd[i] (mod 65521) and s2 = Σ running-s1 (mod 65521). The standard library is scalar (byte-by-byte; see golang/go#68188, the open proposal to add SIMD to the standard library). This package vectorises the inner sum: over a chunk of at most nmax bytes the 16-bit lane sums cannot overflow, so per chunk s1 += Σbytes and s2 += chunkLen*s1_before + Σ weight_i*byte_i with weights chunkLen..1 — the weighted sum being the SIMD core (PMADDUBSW on amd64).

Index

• func Checksum(data []byte) uint32
• func New() hash.Hash32

Functions

func Checksum

func Checksum(data []byte) uint32

Checksum returns the Adler-32 checksum of data. It is bit-for-bit identical to hash/adler32.Checksum.

func New

func New() hash.Hash32

New returns a new hash.Hash32 computing the Adler-32 checksum. Its Sum32 is bit-for-bit identical to hash/adler32.New().Sum32 for the same writes; the bulk of each Write is folded by the SIMD kernel.