README
¶
crc64
A pure-Go, SIMD-accelerated drop-in replacement for the standard library's
hash/crc64. It produces bit-identical CRC-64 checksums — for the predefined
ISO and ECMA polynomials and for any custom polynomial — but folds the bulk
of the input with the host's carryless-multiply unit instead of the scalar
slicing-by-8 table.
No cgo, no GOEXPERIMENT, no assembler intrinsics required: a plain go build
produces the accelerated binary on every supported target.
Why
The standard hash/crc64 is table-based (slicing-by-8). Modern CPUs have a
carryless-multiply instruction that folds CRCs several times faster using the
classic Intel "Fast CRC Computation" approach. This package uses it on all six
of Go's 64-bit SIMD-capable architectures while remaining a verified bit-exact
match for the standard library.
Drop-in usage
Change only the import path:
import (
"github.com/go-simd/crc64" // was "hash/crc64"
)
func main() {
tab := crc64.MakeTable(crc64.ECMA)
sum := crc64.Checksum(data, tab)
h := crc64.New(tab)
h.Write(data)
_ = h.Sum64()
}
The API matches hash/crc64 exactly: Checksum, Update, New, MakeTable,
the ISO/ECMA constants, Size, the Table type (aliased to the stdlib type
so tables are interchangeable), and the hash.Hash64 returned by New (including
encoding.BinaryMarshaler/BinaryUnmarshaler/AppendBinary).
How it works
The data is folded 16 bytes at a time into a single 128-bit reflected
accumulator using carryless multiplication, then reduced to the 64-bit CRC. The
fold constants are derived from the polynomial itself (reflect(x^191 mod P) and
reflect(x^127 mod P)) — there are no copied magic numbers. The short tail
(< 16 bytes) and the table build reuse the standard library, so results are
guaranteed identical.
| Arch | Instruction | Gate |
|---|---|---|
| amd64 | PCLMULQDQ |
cpu.X86.HasPCLMULQDQ |
| arm64 | PMULL / PMULL2 |
cpu.ARM64.HasPMULL |
| ppc64le | VPMSUMD |
VSX (baseline, POWER8+) |
| s390x | VGFMAG |
cpu.S390X.HasVX |
| riscv64 | — (scalar fallback) | Go does not yet expose Zbc |
| loong64 | — (scalar fallback) | no carryless multiply |
On riscv64 and loong64, and on any CPU lacking the relevant instruction, the
package transparently falls back to the standard-library scalar path. The
assembly is generated by go-asmgen; the
generators (kernel_*_gen.go) are checked in alongside the committed .s.
Performance
Honest numbers, ECMA polynomial, single call over a buffer of the given size.
Native arm64 (Apple M-series, PMULL):
| Size | this package | hash/crc64 |
speedup |
|---|---|---|---|
| 1 KiB | 3824 MB/s | 2065 MB/s | 1.9× |
| 16 KiB | 7613 MB/s | 2034 MB/s | 3.7× |
| 1 MiB | 8271 MB/s | 1977 MB/s | 4.2× |
The crossover is a few hundred bytes; below minBulk (512 B) the package uses
the scalar path, so small inputs are never slower than the standard library.
amd64 (PCLMULQDQ): validated for correctness on real x86-64 hardware and
shows the same large-buffer advantage over the standard library (≈2× in a
virtualized measurement environment; native silicon is higher).
ppc64le / s390x: the VPMSUMD and VGFMAG kernels are validated for
correctness under QEMU; native-hardware performance numbers are pending access to
real POWER / Z systems.
Testing
Correctness is gated by FuzzChecksum, which compares against hash/crc64 for
both ISO and ECMA on arbitrary inputs, plus exhaustive length sweeps across all
block boundaries for ISO, ECMA and a custom polynomial. CI runs on native
amd64/arm64 and under QEMU for riscv64, loong64, ppc64le (power9) and s390x, with
a 100 %-statement-coverage gate on every architecture.
License
BSD-3-Clause. See LICENSE.
Documentation
¶
Overview ¶
Package crc64 is a pure-Go, SIMD-accelerated drop-in replacement for the standard library's hash/crc64. It computes bit-identical CRC-64 checksums (both the ISO and ECMA polynomials, and any custom polynomial) but folds the bulk of the input with the host's carryless-multiply unit:
amd64 PCLMULQDQ / VPCLMULQDQ (gated on cpu.X86.HasPCLMULQDQ) arm64 PMULL / PMULL2 (gated on cpu.ARM64.HasPMULL) ppc64le VPMSUMD (VSX, baseline on POWER8+) s390x VGFMG / VGFMAG (gated on cpu.S390X.HasVX)
riscv64 and loong64 fall back to the stdlib-equivalent scalar table code. There is no cgo, no GOEXPERIMENT and no assembler intrinsic requirement: a plain `go build` produces the accelerated binary on every target above.
The API mirrors hash/crc64 exactly, so it can be swapped in by changing only the import path.
Index ¶
Constants ¶
const ( // ISO polynomial, defined in ISO 3309 and used in HDLC. ISO = stdcrc64.ISO // ECMA polynomial, defined in ECMA 182. ECMA = stdcrc64.ECMA )
Predefined polynomials (identical to hash/crc64).
const Size = 8
The size of a CRC-64 checksum in bytes.
Variables ¶
This section is empty.
Functions ¶
func Checksum ¶
Checksum returns the CRC-64 checksum of data using the polynomial represented by the Table.
func New ¶
New creates a new hash.Hash64 computing the CRC-64 checksum using the polynomial represented by the Table. Its Sum method lays the value out in big-endian byte order. The returned Hash64 also implements encoding.BinaryMarshaler and encoding.BinaryUnmarshaler.
The hash uses the SIMD-accelerated kernel for its Write path.
Source Files