base64

README

base64

A drop-in fast path for standard base64, byte- AND error-identical to encoding/base64. It mirrors the stdlib's four encodings — StdEncoding (+/, padded), URLEncoding (-_, padded), RawStdEncoding (+/, no padding) and RawURLEncoding (-_, no padding) — and both encode and decode run a SIMD kernel generated by go-asmgen; the short tail + padding (and, on decode, any block with an invalid byte) reuse the standard library, so the output and every CorruptInputError offset match exactly.

s := base64.EncodeToString(data)            // StdEncoding, same bytes as encoding/base64
b, err := base64.DecodeString(s)            // same bytes + same error/offset as stdlib

u := base64.URLEncoding.EncodeToString(d)   // A-Za-z0-9-_ , padded
r := base64.RawURLEncoding.EncodeToString(d) // A-Za-z0-9-_ , no padding

The four encodings are package-level *Encoding values (StdEncoding, URLEncoding, RawStdEncoding, RawURLEncoding) with the same method set as encoding/base64.Encoding; the top-level Encode/Decode/EncodeToString/… are shortcuts for StdEncoding. The +/ and -_ SIMD kernels are the same kernel with a swapped alphabet table (two LUT entries differ), so URL-safe runs at full SIMD throughput — never the scalar pessimization a post-translate pass would cost. Padding vs raw is purely a tail concern, so each padded/raw pair shares one kernel. Every encoding is verified byte- and error-identical to its encoding/base64 counterpart across n=0..512 (encode + decode, valid + invalid + cross-alphabet input) on all six build targets.

All four encodings (+/ and -_, padded and raw) get the SIMD path on each arch below; the URL (-_) kernels are the +/ kernels with a two-entry alphabet swap (encode: a two-entry change to the ASCII offset-LUT; decode: URL validate/roll LUTs derived by an exhaustive biclique cover + a one-byte roll-index delta mask, both self-verified against every byte 0x00..0xFF in the gen).

op	amd64	arm64	ppc64le	s390x	loong64 / riscv64
encode	AVX2 + SSE2 (Lemire)	NEON (VLD3/VST4 deinterleaving I/O)	VSX (shift-based)	vector facility (shift-based)	scalar (stdlib)
decode	AVX2 + SSE (Muła)	NEON (VLD4/VST3 deinterleaving I/O)	VSX (shift-based)	vector facility (shift-based)	scalar (stdlib)

The encoder is Lemire's vectorised base64: a shuffle spreads the input across 24-bit lanes, the 6-bit indices are pulled out, and a PSHUFB/TBL/VPERM offset-LUT maps each to its ASCII byte (constants via go-asmgen's emit.File.Data). On amd64 the index extraction uses the two-multiply (PMULHUW/PMULLW) trick; on ppc64le/s390x it uses a right-shift-only extraction (per-32-bit-word shifts + masks).

arm64 uses the aklomp/emmansun deinterleaving-I/O design, the structure that makes NEON base64 fast — and it needs no integer vector multiply, so it builds on released Go (no go1.27 gating). Each iteration consumes 48 input bytes and emits 64 chars: a VLD3.P deinterleaving load splits the input into its three byte-planes (lane i of each plane is the 1st/2nd/3rd byte of 24-bit group i), the four 6-bit indices are extracted per byte-plane with plain shifts and inserts (VUSHR/VSLI/VAND, no multiply), four VTBL lookups map each plane's indices through the 64-byte alphabet, and a VST4.P interleaving store writes the chars back in order. The deinterleave/interleave is free in the load/store unit, which is why this ties emmansun/base64 at ~22 GB/s (see Performance) versus the ~6.6 GB/s of the previous VTBL-spread + per-lane-shift kernel. (An earlier go1.27-only VUMULL/VMUL multiply port of the amd64 trick was dropped: it ran ~16 GB/s — faster than the old shift kernel but slower than this VLD3/VST4 path, and it required an unreleased toolchain.)

On amd64/ppc64le/s390x the decoder is Muła's vectorised base64: two nibble-keyed PSHUFB/TBL/VPERM LUTs (lut_lo/lut_hi) both validate each byte (a char is valid iff lo & hi == 0) and produce its 6-bit value via a "roll" offset LUT; the 4×6-bit fields are then packed into 3 bytes — on amd64 with the multiply-add pair (PMADDUBSW+PMADDWD), on ppc64le/s390x with a shift-only per-32-bit-word gather. On arm64 the decoder mirrors the encode side's aklomp/emmansun deinterleaving-I/O design: a VLD4.P load splits 64 chars into four char-planes, a two-table VTBL+VTBX translate maps each char to its 6-bit value (and flags out-of-range bytes), a high-bit (≥0x80) check rejects non-ASCII bytes, a pure-shift pack produces the 3 output bytes, and a VST3.P store writes 48 bytes — 64→48 per iteration, ~3× the stdlib and ~0.9× the emmansun NEON reference (which it strictly out-correctness: emmansun mis-accepts ≥0x80 bytes the stdlib rejects). On ANY invalid byte (whitespace, padding =, non-alphabet, non-ASCII) the block bails to the scalar encoding/base64, so errors and offsets are exact. All six build targets are verified against encoding/base64 (table + FuzzDecode, incl. invalid/whitespace/padding): amd64/arm64 natively, ppc64le and s390x under QEMU (see below).

Six SIMD targets, validated on seven architectures. SIMD acceleration covers six targets (amd64 AVX2, arm64 NEON, riscv64 RVV, loong64 LSX, ppc64le VSX, s390x vector). A seventh architecture — ppc64 (big-endian) — is build+test validated on real POWER9 silicon: it has no VSX build tag, so it exercises the portable generic fallback path, proven bit-exact on a big-endian target distinct from s390x's vector kernel.

ppc64le (VSX) and s390x (vector facility)

Both got a real SIMD encode and decode kernel, generated by go-asmgen. ppc64le is now natively measured on real POWER9 silicon (GCC Compile Farm, VSX, Go 1.26.4, 2026-06-26 — see Performance). s390x is now natively measured on real IBM z15 (VXE2, 2026-07-03, -count=6): ~3.7–4.2× stdlib on ≥1 KiB buffers (scalar fallback wins at ≤16 B) — see Performance.

• ppc64le encode: LXVB16X byte-order-correct load → VPERM spread → VSRW+VAND index extraction → VSUBUBS/VCMPGTUB range bucket → VPERM ASCII LUT → STXVB16X store. Because VSRW interprets each word big-endian, the spread lays out lanes as [b0,b1,b2,0] and the masks/shift-counts are stored big-endian-per-lane so the four indices land directly in their output bytes.
• ppc64le decode: LXVB16X load → two VPERM nibble LUTs (translate + validate; valid iff lo & hi == 0, tested with the record-form VCMPEQUB. against zero → CR6) → VPERM roll-LUT for the 6-bit value → VSRW+VAND shift-only pack → VPERM compaction → STXVB16X.
• s390x encode (big-endian): VL → VPERM spread → VESRLF+VN index extraction → VMNLB/VSB/VCHLB range bucket → VPERM ASCII LUT → VST.
• s390x decode (big-endian): VL → VESRLB/VN nibbles → two VPERM LUTs (translate + validate via VCEQBS against zero, which sets the condition code) → VPERM roll-LUT → VESRLF+VN shift-only pack → VPERM compaction → VST.

The cross-lane VPERM shuffles (the spread/LUT on encode, and the two nibble LUTs

• roll LUT + compaction on decode) are the big-endian risk on s390x; control vectors use big-endian lane numbering (lane 0 = lowest address) and the whole kernel is verified position-dependently under qemu, with FuzzEncode/FuzzDecode confirming byte- and error-identical output to encoding/base64. Each backend's instruction operand orders were confirmed empirically under qemu before assembly (e.g. ppc64 VSRW Vdata,Vshift; s390x VSB is second-minus-first).

Performance

Encode throughput, 1 MiB buffer, native amd64 (GitHub Actions, AMD EPYC 7763, GOAMD64=v1, -count=6, median MB/s; CI-measured because the dev box is arm64):

implementation	kind	MB/s	vs stdlib
encoding/base64 (stdlib)	scalar	~1180	1.0×
cristalhq/base64	pure-Go scalar	~2650	2.2×
emmansun/base64	pure-Go SIMD (AVX2)	~19300	~16×
this package	pure-Go SIMD (AVX2)	~20500	~17×

On amd64 this package leads emmansun/base64 (~5–6% on native CI; the re-bench confirms the direction) — notable, since emmansun is the mature reference. The edge came from a cycle-model-guided optimization: llvm-mca showed the kernel was shuffle-port (Zn3FP1) bound; disassembling emmansun revealed a -4-offset 32-byte load that spreads with a single VPSHUFB (no cross-lane VINSERTI128); combining that load with this package's 2×-unroll was predicted at 2.15 cyc/block (vs emmansun's 2.20) and the benchmark confirmed it. (Block 0 keeps VINSERTI128 — a -4 load there would read before src.) See go-asmgen toolkit/mca.

Re-bench ratios (as of 2026-06-14, -count=6 medians, kernels unchanged)

arch	host	toolchain	ours vs stdlib	ours vs emmansun
amd64	x86_64 QEMU VM (ratio valid, absolute MB/s low)	stable	~3.6×	~1.31× (ours wins)
arm64	Apple Silicon (native)	stable (VLD3/VST4 kernel)	~8.0×	~0.98× (tie)

Verdict update — honest: the "leads emmansun" claim holds only on amd64. On arm64 the new VLD3.P/VST4.P deinterleaving-I/O encode kernel runs at ~22.1 GB/s vs emmansun/base64's ~22.5 GB/s (-count=6 averages on an Apple Silicon native host; modal runs of both sit at ~22.4 GB/s) — a dead heat, ours at ~0.98× emmansun, well inside run-to-run noise. This is a ~3.3× jump over the prior stable shift kernel (~6.6 GB/s) and it builds on released Go. The deinterleaving load/store is the whole lever: emmansun's speed came from VLD3/ VST4 + one TBL per output stream (no multiply), and once our kernel adopts the same structure the two converge. An earlier go1.27-only VUMULL/VMUL multiply port reached ~16 GB/s but was both slower than this path and gated on an unreleased toolchain, so it was removed.

• arm64: NEON VLD3/VST4 deinterleaving-I/O encode kernel on released Go (~8× stdlib, ~tie with emmansun; see above).
• ppc64le: VSX SIMD encode kernel, natively measured on real POWER9 (GCC Compile Farm, 2026-06-26): SIMD encode ~2.1× stdlib (1613 vs 782 MB/s), decode ~2.0× (2090 vs 1057 MB/s); decode also beats emmansun. See below.
• s390x: vector-facility SIMD kernel, natively measured on real IBM z15 (VXE2) (2026-07-03, -count=6): SIMD encode/decode ~3.7–4.2× stdlib on buffers ≥1 KiB; the scalar fallback wins at ≤16-byte small inputs (dispatch
  • non-vectorised tail dominate there). See below.

ppc64le — measured on real POWER9 (VSX, 2026-06-26)

Native hardware measurement on real POWER9 silicon (GCC Compile Farm, https://portal.cfarm.net/ , VSX, Go 1.26.4, 2026-06-26): SIMD encode ~2.1× stdlib (1613 vs 782 MB/s) and SIMD decode ~2.0× (2090 vs 1057 MB/s); decode also beats emmansun/base64. This replaces the earlier llvm-mca estimate for ppc64le — the kernel is now proven on real silicon, not just modeled. (The s390x kernel is likewise now natively measured on real IBM z15 — see the s390x section below.)

riscv64 — measured on real SpacemiT X60 (RVV 1.0): scalar parity

Native hardware measurement on a real SpacemiT X60 (riscv64 RVV 1.0, GCC Compile Farm, Go 1.26.4, 2026-06-26). Honest result: scalar parity on riscv64 — RVV encode 164 vs stdlib 166 MB/s, decode 154 vs 156 MB/s. The byte-shuffle / nibble-spread kernel does not beat the compiler's scalar encoding/base64 on this core: the X60 is a low-power, in-order core and is currently the only widely-available RVV 1.0 silicon. Shuffle-heavy kernels need cross-lane permute throughput that an in-order core lacks; an out-of-order RVV core would likely lift this kernel. The encode/decode wins stay on amd64, arm64 and ppc64le.

s390x — measured on real IBM z15 (VXE2, 2026-07-03)

Native hardware measurement on real IBM z15 silicon (VXE2, native execution, 2026-07-03, -count=6): the vector-facility base64 kernel runs ~3.7–4.2× the stdlib scalar encoding/base64 on buffers ≥1 KiB (encode and decode). At ≤16-byte inputs the scalar fallback wins — dispatch and the non-vectorised tail dominate at that size — so the SIMD path pays off on the larger inputs it targets. This supersedes the llvm-mca cycle-model estimate.

Historical cycle-model estimate (superseded by the measurement above):

arch	cpu model	SIMD cyc/iter	SIMD input-B/cyc	scalar input-B/cyc	est. speedup
s390x	z14 (est.)	9.5 (12 B in)	~1.26	~0.43 (3 B / 7.0 cyc)	~2.9×

The s390x cycle model projected ~2.9×; the native z15 measurement above (~3.7–4.2× on ≥1 KiB) is the authoritative figure and comes in above the conservative model. The ppc64le cycle-model estimate was likewise superseded by the real POWER9 measurement above. Caveats on the historical model: llvm-mca idealizes the frontend (perfect dispatch, no branch misprediction, no cache/store-buffer stalls). All instructions in both loops were accepted and modeled by llvm-mca (no fallbacks).

Decode throughput

Decode throughput, 1 MiB encoded buffer, -count=4 medians (MB/s measured on the encoded input). amd64 numbers come from an emulated x86_64 QEMU VM so the absolute MB/s is low but the ratio is valid; native CI silicon is far faster.

arch	host	ours	stdlib	ours vs stdlib	ours vs emmansun
amd64	x86_64 QEMU VM	~393	~284	~1.38×	~1.31× (ours wins)
arm64	Apple Silicon (M4 Max, native)	~15220	~4900	~3.1×	~0.93×

arm64 decode is the VLD4/VST3 deinterleaving-I/O kernel (the decode analogue of the encode kernel that ties emmansun): re-measured on an Apple M4 Max it runs at ~15.2 GB/s vs stdlib ~4.9 GB/s — ~3.1× (URL-safe matches, ~15.2 GB/s), and is within ~0.9× of emmansun's hand-tuned NEON decode. This supersedes an earlier shift-only arm64 decode kernel (~705 MB/s on M-series, slower than stdlib) — that pessimization is gone; arm64 SIMD decode is now strictly faster than stdlib, so the library ships no decode regression on any SIMD arch. On amd64 the Muła SIMD decode beats both stdlib (~1.4×) and emmansun (~1.3×). ppc64le / s390x decode kernels are validated byte- and error-identical (FuzzDecode); ppc64le is natively measured on real POWER9 (decode ~2.0× stdlib, 2090 vs 1057 MB/s; beats emmansun); s390x is now natively measured on real IBM z15 (VXE2) — decode ~3.7–4.2× stdlib on ≥1 KiB buffers (scalar fallback wins at ≤16 B). riscv64 / loong64 keep the scalar encoding/base64 decode (and encode): on the only widely-available RVV-1.0 silicon (in-order SpacemiT X60) the shuffle-heavy kernel only reaches scalar parity, so shipping a SIMD decode there would be a no-win — the stdlib fallback is deliberate, not a gap (see riscv64 note above).

• cgo wrappers of aklomp/base64 are faster still but need a C toolchain — excluded from this pure-Go comparison.

Coverage

The CI gate enforces 100% coverage of the Go code on every arch job: native amd64 + native arm64, plus QEMU jobs for ppc64le, s390x, and riscv64 (the latter covering the generic decode/encode fallback). Coverage is of the Go statements only: the generated .s SIMD kernels are not measured by go test -cover — they are validated by differential tests against the scalar encoding/base64 reference plus FuzzEncode and FuzzDecode (the latter asserting byte- AND error-identical results, including CorruptInputError offsets, for invalid/whitespace/padding inputs): amd64/arm64 natively; ppc64le and s390x under QEMU.

License

BSD-3-Clause.

Documentation

Overview

Package base64 is a drop-in fast path for standard base64, byte- AND error-identical to encoding/base64. It mirrors the four stdlib encodings — StdEncoding (A-Za-z0-9+/, padded), URLEncoding (A-Za-z0-9-_, padded), RawStdEncoding (+/, no padding) and RawURLEncoding (-_, no padding) — and the bulk of encoding and decoding runs a SIMD kernel generated by go-asmgen (amd64 SSE/AVX2, ppc64le VSX, s390x vector facility, arm64 NEON for both encode and decode); other arches fall back to the standard library, and the short tail/padding always use encoding/base64 so output is byte-identical to the matching base64.Encoding.

The SIMD kernels come in two alphabets (+/ for Std/RawStd, -_ for URL/RawURL); padding is a tail concern handled entirely by the wrapped encoding/base64 encoding, so the padded and raw variants share the same kernel.

Index

• Variables
• func Decode(dst, src []byte) (int, error)
• func DecodeString(s string) ([]byte, error)
• func DecodedLen(n int) int
• func Encode(dst, src []byte)
• func EncodeToString(src []byte) string
• func EncodedLen(n int) int
• type Encoding
  • func (enc *Encoding) Decode(dst, src []byte) (int, error)
  • func (enc *Encoding) DecodeString(s string) ([]byte, error)
  • func (enc *Encoding) DecodedLen(n int) int
  • func (enc *Encoding) Encode(dst, src []byte)
  • func (enc *Encoding) EncodeToString(src []byte) string
  • func (enc *Encoding) EncodedLen(n int) int

Variables

var (
	StdEncoding    = &Encoding{std: base64.StdEncoding, url: false}
	URLEncoding    = &Encoding{std: base64.URLEncoding, url: true}
	RawStdEncoding = &Encoding{std: base64.RawStdEncoding, url: false}
	RawURLEncoding = &Encoding{std: base64.RawURLEncoding, url: true}
)

The four encodings mirror encoding/base64's StdEncoding, URLEncoding, RawStdEncoding and RawURLEncoding exactly.

Functions

func Decode

func Decode(dst, src []byte) (int, error)

Decode decodes StdEncoding base64 src into dst.

func DecodeString

func DecodeString(s string) ([]byte, error)

DecodeString decodes a StdEncoding base64 string.

func DecodedLen

func DecodedLen(n int) int

DecodedLen returns the max decoded length of n StdEncoding base64 bytes.

func Encode

func Encode(dst, src []byte)

Encode writes the StdEncoding base64 of src into dst.

func EncodeToString

func EncodeToString(src []byte) string

EncodeToString returns the StdEncoding base64 of src.

func EncodedLen

func EncodedLen(n int) int

EncodedLen returns the length in bytes of the StdEncoding base64 of n bytes.

Types

type Encoding

type Encoding struct {
	// contains filtered or unexported fields
}

An Encoding is a base64 codec (one of the four package-level encodings). It wraps the matching encoding/base64.Encoding for the tail/padding so every result — including each CorruptInputError offset — is identical to the stdlib.

func (*Encoding) Decode

func (enc *Encoding) Decode(dst, src []byte) (int, error)

Decode decodes base64 src into dst. A SIMD kernel handles the bulk of clean (no-padding, no-whitespace, all-valid) full blocks; the remainder — the final padded quantum and any block containing an invalid byte — is handed to encoding/base64 so the result is byte- and error-identical (CorruptInputError offsets match exactly).

func (*Encoding) DecodeString

func (enc *Encoding) DecodeString(s string) ([]byte, error)

DecodeString decodes a base64 string.

func (*Encoding) DecodedLen

func (enc *Encoding) DecodedLen(n int) int

DecodedLen returns the max decoded length of n base64 bytes.

func (*Encoding) Encode

func (enc *Encoding) Encode(dst, src []byte)

Encode writes the base64 of src into dst (len(dst) >= EncodedLen(len(src))).

func (*Encoding) EncodeToString

func (enc *Encoding) EncodeToString(src []byte) string

EncodeToString returns the base64 of src.

func (*Encoding) EncodedLen

func (enc *Encoding) EncodedLen(n int) int

EncodedLen returns the length in bytes of the base64 encoding of n input bytes.