README
¶
Base64 using SIMD
Translation of Alfred Klomp's base64 library to Go using the currently experimental simd/archsimd package.
This package currently only employs SIMD instructions on AMD64 architecture. It requires Go 1.26 (or above) toolchain with GOEXPERIMENT=simd set in the environment.
Performance
The benchmarks below use standard base64 alphabet with padding and employ the AVX2 instruction set.
This compares the same encode and decode operations using encoding/base64 package from the standard library with this package.
goos: linux
goarch: amd64
pkg: codeberg.org/jongray/base64simd
cpu: Intel(R) Core(TM) i7-6770HQ CPU @ 2.60GHz
│ base64 │ base64simd │
│ sec/op │ sec/op vs base │
Encode/size=10-8 16.97n ± 22% 19.71n ± 8% +16.15% (p=0.035 n=10)
Encode/size=100-8 125.95n ± 31% 45.27n ± 10% -64.06% (p=0.000 n=10)
Encode/size=1000-8 1129.0n ± 8% 176.1n ± 8% -84.40% (p=0.000 n=10)
Encode/size=10000-8 11.946µ ± 7% 1.230µ ± 8% -89.71% (p=0.000 n=10)
Encode/size=100000-8 113.56µ ± 8% 12.36µ ± 6% -89.12% (p=0.000 n=10)
Decode/size=16-8 29.81n ± 10% 22.66n ± 5% -23.97% (p=0.000 n=10)
Decode/size=136-8 129.95n ± 6% 87.42n ± 6% -32.72% (p=0.000 n=10)
Decode/size=1336-8 1154.0n ± 5% 214.0n ± 7% -81.46% (p=0.000 n=10)
Decode/size=13336-8 11.388µ ± 9% 1.715µ ± 5% -84.94% (p=0.000 n=10)
Decode/size=133336-8 114.01µ ± 7% 16.17µ ± 12% -85.82% (p=0.000 n=10)
geomean 1.342µ 350.6n -73.88%
Beware!
Compare the above results to the below using a different CPU. This package decode ns/op is significantly higher that of the standard encoding/base64 package. This suggests there can be differences in a CPU implementation of the SIMD instruction set. Be sure to benchmark in your environment.
goos: linux
goarch: amd64
pkg: codeberg.org/jongray/base64simd
cpu: AMD Ryzen 5 5600X 6-Core Processor
│ base64 │ base64simd │
│ sec/op │ sec/op vs base │
Encode/size=10-12 9.307n ± 3% 11.465n ± 5% +23.19% (p=0.000 n=10)
Encode/size=100-12 59.34n ± 2% 24.98n ± 7% -57.91% (p=0.000 n=10)
Encode/size=1000-12 573.65n ± 1% 80.87n ± 1% -85.90% (p=0.000 n=10)
Encode/size=10000-12 5639.0n ± 1% 567.6n ± 4% -89.93% (p=0.000 n=10)
Encode/size=100000-12 56.680µ ± 1% 5.505µ ± 2% -90.29% (p=0.000 n=10)
Decode/size=16-12 17.17n ± 6% 13.30n ± 9% -22.51% (p=0.001 n=10)
Decode/size=136-12 76.59n ± 6% 236.00n ± 2% +208.13% (p=0.000 n=10)
Decode/size=1336-12 666.5n ± 1% 2508.5n ± 1% +276.37% (p=0.000 n=10)
Decode/size=13336-12 6.504µ ± 1% 25.221µ ± 2% +287.77% (p=0.000 n=10)
Decode/size=133336-12 66.29µ ± 4% 253.13µ ± 1% +281.84% (p=0.000 n=10)
geomean 721.6n 570.3n -20.97%
By forcing the benchmark to use the AVX instruction set, as opposed to AVX2, the results look as expected.
goos: linux
goarch: amd64
pkg: codeberg.org/jongray/base64simd
cpu: AMD Ryzen 5 5600X 6-Core Processor
│ base64 │ base64simd │
│ sec/op │ sec/op vs base │
Encode/size=10-12 9.165n ± 4% 11.155n ± 1% +21.72% (p=0.000 n=10)
Encode/size=100-12 59.00n ± 4% 19.77n ± 2% -66.50% (p=0.000 n=10)
Encode/size=1000-12 577.10n ± 6% 97.69n ± 0% -83.07% (p=0.000 n=10)
Encode/size=10000-12 5657.0n ± 1% 838.6n ± 5% -85.18% (p=0.000 n=10)
Encode/size=100000-12 57.203µ ± 1% 8.213µ ± 4% -85.64% (p=0.000 n=10)
Decode/size=16-12 17.12n ± 5% 12.32n ± 2% -28.09% (p=0.000 n=10)
Decode/size=136-12 76.34n ± 6% 34.73n ± 0% -54.51% (p=0.000 n=10)
Decode/size=1336-12 669.3n ± 1% 136.2n ± 6% -79.64% (p=0.000 n=10)
Decode/size=13336-12 6.537µ ± 2% 1.178µ ± 5% -81.99% (p=0.000 n=10)
Decode/size=133336-12 65.85µ ± 4% 10.98µ ± 2% -83.32% (p=0.000 n=10)
geomean 721.1n 201.8n -72.02%
Documentation
¶
Overview ¶
Package base64simd implements base64 encoding (as specified in RFC 4648) using SIMD instructions (currently experimental, enabled with GOEXPERIMENT=simd environment variable).
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var RawStdEncoding = StdEncoding.withoutPadding()
RawStdEncoding performs standard base64 encoding using the standard 64 character alphabet, but without padding, as defined in RFC 4648.
var RawUrlEncoding = UrlEncoding.withoutPadding()
RawUrlEncoding performs alternate base64url encoding using the URL (and filename) safe alternate 64 character alphabet, but without padding, as defined in RFC 4648.
var StdEncoding = newEncoding("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/")
StdEncoding performs standard base64 encoding using the standard 64 character alphabet as defined in RFC 4648.
var UrlEncoding = newEncoding("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_").urlSafe()
UrlEncoding performs alternate base64url encoding using the URL (and filename) safe alternate 64 character alphabet as defined in RFC 4648.
Functions ¶
This section is empty.
Types ¶
type Encoding ¶
type Encoding struct {
// contains filtered or unexported fields
}
func (*Encoding) Decode ¶
Decode decodes data from source using the encoding e, writing upto encoding/base64.Encoding.DecodedLen(len(source)) bytes to destination and returns the actual number of bytes written.
destination MUST have at least size encoding/base64.Encoding.DecodedLen(len(source)) else Decode will panic.
Example ¶
package main
import (
"encoding/base64"
"fmt"
"codeberg.org/jongray/base64simd"
)
func main() {
sourceBytes := []byte("Zm9vYmFy")
destinationBytes := make([]byte, base64.StdEncoding.DecodedLen(len(sourceBytes)))
n, _ := base64simd.StdEncoding.Decode(destinationBytes, sourceBytes)
fmt.Println(string(destinationBytes[:n]))
}
Output: foobar
func (*Encoding) Encode ¶
Encode encodes data from source using the encoding e, writing encoding/base64.Encoding.EncodedLen(len(source)) bytes to destination.
destination MUST have at least size encoding/base64.Encoding.EncodedLen(len(source)) else Encode will panic.
Example ¶
package main
import (
"encoding/base64"
"fmt"
"codeberg.org/jongray/base64simd"
)
func main() {
sourceBytes := []byte("foobar")
destinationBytes := make([]byte, base64.StdEncoding.EncodedLen(len(sourceBytes)))
base64simd.StdEncoding.Encode(destinationBytes, sourceBytes)
fmt.Println(string(destinationBytes))
}
Output: Zm9vYmFy
Source Files