xorsimd

README

XOR SIMD

Introduction:

• XOR code engine in pure Go.

• High Performance: More than 270GB/s per physics core.

Performance

Performance depends mainly on:

• CPU instruction extension.

• Number of source row vectors.

Platform:

AWS c5d.xlarge (Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz)

All test run on a single Core.

I/O = (src_num + 1) * vector_size / cost

Src Num	Vector size	AVX512 I/O (MB/S)	AVX2 I/O (MB/S)	SSE2 I/O (MB/S)
5	4KB	270403.73	142825.25	74443.91
5	1MB	26948.34	26887.37	26950.65
5	8MB	17881.32	17212.56	16402.97
10	4KB	190445.30	102953.59	53244.04
10	1MB	26424.44	26618.65	26094.39
10	8MB	15471.31	14866.72	13565.80

Documentation

Index

• Variables
• func Bytes(dst, a, b []byte) int
• func Bytes16(dst, a, b []byte)
• func Bytes16Align(dst, a, b []byte)
• func Bytes8(dst, a, b []byte)
• func Bytes8Align(dst, a, b []byte)
• func BytesA(dst, a, b []byte)
• func BytesB(dst, a, b []byte)
• func Encode(dst []byte, src [][]byte) (n int)

Variables

var EnableAVX512 = true

EnableAVX512 may slow down CPU Clock (maybe not). TODO need more research: https://lemire.me/blog/2018/04/19/by-how-much-does-avx-512-slow-down-your-cpu-a-first-experiment/

Functions

func Bytes

func Bytes(dst, a, b []byte) int

Bytes XORs the bytes in a and b into a destination slice. The source and destination may overlap.

Bytes returns the number of bytes encoded, which will be the minimum of len(dst), len(a), len(b).

func Bytes16

func Bytes16(dst, a, b []byte)

Bytes16 XORs of packed 16 Bytes. The slice arguments a, b, dst's lengths are assumed to be at least 16, if not, Bytes16 will panic.

func Bytes16Align

func Bytes16Align(dst, a, b []byte)

Bytes16Align XORs of packed 16 Bytes. The slice arguments a, b, dst's lengths are assumed to be at least 16, if not, Bytes16 will panic.

func Bytes8

func Bytes8(dst, a, b []byte)

Bytes8 XORs of 8 Bytes. The slice arguments a, b, dst's lengths are assumed to be at least 8, if not, Bytes8 will panic.

func Bytes8Align

func Bytes8Align(dst, a, b []byte)

Bytes8Align XORs of 8 Bytes. The slice arguments a, b, dst's lengths are assumed to be at least 8, if not, Bytes8 will panic.

func BytesA

func BytesA(dst, a, b []byte)

BytesA XORs the len(a) bytes in a and b into a destination slice. The destination should have enough space.

It's used for encoding small bytes slices (< dozens bytes), and the slices may not be aligned to 8 bytes or 16 bytes. If the length is big, it's better to use 'func Bytes(dst, a, b []byte)' instead for gain better performance.

func BytesB

func BytesB(dst, a, b []byte)

BytesB XORs the len(b) bytes in a and b into a destination slice. The destination should have enough space.

It's used for encoding small bytes slices (< dozens bytes), and the slices may not be aligned to 8 bytes or 16 bytes. If the length is big, it's better to use 'func Bytes(dst, a, b []byte)' instead for gain better performance.

func Encode

func Encode(dst []byte, src [][]byte) (n int)

Encode encodes elements from source slice into a destination slice. The source and destination may overlap. Encode returns the number of bytes encoded, which will be the minimum of len(src[i]) and len(dst).