goturbopfor

README

Go TurboPFor teaching implementation

This teaching implementation illustrates the accompanying article at https://michael.stapelberg.de/posts/2019-02-05-turbopfor-analysis/

To confirm my understanding of the details of the format, I implemented a pure-Go TurboPFor256 decoder. Note that it is intentionally not optimized as its main goal is to use simple code to teach the TurboPFor256 on-disk format.

If you’re looking to use TurboPFor from Go, I recommend using cgo. cgo’s function call overhead is about 51ns as of Go 1.8, which will easily be offset by TurboPFor’s carefully optimized, vectorized (SSE/AVX) code.

Documentation

Index

• type Decoder
  • func NewDecoder() *Decoder
  • func (d *Decoder) P4ndec256v32(input []byte, output []uint32) (read int)

Types

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder is not safe for concurrent use by multiple goroutines.

func NewDecoder

func NewDecoder() *Decoder

func (*Decoder) P4ndec256v32

func (d *Decoder) P4ndec256v32(input []byte, output []uint32) (read int)

P4ndec256v32 fills output from input, decoding 256 uint32s at a time.

Note that different decoding algorithms are used for the last block, if that block does not contain 256 uint32s.