fpack

package module
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 7, 2023 License: MIT Imports: 12 Imported by: 5

README

fpack

Test

A functional approach to encoding and decoding byte sequences.

Example

// encode
buf, ref, err := Encode(Global(), func(enc *Encoder) error {
    enc.Uint8(42)
    enc.FixString("Hello World!", 2)
    return nil
})
if err != nil {
    panic(err)
}

// ensure release
defer ref.Release()

// decode
var num uint8
var str string
err = Decode(buf, func(dec *Decoder) error {
    num = dec.Uint8()
    str = dec.FixString(2, false)
    return nil
})
if err != nil {
    panic(err)
}

// print
fmt.Println(len(buf))
fmt.Println(num)
fmt.Println(str)

// Output:
// 15
// 42
// Hello World!

Documentation

Overview

Package fpack provides a functional approach to encoding and decoding byte sequences.

Example 
// encode
buf, ref, err := Encode(Global(), func(enc *Encoder) error {
	enc.Uint8(42)
	enc.FixString("Hello World!", 2)
	return nil
})
if err != nil {
	panic(err)
}

// ensure release
defer ref.Release()

// decode
var num uint8
var str string
err = Decode(buf, func(dec *Decoder) error {
	num = dec.Uint8()
	str = dec.FixString(2, false)
	return nil
})
if err != nil {
	panic(err)
}

// print
fmt.Println(len(buf))
fmt.Println(num)
fmt.Println(str)

Output:

15
42
Hello World!

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrBufferTooShort = errors.New("buffer too short")

ErrBufferTooShort is returned if the provided buffer is too short.

View Source 
var ErrEmptyDelimiter = errors.New("empty delimiter")

ErrEmptyDelimiter is returned if a provided delimiter is empty.

View Source 
var ErrInvalidOffset = errors.New("invalid offset")

ErrInvalidOffset is return for offsets that under or overflow the buffer.

View Source 
var ErrInvalidSize = errors.New("invalid size")

ErrInvalidSize is returned if a provided number size is invalid.

View Source 
var ErrNumberOverflow = errors.New("number overflow")

ErrNumberOverflow is returned if a provided number overflows its size.

View Source 
var ErrRemainingBytes = errors.New("remaining bytes")

ErrRemainingBytes is returned if the provided buffer is not fully consumed.

Functions

func Decode 

func Decode(bytes []byte, fn func(dec *Decoder) error) error

Decode will decode data using the provided decoding function. The function is run once to decode the data. It will return ErrBufferTooShort if the buffer was not long enough to read all data, ErrRemainingBytes if the provided buffers has not been full consumed or any error returned by the callback.

func EncodeInto 

func EncodeInto(buf []byte, fn func(enc *Encoder) error) (int, error)

EncodeInto will encode data into the specified byte slice using the provided encoding function. The function is run once to assess the length of the buffer and once to encode the data. Any error returned by the callback is returned immediately. If the provided buffer is too small ErrBufferTooShort is returned.

func Measure 

func Measure(fn func(enc *Encoder) error) (int, error)

Measure will measure the required byte slice to run the provided encoding function. Any error returned by the callback is returned immediately.

func Track added in v0.6.1 

func Track(fn func([]byte))

Track will enable buffer tracking if a function is provided and disable it otherwise. The registered function will receive stack traces for leaked buffers.

Types

type Arena 

type Arena struct {
	// contains filtered or unexported fields
}

Arena is a basic arena allocator that allocates fixed size buffers to provide memory for many small buffers.

func NewArena added in v0.6.0 

func NewArena(pool *Pool, size int) *Arena

NewArena creates and returns a new arena using the specified pool and buffer size. The arena is obtained from a global pool and recycled upon release.

func (*Arena) Clone 

func (a *Arena) Clone(buf []byte) []byte

Clone will return a copy of the provided buffer.

func (*Arena) Get 

func (a *Arena) Get(length int, zero bool) []byte

Get will return a buffer of the provided length.

func (*Arena) Length added in v0.6.0 

func (a *Arena) Length() int

Length returns the total length of the arena.

func (*Arena) Release 

func (a *Arena) Release()

Release will release all returned buffers.

type Buffer added in v0.6.1 

type Buffer struct {
	// contains filtered or unexported fields
}

Buffer is basic buffer that dynamically allocates needed chunks.

func NewBuffer added in v0.6.1 

func NewBuffer(pool *Pool, alloc int) *Buffer

NewBuffer will return a new buffer that uses the provided pool and allocation size to dynamically allocate chunks as needed to hold the data.

func (*Buffer) Length added in v0.6.1 

func (b *Buffer) Length() int

Length returns the buffer length.

func (*Buffer) Range added in v0.6.2 

func (b *Buffer) Range(offset, length int, fn func(offset int, data []byte))

Range will iterate over the buffer in the given range and call the provided function with the offset and data for each chunk.

func (*Buffer) Read added in v0.6.1 

func (b *Buffer) Read(buf []byte) (int, error)

Read implements the io.Reader interface.

func (*Buffer) ReadAt added in v0.6.1 

func (b *Buffer) ReadAt(buf []byte, off int64) (int, error)

ReadAt implements the io.ReaderAt interface.

func (*Buffer) Release added in v0.6.1 

func (b *Buffer) Release()

Release will release the buffer and all memory.

func (*Buffer) Seek added in v0.6.1 

func (b *Buffer) Seek(offset int64, whence int) (int64, error)

Seek implements the io.Seeker interface.

func (*Buffer) Write added in v0.6.1 

func (b *Buffer) Write(buf []byte) (int, error)

Write implements the io.Writer interface.

func (*Buffer) WriteAt added in v0.6.1 

func (b *Buffer) WriteAt(buf []byte, off int64) (int, error)

WriteAt implements the io.WriterAt interface.

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder manages data decoding.

func NewDecoder 

func NewDecoder(buf []byte) *Decoder

NewDecoder will return a new decoder.

func (*Decoder) Bool 

func (d *Decoder) Bool() bool

Bool reads a boolean.

func (*Decoder) Bytes 

func (d *Decoder) Bytes(length int, clone bool) []byte

Bytes reads a raw byte slice. If the byte slice is not cloned it may change if the source byte slice changes.

func (*Decoder) DelBytes 

func (d *Decoder) DelBytes(delim []byte, clone bool) []byte

DelBytes reads a suffix delimited byte slice. If the byte slice is not cloned it may change if the source byte slice changes.

func (*Decoder) DelString 

func (d *Decoder) DelString(delim string, clone bool) string

DelString reads a suffix delimited string. If the string is not cloned it may change if the source byte slice changes.

func (*Decoder) Error 

func (d *Decoder) Error() error

Error will return the current error.

func (*Decoder) FixBytes 

func (d *Decoder) FixBytes(lenSize int, clone bool) []byte

FixBytes reads a fixed length prefixed byte slice. If the byte slice is not cloned it may change if the source byte slice changes.

func (*Decoder) FixString 

func (d *Decoder) FixString(lenSize int, clone bool) string

FixString reads a fixed length prefixed string. If the string is not cloned it may change if the source byte slice changes.

func (*Decoder) Float32 

func (d *Decoder) Float32() float32

Float32 reads a four byte float.

func (*Decoder) Float64 

func (d *Decoder) Float64() float64

Float64 reads an eight byte float.

func (*Decoder) Int 

func (d *Decoder) Int(size int) int64

Int read a one, two, four or eight byte signed integer (two's complement).

func (*Decoder) Int16 

func (d *Decoder) Int16() int16

Int16 reads a two byte signed integer (two's complement).

func (*Decoder) Int32 

func (d *Decoder) Int32() int32

Int32 reads a four byte signed integer (two's complement).

func (*Decoder) Int64 

func (d *Decoder) Int64() int64

Int64 reads an eight byte signed integer (two's complement).

func (*Decoder) Int8 

func (d *Decoder) Int8() int8

Int8 reads a one byte signed integer (two's complement).

func (*Decoder) Length 

func (d *Decoder) Length() int

Length returns the remaining length of the buffer.

func (*Decoder) Remaining 

func (d *Decoder) Remaining() bool

Remaining returns whether more bytes can be decoded.

func (*Decoder) Reset 

func (d *Decoder) Reset(buf []byte)

Reset will reset the decoder.

func (*Decoder) Skip 

func (d *Decoder) Skip(num int)

Skip the specified amount of bytes.

func (*Decoder) String 

func (d *Decoder) String(length int, clone bool) string

String reads a raw string. If the string is not cloned it may change if the source byte slice changes.

func (*Decoder) Tail 

func (d *Decoder) Tail(clone bool) []byte

Tail reads a tail byte slice. If the byte slice is not cloned it may change if the source byte slice changes.

func (*Decoder) TimeUnix added in v0.6.2 

func (d *Decoder) TimeUnix() time.Time

TimeUnix reads a Unix timestamps in seconds.

func (*Decoder) Uint 

func (d *Decoder) Uint(size int) uint64

Uint reads a one, two, four or eight byte unsigned integer.

func (*Decoder) Uint16 

func (d *Decoder) Uint16() uint16

Uint16 reads a two byte unsigned integer.

func (*Decoder) Uint32 

func (d *Decoder) Uint32() uint32

Uint32 reads a four byte unsigned integer.

func (*Decoder) Uint64 

func (d *Decoder) Uint64() uint64

Uint64 reads an eight byte unsigned integer.

func (*Decoder) Uint8 

func (d *Decoder) Uint8() uint8

Uint8 reads a one byte unsigned integer.

func (*Decoder) UseArena added in v0.6.0 

func (d *Decoder) UseArena(arena *Arena)

UseArena will use the specified arena for string and bytes cloning.

func (*Decoder) UseLittleEndian 

func (d *Decoder) UseLittleEndian()

UseLittleEndian will set the used binary byte order to little endian.

func (*Decoder) VarBytes 

func (d *Decoder) VarBytes(clone bool) []byte

VarBytes reads a variable length prefixed byte slice. If the byte slice is not cloned it may change if the source byte slice changes.

func (*Decoder) VarInt 

func (d *Decoder) VarInt() int64

VarInt reads a variable signed integer.

func (*Decoder) VarString 

func (d *Decoder) VarString(clone bool) string

VarString reads a variable length prefixed string. If the string is not cloned it may change if the source byte slice changes.

func (*Decoder) VarUint 

func (d *Decoder) VarUint() uint64

VarUint reads a variable unsigned integer.

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder manages data encoding.

func NewEncoder 

func NewEncoder() *Encoder

NewEncoder will return an encoder.

func (*Encoder) Bool 

func (e *Encoder) Bool(yes bool)

Bool writes a boolean.

func (*Encoder) Bytes 

func (e *Encoder) Bytes(buf []byte)

Bytes writes a raw byte slice.

func (*Encoder) Counting 

func (e *Encoder) Counting() bool

Counting returns whether the encoder is counting.

func (*Encoder) DelBytes 

func (e *Encoder) DelBytes(buf, delim []byte)

DelBytes writes a suffix delimited byte slice.

func (*Encoder) DelString 

func (e *Encoder) DelString(str, delim string)

DelString writes a suffix delimited string.

func (*Encoder) Error added in v0.6.0 

func (e *Encoder) Error() error

Error will return the current error.

func (*Encoder) FixBytes 

func (e *Encoder) FixBytes(buf []byte, lenSize int)

FixBytes writes a fixed length prefixed byte slice.

func (*Encoder) FixString 

func (e *Encoder) FixString(str string, lenSize int)

FixString writes a fixed length prefixed string.

func (*Encoder) Float32 

func (e *Encoder) Float32(num float32)

Float32 writes a four byte float.

func (*Encoder) Float64 

func (e *Encoder) Float64(num float64)

Float64 writes an eight byte float.

func (*Encoder) Int 

func (e *Encoder) Int(n int64, size int)

Int writes a one, two, four or eight byte signed integer (two's complement).

func (*Encoder) Int16 

func (e *Encoder) Int16(num int16)

Int16 writes a two byte signed integer (two's complement).

func (*Encoder) Int32 

func (e *Encoder) Int32(num int32)

Int32 writes a four byte signed integer (two's complement).

func (*Encoder) Int64 

func (e *Encoder) Int64(num int64)

Int64 writes an eight byte signed integer (two's complement).

func (*Encoder) Int8 

func (e *Encoder) Int8(num int8)

Int8 writes a one byte signed integer (two's complement).

func (*Encoder) Length 

func (e *Encoder) Length() int

Length will return the accumulated length.

func (*Encoder) Reset 

func (e *Encoder) Reset(buf []byte)

Reset will reset the encoder. Pass nil so set the encoder to counting mode.

func (*Encoder) Skip 

func (e *Encoder) Skip(num int)

Skip the specified amount of bytes.

func (*Encoder) String 

func (e *Encoder) String(str string)

String writes a raw string.

func (*Encoder) Tail 

func (e *Encoder) Tail(buf []byte)

Tail writes a tail byte slice.

func (*Encoder) TimeUnix added in v0.6.2 

func (e *Encoder) TimeUnix(ts time.Time)

TimeUnix writes a Unix timestamps in seconds.

func (*Encoder) Uint 

func (e *Encoder) Uint(num uint64, size int)

Uint writes a one, two, four or eight byte unsigned integer.

func (*Encoder) Uint16 

func (e *Encoder) Uint16(num uint16)

Uint16 writes a two byte unsigned integer.

func (*Encoder) Uint32 

func (e *Encoder) Uint32(num uint32)

Uint32 writes a four byte unsigned integer.

func (*Encoder) Uint64 

func (e *Encoder) Uint64(num uint64)

Uint64 writes an eight byte unsigned integer.

func (*Encoder) Uint8 

func (e *Encoder) Uint8(num uint8)

Uint8 writes a one byte unsigned integer.

func (*Encoder) UseLittleEndian 

func (e *Encoder) UseLittleEndian()

UseLittleEndian will set the used binary byte order to little endian.

func (*Encoder) VarBytes 

func (e *Encoder) VarBytes(buf []byte)

VarBytes writes a variable length prefixed byte slice.

func (*Encoder) VarInt 

func (e *Encoder) VarInt(num int64)

VarInt writes a variable signed integer.

func (*Encoder) VarString 

func (e *Encoder) VarString(str string)

VarString writes a variable length prefixed string.

func (*Encoder) VarUint 

func (e *Encoder) VarUint(num uint64)

VarUint writes a variable unsigned integer.

type Pool 

type Pool struct {
	// contains filtered or unexported fields
}

Pool is dynamic slice length pool.

func Global 

func Global() *Pool

Global returns the global pool.

func NewPool 

func NewPool() *Pool

NewPool creates and returns a new pool.

func (*Pool) Borrow 

func (p *Pool) Borrow(len int, zero bool) ([]byte, Ref)

Borrow will return a slice that has the specified length. If the requested length is too small or too big, a slice will be allocated. To recycle the slice, it must be released by calling Release() on the returned Ref value. Always release any returned value, even if the slice grows, it is possible to at least return the originally requested slice. If zero is true, the returned slice will be zeroed (but not the full underlying buffer).

Note: For values up to 8 bytes (64 bits) the internal Go arena allocator is used by calling make(). From benchmarks this seems to be faster than calling the pool to borrow and return a value. Also values above 32 MiB are allocated using the Go allocator to ensure not used memory is available to be freed immediately if not used anymore.

func (*Pool) Clone 

func (p *Pool) Clone(slice []byte) ([]byte, Ref)

Clone will copy the provided slice into a borrowed slice.

func (*Pool) Concat 

func (p *Pool) Concat(slices ...[]byte) ([]byte, Ref)

Concat will concatenate the provided byte slices using a borrowed slice.

type Ref 

type Ref struct {
	// contains filtered or unexported fields
}

Ref is a reference to a borrowed slice. A zero reference represents a no-op reference.

func Encode 

func Encode(pool *Pool, fn func(enc *Encoder) error) ([]byte, Ref, error)

Encode will encode data using the provided encoding function. The function is run once to assess the length of the buffer and once to encode the data. Any error returned by the callback is returned immediately.

func (Ref) Release 

func (r Ref) Release()

Release will release the borrowed slice. The function should be called at most once and will panic otherwise.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.