gxring

README

Circular Buffer

This is a circular buffer implementation designed to be used with gigabyte sized buffers. It uses a file as its backing store. The backing file is mmap'd into memory. One can lower the resident memory size of the program by advising where the customer cursor is (uses madvise). The buffer can be loaded again from the backing file if the program exists.

For now please check tests for examples.

Documentation

Index

• Variables
• func GetLittleEndianUint32(b []byte, offset int) uint32
• func GetLittleEndianUint64(b []byte, offset int) uint64
• func PutLittleEndianUint32(b []byte, offset int, v uint32)
• func PutLittleEndianUint64(b []byte, offset int, v uint64)
• type Buffer
  • func Load(filename string) (*Buffer, error)
  • func New(capacity int, filename string) (*Buffer, error)
  • func (b *Buffer) Close() error
  • func (b *Buffer) DontNeed(c Cursor) error
  • func (b *Buffer) Insert(data []byte) error
  • func (b *Buffer) Len() uint64
  • func (b *Buffer) Read(data []byte, c Cursor) (n int, next Cursor, err error)
  • func (b *Buffer) ReadFirst(data []byte) (n int, next Cursor, err error)
  • func (b *Buffer) Remaining(from Cursor) uint64
  • func (b *Buffer) Unmap()
• type Cursor
  • func (c Cursor) Seq() uint64

Variables

var (
	ErrFileExists  = errors.New("buffer: log file already exists")
	ErrPartialData = errors.New("buffer: partial data returned")
	ErrNotArrived  = errors.New("buffer: data has not yet arrived")
)

Functions

func GetLittleEndianUint32

func GetLittleEndianUint32(b []byte, offset int) uint32

func GetLittleEndianUint64

func GetLittleEndianUint64(b []byte, offset int) uint64

func PutLittleEndianUint32

func PutLittleEndianUint32(b []byte, offset int, v uint32)

func PutLittleEndianUint64

func PutLittleEndianUint64(b []byte, offset int, v uint64)

Types

type Buffer

type Buffer struct {
	// contains filtered or unexported fields
}

func Load

func Load(filename string) (*Buffer, error)

func New

func New(capacity int, filename string) (*Buffer, error)

func (*Buffer) Close

func (b *Buffer) Close() error

Close calls b.Unmap and deletes the buffer file.

func (*Buffer) DontNeed

func (b *Buffer) DontNeed(c Cursor) error

DontNeed advises the system about memory no longer needed.

func (*Buffer) Insert

func (b *Buffer) Insert(data []byte) error

Insert adds data into b. Caller is free to recycle the underlying array of the slice.

func (*Buffer) Len

func (b *Buffer) Len() uint64

Len returns the total data bytes stored in the buffer. This does not include the space used for framing.

func (*Buffer) Read

func (b *Buffer) Read(data []byte, c Cursor) (n int, next Cursor, err error)

Read copies record that c points to into data. It returns number of bytes copied, a cursor to the next record, and error if there was any.

If c points to a record that has been ejected from b the call is analogous to b.ReadFirst.

If c points to a record that has not yet arrived an ErrNotArrived error is returned.

If data does not have enough room for record an ErrPartialData error is returned.

func (*Buffer) ReadFirst

func (b *Buffer) ReadFirst(data []byte) (n int, next Cursor, err error)

ReadFirst reads the first (oldest) record in b. It's return values are analogous to Read.

func (*Buffer) Remaining

func (b *Buffer) Remaining(from Cursor) uint64

Remaining returns the total bytes remaining to be read before reading everything in the buffer.

func (*Buffer) Unmap

func (b *Buffer) Unmap()

Unmap unmaps the buffer file from memory.

type Cursor

type Cursor struct {
	// contains filtered or unexported fields
}

func (Cursor) Seq

func (c Cursor) Seq() uint64

Seq returns the sequence number of the record c points to in Buffer. Sequence numbers are sequential.