record

Documentation

Index

• Constants
• Variables
• type Header
• type Reader
  • func NewReader(fs afero.Fs, path string) (*Reader, error)
  • func (r *Reader) Close() error
  • func (r *Reader) ReadKeyAt(offset int64) (*Record, error)
  • func (r *Reader) ReadRecordAt(offset int64) (*Record, error)
  • func (r *Reader) ReadRecordAtStrict(offset int64) (*Record, error)
  • func (r *Reader) ReadValueAt(offset int64) (*Record, error)
• type Record
• type Scanner
  • func NewScanner(fs afero.Fs, path string) (*Scanner, error)
  • func (scanner *Scanner) Close() error
  • func (scanner *Scanner) Scan() (Record, int64, error)
• type Writer
  • func NewBufferedWriter(fs afero.Fs, path string) (*Writer, error)
  • func NewWriter(fs afero.Fs, path string) (*Writer, error)
  • func (w *Writer) Close() error
  • func (w *Writer) Sync() error
  • func (w *Writer) WriteKeyValue(key []byte, value []byte) (int64, error)
  • func (w *Writer) WriteKeyValueWithTs(key []byte, value []byte, ts time.Time) (int64, error)
  • func (w *Writer) WriteTombstone(key []byte) (int64, error)
  • func (w *Writer) WriteTombstoneWithTs(key []byte, ts time.Time) (int64, error)

Constants

const (
	RecordTypeDelete = 0x44
)

Variables

var ErrCrcChecksumMismatch = errors.New("crc checksum does not match stored value")

var ErrKeyTooLarge = errors.New("key too large")

var ErrValueTooLarge = errors.New("value too large")

Types

type Header

type Header struct {
	Timestamp  time.Time
	KeySize    uint32
	ValueSize  uint32
	RecordType uint8
	ValueType  uint8
}

Header contains metadata information about a log record

Timestamp represents the time when the record was created or last modified. KeySize specifies the size in bytes of the record's key. ValueSize specifies the size in bytes of the record's value. RecordType indicates the type of operation (e.g., insert, update, delete). ValueType indicates the data type of the value (e.g., string, integer, blob). Currently it's set to 0x0

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader is responsible for reading log records from a file. This implementation uses ReadAt (that uses pread internally on supported files) and hence is safe to access concurrently

func NewReader

func NewReader(fs afero.Fs, path string) (*Reader, error)

NewReader creates a new Record Reader that opens a file at the specified path for reading log records. It starts reading from the 19th byte in the file (To skip the header)

func (*Reader) Close

func (r *Reader) Close() error

Close closes the underlying file

func (*Reader) ReadKeyAt

func (r *Reader) ReadKeyAt(offset int64) (*Record, error)

ReadKeyAt reads a record at the given offset (from the start of the first record). It only reads and populates the key in the returned record. Value is left empty.

func (*Reader) ReadRecordAt

func (r *Reader) ReadRecordAt(offset int64) (*Record, error)

ReadRecordAt reads a record at the given offset (from the start of the first record). It reads both the key and value from the file, and both the Key and Value in the returned record are valid.

func (*Reader) ReadRecordAtStrict

func (r *Reader) ReadRecordAtStrict(offset int64) (*Record, error)

ReadRecordAtStrict reads a record at the given offset (from the start of the first record). It reads both the key and value from the file, and both the Key and Value in the returned record are valid. It also verifies if the record is valid by computing the CRC checksum

func (*Reader) ReadValueAt

func (r *Reader) ReadValueAt(offset int64) (*Record, error)

ReadValueAt reads a record at the given offset (from the start of the first record). It only reads and populates the value in the returned record. Key is left empty.

type Record

type Record struct {
	Header Header
	Key    []byte
	Value  []byte
	Size   int64
}

Record represents a single key-value pair in the log file. `Key` and `Value` can be empty depending upon the mode through which the record was read. Size represents the total size of the record (header + key + value + crc), it's useful for determining the start of the next record

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner sequentially reads records from the given file. It internally uses a buffered reader to improve performance. This is not meant to be used in Get operation, and is intended to be used for Merge (or other sequential scans of the datafile)

func NewScanner

func NewScanner(fs afero.Fs, path string) (*Scanner, error)

func (*Scanner) Close

func (scanner *Scanner) Close() error

func (*Scanner) Scan

func (scanner *Scanner) Scan() (Record, int64, error)

Scan returns the next record, the offset for the start of the record (from the first record) Note: They Key & Value inside record are backed by a shared buffer, and hence it'll be overwritten the next time Scan is called. If you need the record key / value later, make a copy

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer is responsible for writing log records to the file. There are no locks in this implementation, so it's unsafe to call Writer methods concurrently

func NewBufferedWriter

func NewBufferedWriter(fs afero.Fs, path string) (*Writer, error)

NewBufferedWriter creates a new Buffered Record Writer that opens a file at the specified path for appending logs Note: It uses a bufio.Writer internally, it's good for merging records, but remember to Sync() otherwise data will get lost

func NewWriter

func NewWriter(fs afero.Fs, path string) (*Writer, error)

NewWriter creates a new Record Writer that opens a file at the specified path for appending logs

func (*Writer) Close

func (w *Writer) Close() error

Close closes the underlying file, it also writes any pending changes and syncs the changes to the disk

func (*Writer) Sync

func (w *Writer) Sync() error

Sync flushes any buffered data to the underlying file. It calls sync() on the file

func (*Writer) WriteKeyValue

func (w *Writer) WriteKeyValue(key []byte, value []byte) (int64, error)

WriteKeyValue writes the key-value pair as a new log entry to the file. It does not call sync(), so there is a chance that data might get lost if the system crashes. If you need durability, call Sync() after writing. This function returns the offset of the record in the file, measured from the start of the file

func (*Writer) WriteKeyValueWithTs

func (w *Writer) WriteKeyValueWithTs(key []byte, value []byte, ts time.Time) (int64, error)

func (*Writer) WriteTombstone

func (w *Writer) WriteTombstone(key []byte) (int64, error)

WriteTombstone writes a tombstone value for the specified key, this function is to be used to delete a key from the store. This function returns the offset of the record in the file, measured from the start of the file

func (*Writer) WriteTombstoneWithTs

func (w *Writer) WriteTombstoneWithTs(key []byte, ts time.Time) (int64, error)