delimited

Documentation

Overview

This package implements a simple reader and writer for streams of length-delimited byte records. Each record is written as a varint-encoded length in bytes, followed immediately by the record itself. A stream consists of a sequence of such records packed consecutively without additional padding. No checksums or compression are being used.

Index

• func Copy(sink Sink, src DataSource) error
• type DataSource
• type Reader
  • func NewReader(r io.Reader) *Reader
  • func (r *Reader) Next() ([]byte, error)
  • func (r *Reader) NextProto(pb proto.Message) error
• type Sink
• type Writer
  • func NewWriter(w io.Writer) *Writer
  • func (w Writer) Put(record []byte) error
  • func (w Writer) PutProto(msg proto.Message) error
  • func (w Writer) WriteRecord(record []byte) (int, error)

Functions

func Copy

func Copy(sink Sink, src DataSource) error

Copy copies each record read from src to sink sequentially until src.Next() Note: - returns io.EOF or another error occurs

Types

type DataSource

type DataSource interface {
	// Next will return the next record in the sequence
	// Note:
	// - io.EOF is returned when no further records are available
	// - the slice returned by Next() is only required to be valid until a subsequent call to Next()
	Next() ([]byte, error)
}

A DataSource represents a sequence of records.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader reads length-delimited records from a byte data source

func NewReader

func NewReader(r io.Reader) *Reader

NewReader returns a new delimited Reader for the records in r

func (*Reader) Next

func (r *Reader) Next() ([]byte, error)

Next returns the next length-delimited record from the input Note:

• returns io.EOF if there are no more records available
• returns io.ErrUnexpectedEOF if a short record is found, with a length of n but fewer than n bytes of data
• since there is no resynchronization mechanism, it is generally not possible to recover from a short record in this format

The slice returned is valid only until a subsequent call to Next.

func (*Reader) NextProto

func (r *Reader) NextProto(pb proto.Message) error

NextProto consumes the next available record by calling r.Next and decodes it into protobuf using proto.Unmarshal()

type Sink

type Sink interface {
	// Put delivers a record to the sink.
	Put([]byte) error
}

Sink models a receiver of delimited records

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer outputs delimited records to an io.Writer

func NewWriter

func NewWriter(w io.Writer) *Writer

NewWriter constructs a new delimited Writer that writes records to w.

func (Writer) Put

func (w Writer) Put(record []byte) error

Put writes the specified record to the writer Note:

• equivalent to WriteRecord, but discards the number of bytes written

func (Writer) PutProto

func (w Writer) PutProto(msg proto.Message) error

PutProto encodes and writes the specified proto.Message to the writer

func (Writer) WriteRecord

func (w Writer) WriteRecord(record []byte) (int, error)

WriteRecord writes the specified record to the underlying writer Note:

• returns the total number of bytes written including the length tag