chunkreader

README

chunkreader

Package chunkreader provides an io.Reader wrapper that minimizes IO reads and memory allocations.

Extracted from original implementation in https://github.com/jackc/pgx.

Documentation

Overview

Package chunkreader provides an io.Reader wrapper that minimizes IO reads and memory allocations.

Index

• type ChunkReader
  • func New(r io.Reader) *ChunkReader
  • func NewConfig(r io.Reader, config Config) (*ChunkReader, error)
  • func (r *ChunkReader) Next(n int) (buf []byte, err error)
• type Config

Types

type ChunkReader

type ChunkReader struct {
	// contains filtered or unexported fields
}

ChunkReader is a io.Reader wrapper that minimizes IO reads and memory allocations. It allocates memory in chunks and will read as much as will fit in the current buffer in a single call regardless of how large a read is actually requested. The memory returned via Next is owned by the caller. This avoids the need for an additional copy.

The downside of this approach is that a large buffer can be pinned in memory even if only a small slice is referenced. For example, an entire 4096 byte block could be pinned in memory by even a 1 byte slice. In these rare cases it would be advantageous to copy the bytes to another slice.

func New

func New(r io.Reader) *ChunkReader

New creates and returns a new ChunkReader for r with default configuration.

func NewConfig

func NewConfig(r io.Reader, config Config) (*ChunkReader, error)

NewConfig creates and a new ChunkReader for r configured by config.

func (*ChunkReader) Next

func (r *ChunkReader) Next(n int) (buf []byte, err error)

Next returns buf filled with the next n bytes. The caller gains ownership of buf. It is not necessary to make a copy of buf. If an error occurs, buf will be nil.

type Config

type Config struct {
	MinBufLen int // Minimum buffer length
}

Config contains configuration parameters for ChunkReader.