readahead

README

readahead

This is not an official Google product

readahead is a package that provides readers that enable concurrent reads from seekable or compressed files. It's useful when reading from a network file system (like Google Cloud Storage).

To install: go get github.com/google/readahead

For information on use, see the godoc.

Documentation

Overview

Package readahead provides readers which enable concurrent read from seekable or compressed files. It's useful when reading from a network file system (like Google Cloud Storage).

Index

• func NewConcurrentReader(name string, r io.ReaderAt, chunkSize int, chunkAhead int, numWorkers int) io.ReadCloser
• func NewReader(name string, r io.Reader, chunkSize, chunkAhead int) io.ReadCloser

Functions

func NewConcurrentReader

func NewConcurrentReader(name string, r io.ReaderAt, chunkSize int, chunkAhead int, numWorkers int) io.ReadCloser

NewConcurrentReader creates a new reader with the specified chunk size and number of workers. Name is only used for logging. It reads ahead up to chunkAhead chunks of chunkSize with numWorkers and tries to maintain the readahead buffer.

func NewReader

func NewReader(name string, r io.Reader, chunkSize, chunkAhead int) io.ReadCloser

NewReader creates a readahead reader. It will read up to chunkAhead chunks of chunkSize bytes each and use a separate goroutine for that. It is useful when reading from a compressed stream or from network. If an incoming stream supports io.ReaderAt, NewConcurrentReader is a faster option. Name is only used for logging. The resulting reader must be read to EOF or Close must be called to prevent memory leaks.