samplereader

README

samplereader

Multiple callers can sample data from a io.Reader.

Related / Similar Projects

• github.com/djherbis/fscache

Documentation

Overview

Package samplereader implements a reader mechanism that allows multiple callers to sample some or the entire contents of a source reader, while only reading from the source reader once.

This is admittedly a rather arcane need. Let's say we're reading from stdin. For example:

# cat myfile.ext | myprogram

In this scenario, myprogram wants to detect the type of data in the file/pipe, and then print it out. The input file could be, let's say, a CSV file, or a TSV file.

The obvious approach is to inspect the first few lines of the input, and check if the input is either valid CSV, or valid TSV. After that process, let's say we want to dump out the entire contents of the input.

Package samplereader provides a facility to create a Source from an underlying io.Reader (os.Stdin in this scenario), and spawn multiple readers, each of which can operate independently, in their own goroutines if desired. The underlying source (again, os.Stdin in this scenario) will only be read from once, but its data is available to multiple readers, because that data is cached in memory. That is, until there's only one final reader left, (after invoking Source.Seal) at which point the cache is discarded, and the final reader reads straight from the underlying source.

Index

• Variables
• type ReadCloser
  • func (rc *ReadCloser) Close() error
  • func (rc *ReadCloser) Read(p []byte) (n int, err error)
• type Source
  • func NewSource(r io.Reader) *Source
  • func (s *Source) NewReadCloser() (*ReadCloser, error)
  • func (s *Source) Seal()

Variables

var ErrSealed = errors.New("already sealed")

ErrSealed is returned by Source.NewReadCloser if the source is already sealed.

Types

type ReadCloser

type ReadCloser struct {
	// contains filtered or unexported fields
}

ReadCloser is returned by Source.NewReadCloser.

func (*ReadCloser) Close

func (rc *ReadCloser) Close() error

Close closes this reader. If the parent Source is not sealed, this method is effectively a no-op. If the parent Source is sealed and the is the last remaining reader, the parent Source's underlying io.Reader is closed (if it implements io.Closer),and this ReadCloser switches to "direct mode" reading for the remaining data. Note that subsequent calls are no-op and return the same result.

func (*ReadCloser) Read

func (rc *ReadCloser) Read(p []byte) (n int, err error)

Read implements io.Reader.

type Source

type Source struct {
	// contains filtered or unexported fields
}

Source encapsulates an underlying io.Reader that many callers can read from.

func NewSource

func NewSource(r io.Reader) *Source

NewSource returns a new source with r as the underlying reader.

func (*Source) NewReadCloser

func (s *Source) NewReadCloser() (*ReadCloser, error)

NewReadCloser returns a new ReadCloser for Source. It is the caller's responsibility to close the returned ReadCloser.

func (*Source) Seal

func (s *Source) Seal()

Seal is called to indicate that there will be no more calls to NewReadCloser.