fileset

Documentation

Index

• type Reader
  • func (r *Reader) Close() error
  • func (r *Reader) Next() (*index.Header, error)
  • func (r *Reader) Peek() (*index.Header, error)
  • func (r *Reader) PeekTag() (*index.Tag, error)
  • func (r *Reader) Read(data []byte) (int, error)
  • func (r *Reader) WriteToFiles(w *Writer, pathBound ...string) error
  • func (r *Reader) WriteToTags(w *Writer, tagBound ...string) error
• type Storage
  • func NewStorage(objC obj.Client, chunks *chunk.Storage) *Storage
  • func (s *Storage) NewIndexReader(ctx context.Context, fileSet, idxPrefix string) *index.Reader
  • func (s *Storage) NewIndexWriter(ctx context.Context, fileSet string) *index.Writer
  • func (s *Storage) NewReader(ctx context.Context, fileSet, idxPrefix string) *Reader
  • func (s *Storage) NewWriter(ctx context.Context, fileSet string) *Writer
• type Writer
  • func (w *Writer) Close() error
  • func (w *Writer) StartTag(id string)
  • func (w *Writer) Write(data []byte) (int, error)
  • func (w *Writer) WriteHeader(hdr *index.Header) error

Types

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader reads the serialized format of a fileset.

func (*Reader) Close

func (r *Reader) Close() error

Close closes the reader.

func (*Reader) Next

func (r *Reader) Next() (*index.Header, error)

Next returns the next header, and prepares the file's content for reading.

func (*Reader) Peek

func (r *Reader) Peek() (*index.Header, error)

Peek returns the next header without progressing the reader.

func (*Reader) PeekTag

func (r *Reader) PeekTag() (*index.Tag, error)

PeekTag returns the next tag without progressing the reader.

func (*Reader) Read

func (r *Reader) Read(data []byte) (int, error)

Read reads from the current file in the tar stream.

func (*Reader) WriteToFiles

func (r *Reader) WriteToFiles(w *Writer, pathBound ...string) error

WriteToFiles does an efficient copy of files from a file set reader to writer. The optional pathBound specifies the upper bound (exclusive) for files to copy.

func (*Reader) WriteToTags

func (r *Reader) WriteToTags(w *Writer, tagBound ...string) error

WriteToTags does an efficient copy of tagged content in the current file from a file set reader to writer. The optional tagBound specifies the upper bound (inclusive, but duplicate tags should not exist) for tags to copy.

type Storage

type Storage struct {
	// contains filtered or unexported fields
}

Storage is the abstraction that manages fileset storage.

func NewStorage

func NewStorage(objC obj.Client, chunks *chunk.Storage) *Storage

NewStorage creates a new Storage.

func (*Storage) NewIndexReader

func (s *Storage) NewIndexReader(ctx context.Context, fileSet, idxPrefix string) *index.Reader

NewIndexReader creates a new index.Reader.

func (*Storage) NewIndexWriter

func (s *Storage) NewIndexWriter(ctx context.Context, fileSet string) *index.Writer

NewIndexWriter creates a new index.Writer.

func (*Storage) NewReader

func (s *Storage) NewReader(ctx context.Context, fileSet, idxPrefix string) *Reader

NewReader creates a new Reader.

func (*Storage) NewWriter

func (s *Storage) NewWriter(ctx context.Context, fileSet string) *Writer

NewWriter creates a new Writer.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer writes the serialized format of a fileset. The serialized format of a fileset consists of indexes and content which are both realized as compressed tar stream chunks.

func (*Writer) Close

func (w *Writer) Close() error

Close closes the writer. (bryce) is not closing the tar writer the right choice here? We cannot include it in the last file's range because it will effect the hash. Our indexing is what will exit the reading of the tar stream, not the end of tar entry (two empty 512 bytes).

func (*Writer) StartTag

func (w *Writer) StartTag(id string)

StartTag starts a tag for the next set of bytes (used for the reverse index, mapping file output to datums).

func (*Writer) Write

func (w *Writer) Write(data []byte) (int, error)

Write writes to the current file in the tar stream.

func (*Writer) WriteHeader

func (w *Writer) WriteHeader(hdr *index.Header) error

WriteHeader writes a tar header and prepares to accept the file's contents.