splitter

Documentation

Index

• type Options
• type Segment
• type SegmentInfo
• type Splitter
  • func New(opts Options) (*Splitter, error)
  • func (s *Splitter) Finish(err error)
  • func (s *Splitter) Next(ctx context.Context) (Segment, error)
  • func (s *Splitter) Write(p []byte) (int, error)
• type WriteFinisher

Types

type Options

type Options struct {
	// Split is the plaintext number of bytes to start new segments.
	Split int64

	// Minimum is the plaintext number of bytes necessary to create
	// a remote segment.
	Minimum int64

	// Params controls the encryption used on the plaintext bytes.
	Params storj.EncryptionParameters

	// Key is used to encrypt the encryption keys used to encrypt
	// the data.
	Key *storj.Key

	// PartNumber is the segment's part number if doing multipart
	// uploads, and 0 otherwise.
	PartNumber int32
}

Options controls parameters of how an incoming stream of bytes is split into segments, remote and inline.

type Segment

type Segment interface {
	// Begin returns a metaclient.BatchItem to begin the segment, either inline
	// or remote.
	Begin() metaclient.BatchItem

	// Position returns the segment position.
	Position() metaclient.SegmentPosition

	// Inline returns true if the segment is small enough to be inline.
	Inline() bool

	// Reader returns a fresh io.Reader that reads the data of the segment.
	Reader() io.Reader

	// EncryptETag encrypts the provided etag with the correct encryption
	// keys that the segment is using.
	EncryptETag(eTag []byte) ([]byte, error)

	// Finalize returns a SegmentInfo if the segment is done being read
	// from.
	Finalize() *SegmentInfo

	// DoneReading reports to the segment that we are no longer reading
	// with the provided error to report to writes.
	DoneReading(err error)
}

Segment is an interface describing what operations a segment must provide to be uploaded to the network.

type SegmentInfo

type SegmentInfo struct {
	// Encryption contains the encryption parameters that will be stored
	// on the satellite.
	Encryption metaclient.SegmentEncryption

	// PlainSize is the plaintext number of bytes in the segment.
	PlainSize int64

	// EncryptedSize is the encrypted number of bytes in the segment.
	EncryptedSize int64
}

SegmentInfo is information related to what is necessary to commit the segment.

type Splitter

type Splitter struct {
	// NewBackend lets one swap out the backend used to store segments
	// while they are being uploaded.
	NewBackend func() (buffer.Backend, error)
	// contains filtered or unexported fields
}

Splitter takes an incoming stream of bytes and splits it into encrypted segments.

func New

func New(opts Options) (*Splitter, error)

New constructs a Splitter with the provided Options.

func (*Splitter) Finish

func (s *Splitter) Finish(err error)

Finish informs the Splitter that no more writes are coming, along with any error that may have caused the writes to stop.

func (*Splitter) Next

func (s *Splitter) Next(ctx context.Context) (Segment, error)

Next returns the next Segment split from the stream. If the stream is finished then it will return nil, nil.

func (*Splitter) Write

func (s *Splitter) Write(p []byte) (int, error)

Write appends data into the stream.

type WriteFinisher

type WriteFinisher interface {
	io.Writer
	DoneWriting(error)
}

WriteFinisher is a Writer that can be signalled by the caller when it is done being written do. Subsequent calls to write should return an error after writing is "done".