streams

Documentation

Index

• func DisableDeleteOnCancel(ctx context.Context) context.Context
• type EOFReader
  • func NewEOFReader(r io.Reader) *EOFReader
  • func (r *EOFReader) HasError() bool
  • func (r *EOFReader) IsEOF() bool
  • func (r *EOFReader) Read(p []byte) (n int, err error)
• type ETag
• type Meta
• type Metadata
• type Part
• type PeekThresholdReader
  • func NewPeekThresholdReader(r io.Reader) (pt *PeekThresholdReader)
  • func (pt *PeekThresholdReader) IsLargerThan(thresholdSize int) (bool, error)
  • func (pt *PeekThresholdReader) Read(p []byte) (n int, err error)
• type SizedReader
  • func SizeReader(r io.Reader) *SizedReader
  • func (r *SizedReader) Read(p []byte) (n int, err error)
  • func (r *SizedReader) Size() int64
• type Store
  • func NewStreamStore(metainfo *metaclient.Client, ec ecclient.Client, segmentSize int64, ...) (*Store, error)
  • func (s *Store) Close() error
  • func (s *Store) Get(ctx context.Context, bucket, unencryptedKey string, ...) (rr ranger.Ranger, err error)
  • func (s *Store) Put(ctx context.Context, bucket, unencryptedKey string, data io.Reader, ...) (_ Meta, err error)
  • func (s *Store) PutPart(ctx context.Context, bucket, unencryptedKey string, streamID storj.StreamID, ...) (_ Part, err error)
  • func (s *Store) Ranger(ctx context.Context, response metaclient.DownloadSegmentWithRSResponse) (rr ranger.Ranger, err error)

Functions

func DisableDeleteOnCancel

func DisableDeleteOnCancel(ctx context.Context) context.Context

DisableDeleteOnCancel is now a no-op.

Types

type EOFReader

type EOFReader struct {
	// contains filtered or unexported fields
}

EOFReader holds reader and status of EOF.

func NewEOFReader

func NewEOFReader(r io.Reader) *EOFReader

NewEOFReader keeps track of the state, has the internal reader reached EOF.

func (*EOFReader) HasError

func (r *EOFReader) HasError() bool

HasError returns true if error was returned during reading.

func (*EOFReader) IsEOF

func (r *EOFReader) IsEOF() bool

IsEOF returns true if EOF was returned during reading.

func (*EOFReader) Read

func (r *EOFReader) Read(p []byte) (n int, err error)

type ETag

type ETag interface {
	ETag() []byte
}

ETag interface returns the latest ETag for a part.

type Meta

type Meta struct {
	Modified   time.Time
	Expiration time.Time
	Size       int64
	Data       []byte
}

Meta info about a stream.

type Metadata

type Metadata interface {
	Metadata() ([]byte, error)
}

Metadata interface returns the latest metadata for an object.

type Part

type Part struct {
	PartNumber uint32
	Size       int64
	Modified   time.Time
	ETag       []byte
}

Part info about a part.

type PeekThresholdReader

type PeekThresholdReader struct {
	// contains filtered or unexported fields
}

PeekThresholdReader allows a check to see if the size of a given reader exceeds the maximum inline segment size or not.

func NewPeekThresholdReader

func NewPeekThresholdReader(r io.Reader) (pt *PeekThresholdReader)

NewPeekThresholdReader creates a new instance of PeekThresholdReader.

func (*PeekThresholdReader) IsLargerThan

func (pt *PeekThresholdReader) IsLargerThan(thresholdSize int) (bool, error)

IsLargerThan returns a bool to determine whether a reader's size is larger than the given threshold or not.

func (*PeekThresholdReader) Read

func (pt *PeekThresholdReader) Read(p []byte) (n int, err error)

Read initially reads bytes from the internal buffer, then continues reading from the wrapped data reader. The number of bytes read `n` is returned.

type SizedReader

type SizedReader struct {
	// contains filtered or unexported fields
}

SizedReader allows to check the total number of bytes read so far.

func SizeReader

func SizeReader(r io.Reader) *SizedReader

SizeReader create a new instance of SizedReader.

func (*SizedReader) Read

func (r *SizedReader) Read(p []byte) (n int, err error)

Read implements io.Reader.Read.

func (*SizedReader) Size

func (r *SizedReader) Size() int64

Size returns the total number of bytes read so far.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a store for streams. It implements typedStore as part of an ongoing migration to use typed paths. See the shim for the store that the rest of the world interacts with.

func NewStreamStore

func NewStreamStore(metainfo *metaclient.Client, ec ecclient.Client, segmentSize int64, encStore *encryption.Store, encryptionParameters storj.EncryptionParameters, inlineThreshold int) (*Store, error)

NewStreamStore constructs a stream store.

func (*Store) Close

func (s *Store) Close() error

Close closes the underlying resources passed to the metainfo DB.

func (*Store) Get

func (s *Store) Get(ctx context.Context, bucket, unencryptedKey string, info metaclient.DownloadInfo) (rr ranger.Ranger, err error)

Get returns a ranger that knows what the overall size is (from l/<key>) and then returns the appropriate data from segments s0/<key>, s1/<key>, ..., l/<key>.

func (*Store) Put

func (s *Store) Put(ctx context.Context, bucket, unencryptedKey string, data io.Reader, metadata Metadata, expiration time.Time) (_ Meta, err error)

Put breaks up data as it comes in into s.segmentSize length pieces, then store the first piece at s0/<key>, second piece at s1/<key>, and the *last* piece at l/<key>. Store the given metadata, along with the number of segments, in a new protobuf, in the metadata of l/<key>.

If there is an error, it cleans up any uploaded segment before returning.

func (*Store) PutPart

func (s *Store) PutPart(ctx context.Context, bucket, unencryptedKey string, streamID storj.StreamID, partNumber uint32, eTag ETag, data io.Reader) (_ Part, err error)

PutPart uploads single part.

func (*Store) Ranger

func (s *Store) Ranger(ctx context.Context, response metaclient.DownloadSegmentWithRSResponse) (rr ranger.Ranger, err error)

Ranger creates a ranger for downloading erasure codes from piece store nodes.