pieceupload

Documentation

Index

• Variables
• func UploadOne(longTailCtx, uploadCtx context.Context, manager *Manager, putter PiecePutter, ...) (_ bool, err error)
• type LimitsExchanger
• type Manager
  • func NewManager(exchanger LimitsExchanger, pieceReader PieceReader, segmentID storj.SegmentID, ...) *Manager
  • func (mgr *Manager) NextPiece(ctx context.Context) (_ io.Reader, _ *pb.AddressedOrderLimit, ...)
  • func (mgr *Manager) Results() (storj.SegmentID, []*pb.SegmentPieceUploadResult)
• type PiecePutter
• type PieceReader

Variables

var (
	// ErrDone is returned from the Manager NextPiece when all of the piece
	// uploads have completed.
	ErrDone = errs.New("all pieces have been uploaded")
)

Functions

func UploadOne

func UploadOne(longTailCtx, uploadCtx context.Context, manager *Manager, putter PiecePutter, privateKey storj.PiecePrivateKey) (_ bool, err error)

UploadOne uploads one piece from the manager using the given private key. If it fails, it will attempt to upload another until either the upload context, or the long tail context is cancelled.

Types

type LimitsExchanger

type LimitsExchanger interface {
	ExchangeLimits(ctx context.Context, segmentID storj.SegmentID, pieceNumbers []int) (storj.SegmentID, []*pb.AddressedOrderLimit, error)
}

LimitsExchanger exchanges piece upload limits.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager tracks piece uploads for a segment. It provides callers with piece data and limits and tracks which uploads have been successful (or not). It also manages obtaining new piece upload limits for failed uploads to add resiliency to segment uploads. The manager also keeps track of the upload results, which the caller can use to commit the segment, including the segment ID, which is updated as limits are exchanged.

func NewManager

func NewManager(exchanger LimitsExchanger, pieceReader PieceReader, segmentID storj.SegmentID, limits []*pb.AddressedOrderLimit) *Manager

NewManager returns a new piece upload manager.

func (*Manager) NextPiece

func (mgr *Manager) NextPiece(ctx context.Context) (_ io.Reader, _ *pb.AddressedOrderLimit, _ func(hash *pb.PieceHash, uploaded bool), err error)

NextPiece returns a reader and limit for the next piece to upload. It also returns a callback that the caller uses to indicate success (along with the results) or not. NextPiece may return data with a new limit for a piece that was previously attempted but failed. It will return ErrDone when enough pieces have finished successfully to satisfy the optimal threshold. If NextPiece is unable to exchange limits for failed pieces, it will return an error.

func (*Manager) Results

func (mgr *Manager) Results() (storj.SegmentID, []*pb.SegmentPieceUploadResult)

Results returns the results of each piece successfully updated as well as the segment ID, which may differ from that passed into NewManager if piece limits needed to be exchanged for failed piece uploads.

type PiecePutter

type PiecePutter interface {
	// PutPiece puts a piece using the given limit and private key. The
	// operation can be cancelled using the longTailCtx or uploadCtx is
	// cancelled.
	PutPiece(longTailCtx, uploadCtx context.Context, limit *pb.AddressedOrderLimit, privateKey storj.PiecePrivateKey, data io.ReadCloser) (hash *pb.PieceHash, deprecated *struct{}, err error)
}

PiecePutter puts pieces.

type PieceReader

type PieceReader interface {
	PieceReader(num int) io.Reader
}

PieceReader provides a reader for a piece with the given number.