transfer

Documentation

Overview

Package transfer collects together adapters for uploading and downloading LFS content NOTE: Subject to change, do not rely on this package from outside git-lfs source

Index

• Constants
• func NewCustomAdapterDownloadRequest(oid string, size int64, action *api.LinkRelation) *customAdapterTransferRequest
• func NewCustomAdapterInitRequest(op string, concurrent bool, concurrentTransfers int) *customAdapterInitRequest
• func NewCustomAdapterTerminateRequest() *customAdapterTerminateRequest
• func NewCustomAdapterUploadRequest(oid string, size int64, path string, action *api.LinkRelation) *customAdapterTransferRequest
• type Direction
• type Manifest
  • func ConfigureManifest(m *Manifest, cfg *config.Configuration) *Manifest
  • func NewManifest() *Manifest
  • func (m *Manifest) GetAdapterNames(dir Direction) []string
  • func (m *Manifest) GetDownloadAdapterNames() []string
  • func (m *Manifest) GetUploadAdapterNames() []string
  • func (m *Manifest) NewAdapter(name string, dir Direction) TransferAdapter
  • func (m *Manifest) NewAdapterOrDefault(name string, dir Direction) TransferAdapter
  • func (m *Manifest) NewDownloadAdapter(name string) TransferAdapter
  • func (m *Manifest) NewUploadAdapter(name string) TransferAdapter
  • func (m *Manifest) RegisterNewTransferAdapterFunc(name string, dir Direction, f NewTransferAdapterFunc)
• type NewTransferAdapterFunc
• type Transfer
  • func NewTransfer(name string, obj *api.ObjectResource, path string) *Transfer
• type TransferAdapter
• type TransferProgressCallback
• type TransferResult

Constants

const (
	Upload   = Direction(iota)
	Download = Direction(iota)
)

const (
	TusAdapterName = "tus"
	TusVersion     = "1.0.0"
)

const (
	BasicAdapterName = "basic"
)

Functions

func NewCustomAdapterDownloadRequest

func NewCustomAdapterDownloadRequest(oid string, size int64, action *api.LinkRelation) *customAdapterTransferRequest

func NewCustomAdapterInitRequest

func NewCustomAdapterInitRequest(op string, concurrent bool, concurrentTransfers int) *customAdapterInitRequest

func NewCustomAdapterTerminateRequest

func NewCustomAdapterTerminateRequest() *customAdapterTerminateRequest

func NewCustomAdapterUploadRequest

func NewCustomAdapterUploadRequest(oid string, size int64, path string, action *api.LinkRelation) *customAdapterTransferRequest

Types

type Direction

type Direction int

type Manifest

type Manifest struct {
	// contains filtered or unexported fields
}

func ConfigureManifest

func ConfigureManifest(m *Manifest, cfg *config.Configuration) *Manifest

func NewManifest

func NewManifest() *Manifest

func (*Manifest) GetAdapterNames

func (m *Manifest) GetAdapterNames(dir Direction) []string

GetAdapterNames returns a list of the names of adapters available to be created

func (*Manifest) GetDownloadAdapterNames

func (m *Manifest) GetDownloadAdapterNames() []string

GetDownloadAdapterNames returns a list of the names of download adapters available to be created

func (*Manifest) GetUploadAdapterNames

func (m *Manifest) GetUploadAdapterNames() []string

GetUploadAdapterNames returns a list of the names of upload adapters available to be created

func (*Manifest) NewAdapter

func (m *Manifest) NewAdapter(name string, dir Direction) TransferAdapter

Create a new adapter by name and direction, or nil if doesn't exist

func (*Manifest) NewAdapterOrDefault

func (m *Manifest) NewAdapterOrDefault(name string, dir Direction) TransferAdapter

Create a new adapter by name and direction; default to BasicAdapterName if doesn't exist

func (*Manifest) NewDownloadAdapter

func (m *Manifest) NewDownloadAdapter(name string) TransferAdapter

Create a new download adapter by name, or BasicAdapterName if doesn't exist

func (*Manifest) NewUploadAdapter

func (m *Manifest) NewUploadAdapter(name string) TransferAdapter

Create a new upload adapter by name, or BasicAdapterName if doesn't exist

func (*Manifest) RegisterNewTransferAdapterFunc

func (m *Manifest) RegisterNewTransferAdapterFunc(name string, dir Direction, f NewTransferAdapterFunc)

RegisterNewTransferAdapterFunc registers a new function for creating upload or download adapters. If a function with that name & direction is already registered, it is overridden

type NewTransferAdapterFunc

type NewTransferAdapterFunc func(name string, dir Direction) TransferAdapter

NewTransferAdapterFunc creates new instances of TransferAdapter. Code that wishes to provide new TransferAdapter instances should pass an implementation of this function to RegisterNewTransferAdapterFunc() on a *Manifest. name and dir are to provide context if one func implements many instances

type Transfer

type Transfer struct {
	// Name of the file that triggered this transfer
	Name string
	// Object from API which provides the core data for this transfer
	Object *api.ObjectResource
	// Path for uploads is the source of data to send, for downloads is the
	// location to place the final result
	Path string
}

General struct for both uploads and downloads

func NewTransfer

func NewTransfer(name string, obj *api.ObjectResource, path string) *Transfer

NewTransfer creates a new Transfer instance

type TransferAdapter

type TransferAdapter interface {
	// Name returns the name of this adapter, which is the same for all instances
	// of this type of adapter
	Name() string
	// Direction returns whether this instance is an upload or download instance
	// TransferAdapter instances can only be one or the other, although the same
	// type may be instantiated for each direction
	Direction() Direction
	// Begin a new batch of uploads or downloads. Call this first, followed by
	// one or more Add calls. maxConcurrency controls the number of transfers
	// that may be done at once. The passed in callback will receive updates on
	// progress, and the completion channel will receive completion notifications
	// Either argument may be nil if not required by the client
	Begin(maxConcurrency int, cb TransferProgressCallback, completion chan TransferResult) error
	// Add queues a download/upload, which will complete asynchronously and
	// notify the callbacks given to Begin()
	Add(t *Transfer)
	// Indicate that all transfers have been scheduled and resources can be released
	// once the queued items have completed.
	// This call blocks until all items have been processed
	End()
	// ClearTempStorage clears any temporary files, such as unfinished downloads that
	// would otherwise be resumed
	ClearTempStorage() error
}

TransferAdapter is implemented by types which can upload and/or download LFS file content to a remote store. Each TransferAdapter accepts one or more requests which it may schedule and parallelise in whatever way it chooses, clients of this interface will receive notifications of progress and completion asynchronously. TransferAdapters support transfers in one direction; if an implementation provides support for upload and download, it should be instantiated twice, advertising support for each direction separately. Note that TransferAdapter only implements the actual upload/download of content itself; organising the wider process including calling the API to get URLs, handling progress reporting and retries is the job of the core TransferQueue. This is so that the orchestration remains core & standard but TransferAdapter can be changed to physically transfer to different hosts with less code.

type TransferProgressCallback

type TransferProgressCallback func(name string, totalSize, readSoFar int64, readSinceLast int) error

type TransferResult

type TransferResult struct {
	Transfer *Transfer
	// This will be non-nil if there was an error transferring this item
	Error error
}

Result of a transfer returned through CompletionChannel()