loader

Documentation

Index

• Variables
• type Loader
  • func InitLoader(customS3Endpoint string) Loader
  • func (loader *Loader) DownloadFiles(buckets []string, keys []string, targetDir string) error
  • func (loader *Loader) UploadFiles(bucket string, baseKey string, filepaths []string) error

Variables

var ErrInequalKeyAndBucketLen = errors.New("Foo")

ErrInequalKeyAndBucketLen Error to indicate inequal number of buckets and keys

Types

type Loader

type Loader struct {
	Downloader *s3manager.Downloader
	Uploader   *s3manager.Uploader
}

Loader Data loader to stage data from and to object storage

func InitLoader

func InitLoader(customS3Endpoint string) Loader

InitLoader Initiates a new data loader

func (*Loader) DownloadFiles

func (loader *Loader) DownloadFiles(buckets []string, keys []string, targetDir string) error

DownloadFiles Downloads the files into a directory

func (*Loader) UploadFiles

func (loader *Loader) UploadFiles(bucket string, baseKey string, filepaths []string) error

UploadFiles Uploads the files to the given object storage location