storagesvc

README

StorageSvc

StorageSvc consists of 3 components

• Storage http request handler
• StowClient
• ArchivePruner

StorageSvc

This is the HTTP handler that serves requests to :

• upload archive into a storage
• fetch an archive from storage
• delete archive from storage

StowClient

This is the storage interface layer that interacts with stow package. It provides methods to:

• write a file to storage
• retrieve a file from storage
• delete a file from storage
• get all files on storage

ArchivePruner

This acts like a cron job to clean up orphaned archives from storage. By default configured to run every hour. The value can be set in Values.yaml to any preferred interval.

Documentation

Index

• Constants
• Variables
• func Start(ctx context.Context, clientGen crd.ClientGeneratorInterface, ...) error
• type ArchivePruner
  • func MakeArchivePruner(logger *zap.Logger, clientGen crd.ClientGeneratorInterface, ...) (*ArchivePruner, error)
  • func (pruner *ArchivePruner) Start(ctx context.Context, mgr manager.Interface)
• type Storage
  • func NewLocalStorage(localPath string) Storage
  • func NewS3Storage(args ...string) Storage
• type StorageService
  • func MakeStorageService(logger *zap.Logger, storageClient *StowClient, port int) *StorageService
  • func (ss *StorageService) Start(ctx context.Context, mgr manager.Interface, port int)
• type StorageType
• type StowClient
  • func MakeStowClient(logger *zap.Logger, storage Storage) (*StowClient, error)
• type UploadResponse

Constants

const (
	// StorageTypeLocal is a constant to hold local storate type name literal
	StorageTypeLocal StorageType = "local"
	// StorageTypeS3 is a constant to hold S3 storage type name literal
	StorageTypeS3 StorageType = "s3"
	// PaginationSize is a constant to hold no of pages
	PaginationSize int = 10
)

Variables

var (
	ErrNotFound                = errors.New("not found")
	ErrRetrievingItem          = errors.New("unable to retrieve item")
	ErrOpeningItem             = errors.New("unable to open item")
	ErrWritingFile             = errors.New("unable to write file")
	ErrWritingFileIntoResponse = errors.New("unable to copy item into http response")
)

Functions

func Start

func Start(ctx context.Context, clientGen crd.ClientGeneratorInterface, logger *zap.Logger, storage Storage, mgr manager.Interface, port int) error

Start runs storage service

Types

type ArchivePruner

type ArchivePruner struct {
	// contains filtered or unexported fields
}

func MakeArchivePruner

func MakeArchivePruner(logger *zap.Logger, clientGen crd.ClientGeneratorInterface, stowClient *StowClient, pruneInterval time.Duration) (*ArchivePruner, error)

func (*ArchivePruner) Start

func (pruner *ArchivePruner) Start(ctx context.Context, mgr manager.Interface)

Start starts a go routine that listens to a channel for archive IDs that need to deleted. Also wakes up at regular intervals to make a list of archive IDs that need to be reaped and sends them over to the channel for deletion

type Storage

type Storage interface {
	// contains filtered or unexported methods
}

Storage is an interface to force storage level details implementation.

func NewLocalStorage

func NewLocalStorage(localPath string) Storage

NewLocalStorage return new local storage struct

func NewS3Storage

func NewS3Storage(args ...string) Storage

NewS3Storage returns a new s3 storage struct

type StorageService

type StorageService struct {
	// contains filtered or unexported fields
}

StorageService is a struct to hold all things for storage service

func MakeStorageService

func MakeStorageService(logger *zap.Logger, storageClient *StowClient, port int) *StorageService

func (*StorageService) Start

func (ss *StorageService) Start(ctx context.Context, mgr manager.Interface, port int)

type StorageType

type StorageType string

StorageType contains all different types of supported storage

type StowClient

type StowClient struct {
	// contains filtered or unexported fields
}

StowClient is the wrapper client for stow (Cloud storage abstraction package)

func MakeStowClient

func MakeStowClient(logger *zap.Logger, storage Storage) (*StowClient, error)

MakeStowClient create a new StowClient for given storage

type UploadResponse

type UploadResponse struct {
	ID string `json:"id"`
}