compress

package
v0.0.0-...-b4aeae5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 7, 2024 License: MIT Imports: 28 Imported by: 0

Documentation

Overview

Package compress provides wrappers for Fs and Object which implement compression.

Index

Constants

View Source
const (
	Uncompressed = 0
	Gzip         = 2
)

Compression modes

Variables

This section is empty.

Functions

func NewFs

func NewFs(ctx context.Context, name, rpath string, m configmap.Mapper) (fs.Fs, error)

NewFs constructs an Fs from the path, container:path

Types

type Fs

type Fs struct {
	fs.Fs
	// contains filtered or unexported fields
}

Fs represents a wrapped fs.Fs

func (*Fs) About

func (f *Fs) About(ctx context.Context) (*fs.Usage, error)

About gets quota information from the Fs

func (*Fs) ChangeNotify

func (f *Fs) ChangeNotify(ctx context.Context, notifyFunc func(string, fs.EntryType), pollIntervalChan <-chan time.Duration)

ChangeNotify calls the passed function with a path that has had changes. If the implementation uses polling, it should adhere to the given interval.

func (*Fs) CleanUp

func (f *Fs) CleanUp(ctx context.Context) error

CleanUp the trash in the Fs

Implement this if you have a way of emptying the trash or otherwise cleaning up old versions of files.

func (*Fs) Copy

func (f *Fs) Copy(ctx context.Context, src fs.Object, remote string) (fs.Object, error)

Copy src to this remote using server side copy operations.

This is stored with the remote path given.

It returns the destination Object and a possible error.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantCopy

func (*Fs) DirCacheFlush

func (f *Fs) DirCacheFlush()

DirCacheFlush resets the directory cache - used in testing as an optional interface

func (*Fs) DirMove

func (f *Fs) DirMove(ctx context.Context, src fs.Fs, srcRemote, dstRemote string) error

DirMove moves src, srcRemote to this remote at dstRemote using server side move operations.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantDirMove

If destination exists then return fs.ErrorDirExists

func (*Fs) DirSetModTime

func (f *Fs) DirSetModTime(ctx context.Context, dir string, modTime time.Time) error

DirSetModTime sets the directory modtime for dir

func (*Fs) Features

func (f *Fs) Features() *fs.Features

Features returns the optional features of this Fs

func (*Fs) Hashes

func (f *Fs) Hashes() hash.Set

Hashes returns the supported hash sets.

func (*Fs) List

func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err error)

List the objects and directories in dir into entries. The entries can be returned in any order but should be for a complete directory.

dir should be "" to list the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found. List entries and process them

func (*Fs) ListR

func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (err error)

ListR lists the objects and directories of the Fs starting from dir recursively into out.

dir should be "" to start from the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found.

It should call callback for each tranche of entries read. These need not be returned in any particular order. If callback returns an error then the listing will stop immediately.

Don't implement this unless you have a more efficient way of listing recursively that doing a directory traversal.

func (*Fs) MergeDirs

func (f *Fs) MergeDirs(ctx context.Context, dirs []fs.Directory) error

MergeDirs merges the contents of all the directories passed in into the first one and rmdirs the other directories.

func (*Fs) Mkdir

func (f *Fs) Mkdir(ctx context.Context, dir string) error

Mkdir makes the directory (container, bucket)

Shouldn't return an error if it already exists

func (*Fs) MkdirMetadata

func (f *Fs) MkdirMetadata(ctx context.Context, dir string, metadata fs.Metadata) (fs.Directory, error)

MkdirMetadata makes the root directory of the Fs object

func (*Fs) Move

func (f *Fs) Move(ctx context.Context, src fs.Object, remote string) (fs.Object, error)

Move src to this remote using server side move operations.

This is stored with the remote path given.

It returns the destination Object and a possible error.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantMove

func (*Fs) Name

func (f *Fs) Name() string

Name of the remote (as passed into NewFs)

func (*Fs) NewObject

func (f *Fs) NewObject(ctx context.Context, remote string) (fs.Object, error)

NewObject finds the Object at remote.

func (*Fs) Precision

func (f *Fs) Precision() time.Duration

Precision returns the precision of this Fs

func (f *Fs) PublicLink(ctx context.Context, remote string, duration fs.Duration, unlink bool) (string, error)

PublicLink generates a public link to the remote path (usually readable by anyone)

func (*Fs) Purge

func (f *Fs) Purge(ctx context.Context, dir string) error

Purge all files in the root and the root directory

Implement this if you have a way of deleting all the files quicker than just running Remove() on the result of List()

Return an error if it doesn't exist

func (*Fs) Put

func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

Put in to the remote path with the modTime given of the given size

May create the object even if it returns an error - if so will return the object and the error, otherwise will return nil and the error

func (*Fs) PutStream

func (f *Fs) PutStream(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

PutStream uploads to the remote path with the modTime given of indeterminate size

func (*Fs) Rmdir

func (f *Fs) Rmdir(ctx context.Context, dir string) error

Rmdir removes the directory (container, bucket) if empty

Return an error if it doesn't exist or isn't empty

func (*Fs) Root

func (f *Fs) Root() string

Root of the remote (as passed into NewFs)

func (*Fs) SetWrapper

func (f *Fs) SetWrapper(wrapper fs.Fs)

SetWrapper sets the Fs that is wrapping this Fs

func (*Fs) Shutdown

func (f *Fs) Shutdown(ctx context.Context) error

Shutdown the backend, closing any background tasks and any cached connections.

func (*Fs) String

func (f *Fs) String() string

Return a string version

func (*Fs) UnWrap

func (f *Fs) UnWrap() fs.Fs

UnWrap returns the Fs that this Fs is wrapping

func (*Fs) WrapFs

func (f *Fs) WrapFs() fs.Fs

WrapFs returns the Fs that is wrapping this Fs

type Object

type Object struct {
	fs.Object // Wraps around data object for this object
	// contains filtered or unexported fields
}

Object with external metadata

func (*Object) Fs

func (o *Object) Fs() fs.Info

Fs returns read only access to the Fs that this object is part of

func (*Object) GetTier

func (o *Object) GetTier() string

GetTier returns storage tier or class of the Object

func (*Object) Hash

func (o *Object) Hash(ctx context.Context, ht hash.Type) (string, error)

Hash returns the selected checksum of the file If no checksum is available it returns ""

func (*Object) ID

func (o *Object) ID() string

ID returns the ID of the Object if known, or "" if not

func (*Object) Metadata

func (o *Object) Metadata(ctx context.Context) (fs.Metadata, error)

Metadata returns metadata for an object

It should return nil if there is no Metadata

func (*Object) MimeType

func (o *Object) MimeType(ctx context.Context) string

MimeType returns the MIME type of the file

func (*Object) Open

func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (rc io.ReadCloser, err error)

Open opens the file for read. Call Close() on the returned io.ReadCloser. Note that this call requires quite a bit of overhead.

func (*Object) Remote

func (o *Object) Remote() string

Remote returns the remote path

func (*Object) Remove

func (o *Object) Remove(ctx context.Context) error

Remove removes this object

func (*Object) SetTier

func (o *Object) SetTier(tier string) error

SetTier performs changing storage tier of the Object if multiple storage classes supported

func (*Object) Size

func (o *Object) Size() int64

Size returns the size of the file

func (*Object) String

func (o *Object) String() string

Return a string version

func (*Object) UnWrap

func (o *Object) UnWrap() fs.Object

UnWrap returns the wrapped Object

func (*Object) Update

func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (err error)

Update in to the object with the modTime given of the given size

type ObjectInfo

type ObjectInfo struct {
	// contains filtered or unexported fields
}

ObjectInfo describes a wrapped fs.ObjectInfo for being the source

func (*ObjectInfo) Fs

func (o *ObjectInfo) Fs() fs.Info

Fs returns read only access to the Fs that this object is part of

func (*ObjectInfo) GetTier

func (o *ObjectInfo) GetTier() string

GetTier returns storage tier or class of the Object

func (*ObjectInfo) Hash

func (o *ObjectInfo) Hash(ctx context.Context, ht hash.Type) (string, error)

Hash returns the selected checksum of the file If no checksum is available it returns ""

func (*ObjectInfo) ID

func (o *ObjectInfo) ID() string

ID returns the ID of the Object if known, or "" if not

func (*ObjectInfo) Metadata

func (o *ObjectInfo) Metadata(ctx context.Context) (fs.Metadata, error)

Metadata returns metadata for an object

It should return nil if there is no Metadata

func (*ObjectInfo) MimeType

func (o *ObjectInfo) MimeType(ctx context.Context) string

MimeType returns the content type of the Object if known, or "" if not

func (*ObjectInfo) ModTime

func (o *ObjectInfo) ModTime(ctx context.Context) time.Time

ModTime returns the modification time

func (*ObjectInfo) Remote

func (o *ObjectInfo) Remote() string

Remote returns the remote path

func (*ObjectInfo) Size

func (o *ObjectInfo) Size() int64

Size returns the size of the file

func (*ObjectInfo) Storable

func (o *ObjectInfo) Storable() bool

Storable returns whether object is storable

func (*ObjectInfo) String

func (o *ObjectInfo) String() string

String returns string representation

func (*ObjectInfo) UnWrap

func (o *ObjectInfo) UnWrap() fs.Object

UnWrap returns the Object that this Object is wrapping or nil if it isn't wrapping anything

type ObjectMetadata

type ObjectMetadata struct {
	Mode                int    // Compression mode of the file.
	Size                int64  // Size of the object.
	MD5                 string // MD5 hash of the file.
	MimeType            string // Mime type of the file
	CompressionMetadata sgzip.GzipMetadata
}

ObjectMetadata describes the metadata for an Object.

type Options

type Options struct {
	Remote           string        `config:"remote"`
	CompressionMode  string        `config:"mode"`
	CompressionLevel int           `config:"level"`
	RAMCacheLimit    fs.SizeSuffix `config:"ram_cache_limit"`
}

Options defines the configuration for this backend

type ReadCloserWrapper

type ReadCloserWrapper struct {
	io.Reader
	io.Closer
}

ReadCloserWrapper combines a Reader and a Closer to a ReadCloser

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL