blaze

package
v0.34.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 12, 2024 License: MIT Imports: 27 Imported by: 2

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrExistingLink = xo.BF("existing link")

ErrExistingLink is returned by Attach if a link already exists.

View Source 
var ErrInvalidHandle = xo.BF("invalid handle")

ErrInvalidHandle is returned if the provided handle is invalid.

View Source 
var ErrInvalidPosition = xo.BF("invalid position")

ErrInvalidPosition is returned if a seek resulted in an invalid position.

View Source 
var ErrNotFound = xo.BF("not found")

ErrNotFound is returned if there is no blob for the provided handle.

View Source 
var ErrUsedHandle = xo.BF("used handle")

ErrUsedHandle is returned if the provided handle has already been used.

Functions

func Assign added in v0.34.0 

func Assign(ctx context.Context, store *coal.Store, bucket *Bucket, model coal.Model, field string, newLink *Link) error

Assign will claim/release the specified link on the provided model to match the provided new link. If a link already exists it is released beforehand. A transaction is used to lock and refresh the model, release and claim the file and update the model.

func Attach added in v0.33.1 

func Attach(ctx context.Context, store *coal.Store, bucket *Bucket, model coal.Model, field string, input io.Reader, name, typ string, size int64) error

Attach will upload and claim a file on the provided model. If a link already exists ErrExistingLink is returned. A transaction is used to claim the file and update the model.

func DownloadTo added in v0.28.1 

func DownloadTo(download Download, writer io.Writer) error

DownloadTo will download a blob to the provided writer.

func EnsureService added in v0.33.0 

func EnsureService(name string) coal.Migration

EnsureService will return a migration that ensures that the specified service name is set on any file missing a service name. 

func IsValidLink(requireFilename bool, whitelist ...string) func(sub stick.Subject) error

IsValidLink returns a stick.Validate compatible link rule. 

func IsValidLinks(uniqueFilenames bool, whitelist ...string) func(sub stick.Subject) error

IsValidLinks returns a stick.Validate compatible links rule.

func TestService added in v0.33.0 

func TestService(t Tester, svc Service)

TestService will test the specified service for compatibility.

func TestServiceSeek added in v0.33.0 

func TestServiceSeek(t Tester, svc Service)

TestServiceSeek will test the specified service for seek compatibility.

func UploadFrom added in v0.28.1 

func UploadFrom(upload Upload, reader io.Reader) (int64, error)

UploadFrom will upload a blob from the provided reader.

func ValidateType added in v0.29.0 

func ValidateType(str string, whitelist ...string) error

ValidateType will validate a media type.

Types

type Binding added in v0.29.0 

type Binding struct {
	// The name e.g. "user-avatar".
	Name string

	// The model.
	Model coal.Model

	// The link field on the model.
	Field string

	// The file size limit.
	Limit int64

	// The allowed media types.
	Types []string

	// The forced filename for downloads.
	FileName string
}

Binding describes the binding if a file to a model.

func (*Binding) Validate added in v0.33.0 

func (b *Binding) Validate() error

Validate will validate the binding.

type Bucket added in v0.33.0 

type Bucket struct {
	// contains filtered or unexported fields
}

Bucket provides file storage capabilities.

func NewBucket added in v0.33.0 

func NewBucket(store *coal.Store, notary *heat.Notary, bindings ...*Binding) *Bucket

NewBucket creates a new bucket from a store, notary and binding registry.

func (*Bucket) Claim added in v0.33.0 

func (b *Bucket) Claim(ctx context.Context, model coal.Model, field string) error

Claim will claim the link at the field on the provided model. The claimed link must be persisted in the same transaction as the claim to ensure consistency.

func (*Bucket) ClaimFile added in v0.33.0 

func (b *Bucket) ClaimFile(ctx context.Context, claimKey, binding string, owner coal.ID) (*File, error)

ClaimFile will claim the file referenced by the provided claim key using the specified binding and owner.

Note: For consistency this should be called within a transaction. 

func (b *Bucket) ClaimLink(ctx context.Context, link *Link, binding string, owner coal.ID) error

ClaimLink will claim the provided link under the specified binding. The claimed link must be persisted in the same transaction as the claim to ensure consistency.

func (*Bucket) CleanupFile added in v0.33.0 

func (b *Bucket) CleanupFile(ctx context.Context, id coal.ID) error

CleanupFile will clean up a single file. In the first step, files in the uploading or uploaded state are marked as "deleting". In the second step, blobs of "deleting" files are deleted. In the last step "deleting" files with missing blobs are fully deleted.

func (*Bucket) CleanupTask added in v0.33.0 

func (b *Bucket) CleanupTask(retention time.Duration, batch int) *axe.Task

CleanupTask will return a periodic task that will scan and enqueue jobs that clean up files in the "uploading" or "uploaded" state older than the specified retention as well as files in the "released" and "deleting" state.

func (*Bucket) Decorate added in v0.33.0 

func (b *Bucket) Decorate(ctx context.Context, link *Link) error

Decorate will populate the provided link if a file is available.

func (*Bucket) Decorator added in v0.33.0 

func (b *Bucket) Decorator(fields ...string) *fire.Callback

Decorator will populate all or just the specified link fields.

func (*Bucket) Download added in v0.33.0 

func (b *Bucket) Download(ctx context.Context, viewKey string) (Download, *File, error)

Download will initiate a download for the file referenced by the provided view key.

func (*Bucket) DownloadAction added in v0.33.0 

func (b *Bucket) DownloadAction(timeout time.Duration) *fire.Action

DownloadAction returns an endpoint that allows downloading files using view keys. This action is usually publicly accessible.

func (*Bucket) DownloadFile added in v0.33.0 

func (b *Bucket) DownloadFile(ctx context.Context, id coal.ID) (Download, *File, error)

DownloadFile will initiate a download for the specified file.

func (*Bucket) GetViewKey added in v0.33.0 

func (b *Bucket) GetViewKey(ctx context.Context, file coal.ID) (string, error)

GetViewKey generates and returns a view key for the specified file.

func (*Bucket) MigrateFile added in v0.33.0 

func (b *Bucket) MigrateFile(ctx context.Context, id coal.ID) error

MigrateFile will migrate a single file by re-uploading the specified files blob using on of the configured uploader services and swapping the handle to the original file. On success or error the original or temporary blob is automatically cleaned up.

func (*Bucket) MigrateTask added in v0.33.0 

func (b *Bucket) MigrateTask(services []string, batch int) *axe.Task

MigrateTask will return a periodic task that will scan and enqueue jobs that migrates files using one of the provided services to the current uploader services. Up to specified amount of files are migrated per run.

func (*Bucket) Modifier added in v0.33.0 

func (b *Bucket) Modifier(fields ...string) *fire.Callback

Modifier will handle modifications on all or just the specified link fields.

func (*Bucket) RedirectAction added in v0.34.4 

func (b *Bucket) RedirectAction(field, endpoint string) *fire.Action

RedirectAction will construct an action that will redirect to the specified endpoint with the view key of the file in the specified field. The endpoint should be the relative or absolute URL of the blaze download endpoint.

func (*Bucket) Release added in v0.33.0 

func (b *Bucket) Release(ctx context.Context, model coal.Model, field string) error

Release will release the link at the field on the provided model. The released link must be persisted in the same transaction as the release to ensure consistency.

func (*Bucket) ReleaseFile added in v0.33.0 

func (b *Bucket) ReleaseFile(ctx context.Context, file coal.ID) error

ReleaseFile will release the file with the provided ID.

Note: For consistency this should be called within a transaction. 

func (b *Bucket) ReleaseLink(ctx context.Context, link *Link) error

ReleaseLink will release the provided link. The released link must be persisted in the same transaction as the release to ensure consistency.

func (*Bucket) Upload added in v0.33.0 

func (b *Bucket) Upload(ctx context.Context, name, mediaType string, size int64, cb func(Upload) (int64, error)) (string, *File, error)

Upload will initiate and perform an upload using the provided callback and return a claim key and the uploaded file. Upload must be called outside a transaction to ensure the uploaded file is tracked in case of errors.

func (*Bucket) UploadAction added in v0.33.0 

func (b *Bucket) UploadAction(limit int64, timeout time.Duration) *fire.Action

UploadAction returns an action that provides an upload endpoint that stores files and returns claim keys. The action should be protected and only allow authorized clients.

func (*Bucket) Use added in v0.33.0 

func (b *Bucket) Use(service Service, name string, upload bool)

Use will register the specified service using the provided name. If upload is true the service is used for new uploads.

type ClaimKey added in v0.26.0 

type ClaimKey struct {
	heat.Base `json:"-" heat:"blaze/claim,1h"`

	// The claimable file.
	File coal.ID `json:"file"`

	// The files name.
	Name string `json:"name"`

	// The files size.
	Size int64 `json:"size"`

	// The files media type.
	Type string `json:"type"`
}

ClaimKey is used to authorize file claims.

func (*ClaimKey) Validate added in v0.26.0 

func (k *ClaimKey) Validate() error

Validate will validate the claim key.

type CleanupJob added in v0.28.0 

type CleanupJob struct {
	axe.Base           `json:"-" axe:"blaze/cleanup"`
	stick.NoValidation `json:"-"`
}

CleanupJob is the periodic job enqueued to clean up a bucket.

type Download added in v0.28.1 

type Download interface {
	Seek(offset int64, whence int) (int64, error)
	Read(buf []byte) (int, error)
	Close() error
}

Download handles the download of a blob.

func SeekableDownload added in v0.33.0 

func SeekableDownload(size int64, source func(offset int64) (io.ReadCloser, error)) Download

SeekableDownload returns a download that is seekable from a function that opens a download stream at a given offset. This function is useful to efficiently implemented a seekable download over a service that does not provide native seekable downloads.

type File added in v0.26.0 

type File struct {
	coal.Base `json:"-" bson:",inline" coal:"files"`

	// The current state of the file e.g. "uploading".
	State State `json:"state"`

	// The last time the file was updated.
	Updated time.Time `json:"updated-at" bson:"updated_at"`

	// The name of the file e.g. "logo.png".
	Name string `json:"name"`

	// The media type of the file e.g. "image/png".
	Type string `json:"type"`

	// The size of the file.
	Size int64 `json:"size"`

	// The blob storage service.
	Service string `json:"service"`

	// The service specific blob handle.
	Handle Handle `json:"handle"`

	// The binding of the file.
	Binding string `json:"binding"`

	// The owner of the file.
	Owner *coal.ID `json:"owner"`
}

File tracks uploaded files and their state.

func (*File) Validate added in v0.28.0 

func (f *File) Validate() error

Validate will validate the model.

type GridFS added in v0.26.0 

type GridFS struct {
	// contains filtered or unexported fields
}

GridFS stores blobs in a GridFS bucket.

func NewGridFS added in v0.26.0 

func NewGridFS(bucket *lungo.Bucket) *GridFS

NewGridFS creates a new GridFS service.

Note: The bucket's indexes must already be ensured.

func (*GridFS) Delete added in v0.26.0 

func (g *GridFS) Delete(ctx context.Context, handle Handle) error

Delete implements the Service interface.

func (*GridFS) Download added in v0.26.0 

func (g *GridFS) Download(ctx context.Context, handle Handle) (Download, error)

Download implements the Service interface.

func (*GridFS) Lookup added in v0.33.0 

func (g *GridFS) Lookup(ctx context.Context, handle Handle) (Info, error)

Lookup implements the Service interface.

func (*GridFS) Prepare added in v0.26.0 

func (g *GridFS) Prepare(context.Context) (Handle, error)

Prepare implements the Service interface.

func (*GridFS) Upload added in v0.26.0 

func (g *GridFS) Upload(ctx context.Context, handle Handle, _ Info) (Upload, error)

Upload implements the Service interface.

type Handle added in v0.26.0 

type Handle map[string]interface{}

Handle is a reference to a blob stored in a service.

type Info added in v0.33.0 

type Info struct {
	Size      int64
	MediaType string
}

Info describes a blob stored in a service. 

type Link struct {
	// The local unique link reference. This value is used to map links in lists
	// and thus allow list layout changes. New links must be initialized with a
	// locally unique value.
	Ref string `json:"ref"`

	// The read-only name, type and size of the linked file.
	Name string `json:"name" bson:"-"`
	Type string `json:"type" bson:"-"`
	Size int64  `json:"size" bson:"-"`

	// The key for claiming a file. This value may be set by the client to link
	// a new file.
	ClaimKey string `json:"claim-key" bson:"-"`

	// The read-only key for viewing the linked file.
	ViewKey string `json:"view-key" bson:"-"`

	// The internal reference to the linked file.
	File coal.ID `json:"-" bson:"file_id"`

	// The internal information about the linked file.
	FileName string `json:"-" bson:"name"`
	FileType string `json:"-" bson:"type"`
	FileSize int64  `json:"-" bson:"size"`
}

Link is used to link a file to a model.

func (*Link) Validate added in v0.29.0 

func (l *Link) Validate(requireFilename bool, whitelist ...string) error

Validate will validate the link. 

type Links []Link

Links is a set of links.

func (*Links) UnmarshalBSONValue added in v0.29.1 

func (l *Links) UnmarshalBSONValue(typ bsontype.Type, bytes []byte) error

UnmarshalBSONValue implements the bson.ValueUnmarshaler interface.

func (*Links) UnmarshalJSON added in v0.29.1 

func (l *Links) UnmarshalJSON(bytes []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*Links) Validate added in v0.29.1 

func (l *Links) Validate(uniqueFilenames bool, whitelist ...string) error

Validate will validate the links.

type Memory added in v0.26.0 

type Memory struct {
	// The stored blobs.
	Blobs map[string]*MemoryBlob

	// The next ID.
	Next int
	// contains filtered or unexported fields
}

Memory is a service for testing purposes that stores blobs in memory.

func NewMemory added in v0.26.0 

func NewMemory() *Memory

NewMemory will create a new memory service.

func (*Memory) Delete added in v0.26.0 

func (m *Memory) Delete(_ context.Context, handle Handle) error

Delete implements the Service interface.

func (*Memory) Download added in v0.26.0 

func (m *Memory) Download(_ context.Context, handle Handle) (Download, error)

Download implements the Service interface.

func (*Memory) Lookup added in v0.33.0 

func (m *Memory) Lookup(_ context.Context, handle Handle) (Info, error)

Lookup implements the Service interface.

func (*Memory) Prepare added in v0.26.0 

func (m *Memory) Prepare(context.Context) (Handle, error)

Prepare implements the Service interface.

func (*Memory) Upload added in v0.26.0 

func (m *Memory) Upload(_ context.Context, handle Handle, info Info) (Upload, error)

Upload implements the Service interface.

type MemoryBlob added in v0.29.0 

type MemoryBlob struct {
	Type  string
	Bytes []byte
}

MemoryBlob is a blob stored by the memory service.

type MigrateJob added in v0.33.0 

type MigrateJob struct {
	axe.Base           `json:"-" axe:"blaze/migrate"`
	stick.NoValidation `json:"-"`
}

MigrateJob is the periodic job enqueued to migrate files in a bucket.

type Minio added in v0.33.0 

type Minio struct {
	// contains filtered or unexported fields
}

Minio stores blobs in a S3 compatible bucket.

func NewMinio added in v0.33.0 

func NewMinio(client *minio.Client, bucket string) *Minio

NewMinio creates a new Minio service.

func (*Minio) Delete added in v0.33.0 

func (m *Minio) Delete(ctx context.Context, handle Handle) error

Delete implements the Service interface.

func (*Minio) Download added in v0.33.0 

func (m *Minio) Download(ctx context.Context, handle Handle) (Download, error)

Download implements the Service interface.

func (*Minio) Lookup added in v0.33.0 

func (m *Minio) Lookup(ctx context.Context, handle Handle) (Info, error)

Lookup implements the Service interface.

func (*Minio) Prepare added in v0.33.0 

func (m *Minio) Prepare(context.Context) (Handle, error)

Prepare implements the Service interface.

func (*Minio) Upload added in v0.33.0 

func (m *Minio) Upload(ctx context.Context, handle Handle, info Info) (Upload, error)

Upload implements the Service interface.

type Registry added in v0.33.2 

type Registry struct {
	*stick.Registry[*Binding]
}

Registry is a collection of known bindings.

func NewRegistry added in v0.33.0 

func NewRegistry(bindings ...*Binding) *Registry

NewRegistry returns a binding registry indexed by the binding name and owner/field tuple.

type Service added in v0.26.0 

type Service interface {
	// Prepare should return a new handle for uploading a blob.
	Prepare(ctx context.Context) (Handle, error)

	// Upload should initiate the upload of a blob.
	Upload(ctx context.Context, handle Handle, info Info) (Upload, error)

	// Lookup should retrieve the specified block and return its media type and size.
	Lookup(ctx context.Context, handle Handle) (Info, error)

	// Download should initiate the download of a blob.
	Download(ctx context.Context, handle Handle) (Download, error)

	// Delete should delete the blob.
	Delete(ctx context.Context, handle Handle) error
}

Service is responsible for managing blobs.

type State added in v0.26.0 

type State string

State describes the current state of a file. Usually, the state of a file always moves forward by one step, but it may also jump directly to "deleting". 

const (
	Uploading State = "uploading"
	Uploaded  State = "uploaded"
	Claimed   State = "claimed"
	Released  State = "released"
	Deleting  State = "deleting"
)

The individual states.

func (State) Valid added in v0.28.0 

func (s State) Valid() bool

Valid returns whether the state is valid.

type Tester added in v0.33.0 

type Tester interface {
	Errorf(format string, args ...interface{})
}

Tester is a common interface implemented by test objects.

type Upload added in v0.28.1 

type Upload interface {
	Write(data []byte) (int, error)
	Abort() error
	Close() error
}

Upload handles the upload of a blob.

func PipeUpload added in v0.33.0 

func PipeUpload(fn func(upload io.Reader) error) Upload

PipeUpload returns an upload that pipes data to the reader yielded to the provided callback. This function is useful to upload data to a service that expects a reader. Errors from the callback are returned by the upload either on write or on close.

type ViewKey added in v0.26.0 

type ViewKey struct {
	heat.Base `json:"-" heat:"blaze/view,24h"`

	// The viewable file.
	File coal.ID `json:"file"`
}

ViewKey is used to authorize file views.

func (*ViewKey) Validate added in v0.26.0 

func (k *ViewKey) Validate() error

Validate will validate the view key.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.