uploader

Documentation

Overview

Package uploader manages media files uploaded to a server. The files are assumed to be destined to belong to a parent database object, such as a slideshow, with the files being uploaded asynchronously from a form that modifies the parent. The parent need not exist at the time of upload, and a log is used to maintain consistency between the database and the media files.

Images are resized to fit within limits specified by the server. Videos are converted to MP4 format. Thumbnails are generated for both images and videos.

Note that files are given revision numbers for these reasons: (1) A different name forces browsers to fetch the updated file after its content has been changed. (2) It allows us to upload an file without overwriting the current one, and then forget it if the update form is not submitted. (3) The committed state of an object may be displayed with its files, while a new state is being prepared.

uploader maintains consistency between the parent object and the media files it references, with these limitations:

- Object saved before EndSave is completed : there is a brief period when new media files cannot be displayed. ## Can I check?

• After object saved, and before the bind operation, there is a brief period where it references new files and has deleted files removed, but still references the previous versions of updated files.

Use the uploader as follows:

(1) A web request is received to create or update a parent object: call Begin and add the transaction code as a hidden field in the form. Use NameFromFile to extract the media names shown to users from the media file names.

(2) A media file is uploaded via an AJAX request: call Save with the transaction code. Images are resized and thumbnails generated asynchronously to the request.

(3) A parent object is created, updated or deleted: call SetParent to associate the transaction code with the object. Use CleanName to sanitise user names for media, and use MediaType to check that uploaded file types are acceptable. If the media name is new or changed, call FileFromName to get the file name to be stored in the database. (Changed versions for existing names are handled in step 5.) Call tx.SetNext ensure the next step will be executed, commit the change to the database.

(4) Call DoNext. The uploader waits for any uploads to be processed, and then calls the operation specified by etx.SetNext.

(5) Handle the parent bind request from DoNext.

(i) Call StartBind to begin updating references between the parent and the media files.

(ii) For each media file referenced by the parent, call Bind.File, and record any new or updated references. Save the parent in the database.

(6) After the parent update has been committed, call Bind.End. Any existing files not listed by Bind will be deleted.

When deleting an object, call StartBind (with no request code), delete the object and then call EndBind.

Use Thumbnail to get the file name for a thumbnail image corresponding to a media file.

Index

• Constants
• Variables
• func CleanName(name string) string
• func FileFromName(id etx.TxId, name string) string
• func NameFromFile(fileName string) (string, string, int)
• func Thumbnail(filename string) string
• type Bind
  • func (b *Bind) End() error
  • func (b *Bind) File(fileName string) (string, error)
• type DB
• type OpOrphans
• type Uploader
  • func (up *Uploader) Begin() (string, error)
  • func (up *Uploader) DoNext(tx etx.TxId)
  • func (up *Uploader) ForOperation(opType int) etx.Op
  • func (up *Uploader) Initialise(log *log.Logger, db DB, tm *etx.TM)
  • func (up *Uploader) MediaType(name string) int
  • func (up *Uploader) Name() string
  • func (up *Uploader) Operation(id etx.TxId, opType int, op etx.Op)
  • func (up *Uploader) Save(fh *multipart.FileHeader, tx etx.TxId) (err error, byClient bool)
  • func (up *Uploader) StartBind(parentId int64, tx etx.TxId) *Bind
  • func (up *Uploader) Stop()
  • func (up *Uploader) ValidCode(tx etx.TxId) bool

Constants

const (
	MediaImage = 1
	MediaVideo = 2
	MediaAudio = 3
)

Variables

var WebFiles embed.FS

WebFiles are the package's web resources (templates and static files)

Functions

func CleanName

func CleanName(name string) string

CleanName removes unwanted characters from a filename, to make it safe for display and storage. From https://stackoverflow.com/questions/54461423/efficient-way-to-remove-all-non-alphanumeric-characters-from-large-text. ## This is far more restrictive than we need.

func FileFromName

func FileFromName(id etx.TxId, name string) string

FileFromName returns a stored file name from a user's name for a newly uploaded file. The owner is a transaction code, because the parent object may not exist yet. It has no revision number, so it doesn't overwrite a previous copy yet.

func NameFromFile

func NameFromFile(fileName string) (string, string, int)

NameFromFile returns the owner ID, media file name and revision from a file name. If the revision is 0, the owner is the request, otherwise the owner is a parent object.

func Thumbnail

func Thumbnail(filename string) string

Thumbnail returns the prefixed name for a thumbnail.

Types

type Bind

type Bind struct {
	// contains filtered or unexported fields
}

Context for a sequence of bind calls.

func (*Bind) End

func (b *Bind) End() error

End completes the linking a parent object. It deletes unused files. This includes:

• old versions that have been superseded;
• the upload names (resulting in deletion if the file wasn't referenced in the saved parent);
• files that are no referenced no more.

func (*Bind) File

func (b *Bind) File(fileName string) (string, error)

File is called to check if a file has changed. If so, it links the file to a new version, and returns the new filename. An empty string indicates no change.

type DB

type DB interface {
	Begin() func() // start transaction and return commit function
}

DB is an interface to the database manager that handles parent transactions.

type OpOrphans

type OpOrphans struct {
	// contains filtered or unexported fields
}

type Uploader

type Uploader struct {

	// parameters
	FilePath     string
	MaxW         int
	MaxH         int
	ThumbW       int
	ThumbH       int
	MaxAge       time.Duration // maximum time for a parent update
	SnapshotAt   time.Duration // snapshot time in video (-ve for none)
	AudioTypes   []string
	VideoPackage string // software for video processing: ffmpeg, or a docker-hosted implementation of ffmpeg, for debugging
	VideoTypes   []string
	// contains filtered or unexported fields
}

Uploader holds the parameters and state for uploading files. Typically only one is needed.

func (*Uploader) Begin

func (up *Uploader) Begin() (string, error)

Begin returns an identifier for an update that may include a set of uploads. It expects that a database transaction (needed to write redo records) has been started.

func (*Uploader) DoNext

func (up *Uploader) DoNext(tx etx.TxId)

DoNext executes the parent's operation specified by etx.SetNext, when all uploaded images have been saved.

func (*Uploader) ForOperation

func (up *Uploader) ForOperation(opType int) etx.Op

func (*Uploader) Initialise

func (up *Uploader) Initialise(log *log.Logger, db DB, tm *etx.TM)

Initialise starts the file uploader.

func (*Uploader) MediaType

func (up *Uploader) MediaType(name string) int

MediaType returns the media type. It is 0 if not accepted.

func (*Uploader) Name

func (up *Uploader) Name() string

func (*Uploader) Operation

func (up *Uploader) Operation(id etx.TxId, opType int, op etx.Op)

func (*Uploader) Save

func (up *Uploader) Save(fh *multipart.FileHeader, tx etx.TxId) (err error, byClient bool)

Save decodes an uploaded file, and schedules it to be saved in the filesystem.

func (*Uploader) StartBind

func (up *Uploader) StartBind(parentId int64, tx etx.TxId) *Bind

StartBind initiates linking a parent object to a set of uploaded files, returning a context for calls to Bind and EndBind. It loads updated file versions. tx is 0 when we are deleting a parent object.

func (*Uploader) Stop

func (up *Uploader) Stop()

Stop shuts down the uploader.

func (*Uploader) ValidCode

func (up *Uploader) ValidCode(tx etx.TxId) bool

ValidCode returns false if the transaction code for a set of uploads has expired.