medialibrary

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 23, 2026 License: MIT Imports: 30 Imported by: 0

README

go-medialibrary

tests Go Reference

Attach files to the models of your SaaS application: user avatars, product photos, invoice PDFs. go-medialibrary validates uploads by their real content type, stores them on a local directory or any S3-compatible bucket (AWS S3, Cloudflare R2, MinIO, SeaweedFS), records them in PostgreSQL, and generates image conversions such as thumbnails, synchronously or through your own job queue. Writes are ordered so a database record never points at a missing file.

Install

go get github.com/byteminus/go-medialibrary

Requires Go 1.26 or newer.

Quick start

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	"github.com/jackc/pgx/v5/pgxpool"

	medialibrary "github.com/byteminus/go-medialibrary"
	"github.com/byteminus/go-medialibrary/localdisk"
	"github.com/byteminus/go-medialibrary/mediahttp"
	"github.com/byteminus/go-medialibrary/pgxstore"
)

func main() {
	ctx := context.Background()
	pool, err := pgxpool.New(ctx, "postgres://localhost/app")
	if err != nil {
		log.Fatal(err)
	}
	store, err := pgxstore.New(pool)
	if err != nil {
		log.Fatal(err)
	}
	if err := store.CreateSchema(ctx); err != nil {
		log.Fatal(err)
	}
	disk, err := localdisk.New("/var/lib/app/media", localdisk.Options{BaseURL: "/media"})
	if err != nil {
		log.Fatal(err)
	}

	lib, err := medialibrary.New(medialibrary.Config{
		Store: store,
		Disks: map[string]medialibrary.Disk{"local": disk},
		Collections: []medialibrary.Collection{
			{
				OwnerType:  "user",
				Name:       "avatar",
				AcceptMIME: []string{"image/jpeg", "image/png", "image/webp"},
				MaxSize:    5 << 20,
				Single:     true,
				Conversions: []medialibrary.Conversion{
					{Name: "thumb", Width: 128, Height: 128, Fit: medialibrary.FitCrop, Format: medialibrary.FormatJPEG, Quality: 85},
				},
			},
			{OwnerType: "invoice", Name: "documents", AcceptMIME: []string{"application/pdf"}, MaxFiles: 10},
		},
	})
	if err != nil {
		log.Fatal(err)
	}

	uploader := &mediahttp.Uploader{Library: lib, MaxRequestSize: 6 << 20}
	http.HandleFunc("POST /users/{id}/avatar", func(w http.ResponseWriter, r *http.Request) {
		m, err := uploader.Upload(w, r, medialibrary.Ref{Type: "user", ID: r.PathValue("id")}, "avatar")
		if err != nil {
			http.Error(w, err.Error(), mediahttp.StatusCode(err))
			return
		}
		thumb, err := lib.URL(m, "thumb")
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}
		fmt.Fprintln(w, thumb)
	})
	http.Handle("GET /media/", http.StripPrefix("/media", disk.Handler(localdisk.HandlerOptions{})))
	srv := &http.Server{Addr: ":8080", ReadHeaderTimeout: 10 * time.Second}
	log.Fatal(srv.ListenAndServe())
}

Concepts

Media and owners. A Media record belongs to an owner, a Ref of model type and ID (Ref{Type: "user", ID: "42"}), and to a named collection. It holds a UUID v7 ID, a display name, the sanitized file name, the content type detected from the bytes, size, disk and path, SHA-256 checksum, custom JSON properties, the state of each conversion, its position in the collection, and timestamps.

Collections are declared in code per owner type: accepted content types ("image/*" wildcards allowed), maximum size, Single (adding replaces the existing file atomically), MaxFiles, the disk to use and the conversions to run. Limits are enforced by the store under a per-collection lock, so they hold under concurrent uploads.

Validation. The content type comes from http.DetectContentType (or your own Config.DetectContentType) applied to the first 512 bytes, never from the client's file name or header. SVG and HTML are detected as XML, HTML or text, so an image/* collection rejects them. File names are reduced to letters, digits, ., - and _ (SanitizeFileName), and the extension is made to agree with the detected type: a PNG uploaded as x.html is stored as x.png, text uploaded as notes.svg as notes.txt, so servers that pick types from extensions cannot be tricked either. Uploads are spooled to a temporary file while the size limit is enforced (reading stops one byte past it) and the checksum computed, so nothing reaches a disk unvalidated; the temporary file is removed on every path.

Listing. lib.List(ctx, owner, medialibrary.ListOptions{Collection: "photos", Limit: 20, After: cursor}) returns one page and the cursor of the next (nil on the last page). Pagination is keyset-based on collection, position and ID; Limit 0 means DefaultPageSize (50) and values above MaxPageSize (500) are capped.

Serving. Public files of a local disk can be served with localdisk.Disk.Handler. Private media belong on a disk without a public URL and are reached either through lib.SignedURL (S3 presigned URLs: signed, expiring after at most seven days, unguessable) or through mediahttp.Server, which streams a record's file after your own authorization check. Both servers send the recorded content type with X-Content-Type-Options: nosniff and a sandboxing Content-Security-Policy, and serve everything medialibrary.InlineSafe rejects, HTML, SVG and XML included, as Content-Disposition: attachment with RFC 6266 file names.

Disks implement Disk: Put (atomic, with a declared size), Open, Delete (idempotent), Exists, URL and Walk; SignedURLer adds time-limited URLs.

  • localdisk stores below a root directory through os.Root, so no path, .. or symbolic link can escape it, and writes via a temporary file and rename. Disk.Handler serves the files with the content type detected from the content, X-Content-Type-Options: nosniff, a sandboxing Content-Security-Policy, Cache-Control, ETag and Last-Modified validators, and range requests; types that browsers could execute are sent as attachments.
  • s3disk uses the AWS SDK for Go v2 and any S3-compatible service: set BaseEndpoint (and UsePathStyle for MinIO or SeaweedFS) on the s3.Options you build the client with. It supports a key prefix, a public URL base and presigned GET URLs. Files above PartSize (16 MiB by default) are sent as multipart uploads, which are aborted on any error or cancellation; add a bucket lifecycle rule that aborts incomplete multipart uploads for processes that die mid-upload. Every request is bounded by RequestTimeout (two minutes by default). Objects are stored with their content type and a Content-Disposition that is attachment for types browsers could execute, so public and presigned URLs never render an uploaded HTML page. For services that reject the SDK's default request checksums, set RequestChecksumCalculation: aws.RequestChecksumCalculationWhenRequired on the client.

Conversions resize JPEG, PNG, GIF (first frame) and WebP images with golang.org/x/image/draw (Catmull-Rom) and encode JPEG (with a quality setting) or PNG:

Fit Result
FitContain Scaled down to fit inside the box, aspect ratio kept, never enlarged. Width or height may be zero.
FitFill Exactly width x height: scaled to fit, the rest padded with Background (transparent for PNG, white for JPEG by default).
FitCrop Exactly width x height: scaled to cover the box, the overflow cropped around the centre.

Image dimensions are read from the header first, and images above Config.MaxPixels (default 50 megapixels) are refused before any pixel buffer is allocated, which defuses decompression bombs. The header also gives the memory the decoder will hold, by format, colour model, bit depth and progressive coding (a progressive JPEG keeps a 4-byte coefficient per sample, a 16-bit PNG 8 bytes per pixel), and images above Config.MaxDecodeMemory (default 256 MiB) are refused too; at most Config.MaxConcurrentDecodes (default 2) images are decoded at once. The header and the pixels are read from one stream, so the size checked is the size decoded, and New refuses conversions whose box exceeds MaxPixels and conversion names that differ only in case. A failing conversion is recorded as failed with its error and does not fail the upload.

By default conversions run inside Add. Configure a Queue to run them in your own job system: Add records them as pending and enqueues a ConversionJob (plain data, easy to serialize); your worker calls lib.RunConversions(ctx, job). RegenerateConversions re-runs some or all conversions, for example after you change their settings, and drops results of conversions you removed. During a regeneration the previous file keeps being served.

Ordering of database and disk writes. Files are written before the record that references them is inserted or updated, and deleted only after the record stops referencing them:

  • Add: write original and conversions, then insert the record. If the insert fails, the files are deleted again.
  • Delete, replacement in a Single collection, changed or removed conversions: update the database first, delete the files afterwards.

So a failure (a crash, a lost connection, a rolled-back transaction) can leave an unreferenced file, but never a record pointing at a missing file. The trade-off is wasted disk work on failed inserts and orphans that need a sweep: run lib.CleanOrphans(ctx, medialibrary.CleanOptions{}) periodically. It walks the library's paths (<prefix>/<media id>/<file> and <prefix>/<media id>/conversions/<name>-<conversion>-<run token><ext>) and deletes files no record references, re-reading the record just before each deletion. Every conversion run writes new paths, so a retried run never rewrites a file the sweep has judged, and a record on any disk keeps all files of its ID on the other disks, which may be aliases of one storage. Files younger than Config.OrphanGrace (one hour by default), by modification time, by the time in their media ID or by the time in their run token, are left alone, and Add and the conversion runs abandon their work with ErrWriteWindowExceeded when writing takes longer than half of that period, so the files of an upload in flight are never deleted. Use the same OrphanGrace in every process, and commit caller transactions within half of it. Give the library its own PathPrefix or disk.

PostgreSQL

pgxstore keeps records in one table, media by default (pgxstore.WithTable("app.media") for another name or schema). CreateSchema runs this idempotent DDL; copy it into your migrations if you prefer:

CREATE TABLE IF NOT EXISTS "media" (
	id uuid PRIMARY KEY CHECK (id <> '00000000-0000-0000-0000-000000000000'),
	owner_type text NOT NULL CHECK (owner_type <> ''),
	owner_id text NOT NULL CHECK (owner_id <> ''),
	collection text NOT NULL CHECK (collection <> ''),
	name text NOT NULL,
	file_name text NOT NULL CHECK (file_name <> ''),
	mime_type text NOT NULL CHECK (mime_type <> ''),
	size bigint NOT NULL CHECK (size >= 0),
	disk text NOT NULL CHECK (disk <> ''),
	path text NOT NULL CHECK (path <> ''),
	checksum text NOT NULL CHECK (checksum ~ '^[0-9a-f]{64}$'),
	properties jsonb NOT NULL DEFAULT '{}' CHECK (jsonb_typeof(properties) = 'object'),
	conversions jsonb NOT NULL DEFAULT '{}' CHECK (jsonb_typeof(conversions) = 'object'),
	sort_order integer NOT NULL CHECK (sort_order >= 1),
	created_at timestamptz NOT NULL,
	updated_at timestamptz NOT NULL,
	UNIQUE (disk, path),
	UNIQUE (owner_type, owner_id, collection, sort_order) DEFERRABLE INITIALLY IMMEDIATE
);

Inserts and reorders take a transaction-scoped advisory lock per owner and collection, so MaxFiles, Single and ordering hold across processes: of two concurrent adds into a collection with one free slot, the second waits for the first and then fails with ErrCollectionFull. This relies on READ COMMITTED, PostgreSQL's default; in a stricter caller transaction the unique order constraint still rejects the second add, with ErrConflict. Collections are listed in byte order (COLLATE "C") whatever the database locale.

Joining your transaction. WithConn resolves the connection per call; return the pgx.Tx your request carries and media writes commit or roll back with your own. The store's Querier includes Begin, which on a pgx.Tx opens a savepoint, so a failed media write does not abort your transaction. Because files must not be deleted before your commit, also set Config.AfterCommit:

type txKey struct{}

type txState struct {
	tx    pgx.Tx
	after []func(context.Context)
}

store, err := pgxstore.New(pool, pgxstore.WithConn(func(ctx context.Context) pgxstore.Querier {
	if st, ok := ctx.Value(txKey{}).(*txState); ok {
		return st.tx
	}
	return nil // use the pool
}))

lib, err := medialibrary.New(medialibrary.Config{
	Store: store,
	// ...
	AfterCommit: func(ctx context.Context, fn func(context.Context)) {
		if st, ok := ctx.Value(txKey{}).(*txState); ok {
			st.after = append(st.after, fn) // run these after tx.Commit succeeds
			return
		}
		fn(ctx)
	},
})

A Queue implementation receives the same context in Enqueue, so it can insert its job in the same transaction.

API overview

Package Contents
medialibrary New(Config); Library methods Add, Get, List, Delete, Reorder, UpdateProperties, RegenerateConversions, RunConversions, CleanOrphans, URL, SignedURL, Open; types Media, Ref, Collection, Conversion, ConversionResult, Fit, Format, ListOptions, Cursor, Queue, ConversionJob, Store, Disk, SignedURLer; helpers SanitizeFileName, IsConvertible, ValidatePath, InlineSafe, ContentDisposition; the family sentinels ErrNotFound, ErrInvalid, ErrConflict and specific errors wrapping them, such as ErrMimeTypeNotAllowed, ErrFileTooLarge, ErrCollectionFull.
memstore In-memory Store for tests and examples (not transactional).
pgxstore PostgreSQL Store: New, WithTable, WithConn, Schema, CreateSchema.
storetest Run(t, newStore): conformance suite for any Store.
localdisk Local directory Disk and its file server, Disk.Handler.
s3disk S3-compatible Disk with multipart uploads, request timeouts and presigned URLs.
disktest Run(t, newDisk): conformance suite for any Disk.
mediahttp Uploader for streaming multipart uploads with a request size limit; Server for serving media with safe headers; StatusCode maps errors to HTTP statuses.

All errors are sentinels, matched with errors.Is; messages start with the package name. Every error for missing, invalid or conflicting input wraps medialibrary.ErrNotFound, ErrInvalid or ErrConflict, like the other Byte Minus packages. The package reads no actor or tenant from the context, so there are no context helpers to wire.

Testing

go test -race ./...

PostgreSQL tests run when TEST_DATABASE_URL is set and are skipped otherwise (in CI, where CI is set, they fail instead):

TEST_DATABASE_URL=postgres://postgres:postgres@localhost:5432/medialibrary?sslmode=disable go test -race ./...

s3disk is unit-tested against an in-process fake of the S3 API. Its integration tests run against a real S3-compatible endpoint configured with S3_TEST_ENDPOINT, S3_TEST_BUCKET, S3_TEST_ACCESS_KEY_ID, S3_TEST_SECRET_ACCESS_KEY and optionally S3_TEST_REGION and S3_TEST_PATH_STYLE=1. CI runs them against both S3Mock and SeaweedFS (started from testdata/seaweedfs/s3.json); signature and expiry checks need a server that verifies signatures, such as SeaweedFS:

docker run -d -p 8333:8333 -v "$PWD/testdata/seaweedfs:/etc/seaweedfs:ro" chrislusf/seaweedfs:4.48 \
  server -s3 -s3.config=/etc/seaweedfs/s3.json -dir=/data -volume.max=100 -master.volumeSizeLimitMB=64
S3_TEST_ENDPOINT=http://localhost:8333 S3_TEST_BUCKET=medialibrary-test S3_TEST_ACCESS_KEY_ID=byteminus \
  S3_TEST_SECRET_ACCESS_KEY=byteminus-secret S3_TEST_PATH_STYLE=1 go test -race ./s3disk

Use storetest.Run and disktest.Run to check your own implementations.

Roadmap

  • Responsive image sets (srcset) and output formats beyond JPEG and PNG (WebP and AVIF encoding).
  • EXIF orientation handling and metadata stripping for JPEG originals.
  • Conversions for non-image files (PDF and video previews).
  • Uploading several files from one multipart request, and resumable or direct-to-S3 (presigned PUT) uploads.
  • Renaming a media item's display name, and moving media between owners, collections and disks.
  • A file server for S3 disks that proxies private objects.
  • Integrity verification that re-hashes stored files against their checksum.

Stability

This package follows semantic versioning. Before v1.0.0, a minor release may contain breaking changes; each one is listed in CHANGELOG.md with upgrade steps, including SQL for schema changes. From v1.0.0, the exported API and the PostgreSQL schema change only in backwards-compatible ways until the next major version.

License

MIT, see LICENSE.

Documentation

Overview

Package medialibrary attaches files to the models of a SaaS application: avatars of users, photos of products, documents of invoices. It validates uploads, stores them on a local or S3-compatible disk, records them in a database and generates image conversions such as thumbnails.

Concepts

A media item (Media) belongs to an owner, identified by a Ref of model type and ID, and to a named Collection of that owner type. Collections are declared in code: which content types they accept, how large files may be, whether they hold a single file (adding replaces it) or a bounded number, which disk they use, and which Conversions run for images.

A Library ties together a Store (records; see the memstore and pgxstore packages), one or more Disks (content; see localdisk and s3disk) and the collections: 

lib, err := medialibrary.New(medialibrary.Config{
	Store: store,
	Disks: map[string]medialibrary.Disk{"local": disk},
	Collections: []medialibrary.Collection{{
		OwnerType:  "user",
		Name:       "avatar",
		AcceptMIME: []string{"image/*"},
		MaxSize:    5 << 20,
		Single:     true,
		Conversions: []medialibrary.Conversion{
			{Name: "thumb", Width: 128, Height: 128, Fit: medialibrary.FitCrop},
		},
	}},
})
m, err := lib.Add(ctx, medialibrary.Ref{Type: "user", ID: "42"}, "avatar", file, "me.jpg")

List returns an owner's media one page at a time (keyset pagination, ListOptions.Limit up to MaxPageSize) together with the cursor of the next page.

Safety

The content type is detected from the file's bytes and never taken from the client, and the stored file name's extension is corrected to match it, so a PNG named "x.html" is stored as "x.png". File names are sanitized (SanitizeFileName) and every file of a media item lives under a directory named by its UUID. Uploads are spooled to a temporary file while their size is limited and their SHA-256 computed, so nothing reaches a disk before it has been validated. Images are measured from their header before being decoded, and refused above Config.MaxPixels, which defuses decompression bombs; the memory each decoder would hold is estimated from the header and capped by Config.MaxDecodeMemory, and conversion sizes are bounded by the pixel limit. HTML, SVG and XML are never served inline by the bundled servers (see InlineSafe, mediahttp.Server and localdisk.Disk.Handler), and private media are reached through signed, expiring URLs or through mediahttp.Server after the application's own authorization.

Consistency between database and disk

Writes are ordered so a failure never leaves a record pointing at a missing file: files are written before the record that references them is inserted or updated, and deleted only after the record stops referencing them. The price is that failures can leave unreferenced files behind; Library.CleanOrphans finds and removes them and should run periodically. It only deletes files older than Config.OrphanGrace, and writers give up when their file writes take longer than half of it, so the files of an upload in flight are never mistaken for orphans. When records are written inside a caller's transaction, set Config.AfterCommit so file deletion waits for the commit.

Errors

Errors wrap one of ErrNotFound, ErrInvalid and ErrConflict, shared by every Byte Minus package, as well as a specific error such as ErrCollectionFull; match either with errors.Is.

Conversions

Conversions resize JPEG, PNG, GIF (first frame) and WebP images to JPEG or PNG, fitting, padding or cropping them to a box. They run during Add by default. With a Queue configured they are recorded as pending and handed to the application's job system, whose worker calls RunConversions. The status of each conversion is kept on the record.

All types in this package are safe for concurrent use once constructed.

Example 
package main

import (
	"bytes"
	"context"
	"fmt"
	"image"
	"image/png"
	"log"
	"os"
	"strings"

	medialibrary "github.com/byteminus/go-medialibrary"
	"github.com/byteminus/go-medialibrary/localdisk"
	"github.com/byteminus/go-medialibrary/memstore"
)

// samplePNG returns a w x h PNG standing in for an uploaded photo.
func samplePNG(w, h int) []byte {
	var buf bytes.Buffer
	if err := png.Encode(&buf, image.NewRGBA(image.Rect(0, 0, w, h))); err != nil {
		log.Fatal(err)
	}
	return buf.Bytes()
}

func main() {
	ctx := context.Background()
	dir, err := os.MkdirTemp("", "media-example")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()

	disk, err := localdisk.New(dir, localdisk.Options{BaseURL: "https://example.com/files"})
	if err != nil {
		log.Fatal(err)
	}
	lib, err := medialibrary.New(medialibrary.Config{
		Store: memstore.New(),
		Disks: map[string]medialibrary.Disk{"local": disk},
		Collections: []medialibrary.Collection{{
			OwnerType:  "user",
			Name:       "avatar",
			AcceptMIME: []string{"image/*"},
			MaxSize:    5 << 20,
			Single:     true,
			Conversions: []medialibrary.Conversion{
				{Name: "thumb", Width: 64, Height: 64, Fit: medialibrary.FitCrop, Format: medialibrary.FormatJPEG},
			},
		}},
	})
	if err != nil {
		log.Fatal(err)
	}

	owner := medialibrary.Ref{Type: "user", ID: "42"}
	// The client claims a GIF; the content decides, extension included.
	m, err := lib.Add(ctx, owner, "avatar", bytes.NewReader(samplePNG(300, 200)), "My Avatar.gif")
	if err != nil {
		log.Fatal(err)
	}
	thumb := m.Conversions["thumb"]
	fmt.Println(m.FileName, m.MimeType)
	fmt.Println(thumb.Status, thumb.Width, thumb.Height, thumb.MimeType)

	url, err := lib.URL(m, "thumb")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(strings.HasPrefix(url, "https://example.com/files/"+m.ID.String()+"/conversions/"))

	// Single collections replace their file.
	if _, err := lib.Add(ctx, owner, "avatar", bytes.NewReader(samplePNG(10, 10)), "new.png"); err != nil {
		log.Fatal(err)
	}
	list, next, err := lib.List(ctx, owner, medialibrary.ListOptions{Collection: "avatar"})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(len(list), list[0].FileName, next == nil)
}

Output:
My-Avatar.png image/png
done 64 64 image/jpeg
true
1 new.png true

Index

Examples

Constants

View Source 
const (
	// DefaultPageSize is the page size used when ListOptions.Limit is zero.
	DefaultPageSize = 50
	// MaxPageSize caps ListOptions.Limit.
	MaxPageSize = 500
)

Page sizes of Library.List.

View Source 
const DefaultMaxConcurrentDecodes = 2

DefaultMaxConcurrentDecodes is the number of images decoded at once when Config.MaxConcurrentDecodes is zero. With DefaultMaxDecodeMemory it caps the memory held by decoders at about 512 MiB per Library.

View Source 
const DefaultMaxDecodeMemory = 256 << 20

DefaultMaxDecodeMemory is the decode memory limit used when Config.MaxDecodeMemory is zero: 256 MiB, which admits a 50 megapixel baseline JPEG or 8-bit PNG but refuses the same size as a progressive JPEG with full-resolution colour or as a 16-bit PNG.

View Source 
const DefaultMaxPixels = 50_000_000

DefaultMaxPixels is the pixel limit used when Config.MaxPixels is zero: 50 megapixels, enough for current phone and camera photos.

View Source 
const DefaultMaxSize = 25 << 20

DefaultMaxSize is the upload limit used when neither the collection nor Config.MaxSize sets one: 25 MiB.

View Source 
const DefaultOrphanGrace = time.Hour

DefaultOrphanGrace is the OrphanGrace used when Config.OrphanGrace is zero: CleanOrphans leaves files younger than an hour alone, and writes must reach their record within half an hour.

View Source 
const MaxPropertiesSize = 64 << 10

MaxPropertiesSize is the largest JSON encoding of custom properties that UpdateProperties accepts: 64 KiB. Properties often come from clients, and every read of the record carries them.

Variables

View Source 
var (
	// ErrNotFound means the record or file does not exist.
	ErrNotFound = errors.New("medialibrary: not found")
	// ErrInvalid means the input failed validation.
	ErrInvalid = errors.New("medialibrary: invalid input")
	// ErrConflict means a uniqueness violation, or that the state of a
	// record or collection no longer allows the operation.
	ErrConflict = errors.New("medialibrary: conflict")
)

The three family sentinels. Every other error of this module that describes a missing record, invalid input or a conflicting state wraps one of them, so an application maps the errors of every Byte Minus package to HTTP statuses the same way, while errors.Is still reaches the more specific errors below.

View Source 
var (
	// ErrInvalidConfig means New, or a store or disk constructor, was given
	// an unusable configuration. It wraps ErrInvalid.
	ErrInvalidConfig = fmt.Errorf("medialibrary: invalid config: %w", ErrInvalid)
	// ErrEnqueue means the media was stored but its conversions could not be
	// handed to the Queue; they stay pending until regenerated.
	ErrEnqueue = errors.New("medialibrary: enqueue conversions")
	// ErrInvalidMedia means a record violates a store integrity constraint.
	// It wraps ErrInvalid.
	ErrInvalidMedia = fmt.Errorf("medialibrary: invalid media: %w", ErrInvalid)
	// ErrInvalidOwner means the owner reference is incomplete. It wraps
	// ErrInvalid.
	ErrInvalidOwner = fmt.Errorf("medialibrary: invalid owner: %w", ErrInvalid)
	// ErrUnknownCollection means no collection with that name is defined for
	// the owner type. It wraps ErrInvalid.
	ErrUnknownCollection = fmt.Errorf("medialibrary: unknown collection: %w", ErrInvalid)
	// ErrUnknownConversion means the collection defines no conversion with
	// that name. It wraps ErrInvalid.
	ErrUnknownConversion = fmt.Errorf("medialibrary: unknown conversion: %w", ErrInvalid)
	// ErrUnknownDisk means a collection or record names a disk that is not
	// configured. It wraps ErrInvalid.
	ErrUnknownDisk = fmt.Errorf("medialibrary: unknown disk: %w", ErrInvalid)
	// ErrInvalidFileName means the file name is empty after sanitizing. It
	// wraps ErrInvalid.
	ErrInvalidFileName = fmt.Errorf("medialibrary: invalid file name: %w", ErrInvalid)
	// ErrEmptyFile means the upload contained no bytes. It wraps ErrInvalid.
	ErrEmptyFile = fmt.Errorf("medialibrary: empty file: %w", ErrInvalid)
	// ErrFileTooLarge means the upload exceeds the collection's MaxSize. It
	// wraps ErrInvalid.
	ErrFileTooLarge = fmt.Errorf("medialibrary: file too large: %w", ErrInvalid)
	// ErrMimeTypeNotAllowed means the detected content type is not accepted
	// by the collection. It wraps ErrInvalid.
	ErrMimeTypeNotAllowed = fmt.Errorf("medialibrary: mime type not allowed: %w", ErrInvalid)
	// ErrCollectionFull means the collection already holds MaxFiles items.
	// It wraps ErrConflict.
	ErrCollectionFull = fmt.Errorf("medialibrary: collection is full: %w", ErrConflict)
	// ErrInvalidOrder means a reorder request does not list every item of the
	// collection exactly once. It wraps ErrInvalid.
	ErrInvalidOrder = fmt.Errorf("medialibrary: invalid order: %w", ErrInvalid)
	// ErrImageTooLarge means an image exceeds the configured pixel limit and
	// was not decoded. It wraps ErrInvalid.
	ErrImageTooLarge = fmt.Errorf("medialibrary: image exceeds pixel limit: %w", ErrInvalid)
	// ErrUnsupportedImage means the content cannot be decoded for
	// conversions. It wraps ErrInvalid.
	ErrUnsupportedImage = fmt.Errorf("medialibrary: unsupported image format: %w", ErrInvalid)
	// ErrConversionNotReady means the conversion has not produced a file
	// yet. It wraps ErrNotFound.
	ErrConversionNotReady = fmt.Errorf("medialibrary: conversion not ready: %w", ErrNotFound)
	// ErrFileNotFound means a disk has no file at the path. It wraps
	// ErrNotFound.
	ErrFileNotFound = fmt.Errorf("medialibrary: file not found: %w", ErrNotFound)
	// ErrInvalidPath means a disk path is empty, absolute or escapes the
	// disk. It wraps ErrInvalid.
	ErrInvalidPath = fmt.Errorf("medialibrary: invalid path: %w", ErrInvalid)
	// ErrNoPublicURL means the disk has no public URL configured. It wraps
	// ErrInvalid.
	ErrNoPublicURL = fmt.Errorf("medialibrary: disk has no public url: %w", ErrInvalid)
	// ErrSignedURLUnsupported means the disk cannot issue signed URLs. It
	// wraps ErrInvalid.
	ErrSignedURLUnsupported = fmt.Errorf("medialibrary: disk does not support signed urls: %w", ErrInvalid)
	// ErrSizeMismatch means a disk received a different number of bytes than
	// the caller declared. It wraps ErrInvalid.
	ErrSizeMismatch = fmt.Errorf("medialibrary: size mismatch: %w", ErrInvalid)
	// ErrWriteWindowExceeded means writing the files of an upload or of a
	// conversion run took longer than half of Config.OrphanGrace. The
	// operation is abandoned before its record is written, because a record
	// committed later could reference files CleanOrphans already considers
	// orphans. It wraps ErrConflict.
	ErrWriteWindowExceeded = fmt.Errorf("medialibrary: file writes exceeded the orphan grace window: %w", ErrConflict)
)

Errors returned by the library, its stores and its disks. Match them with errors.Is, since most are wrapped with detail.

Functions

func ContentDisposition added in v0.2.0 

func ContentDisposition(disposition, fileName string) string

ContentDisposition returns a Content-Disposition header value of the given disposition type ("inline" or "attachment"; anything else is treated as "attachment") naming fileName, following RFC 6266. An ASCII fallback goes in the filename parameter, and names with other characters are also given in UTF-8 in the filename* parameter (RFC 8187). Control characters, quotes and backslashes never reach the header, so a file name cannot inject header fields or parameters. An empty name yields the bare disposition type.

func InlineSafe added in v0.2.0 

func InlineSafe(mimeType string) bool

InlineSafe reports whether files of the given content type (parameters such as charset are ignored) can be displayed in the browser without letting them run script on the serving origin. Everything else should be served with "Content-Disposition: attachment". The localdisk, s3disk and mediahttp packages use it as their default.

func IsConvertible 

func IsConvertible(mimeType string) bool

IsConvertible reports whether conversions can be generated for files of the given content type: JPEG, PNG, GIF (first frame) and WebP.

func SanitizeFileName 

func SanitizeFileName(name string) string

SanitizeFileName turns a client-supplied file name into one that is safe to use as a storage path component and in a URL. It drops any directory part (with either slash), keeps letters (with combining marks), digits, '.', '-' and '_', replaces runs of anything else with a single '-', strips leading and trailing dots and dashes so the result is never hidden, option-like or a "." or ".." path segment, and truncates names longer than 200 bytes while keeping the extension. The result is empty when nothing usable remains. Add additionally replaces an extension that does not match the detected content type.

Example 
package main

import (
	"fmt"

	medialibrary "github.com/byteminus/go-medialibrary"
)

func main() {
	fmt.Println(medialibrary.SanitizeFileName("../../etc/passwd"))
	fmt.Println(medialibrary.SanitizeFileName(`C:\Users\me\Quarterly report (final).pdf`))
	fmt.Println(medialibrary.SanitizeFileName(".env"))
}

Output:
passwd
Quarterly-report-final-.pdf
env

func ValidatePath 

func ValidatePath(p string) error

ValidatePath reports whether p is a well-formed disk path: relative, slash-separated, without empty, "." or ".." segments, backslashes or NUL bytes. Disk implementations call it before touching storage; errors wrap ErrInvalidPath.

Types

type CleanOptions 

type CleanOptions struct {
	// Disks limits the sweep to the named disks; empty sweeps all.
	Disks []string
	// DryRun reports orphans without deleting them.
	DryRun bool
}

CleanOptions controls CleanOrphans. The zero value sweeps every disk.

type CleanReport 

type CleanReport struct {
	// Scanned counts the files that follow the library's layout.
	Scanned int
	// Orphans lists the unreferenced files found, deleted unless DryRun.
	Orphans []OrphanFile
}

CleanReport summarizes a CleanOrphans run.

type Collection 

type Collection struct {
	// OwnerType is the Ref.Type this collection belongs to.
	OwnerType string
	// Name is unique per owner type.
	Name string
	// AcceptMIME lists the accepted content types, matched against the type
	// detected from the file content. An entry may end in "/*" to accept a
	// whole family, such as "image/*". Empty accepts any type.
	AcceptMIME []string
	// MaxSize is the largest accepted file in bytes; zero uses
	// Config.MaxSize.
	MaxSize int64
	// Single makes the collection hold at most one file: adding a file
	// replaces the existing one atomically.
	Single bool
	// MaxFiles caps the number of files; zero means unlimited. It is ignored
	// when Single is set.
	MaxFiles int
	// Disk names the disk files are stored on; empty uses Config.DefaultDisk.
	Disk string
	// Conversions run for every image added to the collection.
	Conversions []Conversion
}

Collection defines a named group of files for one owner type, such as the "avatar" collection of "user". Collections are declared in code and passed to New; they are never stored.

type Config 

type Config struct {
	// Store keeps the media records.
	Store Store
	// Disks maps disk names to implementations.
	Disks map[string]Disk
	// DefaultDisk is used by collections that name no disk. It may be empty
	// when exactly one disk is configured.
	DefaultDisk string
	// Collections declares every collection of every owner type.
	Collections []Collection
	// MaxSize is the default upload limit in bytes; zero means
	// DefaultMaxSize.
	MaxSize int64
	// MaxPixels refuses to decode images with more pixels than this, and
	// bounds the output size of every conversion; zero means
	// DefaultMaxPixels.
	MaxPixels int64
	// MaxDecodeMemory refuses to decode images whose decoder would allocate
	// more than this many bytes, estimated from the header by format, colour
	// model, bit depth and, for JPEG, progressive coding; zero means
	// DefaultMaxDecodeMemory. Conversion output adds up to 8 bytes per pixel
	// of the largest conversion box.
	MaxDecodeMemory int64
	// MaxConcurrentDecodes caps how many images the Library decodes at once;
	// further conversions wait. Zero means DefaultMaxConcurrentDecodes.
	MaxConcurrentDecodes int
	// PathPrefix is prepended to every storage path, such as "media". Give
	// the library a prefix or a disk of its own: CleanOrphans deletes
	// unreferenced files under it that follow the library's path layout.
	PathPrefix string
	// Queue runs conversions asynchronously; nil runs them inside Add.
	Queue Queue
	// TempDir holds uploads while they are validated and hashed; empty
	// means os.TempDir().
	TempDir string
	// DetectContentType returns the content type of a file from its first
	// 512 bytes; nil means http.DetectContentType. Parameters such as
	// charset are dropped from the result.
	DetectContentType func(head []byte) string
	// OrphanGrace is how old an unreferenced file must be before
	// CleanOrphans deletes it; zero means DefaultOrphanGrace. Add,
	// RegenerateConversions and RunConversions abandon their work with
	// ErrWriteWindowExceeded when writing files takes longer than half of
	// it, so a file younger than OrphanGrace may still be about to be
	// referenced, and an older one never is. Every process that writes or
	// cleans the same disks must use the same value, and a caller
	// transaction joined through the store must commit within half of it.
	OrphanGrace time.Duration
	// AfterCommit schedules work that must only happen once the caller's
	// database transaction has committed: deleting the files of removed or
	// replaced media. Nil runs fn immediately, which is right unless the
	// store joins a caller transaction (pgxstore WithConn). In that case
	// pass a function that runs fn after commit, or a rollback could leave
	// records pointing at deleted files.
	AfterCommit func(ctx context.Context, fn func(ctx context.Context))
	// Logger receives cleanup failures that do not fail the operation, such
	// as an unreachable disk while removing a replaced file. Nil discards.
	Logger *slog.Logger
	// Now returns the current time; nil means time.Now.
	Now func() time.Time
}

Config configures a Library. Store and Disks are required; every other field has a working zero value.

type Conversion 

type Conversion struct {
	// Name is unique within the collection and part of the file path.
	Name   string
	Width  int
	Height int
	Fit    Fit
	Format Format
	// Quality is the JPEG quality from 1 to 100; zero means 85. It is
	// ignored for PNG.
	Quality int
	// Background fills padding (FitFill) and replaces transparency when
	// encoding JPEG. Nil means transparent for PNG and white for JPEG.
	Background color.Color
}

Conversion is a named derived image, such as a "thumb" of 200 x 200 pixels, generated from every image added to a collection.

type ConversionJob 

type ConversionJob struct {
	MediaID uuid.UUID `json:"media_id"`
	// Conversions names the conversions to run; empty means all conversions
	// of the media's collection.
	Conversions []string `json:"conversions,omitempty"`
}

ConversionJob asks a worker to generate conversions for one media item. It is plain data so it can be serialized into any job system.

type ConversionResult 

type ConversionResult struct {
	Status    ConversionStatus `json:"status"`
	Path      string           `json:"path,omitempty"`
	MimeType  string           `json:"mime_type,omitempty"`
	Size      int64            `json:"size,omitempty"`
	Width     int              `json:"width,omitempty"`
	Height    int              `json:"height,omitempty"`
	Error     string           `json:"error,omitempty"`
	UpdatedAt time.Time        `json:"updated_at"`
}

ConversionResult records the outcome of one named conversion of a media item. Path is set once a file has been written; while a regeneration is pending it keeps pointing at the previous file so URLs stay valid.

type ConversionStatus 

type ConversionStatus string

ConversionStatus is the lifecycle state of one generated conversion. 

const (
	ConversionPending ConversionStatus = "pending"
	ConversionDone    ConversionStatus = "done"
	ConversionFailed  ConversionStatus = "failed"
)

Conversion statuses. A conversion is pending from the moment it is scheduled until a worker finishes it, then done or failed.

type CreateOptions 

type CreateOptions struct {
	// Replace deletes the collection's existing items in the same step.
	Replace bool
	// MaxFiles caps the number of items in the collection; zero means
	// unlimited.
	MaxFiles int
}

CreateOptions carries the collection rules a Store enforces atomically.

type Cursor added in v0.2.0 

type Cursor struct {
	Collection string    `json:"collection"`
	Order      int       `json:"order"`
	ID         uuid.UUID `json:"id"`
}

Cursor is a keyset position in the order List returns media: collection, then Order, then ID. The triple is unique, so pages neither repeat nor skip items while others are added or deleted. A Reorder between two pages moves items between positions, and the next page follows the new order.

type Disk 

type Disk interface {
	// Put stores size bytes read from r at path, replacing any existing file.
	// If r yields a different number of bytes, Put fails with an error
	// wrapping ErrSizeMismatch and leaves no file behind.
	Put(ctx context.Context, path string, r io.Reader, size int64, contentType string) error
	// Open returns the content stored at path.
	Open(ctx context.Context, path string) (io.ReadCloser, error)
	// Delete removes the file at path. Deleting a missing file succeeds, so
	// cleanup can be retried safely.
	Delete(ctx context.Context, path string) error
	// Exists reports whether a file is stored at path.
	Exists(ctx context.Context, path string) (bool, error)
	// URL returns the public URL of path, or an error wrapping
	// ErrNoPublicURL when the disk is not publicly served.
	URL(path string) (string, error)
	// Walk calls fn for every file whose path starts with prefix, in no
	// particular order. Returning an error from fn stops the walk and Walk
	// returns that error.
	Walk(ctx context.Context, prefix string, fn func(FileInfo) error) error
}

Disk stores file content under slash-separated relative paths such as "media/0192.../avatar.png". Implementations must be safe for concurrent use, must make Put atomic (readers see either the old content or the new content, never a partial file), and must report a missing file from Open as an error wrapping ErrFileNotFound.

type FileInfo 

type FileInfo struct {
	Path    string
	Size    int64
	ModTime time.Time
}

FileInfo describes one file found by Disk.Walk.

type Fit 

type Fit int

Fit selects how a conversion maps the source image onto Width x Height. 

const (
	// FitContain scales the image down to fit within the box, keeping its
	// aspect ratio. It never enlarges, and a zero Width or Height leaves
	// that dimension unconstrained.
	FitContain Fit = iota
	// FitFill scales the image to fit within the box, keeping its aspect
	// ratio, and pads the rest of the box with Background, so the output is
	// exactly Width x Height.
	FitFill
	// FitCrop scales the image to cover the box, keeping its aspect ratio,
	// and crops the overflow around the center, so the output is exactly
	// Width x Height.
	FitCrop
)

Fit modes.

func (Fit) String 

func (f Fit) String() string

String returns the lowercase name of the fit mode.

type Format 

type Format int

Format is the encoding of a conversion's output. 

const (
	// FormatJPEG encodes as baseline JPEG; transparency is flattened onto
	// the conversion's Background, or white when it is nil.
	FormatJPEG Format = iota
	// FormatPNG encodes as PNG and keeps transparency.
	FormatPNG
)

Output formats.

func (Format) String 

func (f Format) String() string

String returns the lowercase name of the format.

type Library 

type Library struct {
	// contains filtered or unexported fields
}

Library is the entry point for attaching files to models. It is safe for concurrent use; its configuration is fixed at construction.

func New 

func New(cfg Config) (*Library, error)

New validates cfg and returns a Library. Configuration errors wrap ErrInvalidConfig. The Collections slice and the Disks map are copied, so later changes to cfg do not affect the Library.

func (*Library) Add 

func (l *Library) Add(ctx context.Context, owner Ref, collection string, r io.Reader, fileName string) (*Media, error)

Add stores the content of r as a new media item of owner's collection.

The content type is detected from the bytes, never taken from the client, and checked against the collection before the rest of the upload is read. The file name's extension is made to agree with that type (see SanitizeFileName and Media.FileName), so no server that infers types from extensions can be tricked into serving the file as something else. The upload is spooled to a temporary file while its size is limited and its SHA-256 computed, so nothing reaches a disk until it has been fully validated; reading stops one byte past the limit, and the temporary file is removed on every path.

Ordering: the original file and any synchronous conversions are written to the disk first and the record is inserted last. If the insert fails (for example ErrCollectionFull under a concurrent upload), the written files are deleted again. A crash between the two steps therefore leaves at worst an unreferenced file, which CleanOrphans removes, and never a record pointing at a missing file. The cost is that a failed insert does the disk work for nothing, and that orphans need a periodic sweep.

Writing the files and the record must finish within half of Config.OrphanGrace, or Add fails with ErrWriteWindowExceeded and removes what it wrote, so CleanOrphans never deletes files of an upload that is still in flight.

For a Single collection the previous item is replaced in the same atomic store operation, and its files are deleted afterwards through Config.AfterCommit.

Conversions of convertible images run before the insert when no Queue is configured; a conversion that fails is recorded as failed and does not fail the upload. With a Queue they are recorded as pending and enqueued after the insert. If enqueuing fails, Add returns the stored media together with an error wrapping ErrEnqueue.

func (*Library) CleanOrphans 

func (l *Library) CleanOrphans(ctx context.Context, opts CleanOptions) (CleanReport, error)

CleanOrphans deletes files that no media record references: leftovers of uploads whose insert never happened, of deletions whose file removal failed, and of conversions whose output path changed. It walks the library's path prefix on each disk and only considers files laid out as "<id>/<file>" or "<id>/conversions/<file>", so unrelated files are left alone. Run it periodically, for example daily from a scheduled job.

Files of committed media are never deleted: a file is an orphan only if the record of its directory, read during the sweep and again just before that file's deletion, does not reference it; and a record on any disk keeps every file of its ID on the other disks, which may be aliases of the same storage. Conversion runs write new paths each time, so no file the sweep has judged can be made current again by a retry. Files of uploads and conversions still in flight are never deleted either: a candidate must be older than Config.OrphanGrace by its modification time, by the creation time encoded in its media ID and, for a conversion, by the start time encoded in its file name, while writers give up after half of that period without committing. An unknown disk name fails with ErrUnknownDisk.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"strings"
	"time"

	medialibrary "github.com/byteminus/go-medialibrary"
	"github.com/byteminus/go-medialibrary/localdisk"
	"github.com/byteminus/go-medialibrary/memstore"
)

func main() {
	ctx := context.Background()
	dir, err := os.MkdirTemp("", "media-example")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()
	disk, err := localdisk.New(dir, localdisk.Options{})
	if err != nil {
		log.Fatal(err)
	}
	store := memstore.New()
	cfg := medialibrary.Config{
		Store:       store,
		Disks:       map[string]medialibrary.Disk{"local": disk},
		PathPrefix:  "media",
		Collections: []medialibrary.Collection{{OwnerType: "user", Name: "files"}},
	}
	lib, err := medialibrary.New(cfg)
	if err != nil {
		log.Fatal(err)
	}
	if _, err := lib.Add(ctx, medialibrary.Ref{Type: "user", ID: "1"}, "files", strings.NewReader("kept"), "kept.txt"); err != nil {
		log.Fatal(err)
	}
	// A crash between writing a file and inserting its record leaves this.
	if err := disk.Put(ctx, "media/0192f0c4-7c00-7000-8000-000000000001/lost.txt", strings.NewReader("lost"), 4, "text/plain"); err != nil {
		log.Fatal(err)
	}

	report, err := lib.CleanOrphans(ctx, medialibrary.CleanOptions{})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("orphans now:", len(report.Orphans))

	// The scheduled job that runs hours later removes it.
	cfg.Now = func() time.Time { return time.Now().Add(2 * medialibrary.DefaultOrphanGrace) }
	later, err := medialibrary.New(cfg)
	if err != nil {
		log.Fatal(err)
	}
	report, err = later.CleanOrphans(ctx, medialibrary.CleanOptions{})
	if err != nil {
		log.Fatal(err)
	}
	for _, o := range report.Orphans {
		fmt.Println("deleted:", o.Path)
	}
}

Output:
orphans now: 0
deleted: media/0192f0c4-7c00-7000-8000-000000000001/lost.txt

func (*Library) Delete 

func (l *Library) Delete(ctx context.Context, id uuid.UUID) error

Delete removes the media record and then its files. The record goes first: if deleting a file fails, the result is an unreferenced file that CleanOrphans removes later, never a record pointing at a missing file. File deletion runs through Config.AfterCommit and its failures are logged, not returned.

func (*Library) Get 

func (l *Library) Get(ctx context.Context, id uuid.UUID) (*Media, error)

Get returns the media item with the given ID or an error wrapping ErrNotFound.

func (*Library) List 

func (l *Library) List(ctx context.Context, owner Ref, opts ListOptions) (items []*Media, next *Cursor, err error)

List returns one page of the owner's media, ordered by collection, then position, then ID, and the cursor of the next page, which is nil on the last page. An incomplete owner fails with ErrInvalidOwner and an undefined opts.Collection with ErrUnknownCollection.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"strings"

	medialibrary "github.com/byteminus/go-medialibrary"
	"github.com/byteminus/go-medialibrary/localdisk"
	"github.com/byteminus/go-medialibrary/memstore"
)

func main() {
	ctx := context.Background()
	dir, err := os.MkdirTemp("", "media-example")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()
	disk, err := localdisk.New(dir, localdisk.Options{})
	if err != nil {
		log.Fatal(err)
	}
	lib, err := medialibrary.New(medialibrary.Config{
		Store:       memstore.New(),
		Disks:       map[string]medialibrary.Disk{"local": disk},
		Collections: []medialibrary.Collection{{OwnerType: "invoice", Name: "documents"}},
	})
	if err != nil {
		log.Fatal(err)
	}
	owner := medialibrary.Ref{Type: "invoice", ID: "2026-0042"}
	for _, name := range []string{"a.pdf", "b.pdf", "c.pdf"} {
		if _, err := lib.Add(ctx, owner, "documents", strings.NewReader("%PDF-1.7 "+name), name); err != nil {
			log.Fatal(err)
		}
	}

	// Read the collection two items at a time, as an API handler would
	// across requests, passing each page's cursor to the next call.
	opts := medialibrary.ListOptions{Collection: "documents", Limit: 2}
	for {
		page, next, err := lib.List(ctx, owner, opts)
		if err != nil {
			log.Fatal(err)
		}
		names := make([]string, len(page))
		for i, m := range page {
			names[i] = m.FileName
		}
		fmt.Println(strings.Join(names, " "))
		if next == nil {
			break
		}
		opts.After = next
	}
}

Output:
a.pdf b.pdf
c.pdf

func (*Library) Open 

func (l *Library) Open(ctx context.Context, m *Media, conversion string) (io.ReadCloser, error)

Open returns the content of the original file (conversion "") or of a conversion; the caller closes it. A missing file yields ErrFileNotFound.

func (*Library) RegenerateConversions 

func (l *Library) RegenerateConversions(ctx context.Context, id uuid.UUID, names ...string) (*Media, error)

RegenerateConversions generates the named conversions of a media item again, or all conversions of its collection when no names are given. In the latter case results for conversions no longer defined are removed along with their files.

With a Queue configured, the conversions are marked pending (keeping their previous files and paths, so URLs stay valid) and a job is enqueued; otherwise they run before RegenerateConversions returns. The media must be a convertible image (see IsConvertible).

func (*Library) Reorder 

func (l *Library) Reorder(ctx context.Context, owner Ref, collection string, ids []uuid.UUID) error

Reorder sets the order of an owner's collection to the order of ids, which must list every item of the collection exactly once.

func (*Library) RunConversions 

func (l *Library) RunConversions(ctx context.Context, job ConversionJob) (*Media, error)

RunConversions generates the conversions a job asks for, synchronously. Application workers call it for jobs received through Queue. A media item deleted after the job was enqueued yields an error wrapping ErrNotFound, which a worker should treat as done. Like Add, it fails with ErrWriteWindowExceeded when writing takes longer than half of Config.OrphanGrace; the job can simply be retried.

func (*Library) SignedURL 

func (l *Library) SignedURL(ctx context.Context, m *Media, conversion string, ttl time.Duration) (string, error)

SignedURL returns a URL that grants read access to the original file (conversion "") or a conversion until ttl elapses. ttl must be positive, or SignedURL fails with ErrInvalid; disks may impose a maximum. A disk that cannot sign URLs yields ErrSignedURLUnsupported.

func (*Library) URL 

func (l *Library) URL(m *Media, conversion string) (string, error)

URL returns the public URL of the original file (conversion "") or of a generated conversion. It fails with ErrUnknownConversion, ErrConversionNotReady, ErrUnknownDisk or, for a disk that is not publicly served, ErrNoPublicURL. Private media belong on such a disk and are reached through SignedURL or mediahttp.Server.

func (*Library) UpdateProperties 

func (l *Library) UpdateProperties(ctx context.Context, id uuid.UUID, props map[string]any) (*Media, error)

UpdateProperties replaces the custom properties of a media item; nil clears them. The properties must be representable as a JSON object of at most MaxPropertiesSize bytes, or UpdateProperties fails with an error wrapping ErrInvalidMedia. They are stored as JSON, so numbers read back as float64.

type ListOptions added in v0.2.0 

type ListOptions struct {
	// Collection restricts the page to one collection; empty lists every
	// collection of the owner.
	Collection string
	// Limit is the page size; zero means DefaultPageSize, and values above
	// MaxPageSize are capped.
	Limit int
	// After continues the listing after this position, normally the cursor
	// returned with the previous page; nil starts at the beginning.
	After *Cursor
}

ListOptions selects one page of an owner's media. The zero value selects the first DefaultPageSize items of all collections.

type Media 

type Media struct {
	ID         uuid.UUID
	Owner      Ref
	Collection string
	// Name is a display name, initially the file name without its extension.
	Name string
	// FileName is the sanitized client file name; see SanitizeFileName.
	FileName string
	// MimeType is detected from the file content at upload time.
	MimeType string
	Size     int64
	Disk     string
	Path     string
	// Checksum is the lowercase hex SHA-256 of the original file.
	Checksum    string
	Properties  map[string]any
	Conversions map[string]ConversionResult
	// Order is the 1-based position within the owner's collection.
	Order     int
	CreatedAt time.Time
	UpdatedAt time.Time
}

Media is one stored file attached to an owner.

func (*Media) Clone 

func (m *Media) Clone() *Media

Clone returns a deep copy of m, so callers can modify the copy without affecting records held by a store.

func (*Media) Validate 

func (m *Media) Validate() error

Validate reports whether m satisfies the integrity constraints every Store enforces. It mirrors the CHECK constraints of the PostgreSQL schema so that third-party stores can apply the same rules; errors wrap ErrInvalidMedia.

type OrphanFile 

type OrphanFile struct {
	Disk string
	Path string
	Size int64
}

OrphanFile identifies one unreferenced file.

type Queue 

type Queue interface {
	// Enqueue schedules job. It must be safe for concurrent use. An error
	// is returned to the caller of Add or RegenerateConversions wrapped in
	// ErrEnqueue; the conversions stay pending.
	Enqueue(ctx context.Context, job ConversionJob) error
}

Queue hands conversion work to the application's own job system; the application implements it. When a Queue is configured, Add and RegenerateConversions record the conversions as pending and enqueue a job; the application's worker then calls Library.RunConversions with that job. Enqueue receives the caller's context, so an implementation can join a transaction carried there and make the job commit or roll back together with the media record.

Example 
package main

import (
	"bytes"
	"context"
	"fmt"
	"image"
	"image/png"
	"log"
	"os"

	medialibrary "github.com/byteminus/go-medialibrary"
	"github.com/byteminus/go-medialibrary/localdisk"
	"github.com/byteminus/go-medialibrary/memstore"
)

// samplePNG returns a w x h PNG standing in for an uploaded photo.
func samplePNG(w, h int) []byte {
	var buf bytes.Buffer
	if err := png.Encode(&buf, image.NewRGBA(image.Rect(0, 0, w, h))); err != nil {
		log.Fatal(err)
	}
	return buf.Bytes()
}

// queue stands in for an application's job system.
type queue struct{ jobs []medialibrary.ConversionJob }

func (q *queue) Enqueue(_ context.Context, job medialibrary.ConversionJob) error {
	q.jobs = append(q.jobs, job)
	return nil
}

func main() {
	ctx := context.Background()
	dir, err := os.MkdirTemp("", "media-example")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()
	disk, err := localdisk.New(dir, localdisk.Options{})
	if err != nil {
		log.Fatal(err)
	}
	jobs := &queue{}
	lib, err := medialibrary.New(medialibrary.Config{
		Store: memstore.New(),
		Disks: map[string]medialibrary.Disk{"local": disk},
		Queue: jobs,
		Collections: []medialibrary.Collection{{
			OwnerType: "product",
			Name:      "photos",
			Conversions: []medialibrary.Conversion{
				{Name: "card", Width: 320, Height: 240, Fit: medialibrary.FitFill, Format: medialibrary.FormatPNG},
			},
		}},
	})
	if err != nil {
		log.Fatal(err)
	}

	m, err := lib.Add(ctx, medialibrary.Ref{Type: "product", ID: "p-1"}, "photos", bytes.NewReader(samplePNG(800, 800)), "shoe.png")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(m.Conversions["card"].Status)

	// The application's worker receives the job and runs it.
	done, err := lib.RunConversions(ctx, jobs.jobs[0])
	if err != nil {
		log.Fatal(err)
	}
	card := done.Conversions["card"]
	fmt.Println(card.Status, card.Width, card.Height)
}

Output:
pending
done 320 240

type Ref 

type Ref struct {
	Type string `json:"type"`
	ID   string `json:"id"`
}

Ref identifies a record or party by type and ID, such as the model a media item is attached to: Ref{Type: "user", ID: "42"}. Type selects the collection definitions that apply; ID is opaque to the library. A stored Ref always has both fields set.

func (Ref) IsZero added in v0.2.0 

func (r Ref) IsZero() bool

IsZero reports whether both fields are empty.

func (Ref) String 

func (r Ref) String() string

String returns the reference as "type:id", for logs and display only. Never use it as a map, cache or database key: a Type containing ":" would collide with another reference. Use the struct itself instead.

type SignedURLer 

type SignedURLer interface {
	// SignedURL returns a URL granting read access to path until ttl elapses.
	SignedURL(ctx context.Context, path string, ttl time.Duration) (string, error)
}

SignedURLer is implemented by disks that can issue time-limited URLs for private files. Library.SignedURL uses it when the media's disk supports it.

type Store 

type Store interface {
	// Create inserts m after checking m.Validate, assigning m.Order to one
	// past the highest order in its owner's collection. The order assigned
	// by the store overrides m.Order. With opts.Replace every existing item
	// of the collection is deleted in the same atomic step and returned.
	// With opts.MaxFiles > 0, Create fails with an error wrapping
	// ErrCollectionFull when the collection would exceed it. A duplicate ID
	// or disk and path fails with an error wrapping ErrConflict, and an
	// invalid record with one wrapping ErrInvalidMedia.
	Create(ctx context.Context, m *Media, opts CreateOptions) (replaced []*Media, err error)
	// Get returns the record with the given ID, or an error wrapping
	// ErrNotFound.
	Get(ctx context.Context, id uuid.UUID) (*Media, error)
	// List returns at most opts.Limit of the owner's records, in one
	// collection or in all collections when opts.Collection is empty,
	// ordered by collection (comparing bytes), then Order, then ID (comparing
	// bytes), starting after opts.After when it is set. The library always
	// passes a positive Limit, which the store applies as given.
	List(ctx context.Context, owner Ref, opts ListOptions) ([]*Media, error)
	// Delete removes the record and returns it, or fails with an error
	// wrapping ErrNotFound.
	Delete(ctx context.Context, id uuid.UUID) (*Media, error)
	// Reorder sets the order of the collection's items to the position of
	// their ID in ids, starting at 1. ids must list every item of the
	// collection exactly once, or Reorder fails with an error wrapping
	// ErrInvalidOrder.
	Reorder(ctx context.Context, owner Ref, collection string, ids []uuid.UUID, at time.Time) error
	// UpdateProperties replaces the custom properties of a record; nil
	// stores an empty object. A missing record fails with an error wrapping
	// ErrNotFound.
	UpdateProperties(ctx context.Context, id uuid.UUID, props map[string]any, at time.Time) (*Media, error)
	// PatchConversions sets the given conversion results and removes the
	// named ones in a single atomic update, leaving other conversions
	// untouched, so concurrent workers finishing different conversions do
	// not overwrite each other. A missing record fails with an error
	// wrapping ErrNotFound.
	PatchConversions(ctx context.Context, id uuid.UUID, set map[string]ConversionResult, remove []string, at time.Time) (*Media, error)
}

Store persists media records. Applications normally use memstore or pgxstore; implement Store only to keep records elsewhere, and verify the implementation with the storetest package.

Implementations must be safe for concurrent use and must serialize the operations that depend on the contents of a collection (Create and Reorder) per owner and collection, so that limits and ordering hold under concurrent requests. Records passed in and returned are owned by the caller; a store never retains or shares them. Times are stored at microsecond precision and returned in UTC. Every method honours ctx.

Source Files

View all Source files

Directories

Path Synopsis
Package disktest is a conformance suite for medialibrary.Disk implementations.
 Package disktest is a conformance suite for medialibrary.Disk implementations.
Package localdisk is a medialibrary.Disk that stores files in a directory on the local filesystem.
 Package localdisk is a medialibrary.Disk that stores files in a directory on the local filesystem.
Package mediahttp connects medialibrary to net/http: a streaming multipart upload helper with a request size limit, a Server that streams media to clients with headers that keep hostile uploads inert, and a mapping from library errors to HTTP status codes.
 Package mediahttp connects medialibrary to net/http: a streaming multipart upload helper with a request size limit, a Server that streams media to clients with headers that keep hostile uploads inert, and a mapping from library errors to HTTP status codes.
Package memstore is an in-memory medialibrary.Store for tests, examples and prototypes.
 Package memstore is an in-memory medialibrary.Store for tests, examples and prototypes.
Package pgxstore is a PostgreSQL medialibrary.Store built on pgx v5.
 Package pgxstore is a PostgreSQL medialibrary.Store built on pgx v5.
Package s3disk is a medialibrary.Disk backed by Amazon S3 or an S3-compatible service such as Cloudflare R2, MinIO or SeaweedFS, built on the AWS SDK for Go v2.
 Package s3disk is a medialibrary.Disk backed by Amazon S3 or an S3-compatible service such as Cloudflare R2, MinIO or SeaweedFS, built on the AWS SDK for Go v2.
Package storetest is a conformance suite for medialibrary.Store implementations.
 Package storetest is a conformance suite for medialibrary.Store implementations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.