storage

Documentation

Overview

Package storage abstracts where persisted stores live. A Blob is a minimal object store (put, get, delete, list of byte blobs by key) with two implementations: the local filesystem and any S3-compatible service (AWS S3, MinIO, Cloudflare R2, and similar). The S3 client is implemented on the standard library with hand-rolled SigV4 signing, so there is no heavy SDK dependency.

Index

• Variables
• type Blob
• type Local
  • func NewLocal(dir string) (*Local, error)
  • func (l *Local) Delete(_ context.Context, key string) error
  • func (l *Local) Get(_ context.Context, key string) ([]byte, error)
  • func (l *Local) List(_ context.Context, prefix string) ([]string, error)
  • func (l *Local) Put(_ context.Context, key string, data []byte) error
• type S3
  • func NewS3(cfg S3Config) (*S3, error)
  • func (s *S3) Delete(ctx context.Context, key string) error
  • func (s *S3) Get(ctx context.Context, key string) ([]byte, error)
  • func (s *S3) List(ctx context.Context, prefix string) ([]string, error)
  • func (s *S3) Put(ctx context.Context, key string, data []byte) error
• type S3Config

Variables

var ErrNotExist = errors.New("storage: key does not exist")

ErrNotExist is returned by Get and Delete when a key is absent.

Types

type Blob

type Blob interface {
	Put(ctx context.Context, key string, data []byte) error
	Get(ctx context.Context, key string) ([]byte, error)
	Delete(ctx context.Context, key string) error
	List(ctx context.Context, prefix string) ([]string, error)
}

Blob is a small object store keyed by string. Implementations must be safe for concurrent use.

type Local

type Local struct {
	// contains filtered or unexported fields
}

Local is a Blob backed by a directory on the local filesystem. Keys map to file names; subdirectories are created as needed.

func NewLocal

func NewLocal(dir string) (*Local, error)

NewLocal creates a Local blob rooted at dir, creating it if necessary.

func (*Local) Delete

func (l *Local) Delete(_ context.Context, key string) error

Delete removes the blob, mapping a missing file to ErrNotExist.

func (*Local) Get

func (l *Local) Get(_ context.Context, key string) ([]byte, error)

Get reads the blob, mapping a missing file to ErrNotExist.

func (*Local) List

func (l *Local) List(_ context.Context, prefix string) ([]string, error)

List returns the keys under the directory that start with prefix, sorted.

func (*Local) Put

func (l *Local) Put(_ context.Context, key string, data []byte) error

Put writes the blob, replacing it atomically (write to a temp file then rename) so a crash never leaves a half-written store.

type S3

type S3 struct {
	// contains filtered or unexported fields
}

S3 is a Blob backed by an S3-compatible service. It uses path-style addressing and SigV4 request signing, implemented on the standard library so no AWS SDK is required. It works with AWS S3, MinIO, Cloudflare R2, and similar services.

func NewS3

func NewS3(cfg S3Config) (*S3, error)

NewS3 creates an S3 blob from cfg.

func (*S3) Delete

func (s *S3) Delete(ctx context.Context, key string) error

Delete removes the object under key.

func (*S3) Get

func (s *S3) Get(ctx context.Context, key string) ([]byte, error)

Get downloads the object under key.

func (*S3) List

func (s *S3) List(ctx context.Context, prefix string) ([]string, error)

List returns object keys (with the configured prefix stripped) that start with prefix.

func (*S3) Put

func (s *S3) Put(ctx context.Context, key string, data []byte) error

Put uploads data under key.

type S3Config

type S3Config struct {
	Endpoint  string // e.g. https://s3.us-east-1.amazonaws.com or http://localhost:9000
	Bucket    string
	Region    string // e.g. us-east-1
	AccessKey string
	SecretKey string
	// Prefix is an optional key prefix (a folder within the bucket).
	Prefix string
}

S3Config configures an S3-compatible object store.