Documentation
¶
Overview ¶
Package storage provides file/object storage backends for the upload battery.
Three concrete backends ship, each constructed directly (there is no registry or factory indirection):
- NewLocalStorage(dir, ...) — the local filesystem.
- NewMemoryStorage(...) — an in-process store for tests and ephemeral data.
- NewS3Storage(bucket, reg) — S3 (or S3-compatible) object storage.
All implement the Storage interface (a re-export of upload.Storage), which covers save / get / delete / list. LocalStorage and MemoryStorage also implement RangeGetter (re-export of upload.RangeGetter) so HTTP range requests can be answered; S3Storage declines and is instead paired with a presigner (WithPresigner) so uploads/downloads bypass the app entirely.
Content checksums: SaveWithChecksum writes an object plus a checksum sidecar so a later read can detect bit-rot or an interrupted write.
Keys are validated (DefaultKeyValidator) to reject path-traversal and other forbidden sequences before they reach a backend.
Index ¶
- Variables
- func VerifyChecksum(ctx context.Context, s Storage, key, wantSHA256 string) error
- type DefaultKeyValidator
- type FileMeta
- type KeyValidator
- type LocalOption
- type LocalStorage
- func (ls *LocalStorage) Delete(ctx context.Context, key string) error
- func (ls *LocalStorage) Exists(ctx context.Context, key string) (bool, error)
- func (ls *LocalStorage) Get(ctx context.Context, key string) (io.ReadCloser, error)
- func (ls *LocalStorage) GetRange(ctx context.Context, key string) (io.ReadSeekCloser, error)
- func (ls *LocalStorage) Save(ctx context.Context, key string, r io.Reader) error
- type MemoryStorage
- func (ms *MemoryStorage) Delete(_ context.Context, key string) error
- func (ms *MemoryStorage) Exists(_ context.Context, key string) (bool, error)
- func (ms *MemoryStorage) Get(_ context.Context, key string) (io.ReadCloser, error)
- func (ms *MemoryStorage) GetRange(_ context.Context, key string) (io.ReadSeekCloser, error)
- func (ms *MemoryStorage) Save(_ context.Context, key string, r io.Reader) error
- type Presigner
- type RangeGetter
- type ReadCloser
- type S3Client
- type S3Option
- type S3Storage
- func (s *S3Storage) Delete(ctx context.Context, key string) error
- func (s *S3Storage) Exists(ctx context.Context, key string) (bool, error)
- func (s *S3Storage) Get(ctx context.Context, key string) (io.ReadCloser, error)
- func (s *S3Storage) PresignedGetURL(ctx context.Context, key string, expires time.Duration) (*url.URL, error)
- func (s *S3Storage) PresignedPutURL(ctx context.Context, key string, expires time.Duration) (*url.URL, error)
- func (s *S3Storage) Save(ctx context.Context, key string, r io.Reader) error
- type SaveResult
- type Storage
Constants ¶
This section is empty.
Variables ¶
var ErrChecksumMismatch = errors.New("storage: checksum mismatch")
ErrChecksumMismatch is the sentinel returned (wrapped) by VerifyChecksum when an object's actual SHA-256 digest does not match the expected digest.
Functions ¶
func VerifyChecksum ¶ added in v0.14.0
VerifyChecksum re-reads key from s and compares the content's SHA-256 digest with wantSHA256. wantSHA256 must be exactly 64 hexadecimal characters; both lower- and uppercase hex are accepted. It returns nil on a match, an error wrapping ErrChecksumMismatch (carrying the key and the got/want digests) on a mismatch, and the underlying error if the object cannot be read or wantSHA256 is malformed.
Types ¶
type DefaultKeyValidator ¶
type DefaultKeyValidator struct{}
DefaultKeyValidator implements basic key validation.
func (DefaultKeyValidator) ValidateKey ¶
func (DefaultKeyValidator) ValidateKey(key string) error
ValidateKey checks that a key does not contain path traversal sequences.
type KeyValidator ¶
KeyValidator validates storage keys to prevent path traversal and other attacks.
type LocalOption ¶
type LocalOption func(*LocalStorage)
LocalOption configures a LocalStorage instance.
func WithPermissions ¶
func WithPermissions(mode os.FileMode) LocalOption
WithPermissions sets the file permission mode for saved files.
func WithTempDir ¶
func WithTempDir(dir string) LocalOption
WithTempDir sets a custom temporary directory for atomic writes.
type LocalStorage ¶
type LocalStorage struct {
BaseDir string
// contains filtered or unexported fields
}
LocalStorage implements Storage backed by the local filesystem. Writes are atomic: data is first written to a temporary file, then renamed to the final path.
func NewLocalStorage ¶
func NewLocalStorage(baseDir string, opts ...LocalOption) *LocalStorage
NewLocalStorage creates a LocalStorage rooted at baseDir. The directory is created if it does not exist.
func (*LocalStorage) Delete ¶
func (ls *LocalStorage) Delete(ctx context.Context, key string) error
Delete removes the file identified by key from the filesystem. It is not an error if the file does not exist.
func (*LocalStorage) Get ¶
func (ls *LocalStorage) Get(ctx context.Context, key string) (io.ReadCloser, error)
Get opens the file identified by key and returns a ReadCloser for its contents.
func (*LocalStorage) GetRange ¶ added in v0.46.0
func (ls *LocalStorage) GetRange(ctx context.Context, key string) (io.ReadSeekCloser, error)
GetRange implements upload.RangeGetter, exposing the seekability the local backend already has: Get opens an *os.File and then discards Seek through the io.ReadCloser return type. Key validation runs through the same fullPath call, not a parallel one.
type MemoryStorage ¶
type MemoryStorage struct {
// contains filtered or unexported fields
}
MemoryStorage implements Storage backed by an in-memory map. It is safe for concurrent use via sync.RWMutex.
func NewMemoryStorage ¶
func NewMemoryStorage() *MemoryStorage
NewMemoryStorage creates a new empty MemoryStorage.
func (*MemoryStorage) Delete ¶
func (ms *MemoryStorage) Delete(_ context.Context, key string) error
Delete removes the file identified by key. It is not an error if the key does not exist.
func (*MemoryStorage) Get ¶
func (ms *MemoryStorage) Get(_ context.Context, key string) (io.ReadCloser, error)
Get returns a ReadCloser for the file identified by key.
func (*MemoryStorage) GetRange ¶ added in v0.46.0
func (ms *MemoryStorage) GetRange(_ context.Context, key string) (io.ReadSeekCloser, error)
GetRange implements upload.RangeGetter. The bytes are already in memory, so a *bytes.Reader satisfies Seek for free — the wrapper only supplies the no-op Close.
type Presigner ¶
type Presigner interface {
PresignGet(ctx context.Context, bucket, key string, expires time.Duration) (*url.URL, error)
PresignPut(ctx context.Context, bucket, key string, expires time.Duration) (*url.URL, error)
}
Presigner generates presigned URLs for direct browser uploads/downloads.
type RangeGetter ¶ added in v0.46.0
type RangeGetter = upload.RangeGetter
RangeGetter is a re-export of upload.RangeGetter, the optional capability a backend implements to expose seekable reads so HTTP range requests can be answered. LocalStorage and MemoryStorage implement it; S3Storage declines — a network-backed store would have to buffer the whole object to satisfy Seek, and WithPresigner lets the transfer bypass the app entirely.
type ReadCloser ¶
ReadCloser wraps an io.Reader to implement io.ReadCloser with a no-op Close.
type S3Client ¶
type S3Client interface {
PutObject(ctx context.Context, bucket, key string, r io.Reader, size int64, contentType string) error
GetObject(ctx context.Context, bucket, key string) (io.ReadCloser, error)
DeleteObject(ctx context.Context, bucket, key string) error
HeadObject(ctx context.Context, bucket, key string) (bool, error)
}
S3Client is a minimal interface for S3-compatible operations. This avoids importing the AWS SDK directly; callers provide their own implementation or use the PresignedURL field for direct browser uploads.
type S3Option ¶
type S3Option func(*S3Storage)
S3Option configures an S3Storage instance.
func WithPresigner ¶
WithPresigner sets the URL presigner for generating presigned URLs.
func WithS3Client ¶
WithS3Client sets the S3 client implementation.
func WithS3Endpoint ¶
WithS3Endpoint sets a custom S3-compatible endpoint.
type S3Storage ¶
type S3Storage struct {
Bucket string
Region string
Endpoint string
Client S3Client
// contains filtered or unexported fields
}
S3Storage implements Storage backed by an S3-compatible object store. It uses a minimal S3Client interface so no AWS SDK is imported directly.
func NewS3Storage ¶
NewS3Storage creates a new S3Storage for the given bucket and region. Use WithS3Client to inject an actual client before calling Save/Get/etc.
func (*S3Storage) PresignedGetURL ¶
func (s *S3Storage) PresignedGetURL(ctx context.Context, key string, expires time.Duration) (*url.URL, error)
PresignedGetURL returns a presigned URL for downloading the object.
type SaveResult ¶ added in v0.14.0
type SaveResult struct {
// Size is the number of bytes written.
Size int64
// SHA256 is the lowercase hex SHA-256 digest of the stored content.
SHA256 string
}
SaveResult reports what SaveWithChecksum wrote.
func SaveWithChecksum ¶ added in v0.14.0
SaveWithChecksum stores r under key via s.Save while teeing the stream through a SHA-256 hasher, so the content is read exactly once and no backend changes are required. It returns a SaveResult carrying the byte count and the lowercase hex digest. On a Save error it returns the zero SaveResult and the error from s.Save; the stream is not buffered in memory, so it works for arbitrarily large objects.
Source Files