Documentation
¶
Overview ¶
Package storage provides object storage abstractions, adapter-agnostic helpers, and safety defaults for the Keel platform.
Architecture ¶
The package is organised into three layers:
- Core contract (this package): the Store interface, Object descriptor, NewKey key generator, DefaultValidationHook, sentinel errors, and helper functions.
- Adapters (sub-packages): local (filesystem) and s3 (S3-compatible object store). Each adapter satisfies the Store interface and is safe for concurrent use.
- Helpers (helpers.go): Upload and Download wrappers that handle key generation and object construction in one call, plus composable ValidationHook factories.
Security model ¶
The storage package enforces the following properties (security-baseline.md §10):
- Server-controlled keys: NewKey always generates a UUID-based path; clients must never construct or supply storage keys directly.
- Content validation: adapters call ValidationHook before writing any bytes. DefaultValidationHook blocks executable content types. Replace it only with an explicit, narrowly-scoped hook.
- Path traversal prevention: the local adapter validates every key against the root directory and rejects absolute paths and ".." segments.
- Executable content denied by default: application/x-executable, application/x-sh, and related types are denied unless the service opts in with a custom hook.
Key format ¶
<prefix>/<YYYY>/<MM>/<DD>/<UUIDv7><ext> // Example: "reports/2026/03/12/018e5c9a-1234-7abc-8def-0123456789ab.pdf"
Quick start ¶
// 1. Choose an adapter.
adapter, err := local.New(local.Config{Root: "/var/data/uploads"})
if err != nil { /* handle */ }
var store storage.Store = adapter
// 2. Upload using the helper (recommended).
result, err := storage.Upload(ctx, store, storage.UploadRequest{
Prefix: "reports",
Ext: ".pdf",
ContentType: "application/pdf",
Size: r.ContentLength,
Body: r.Body,
})
if err != nil { /* handle */ }
savedKey := result.Key // persist this in your database
// 3. Download.
obj, rc, err := storage.Download(ctx, store, savedKey)
if err != nil { /* handle */ }
defer rc.Close()
// 4. Delete.
if err := store.Delete(ctx, savedKey); err != nil { /* handle */ }
Validation hooks ¶
Use AllowedTypesHook when the expected MIME types are known in advance:
hook := storage.AllowedTypesHook("image/png", "image/jpeg", "image/webp")
adapter, _ := local.New(local.Config{Root: "/data", ValidationHook: hook})
Use ChainHooks to combine multiple hooks:
hook := storage.ChainHooks(
storage.DefaultValidationHook,
storage.BlockedTypesHook("text/csv"),
)
Error handling ¶
All errors wrap one of the package sentinel errors; use errors.Is to match them:
var (
storage.ErrNotFound // object does not exist
storage.ErrValidation // content validation failed
storage.ErrKeyConflict // key already exists (adapters that deny overwrite)
)
helpers.go provides upload/download convenience wrappers and hook factories for platform/storage adapters.
These helpers are the recommended way to perform single-file operations. They handle key generation, object construction, and error context in one call.
Package storage provides object storage abstractions and safe adapter defaults.
The package defines the Store interface and the Object type that all storage adapters must implement. Storage keys are always server-generated; clients must never control final storage paths (see security-baseline.md §10).
Content validation belongs at the upload boundary. Adapters may provide a ValidationHook to deny uploads before any bytes reach the backend.
Usage:
var store storage.Store = local.New(cfg)
key, err := storage.NewKey("reports", ".pdf")
if err != nil {
return err
}
obj := storage.Object{
Key: key,
ContentType: "application/pdf",
Size: r.ContentLength,
}
if err := store.Put(ctx, obj, body); err != nil {
return err
}
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNotFound is returned when the requested object key does not exist. ErrNotFound = errors.New("storage: object not found") // ErrValidation is returned when an upload is rejected by the content // validation hook or adapter-level size/type constraints. ErrValidation = errors.New("storage: validation failed") // ErrKeyConflict is returned by adapters that do not allow overwrites when // an object with the same key already exists. ErrKeyConflict = errors.New("storage: key already exists") )
Sentinel errors returned by Store implementations and validation hooks. Callers should use errors.Is to check for these.
Functions ¶
func NewKey ¶
NewKey generates a server-controlled storage key for an object. prefix organizes objects into logical namespaces (e.g. "reports", "avatars"). ext is the file extension including the leading dot (e.g. ".pdf", ".png"). Pass an empty ext string if no extension is required.
The resulting key has the form:
<prefix>/<date>/<uuid><ext>
Example: reports/2026/03/12/a1b2c3d4-....pdf
Clients must never construct or modify storage keys. The server always generates keys to prevent path traversal and key collision attacks.
Types ¶
type Object ¶
type Object struct {
// Key is the server-assigned storage path. Clients must never supply this directly.
Key string
// Size is the object size in bytes. May be -1 when unknown (streaming upload).
Size int64
// ContentType is the MIME content type validated at the upload boundary.
ContentType string
// Metadata holds optional, non-sensitive key-value pairs attached to the object.
// Do not store credentials, PII, or secrets in metadata.
Metadata map[string]string
}
Object describes a stored object and its associated metadata. Key is always server-generated and must not be provided by the client.
type Store ¶
type Store interface {
// Put persists the object with body content to the backend.
// obj.Key must be server-generated (use NewKey). Returns an error if the
// key already exists and the adapter does not support overwrite.
Put(ctx context.Context, obj Object, body io.Reader) error
// Get retrieves the object descriptor and its body reader for the given key.
// The caller is responsible for closing the returned io.ReadCloser.
Get(ctx context.Context, key string) (Object, io.ReadCloser, error)
// Delete removes the object for the given key from the backend.
// Deleting a non-existent key returns ErrNotFound.
Delete(ctx context.Context, key string) error
}
Store is the primary abstraction for object persistence operations. All implementations must be safe for concurrent use.
type UploadRequest ¶
type UploadRequest struct {
// Prefix is the logical namespace for the object (e.g. "avatars", "reports").
// Must not contain path separators or dots.
Prefix string
// Ext is the file extension including the leading dot (e.g. ".pdf").
// Pass empty string for no extension.
Ext string
// ContentType is the MIME type declared by the caller.
ContentType string
// Size is the declared content length. Use -1 when unknown (streaming).
Size int64
// Metadata holds optional, non-sensitive key-value pairs.
// Do not include credentials, PII, or secrets.
Metadata map[string]string
// Body is the content reader for this upload.
// Must not be nil.
Body io.Reader
}
UploadRequest describes a single upload operation. The storage key is always server-generated from Prefix and Ext. Callers must not supply a pre-built key.
type UploadResult ¶
type UploadResult struct {
// Key is the server-generated storage key assigned to this object.
Key string
// Object is the descriptor stored by the backend.
Object Object
}
UploadResult carries the outcome of a successful Upload call.
func Upload ¶
func Upload(ctx context.Context, store Store, req UploadRequest) (UploadResult, error)
Upload is a convenience wrapper that generates a server-controlled key, constructs the Object descriptor, and calls Store.Put in one operation.
The generated key is returned in UploadResult.Key. Callers must persist this key to retrieve or delete the object later.
Upload fails if req.Prefix is empty, req.Body is nil, or the adapter rejects the upload (ErrValidation, ErrKeyConflict).
type ValidationHook ¶
ValidationHook is called by adapters before accepting an upload. Adapters that support content inspection should call this hook with the detected MIME type and the declared ContentType from the Object.
Return an error to reject the upload. The error will be wrapped as ErrValidation before propagating to the caller.
var DefaultValidationHook ValidationHook = func(detected, declared string, obj Object) error { for _, blocked := range blockedContentTypes { if strings.EqualFold(detected, blocked) || strings.EqualFold(declared, blocked) { return fmt.Errorf("%w: content type %q is not permitted by default", ErrValidation, detected) } } return nil }
DefaultValidationHook is a safe default that blocks executable content types unless the implementing service explicitly replaces it. This implements the security-baseline.md §10 rule: executable content must be blocked unless explicitly required.
func AllowedTypesHook ¶
func AllowedTypesHook(allowed ...string) ValidationHook
AllowedTypesHook returns a ValidationHook that only permits the listed MIME types. Both detected and declared types must be in the allowlist. This is more restrictive than DefaultValidationHook and is recommended for contexts where the expected content types are known in advance.
func BlockedTypesHook ¶
func BlockedTypesHook(blocked ...string) ValidationHook
BlockedTypesHook returns a ValidationHook that rejects any of the listed MIME types. Use this to extend the default deny-list with additional types. Combine with DefaultValidationHook by chaining: ChainHooks(DefaultValidationHook, BlockedTypesHook(...)).
func ChainHooks ¶
func ChainHooks(hooks ...ValidationHook) ValidationHook
ChainHooks returns a ValidationHook that calls each hook in order, stopping at the first error.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package local implements the platform/storage Store interface using the local filesystem.
|
Package local implements the platform/storage Store interface using the local filesystem. |
|
Package s3 implements platform/storage Store for S3-compatible backends.
|
Package s3 implements platform/storage Store for S3-compatible backends. |