storage

package module
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 24, 2026 License: MIT Imports: 8 Imported by: 0

README

storage

Multi-provider file storage with a unified interface — supports local filesystem, S3, and Supabase.

Install

go get github.com/kbukum/gokit/storage@latest

Quick Start

import (
    "github.com/kbukum/gokit/storage"
    _ "github.com/kbukum/gokit/storage/s3"       // register s3 provider
    _ "github.com/kbukum/gokit/storage/supabase"  // register supabase provider
    "github.com/kbukum/gokit/logger"
)

log := logger.New()
comp := storage.NewComponent(storage.Config{
    Enabled:  true,
    Provider: "s3",
    Bucket:   "my-bucket",
    Region:   "us-east-1",
}, log)

comp.Start(ctx)
defer comp.Stop(ctx)

s := comp.Storage()
s.Upload(ctx, "docs/file.pdf", reader)
url, _ := s.URL(ctx, "docs/file.pdf")
files, _ := s.List(ctx, "docs/")

Key Types & Functions

Symbol Description
Storage Interface — Upload, Download, Delete, Exists, URL, List
FileInfo Path, Size, LastModified, ContentType
Config Provider name, bucket, region, endpoint, credentials, max file size
Component Managed lifecycle — Start, Stop, Health
NewComponent(cfg, log) Create a managed storage component
New(cfg, log) Create a Storage directly via registered factory
Providers
Package Factory
storage/local local.NewStorage(basePath)
storage/s3 s3.NewStorage(ctx, s3.Config{Region, Bucket, ...})
storage/supabase supabase.NewStorage(supabase.Config{URL, Bucket, ...})

← Back to main gokit README

Documentation

Overview

Package storage provides byte-oriented convenience types that wrap the streaming Storage interface with []byte operations.

Package storage provides object storage abstractions with pluggable backends for gokit applications.

It defines interfaces for common storage operations (upload, download, delete, list) and follows gokit's component pattern with lifecycle management.

Backends

  • storage/s3: Amazon S3 and S3-compatible storage
  • storage/local: Local filesystem storage for development/testing
  • storage/supabase: Supabase Storage integration

Configuration

Backend selection and settings are provided via Config: 

storage:
  provider: "s3"
  s3:
    bucket: "my-bucket"
    region: "us-east-1"

Package storage provides interfaces and implementations for object storage. Supported providers: local filesystem, Amazon S3 (and S3-compatible services).

Index

Constants

View Source 
const (
	ProviderLocal    = "local"
	ProviderS3       = "s3"
	ProviderSupabase = "supabase"
)

Provider constants for well-known storage backends.

View Source 
const (
	DefaultProvider     = ProviderLocal
	DefaultMaxFileSize  = int64(100 * 1024 * 1024) // 100 MB
	DefaultPresignedTTL = time.Hour
)

Default configuration values.

Variables

This section is empty.

Functions

func RegisterFactory 

func RegisterFactory(name string, f StorageFactory)

RegisterFactory registers a storage backend factory for the given provider name. Implementation packages call this (typically in an init function) to make themselves available to the New constructor.

Types

type BucketDescriber 

type BucketDescriber interface {
	GetBucket() string
}

BucketDescriber is optionally implemented by provider configs that use a bucket.

type ByteClient 

type ByteClient interface {
	// Upload stores data at the given path.
	Upload(ctx context.Context, path string, data []byte) error

	// Download retrieves data from the given path.
	Download(ctx context.Context, path string) ([]byte, error)

	// Delete removes the object at the given path.
	Delete(ctx context.Context, path string) error

	// Exists checks whether an object exists at the given path.
	Exists(ctx context.Context, path string) (bool, error)

	// List returns metadata for all objects whose path starts with prefix.
	List(ctx context.Context, prefix string) ([]ObjectInfo, error)
}

ByteClient provides a []byte-oriented interface for storage operations. This is useful for callers that work with in-memory data rather than streams.

func NewByteClient 

func NewByteClient(s Storage) ByteClient

NewByteClient wraps a streaming Storage implementation with []byte convenience methods.

type Component 

type Component struct {
	// contains filtered or unexported fields
}

Component wraps Storage and implements component.Component for lifecycle management.

func NewComponent 

func NewComponent(cfg Config, providerCfg any, log *logger.Logger) *Component

NewComponent creates a storage component for use with the component registry.

func (*Component) Describe 

func (c *Component) Describe() component.Description

Describe returns infrastructure summary info for the bootstrap display.

func (*Component) Health 

func (c *Component) Health(ctx context.Context) component.Health

Health returns the current health status of the storage component.

func (*Component) Name 

func (c *Component) Name() string

Name returns the component name.

func (*Component) Start 

func (c *Component) Start(_ context.Context) error

Start initializes the storage backend.

func (*Component) Stop 

func (c *Component) Stop(_ context.Context) error

Stop gracefully shuts down the storage component.

func (*Component) Storage 

func (c *Component) Storage() Storage

Storage returns the underlying Storage, or nil if not started.

type Config 

type Config struct {
	// Provider selects the storage backend (e.g. "local", "s3", "supabase").
	Provider string `mapstructure:"provider" json:"provider"`

	// Enabled controls whether the storage component is active.
	Enabled bool `mapstructure:"enabled" json:"enabled"`

	// MaxFileSize is the maximum allowed file size in bytes.
	MaxFileSize int64 `mapstructure:"max_file_size" json:"max_file_size"`

	// PublicURL is the base URL for public access to stored objects.
	PublicURL string `mapstructure:"public_url" json:"public_url"`

	// PresignedTTL is the default duration for pre-signed URLs.
	PresignedTTL time.Duration `mapstructure:"presigned_ttl" json:"presigned_ttl"`

	// AllowedTypes is an optional whitelist of allowed MIME types for uploads.
	AllowedTypes []string `mapstructure:"allowed_types" json:"allowed_types"`
}

Config holds provider-agnostic storage configuration. Provider-specific settings are passed separately via providerCfg (any).

func (*Config) ApplyDefaults 

func (c *Config) ApplyDefaults()

ApplyDefaults fills in zero-valued fields with sensible defaults.

func (*Config) Validate 

func (c *Config) Validate() error

Validate checks that the core configuration is valid. Provider-specific validation is handled by each provider's own config.

type DeleteProvider added in v0.1.1 

type DeleteProvider struct {
	// contains filtered or unexported fields
}

DeleteProvider wraps Storage.Delete as a RequestResponse provider.

func NewDeleteProvider added in v0.1.1 

func NewDeleteProvider(name string, s Storage) *DeleteProvider

NewDeleteProvider creates a RequestResponse provider for deletes.

func (*DeleteProvider) Execute added in v0.1.1 

func (p *DeleteProvider) Execute(ctx context.Context, req DeleteRequest) (struct{}, error)

func (*DeleteProvider) IsAvailable added in v0.1.1 

func (p *DeleteProvider) IsAvailable(_ context.Context) bool

func (*DeleteProvider) Name added in v0.1.1 

func (p *DeleteProvider) Name() string

type DeleteRequest added in v0.1.1 

type DeleteRequest struct {
	Path string
}

DeleteRequest describes a storage delete operation.

type DownloadProvider added in v0.1.1 

type DownloadProvider struct {
	// contains filtered or unexported fields
}

DownloadProvider wraps Storage.Download as a RequestResponse provider.

func NewDownloadProvider added in v0.1.1 

func NewDownloadProvider(name string, s Storage) *DownloadProvider

NewDownloadProvider creates a RequestResponse provider for downloads.

func (*DownloadProvider) Execute added in v0.1.1 

func (p *DownloadProvider) Execute(ctx context.Context, req DownloadRequest) (*DownloadResponse, error)

func (*DownloadProvider) IsAvailable added in v0.1.1 

func (p *DownloadProvider) IsAvailable(_ context.Context) bool

func (*DownloadProvider) Name added in v0.1.1 

func (p *DownloadProvider) Name() string

type DownloadRequest added in v0.1.1 

type DownloadRequest struct {
	Path string
}

DownloadRequest describes a storage download operation.

type DownloadResponse added in v0.1.1 

type DownloadResponse struct {
	Body io.ReadCloser
}

DownloadResponse holds the result of a download operation.

type ExistsProvider added in v0.1.1 

type ExistsProvider struct {
	// contains filtered or unexported fields
}

ExistsProvider wraps Storage.Exists as a RequestResponse provider.

func NewExistsProvider added in v0.1.1 

func NewExistsProvider(name string, s Storage) *ExistsProvider

NewExistsProvider creates a RequestResponse provider for existence checks.

func (*ExistsProvider) Execute added in v0.1.1 

func (p *ExistsProvider) Execute(ctx context.Context, req ExistsRequest) (*ExistsResponse, error)

func (*ExistsProvider) IsAvailable added in v0.1.1 

func (p *ExistsProvider) IsAvailable(_ context.Context) bool

func (*ExistsProvider) Name added in v0.1.1 

func (p *ExistsProvider) Name() string

type ExistsRequest added in v0.1.1 

type ExistsRequest struct {
	Path string
}

ExistsRequest describes a storage existence check.

type ExistsResponse added in v0.1.1 

type ExistsResponse struct {
	Exists bool
}

ExistsResponse holds the result of an existence check.

type FileInfo 

type FileInfo struct {
	Path         string
	Size         int64
	LastModified time.Time
	ContentType  string
}

FileInfo contains metadata about a stored object.

type ObjectInfo 

type ObjectInfo struct {
	Key  string // Object path/key
	Size int64  // Size in bytes
}

ObjectInfo contains minimal metadata about a stored object. This is a simplified alternative to FileInfo for callers that only need key and size information.

type SignedURLProvider 

type SignedURLProvider interface {
	// SignedURL returns a pre-signed URL valid for the specified duration.
	SignedURL(ctx context.Context, path string, expiry time.Duration) (string, error)
}

SignedURLProvider is optionally implemented by storage backends that support generating time-limited signed URLs for private object access.

type Storage 

type Storage interface {
	// Upload writes data from reader to the given path.
	Upload(ctx context.Context, path string, reader io.Reader) error

	// Download returns a reader for the object at the given path.
	// The caller is responsible for closing the returned ReadCloser.
	Download(ctx context.Context, path string) (io.ReadCloser, error)

	// Delete removes the object at the given path.
	// Returns nil if the object does not exist.
	Delete(ctx context.Context, path string) error

	// Exists checks whether an object exists at the given path.
	Exists(ctx context.Context, path string) (bool, error)

	// URL returns a public URL for accessing the object at the given path.
	URL(ctx context.Context, path string) (string, error)

	// List returns metadata for all objects whose path starts with prefix.
	List(ctx context.Context, prefix string) ([]FileInfo, error)
}

Storage defines the interface for object storage operations.

func New 

func New(cfg Config, providerCfg any, log *logger.Logger) (Storage, error)

New creates a Storage implementation based on the given Config. The provider field determines which backend is used. providerCfg carries provider-specific settings (e.g. *s3.Config, *supabase.Config). Ensure the desired provider package has been imported (e.g. _ "github.com/kbukum/gokit/storage/local") so its factory is registered.

type StorageFactory 

type StorageFactory func(cfg Config, providerCfg any, log *logger.Logger) (Storage, error)

StorageFactory creates a Storage implementation from core config and provider-specific configuration. Each provider type-asserts providerCfg to its own config type.

type UploadProvider added in v0.1.1 

type UploadProvider struct {
	// contains filtered or unexported fields
}

UploadProvider wraps Storage.Upload as a RequestResponse provider.

func NewUploadProvider added in v0.1.1 

func NewUploadProvider(name string, s Storage) *UploadProvider

NewUploadProvider creates a RequestResponse provider for uploads.

func (*UploadProvider) Execute added in v0.1.1 

func (p *UploadProvider) Execute(ctx context.Context, req UploadRequest) (struct{}, error)

func (*UploadProvider) IsAvailable added in v0.1.1 

func (p *UploadProvider) IsAvailable(_ context.Context) bool

func (*UploadProvider) Name added in v0.1.1 

func (p *UploadProvider) Name() string

type UploadRequest added in v0.1.1 

type UploadRequest struct {
	Path   string
	Reader io.Reader
}

UploadRequest describes a storage upload operation.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.