backend

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 30, 2026 License: AGPL-3.0 Imports: 8 Imported by: 0

Documentation

Overview

Package backend defines the single abstraction through which all S3 access flows — this is the most important file in the project. Concrete implementations live in sibling sub-packages (s3v4, minio, garage, seaweedfs, memory).

Index

Constants

This section is empty.

Variables

View Source 
var ErrNotRegistered = fmt.Errorf("backend: not registered")

ErrNotRegistered is returned by Unregister and Replace when no entry exists for the given id.

Functions

This section is empty.

Types

type AccessKey 

type AccessKey struct {
	ID        string
	User      string
	Enabled   bool
	CreatedAt time.Time
	ExpiresAt *time.Time
}

type AccessKeyOpts 

type AccessKeyOpts struct {
	ExpiresAt *time.Time
}

type AdminBackend 

type AdminBackend interface {
	ListUsers(ctx context.Context) ([]User, error)
	CreateUser(ctx context.Context, name string) (accessKey, secretKey string, err error)
	DeleteUser(ctx context.Context, name string) error
	ListAccessKeys(ctx context.Context, user string) ([]AccessKey, error)
	CreateAccessKey(ctx context.Context, user string, opts AccessKeyOpts) (AccessKey, error)
	RevokeAccessKey(ctx context.Context, user, accessKeyID string) error
	ListPolicies(ctx context.Context) ([]Policy, error)
	GetPolicy(ctx context.Context, name string) (Policy, error)
	PutPolicy(ctx context.Context, name, document string) error
	AttachPolicy(ctx context.Context, user, policy string) error
}

AdminBackend is optional and provides backend-native admin features. Proxy-layer features (sharing, audit, quotas) are handled separately and work regardless of whether Admin() returns nil.

type Backend 

type Backend interface {
	// Identity
	ID() string
	DisplayName() string
	Capabilities() Capabilities

	// Bucket ops
	ListBuckets(ctx context.Context) ([]Bucket, error)
	CreateBucket(ctx context.Context, name, region string) error
	DeleteBucket(ctx context.Context, name string) error
	HeadBucket(ctx context.Context, name string) (BucketInfo, error)

	// Bucket config
	GetBucketVersioning(ctx context.Context, bucket string) (bool, error)
	SetBucketVersioning(ctx context.Context, bucket string, enabled bool) error
	GetBucketCORS(ctx context.Context, bucket string) ([]CORSRule, error)
	SetBucketCORS(ctx context.Context, bucket string, rules []CORSRule) error
	GetBucketPolicy(ctx context.Context, bucket string) (string, error)
	SetBucketPolicy(ctx context.Context, bucket string, policy string) error
	DeleteBucketPolicy(ctx context.Context, bucket string) error
	GetBucketLifecycle(ctx context.Context, bucket string) ([]LifecycleRule, error)
	SetBucketLifecycle(ctx context.Context, bucket string, rules []LifecycleRule) error

	// Object ops
	ListObjects(ctx context.Context, req ListObjectsRequest) (ListObjectsResult, error)
	GetObject(ctx context.Context, bucket, key, versionID string, rng *Range) (ObjectReader, error)
	HeadObject(ctx context.Context, bucket, key, versionID string) (ObjectInfo, error)
	// ListObjectVersions returns versions and delete markers under keyPrefix.
	// Pass the exact object key as keyPrefix to list a single object's
	// history — callers must filter to exact key matches if they want to
	// avoid sibling-prefix contamination.
	ListObjectVersions(ctx context.Context, bucket, keyPrefix string) ([]ObjectVersion, error)
	PutObject(ctx context.Context, req PutObjectRequest) (ObjectInfo, error)
	DeleteObject(ctx context.Context, bucket, key, versionID string) error
	DeleteObjects(ctx context.Context, bucket string, keys []ObjectIdentifier) (DeleteObjectsResult, error)
	CopyObject(ctx context.Context, src, dst ObjectRef, metadata map[string]string) error

	// Object tags and user metadata. Drivers MAY return a "not supported"
	// error if the backend advertises Capabilities.Tagging == false.
	//
	// SetObjectTagging with an empty map removes all tags (on S3: mapped to
	// DeleteObjectTagging). UpdateObjectMetadata is an in-place rewrite of
	// the user metadata — on S3 it's a self-copy with MetadataDirective =
	// REPLACE, which creates a new version on versioned buckets.
	GetObjectTagging(ctx context.Context, bucket, key, versionID string) (map[string]string, error)
	SetObjectTagging(ctx context.Context, bucket, key, versionID string, tags map[string]string) error
	UpdateObjectMetadata(ctx context.Context, bucket, key string, metadata map[string]string) error

	// Multipart
	CreateMultipart(ctx context.Context, bucket, key, contentType string, metadata map[string]string) (uploadID string, err error)
	UploadPart(ctx context.Context, bucket, key, uploadID string, partNum int, r io.Reader, size int64) (etag string, err error)
	CompleteMultipart(ctx context.Context, bucket, key, uploadID string, parts []CompletedPart) (ObjectInfo, error)
	AbortMultipart(ctx context.Context, bucket, key, uploadID string) error
	ListMultipartUploads(ctx context.Context, bucket, prefix string) ([]MultipartUpload, error)

	// Presigned URLs
	PresignGet(ctx context.Context, bucket, key string, ttl time.Duration) (string, error)
	PresignPut(ctx context.Context, bucket, key string, ttl time.Duration, contentType string) (string, error)

	// Optional admin interface. May return nil.
	Admin() AdminBackend
}

Backend is the per-target S3-compatible driver.

type Bucket 

type Bucket struct {
	Name      string
	CreatedAt time.Time
}

type BucketInfo 

type BucketInfo struct {
	Name   string
	Region string
	Exists bool
}

type CORSRule 

type CORSRule struct {
	AllowedOrigins []string
	AllowedMethods []string
	AllowedHeaders []string
	ExposeHeaders  []string
	MaxAgeSeconds  int
}

type Capabilities 

type Capabilities struct {
	Versioning        bool `json:"versioning"`
	ObjectLock        bool `json:"object_lock"`
	Lifecycle         bool `json:"lifecycle"`
	BucketPolicy      bool `json:"bucket_policy"`
	CORS              bool `json:"cors"`
	Tagging           bool `json:"tagging"`
	ServerSideEncrypt bool `json:"server_side_encrypt"`
	// AdminAPI is "" when no backend-native admin API is available. Known
	// values: "minio", "garage", "seaweedfs".
	AdminAPI          string `json:"admin_api"`
	MaxMultipartParts int    `json:"max_multipart_parts"`
	MaxPartSizeBytes  int64  `json:"max_part_size_bytes"`
}

Capabilities advertises what a backend can do. The UI uses this to hide features the backend does not support.

type CompletedPart 

type CompletedPart struct {
	PartNumber int
	ETag       string
}

type DeleteError 

type DeleteError struct {
	Key       string `json:"key"`
	VersionID string `json:"version_id,omitempty"`
	Code      string `json:"code,omitempty"`
	Message   string `json:"message,omitempty"`
}

type DeleteObjectsResult 

type DeleteObjectsResult struct {
	Deleted []ObjectIdentifier `json:"deleted"`
	Errors  []DeleteError      `json:"errors,omitempty"`
}

type Entry 

type Entry struct {
	Backend Backend
	Source  Source
	Status  Status
}

Entry is the public view of a registered backend plus its status.

type LifecycleRule 

type LifecycleRule struct {
	ID                     string
	Prefix                 string
	Enabled                bool
	ExpirationDays         int
	NoncurrentExpireDays   int
	AbortIncompleteDays    int
	TransitionDays         int
	TransitionStorageClass string
}

type ListObjectsRequest 

type ListObjectsRequest struct {
	Bucket            string
	Prefix            string
	Delimiter         string
	ContinuationToken string
	MaxKeys           int
}

type ListObjectsResult 

type ListObjectsResult struct {
	Objects               []ObjectInfo
	CommonPrefixes        []string
	NextContinuationToken string
	IsTruncated           bool
}

type MultipartUpload 

type MultipartUpload struct {
	Key       string
	UploadID  string
	Initiated time.Time
}

type ObjectIdentifier 

type ObjectIdentifier struct {
	Key       string `json:"key"`
	VersionID string `json:"version_id,omitempty"`
}

type ObjectInfo 

type ObjectInfo struct {
	Key          string
	Size         int64
	ETag         string
	ContentType  string
	StorageClass string
	VersionID    string
	LastModified time.Time
	Metadata     map[string]string
}

type ObjectReader 

type ObjectReader interface {
	io.ReadCloser
	Info() ObjectInfo
}

ObjectReader is returned from GetObject. Callers must Close.

type ObjectRef 

type ObjectRef struct {
	Bucket    string
	Key       string
	VersionID string
}

type ObjectVersion 

type ObjectVersion struct {
	Key            string
	VersionID      string
	IsLatest       bool
	IsDeleteMarker bool
	Size           int64
	ETag           string
	StorageClass   string
	LastModified   time.Time
}

ObjectVersion describes one row from ListObjectVersions. Covers both regular versions and delete markers (IsDeleteMarker=true; Size/ETag zero).

type Policy 

type Policy struct {
	Name     string
	Document string // JSON
}

type ProbeRecord 

type ProbeRecord struct {
	At      time.Time
	Healthy bool
	Latency time.Duration
	Error   string
}

ProbeRecord is one entry of the per-backend probe-history ring.

type ProxyTarget 

type ProxyTarget struct {
	Endpoint  *url.URL
	Region    string
	PathStyle bool
	AccessKey string
	SecretKey string
}

ProxyTarget is the raw forwarding handle the S3 proxy needs to re-sign inbound tenant requests with admin credentials and dispatch them upstream. The Backend interface is intentionally high-level (Get/Put/List etc.); the proxy bypasses it because it must forward arbitrary S3 verbs, including operations the Backend interface does not model.

SecretKey is the unsealed admin secret. It lives in process memory only — the registry holds it after sealed-secret hydration at boot.

type ProxyTargetProvider 

type ProxyTargetProvider interface {
	ProxyTarget() (ProxyTarget, error)
}

ProxyTargetProvider is implemented by drivers whose backing store can be reached via raw S3-API forwarding. The s3v4 driver implements it; in-memory test drivers do not.

type PutObjectRequest 

type PutObjectRequest struct {
	Bucket      string
	Key         string
	Body        io.Reader
	Size        int64
	ContentType string
	Metadata    map[string]string
}

type Range 

type Range struct {
	Start, End int64 // inclusive. End<0 means "to the end".
}

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds the set of configured Backends, keyed by their stable ID, alongside a probe-driven health Status. It is safe for concurrent use.

The Get hot path is lock-free: it loads a snapshot of the (id → Backend) map via atomic.Pointer and indexes into it directly. Mutations (Register, Unregister, Replace) take the slow-path mutex, copy-mutate the map, and atomically publish the new pointer. Status / List / History reads still use the mutex because they access mutable per-entry fields.

func NewRegistry 

func NewRegistry() *Registry

func (*Registry) Get 

func (r *Registry) Get(id string) (Backend, bool)

func (*Registry) History 

func (r *Registry) History(id string) []ProbeRecord

History returns a copy of the probe history for one backend, oldest first. Empty when nothing has probed yet.

func (*Registry) List 

func (r *Registry) List() []Entry

List returns all registered backends paired with their last-known status, sorted by ID for deterministic output.

func (*Registry) ProbeAll 

func (r *Registry) ProbeAll(ctx context.Context, timeout time.Duration)

ProbeAll probes every registered backend in parallel and stores results + a history record per backend.

func (*Registry) ProxyTarget 

func (r *Registry) ProxyTarget(id string) (ProxyTarget, bool, error)

ProxyTarget returns the forwarding handle for a registered backend, or (zero, false) if no backend is registered under id, or if the registered backend's driver does not implement ProxyTargetProvider.

This is the only seam between the proxy and the registry. Keeping it scoped to a sibling interface (rather than baking ProxyTarget() into Backend itself) avoids forcing every driver — including in-memory test stubs — to model raw-forward semantics they don't have.

func (*Registry) Register 

func (r *Registry) Register(b Backend) error

Register adds a config-sourced backend (the historical default). Use RegisterWithSource for DB-sourced entries so the admin API can tell them apart later.

func (*Registry) RegisterWithSource 

func (r *Registry) RegisterWithSource(b Backend, source Source) error

RegisterWithSource adds a backend tagged with its origin. Returns an error if the ID is empty, nil, or already registered.

func (*Registry) Replace 

func (r *Registry) Replace(id string, b Backend) error

Replace swaps the Backend for an existing id while preserving its Source. Status and probe history are reset because the new client may point at a different endpoint or use new credentials, making historical readings misleading. Returns ErrNotRegistered if the id is unknown, or an error if the new backend's ID doesn't match.

func (*Registry) SetStatus 

func (r *Registry) SetStatus(id string, s Status)

func (*Registry) Source 

func (r *Registry) Source(id string) (Source, bool)

Source reports how the entry was registered. Returns ("", false) when the id is unknown.

func (*Registry) Status 

func (r *Registry) Status(id string) (Status, bool)

func (*Registry) Unregister 

func (r *Registry) Unregister(id string) error

Unregister removes a backend from the registry, dropping its status and probe history. Returns ErrNotRegistered if the id was unknown.

type Source 

type Source string

Source distinguishes how a backend got into the registry. The admin API uses this to gate edits — only SourceDB entries can be modified through the UI; SourceConfig entries are owned by config.yaml and read-only. 

const (
	SourceConfig Source = "config"
	SourceDB     Source = "db"
)

type Status 

type Status struct {
	Healthy     bool
	LastProbeAt time.Time
	LastError   string
	LastLatency time.Duration
}

Status is the last-known probe result for a backend.

func Probe 

func Probe(ctx context.Context, b Backend, timeout time.Duration) Status

Probe runs ListBuckets against one backend with a timeout. It does not modify registry state — callers typically feed the result into SetStatus.

type User 

type User struct {
	Name      string
	Enabled   bool
	CreatedAt time.Time
}

Source Files

View all Source files

Directories

Path Synopsis
Package memory is an in-process Backend implementation for tests.
 Package memory is an in-process Backend implementation for tests.
Package s3v4 is the generic AWS SDK v2 driver.
 Package s3v4 is the generic AWS SDK v2 driver.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.