stores

package module
v0.1.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2026 License: MIT Imports: 4 Imported by: 0

README

stores

Unified object storage abstraction across 9 backends with zero cloud SDK dependencies in the core module.

Structure

stores/
├── stores.go          # Core: Store interface, StoreError, sentinel errors
├── private.go         # PrivateURLSigner, DirectUploadPresigner, helpers
├── local/             # Local filesystem (no cloud SDK)
├── s3/                # AWS S3 / S3-compatible
├── oss/               # Alibaba Cloud OSS
├── cos/               # Tencent Cloud COS
├── minio/             # MinIO
├── kodo/              # Qiniu Kodo
├── tos/               # Volcengine TOS
├── obs/               # Huawei Cloud OBS (S3-compatible)
└── ks3/               # Kingsoft Cloud KS3

Design: Per-Provider Modules

Each provider is a separate Go module with its own go.mod. Applications only import the cloud SDK they actually use — no transitive dependencies from providers you don't need.

# Only need S3?
go get github.com/LingByte/ling-base/stores/s3

# Only need Alibaba Cloud OSS?
go get github.com/LingByte/ling-base/stores/oss

# Only need local filesystem (no cloud SDK at all)?
go get github.com/LingByte/ling-base/stores/local

The core stores package has zero cloud SDK dependencies — it only defines the Store interface, error types, and signing/presign helpers.

Quick Start

Local Filesystem
import "github.com/LingByte/ling-base/stores/local"

s := local.New(local.Config{
    Root:       "/var/uploads",
    NewDirPerm: 0755,
})

// Write
s.Write("photos/cat.jpg", file)

// Read
r, size, err := s.Read("photos/cat.jpg")
defer r.Close()

// Delete
s.Delete("photos/cat.jpg")

// Exists
ok, _ := s.Exists("photos/cat.jpg")
Amazon S3
import "github.com/LingByte/ling-base/stores/s3"

s := s3.New(s3.Config{
    Region:          "us-east-1",
    AccessKeyID:     "your-access-key-id",
    AccessKeySecret: "your-secret-access-key",
    BucketName:      "my-bucket",
})

s.Write("file.txt", reader)
r, size, _ := s.Read("file.txt")
Alibaba Cloud OSS
import "github.com/LingByte/ling-base/stores/oss"

s := oss.New(oss.Config{
    AccessKeyID:     "your-access-key-id",
    AccessKeySecret: "your-access-key-secret",
    Endpoint:        "oss-cn-hangzhou.aliyuncs.com",
    BucketName:      "my-bucket",
})
Tencent Cloud COS
import "github.com/LingByte/ling-base/stores/cos"

s := cos.New(cos.Config{
    SecretID:   "your-secret-id",
    SecretKey:  "your-secret-key",
    Region:     "ap-guangzhou",
    BucketName: "my-bucket-1250000000",
})
MinIO
import "github.com/LingByte/ling-base/stores/minio"

s := minio.New(minio.Config{
    Endpoint:  "minio.local:9000",
    AccessKey: "minioadmin",
    SecretKey: "minioadmin",
    Bucket:    "my-bucket",
    UseSSL:    false,
})
Qiniu Kodo
import "github.com/LingByte/ling-base/stores/kodo"

s := kodo.New(kodo.Config{
    AccessKey:  "your-access-key",
    SecretKey:  "your-secret-key",
    BucketName: "my-bucket",
    Domain:     "https://cdn.example.com",
    Private:    true,
})
Volcengine TOS
import "github.com/LingByte/ling-base/stores/tos"

s := tos.New(tos.Config{
    Endpoint:        "https://tos-cn-beijing.volces.com",
    Region:          "cn-beijing",
    AccessKeyID:     "your-access-key-id",
    AccessKeySecret: "your-access-key-secret",
    BucketName:      "my-bucket",
})
Huawei Cloud OBS
import "github.com/LingByte/ling-base/stores/obs"

s := obs.New(obs.Config{
    Endpoint:        "https://obs.cn-north-4.myhuaweicloud.com",
    Region:          "cn-north-4",
    AccessKeyID:     "your-access-key-id",
    AccessKeySecret: "your-access-key-secret",
    BucketName:      "my-bucket",
})
Kingsoft Cloud KS3
import "github.com/LingByte/ling-base/stores/ks3"

s := ks3.New(ks3.Config{
    Endpoint:        "https://ks3-cn-beijing.ksyuncs.com",
    Region:          "BEIJING",
    AccessKeyID:     "your-access-key-id",
    AccessKeySecret: "your-access-key-secret",
    BucketName:      "my-bucket",
})

Store Interface

Every backend implements the same 5-method interface:

type Store interface {
    Read(key string) (io.ReadCloser, int64, error)
    Write(key string, r io.Reader) error
    Delete(key string) error
    Exists(key string) (bool, error)
    PublicURL(key string) string
}

Private URLs & Direct Upload

Backends that support private buckets implement PrivateURLSigner:

if signer, ok := s.(stores.PrivateURLSigner); ok {
    url, _ := signer.SignedURL("file.txt", time.Hour)
}

Or use the helper:

url, err := stores.SignedURL(s, "file.txt", time.Hour)

Backends that support client direct upload implement DirectUploadPresigner:

du, err := stores.PresignUpload(s, "file.txt", "image/jpeg", time.Hour)
// du.Method, du.URL, du.Headers, du.Form, du.FileField

Configuration

All configuration is explicit — no environment variables are read inside the library. Each provider has its own Config struct with only the fields it needs.

Testing

# Core package (no cloud SDK needed)
cd stores && go test -cover

# Per-provider modules
cd stores/local && go test -cover
cd stores/s3    && go test -cover
cd stores/oss   && go test -cover
# ... etc

Coverage notes:

  • stores (core): ~90%
  • stores/local: ~81% (full filesystem CRUD tested)
  • Cloud providers: constructor + PublicURL paths tested locally; actual cloud API calls require real credentials and are not unit-tested.

Documentation

Overview

Package stores provides a unified storage abstraction layer supporting multiple cloud object storage backends and local file system storage.

All backends implement the Store interface with five operations: Read, Write, Delete, Exists, and PublicURL. Configuration is injected explicitly via provider-specific Config structs — no environment variables are read inside the library.

Each provider lives in its own Go module so applications only import the cloud SDK they actually use: 

stores/local   - Local filesystem (no cloud SDK)
stores/s3      - AWS S3 / S3 compatible
stores/oss     - Alibaba Cloud OSS
stores/cos     - Tencent Cloud COS
stores/minio   - MinIO / S3 compatible
stores/kodo    - Qiniu Kodo
stores/tos     - Volcengine TOS
stores/obs     - Huawei Cloud OBS
stores/ks3     - Kingsoft Cloud KS3

Index

Constants

View Source 
const (
	DefaultSignedURLTTL    = 1 * time.Hour
	DefaultDirectUploadTTL = 1 * time.Hour
	MaxPresignTTL          = 24 * time.Hour
)

TTL bounds for signed URLs / direct-upload credentials. Callers passing a non-positive TTL get the default; anything above the max is clamped so a bad caller can never mint week-long private links by accident.

View Source 
const DefaultUploadDir = "uploads"

DefaultUploadDir is the default local upload directory.

Variables

View Source 
var (
	// ErrInvalidPath is returned when a key resolves outside the allowed
	// root directory (path traversal) or is otherwise malformed.
	ErrInvalidPath = errors.New("invalid storage path")

	// ErrAttachmentNotExist is returned when an object is not found.
	ErrAttachmentNotExist = errors.New("attachment does not exist")
)

Sentinel errors.

View Source 
var ErrDirectUploadUnsupported = &StoreError{Code: http.StatusNotImplemented, Message: "direct upload not supported by this storage backend"}

ErrDirectUploadUnsupported is returned by PresignUpload when the configured backend cannot issue client direct-upload credentials (e.g. local disk).

View Source 
var ErrStatsUnsupported = &StoreError{Code: 501, Message: "statistics not supported by this storage backend"}

ErrStatsUnsupported is returned when the backend cannot provide the requested statistics dimension (e.g. local filesystem has no CDN).

Functions

func SignedURL 

func SignedURL(s Store, key string, expires time.Duration) (string, error)

SignedURL returns an expiring access URL for key. Stores without private signing (e.g. local disk, public buckets) fall back to PublicURL.

func SupportsManagement added in v0.1.1 

func SupportsManagement(s Store) bool

SupportsManagement reports whether the store implements ObjectStorageManager.

func SupportsMultipart added in v0.1.1 

func SupportsMultipart(s Store) bool

SupportsMultipart reports whether the store implements MultipartUploader.

func SupportsStats added in v0.1.2 

func SupportsStats(s Store) bool

SupportsStats reports whether the store implements StorageStatsProvider.

Types

type APIStatsPoint added in v0.1.2 

type APIStatsPoint struct {
	Timestamp      time.Time `json:"timestamp"`
	TotalRequests  int64     `json:"totalRequests"`
	GetRequests    int64     `json:"getRequests"`
	PutRequests    int64     `json:"putRequests"`
	DeleteRequests int64     `json:"deleteRequests"`
	HeadRequests   int64     `json:"headRequests"`
	UploadBytes    int64     `json:"uploadBytes"`   // bytes uploaded
	DownloadBytes  int64     `json:"downloadBytes"` // bytes downloaded
	ErrorRequests  int64     `json:"errorRequests"` // 4xx+5xx
}

APIStatsPoint is a single time-series data point for API stats.

type APIStatsRequest added in v0.1.2 

type APIStatsRequest struct {
	Bucket      string      `json:"bucket,omitempty"`
	Range       TimeRange   `json:"range"`
	Granularity Granularity `json:"granularity,omitempty"`
}

APIStatsRequest holds parameters for querying API request statistics.

type APIStatsResponse added in v0.1.2 

type APIStatsResponse struct {
	Points  []APIStatsPoint `json:"points"`
	Summary APIStatsSummary `json:"summary"`
}

APIStatsResponse holds API request statistics time-series data.

func GetAPIRequestStats added in v0.1.2 

func GetAPIRequestStats(s Store, req *APIStatsRequest) (*APIStatsResponse, error)

GetAPIRequestStats is a convenience helper for API request statistics.

type APIStatsSummary added in v0.1.2 

type APIStatsSummary struct {
	TotalRequests int64   `json:"totalRequests"`
	UploadBytes   int64   `json:"uploadBytes"`
	DownloadBytes int64   `json:"downloadBytes"`
	ErrorRequests int64   `json:"errorRequests"`
	ErrorRate     float64 `json:"errorRate"` // 0..1
}

APIStatsSummary aggregates API stats over the full query range.

type BucketInfo added in v0.1.1 

type BucketInfo struct {
	Name         string            `json:"name"`         // bucket name
	Region       string            `json:"region"`       // bucket region
	CreatedAt    time.Time         `json:"createdAt"`    // creation time
	IsPrivate    bool              `json:"isPrivate"`    // whether the bucket is private
	Domains      []string          `json:"domains"`      // bound domain names
	Tags         map[string]string `json:"tags"`         // bucket tags
	StorageClass string            `json:"storageClass"` // storage class (e.g. STANDARD)
	Versioning   bool              `json:"versioning"`   // whether versioning is enabled
}

BucketInfo holds metadata about a storage bucket.

type BucketStats added in v0.1.2 

type BucketStats struct {
	Bucket         string              `json:"bucket"`
	Region         string              `json:"region,omitempty"`
	Size           int64               `json:"size"`        // total storage in bytes
	ObjectCount    int64               `json:"objectCount"` // number of objects
	UpdatedAt      time.Time           `json:"updatedAt"`   // when the snapshot was taken
	StorageClasses []StorageClassUsage `json:"storageClasses,omitempty"`
}

BucketStats is the current storage usage snapshot for a bucket.

func GetBucketStats added in v0.1.2 

func GetBucketStats(s Store, bucket string) (*BucketStats, error)

GetBucketStats is a convenience helper that calls GetBucketStats on the store if it implements StorageStatsProvider, otherwise returns ErrStatsUnsupported.

type CDNStatsPoint added in v0.1.2 

type CDNStatsPoint struct {
	Timestamp    time.Time `json:"timestamp"`
	Traffic      int64     `json:"traffic"`      // bytes served in this interval
	Bandwidth    float64   `json:"bandwidth"`    // peak bandwidth in bps
	Requests     int64     `json:"requests"`     // total requests
	HitRequests  int64     `json:"hitRequests"`  // cache hit requests
	MissRequests int64     `json:"missRequests"` // cache miss requests
}

CDNStatsPoint is a single time-series data point for CDN stats.

type CDNStatsRequest added in v0.1.2 

type CDNStatsRequest struct {
	Bucket      string      `json:"bucket,omitempty"`      // filter by bucket (some providers)
	Domains     []string    `json:"domains,omitempty"`     // filter by CDN domain(s)
	Range       TimeRange   `json:"range"`                 // query interval
	Granularity Granularity `json:"granularity,omitempty"` // aggregation interval
}

CDNStatsRequest holds parameters for querying CDN statistics.

type CDNStatsResponse added in v0.1.2 

type CDNStatsResponse struct {
	Domains []string        `json:"domains,omitempty"`
	Points  []CDNStatsPoint `json:"points"`
	Summary CDNStatsSummary `json:"summary"`
}

CDNStatsResponse holds CDN statistics time-series data.

func GetCDNStats added in v0.1.2 

func GetCDNStats(s Store, req *CDNStatsRequest) (*CDNStatsResponse, error)

GetCDNStats is a convenience helper for CDN statistics.

type CDNStatsSummary added in v0.1.2 

type CDNStatsSummary struct {
	TotalTraffic     int64         `json:"totalTraffic"`          // total bytes
	TotalRequests    int64         `json:"totalRequests"`         // total requests
	TotalHitRequests int64         `json:"totalHitRequests"`      // total cache hits
	HitRatio         float64       `json:"hitRatio"`              // 0..1
	AvgBandwidth     float64       `json:"avgBandwidth"`          // average bandwidth in bps
	PeakBandwidth    float64       `json:"peakBandwidth"`         // peak bandwidth in bps
	StatusCodes      map[int]int64 `json:"statusCodes,omitempty"` // HTTP status code → count
}

CDNStatsSummary aggregates CDN stats over the full query range.

type CompleteMultipartUploadRequest added in v0.1.1 

type CompleteMultipartUploadRequest struct {
	UploadID string          `json:"uploadId"`
	Parts    []CompletedPart `json:"parts"`
}

CompleteMultipartUploadRequest holds parameters for completing a multipart upload.

type CompleteMultipartUploadResponse added in v0.1.1 

type CompleteMultipartUploadResponse struct {
	Location string `json:"location"`
	Bucket   string `json:"bucket"`
	Key      string `json:"key"`
	ETag     string `json:"etag"`
}

CompleteMultipartUploadResponse holds the result of completing a multipart upload.

type CompletedPart added in v0.1.1 

type CompletedPart struct {
	PartNumber int    `json:"partNumber"`
	ETag       string `json:"etag"`
}

CompletedPart represents a single completed part in a multipart upload.

type CopyObjectRequest added in v0.1.1 

type CopyObjectRequest struct {
	SrcBucket   string            `json:"srcBucket"`
	SrcKey      string            `json:"srcKey"`
	DestBucket  string            `json:"destBucket"`
	DestKey     string            `json:"destKey"`
	ContentType string            `json:"contentType,omitempty"`
	Metadata    map[string]string `json:"metadata,omitempty"`
}

CopyObjectRequest holds parameters for copying an object.

type CreateBucketRequest added in v0.1.1 

type CreateBucketRequest struct {
	Name         string            `json:"name"`
	Region       string            `json:"region,omitempty"`
	StorageClass string            `json:"storageClass,omitempty"`
	Tags         map[string]string `json:"tags,omitempty"`
	IsPrivate    bool              `json:"isPrivate"`
}

CreateBucketRequest holds parameters for creating a bucket.

type DirectUpload 

type DirectUpload struct {
	Provider  string            `json:"provider"`
	Method    string            `json:"method"`
	URL       string            `json:"url"`
	Headers   map[string]string `json:"headers,omitempty"`
	Form      map[string]string `json:"form,omitempty"`
	FileField string            `json:"fileField,omitempty"`
	Key       string            `json:"key"`
	ExpiresAt time.Time         `json:"expiresAt"`
}

DirectUpload describes how a client must perform a direct upload. Method "PUT" → send the raw body to URL with Headers. Method "POST" → send multipart/form-data to URL with Form fields plus the file under FileField (Qiniu-style form upload).

func PresignUpload 

func PresignUpload(s Store, key, contentType string, expires time.Duration) (*DirectUpload, error)

PresignUpload issues client direct-upload credentials for key, or ErrDirectUploadUnsupported when the backend cannot (callers then use the regular server-side upload path).

type DirectUploadPresigner 

type DirectUploadPresigner interface {
	// PresignUpload issues one-shot upload credentials for key. contentType
	// may be empty; when set, backends that sign it require the client to
	// send the same Content-Type header.
	PresignUpload(key, contentType string, expires time.Duration) (*DirectUpload, error)
}

DirectUploadPresigner is implemented by stores that support client direct upload (browser/device → storage) without proxying the object bytes through this server.

type FileInfo added in v0.1.1 

type FileInfo struct {
	Key          string            `json:"key"`          // object key
	Size         int64             `json:"size"`         // object size in bytes
	LastModified time.Time         `json:"lastModified"` // last modification time
	ETag         string            `json:"etag"`         // ETag / hash
	ContentType  string            `json:"contentType"`  // MIME type
	Metadata     map[string]string `json:"metadata"`     // user-defined metadata
	StorageClass string            `json:"storageClass"` // storage class
	VersionID    string            `json:"versionId"`    // version ID (if versioning enabled)
	IsLatest     bool              `json:"isLatest"`     // whether this is the latest version
	PublicURL    string            `json:"publicURL"`    // public access URL
}

FileInfo holds metadata about a stored object.

type Granularity added in v0.1.2 

type Granularity string

Granularity is the aggregation interval for time-series data points. 

const (
	Granularity5Min  Granularity = "5min"  // 5-minute buckets
	GranularityHour  Granularity = "hour"  // hourly buckets
	GranularityDay   Granularity = "day"   // daily buckets
	GranularityMonth Granularity = "month" // monthly buckets
)

type InitiateMultipartUploadRequest added in v0.1.1 

type InitiateMultipartUploadRequest struct {
	Key         string            `json:"key"`
	ContentType string            `json:"contentType,omitempty"`
	Metadata    map[string]string `json:"metadata,omitempty"`
}

InitiateMultipartUploadRequest holds parameters for initiating a multipart upload.

type InitiateMultipartUploadResponse added in v0.1.1 

type InitiateMultipartUploadResponse struct {
	UploadID string `json:"uploadId"`
	Key      string `json:"key"`
}

InitiateMultipartUploadResponse holds the result of initiating a multipart upload.

type ListBucketsRequest added in v0.1.1 

type ListBucketsRequest struct {
	Region  string `json:"region,omitempty"`  // filter by region
	Prefix  string `json:"prefix,omitempty"`  // filter by name prefix
	MaxKeys int    `json:"maxKeys,omitempty"` // max buckets to return (0 = all)
}

ListBucketsRequest holds parameters for listing buckets.

type ListBucketsResponse added in v0.1.1 

type ListBucketsResponse struct {
	Buckets     []BucketInfo `json:"buckets"`
	IsTruncated bool         `json:"isTruncated"`
	NextMarker  string       `json:"nextMarker,omitempty"`
}

ListBucketsResponse holds the result of listing buckets.

type ListFilesRequest added in v0.1.1 

type ListFilesRequest struct {
	Prefix    string `json:"prefix,omitempty"`    // prefix filter
	Marker    string `json:"marker,omitempty"`    // pagination marker
	Limit     int    `json:"limit,omitempty"`     // max objects to return
	Delimiter string `json:"delimiter,omitempty"` // directory delimiter (e.g. "/")
}

ListFilesRequest holds parameters for listing objects in a bucket.

type ListFilesResponse added in v0.1.1 

type ListFilesResponse struct {
	Files          []FileInfo `json:"files"`
	Marker         string     `json:"marker,omitempty"`
	IsTruncated    bool       `json:"isTruncated"`
	CommonPrefixes []string   `json:"commonPrefixes,omitempty"` // virtual directories
}

ListFilesResponse holds the result of listing objects.

type ListPartsRequest added in v0.1.1 

type ListPartsRequest struct {
	UploadID         string `json:"uploadId"`
	MaxParts         int    `json:"maxParts,omitempty"`
	PartNumberMarker int    `json:"partNumberMarker,omitempty"`
}

ListPartsRequest holds parameters for listing parts of a multipart upload.

type ListPartsResponse added in v0.1.1 

type ListPartsResponse struct {
	Bucket               string          `json:"bucket"`
	Key                  string          `json:"key"`
	UploadID             string          `json:"uploadId"`
	MaxParts             int             `json:"maxParts"`
	IsTruncated          bool            `json:"isTruncated"`
	PartNumberMarker     int             `json:"partNumberMarker"`
	NextPartNumberMarker int             `json:"nextPartNumberMarker"`
	Parts                []CompletedPart `json:"parts"`
}

ListPartsResponse holds the result of listing parts.

type MultipartUploader added in v0.1.1 

type MultipartUploader interface {
	// InitiateMultipartUpload starts a multipart upload and returns
	// an upload ID.
	InitiateMultipartUpload(bucket string, req *InitiateMultipartUploadRequest) (*InitiateMultipartUploadResponse, error)

	// UploadPart uploads a single part of a multipart upload.
	UploadPart(bucket, key string, req *UploadPartRequest) (*UploadPartResponse, error)

	// CompleteMultipartUpload finalizes a multipart upload by combining
	// all uploaded parts.
	CompleteMultipartUpload(bucket, key string, req *CompleteMultipartUploadRequest) (*CompleteMultipartUploadResponse, error)

	// AbortMultipartUpload cancels a multipart upload and discards
	// uploaded parts.
	AbortMultipartUpload(bucket, key, uploadID string) error

	// ListParts lists the parts that have been uploaded for a multipart
	// upload.
	ListParts(bucket, key string, req *ListPartsRequest) (*ListPartsResponse, error)
}

MultipartUploader is implemented by backends that support multipart upload for large files. This allows uploading files in parts, resuming interrupted uploads, and parallel part uploads.

func AsMultipartUploader added in v0.1.1 

func AsMultipartUploader(s Store) MultipartUploader

AsMultipartUploader returns the given store as a MultipartUploader, or nil if the store does not support multipart upload.

type ObjectStorageManager added in v0.1.1 

type ObjectStorageManager interface {
	Store

	// ListBuckets returns all buckets the configured credentials can
	// access, optionally filtered by the request parameters.
	ListBuckets(req *ListBucketsRequest) (*ListBucketsResponse, error)

	// CreateBucket creates a new bucket.
	CreateBucket(req *CreateBucketRequest) error

	// DeleteBucket deletes an empty bucket. Returns an error if the
	// bucket is not empty or does not exist.
	DeleteBucket(bucket string) error

	// GetBucketInfo returns metadata about a bucket.
	GetBucketInfo(bucket string) (*BucketInfo, error)

	// SetBucketPrivate sets the access control of a bucket.
	SetBucketPrivate(bucket string, isPrivate bool) error

	// GetBucketDomains returns the domain names bound to a bucket.
	GetBucketDomains(bucket string) ([]string, error)

	// ListFiles lists objects in a bucket with optional prefix,
	// delimiter, and pagination.
	ListFiles(bucket string, req *ListFilesRequest) (*ListFilesResponse, error)

	// GetFileInfo returns metadata for a single object.
	GetFileInfo(bucket, key string) (*FileInfo, error)

	// UploadFile uploads data to a bucket. Unlike Write, this method
	// accepts an explicit size hint and bucket parameter.
	UploadFile(bucket, key string, reader io.Reader, size int64) error

	// DeleteFile deletes an object from a bucket.
	DeleteFile(bucket, key string) error

	// CopyFile copies an object within or across buckets.
	CopyFile(req *CopyObjectRequest) error

	// MoveFile moves (renames) an object within or across buckets.
	MoveFile(req *CopyObjectRequest) error

	// GetFileURL returns an expiring URL for accessing a (possibly
	// private) object.
	GetFileURL(bucket, key string, expires time.Duration) (string, error)
}

ObjectStorageManager extends Store with administrative operations for managing buckets, listing objects with metadata, copying/moving objects, and generating expiring URLs. Not all backends implement every operation — callers should check with a type assertion or use the helper functions.

The bucket parameter may be empty for backends that operate on a single pre-configured bucket (e.g. the local store or a single-bucket cloud config); in that case the backend uses its default bucket.

func AsManager added in v0.1.1 

func AsManager(s Store) ObjectStorageManager

AsManager returns the given store as an ObjectStorageManager, or nil if the store does not implement the management interface.

type OriginStatsPoint added in v0.1.2 

type OriginStatsPoint struct {
	Timestamp      time.Time `json:"timestamp"`
	OriginTraffic  int64     `json:"originTraffic"`  // bytes pulled from origin
	OriginRequests int64     `json:"originRequests"` // requests to origin
	FailedRequests int64     `json:"failedRequests"` // failed origin requests
}

OriginStatsPoint is a single time-series data point for origin-fetch stats.

type OriginStatsRequest added in v0.1.2 

type OriginStatsRequest struct {
	Bucket      string      `json:"bucket,omitempty"`
	Domains     []string    `json:"domains,omitempty"`
	Range       TimeRange   `json:"range"`
	Granularity Granularity `json:"granularity,omitempty"`
}

OriginStatsRequest holds parameters for querying origin-fetch statistics.

type OriginStatsResponse added in v0.1.2 

type OriginStatsResponse struct {
	Points  []OriginStatsPoint `json:"points"`
	Summary OriginStatsSummary `json:"summary"`
}

OriginStatsResponse holds origin-fetch statistics time-series data.

func GetOriginFetchStats added in v0.1.2 

func GetOriginFetchStats(s Store, req *OriginStatsRequest) (*OriginStatsResponse, error)

GetOriginFetchStats is a convenience helper for origin-fetch statistics.

type OriginStatsSummary added in v0.1.2 

type OriginStatsSummary struct {
	TotalOriginTraffic  int64   `json:"totalOriginTraffic"`
	TotalOriginRequests int64   `json:"totalOriginRequests"`
	TotalFailedRequests int64   `json:"totalFailedRequests"`
	FailureRate         float64 `json:"failureRate"` // 0..1
}

OriginStatsSummary aggregates origin-fetch stats over the full query range.

type PrivateURLSigner 

type PrivateURLSigner interface {
	SignedURL(key string, expires time.Duration) (string, error)
}

PrivateURLSigner is implemented by stores that can mint expiring signed GET URLs for objects in a private bucket.

type StorageClassUsage added in v0.1.2 

type StorageClassUsage struct {
	Class       string `json:"class"`       // storage class name
	Size        int64  `json:"size"`        // bytes in this class
	ObjectCount int64  `json:"objectCount"` // objects in this class
}

StorageClassUsage breaks down storage by class (e.g. STANDARD, IA, ARCHIVE).

type StorageStatsProvider added in v0.1.2 

type StorageStatsProvider interface {

	// GetBucketStats returns the current storage usage snapshot for a
	// bucket: total size, object count, and storage-class distribution.
	GetBucketStats(bucket string) (*BucketStats, error)

	// GetCDNStats returns CDN traffic, bandwidth, request, and cache-hit
	// statistics over the given time range. If the backend has no CDN
	// integration it returns ErrStatsUnsupported.
	GetCDNStats(req *CDNStatsRequest) (*CDNStatsResponse, error)

	// GetAPIRequestStats returns API call counts, upload/download traffic,
	// and error rates over the given time range.
	GetAPIRequestStats(req *APIStatsRequest) (*APIStatsResponse, error)

	// GetOriginFetchStats returns origin-pull traffic, request counts, and
	// failure rates over the given time range. If the backend has no CDN
	// it returns ErrStatsUnsupported.
	GetOriginFetchStats(req *OriginStatsRequest) (*OriginStatsResponse, error)
}

StorageStatsProvider is implemented by backends that can report storage, CDN, API request, and origin-fetch statistics. Not all backends support every dimension — callers should check with SupportsStats or use the helper functions.

Methods that accept a TimeRange return time-series data points; methods without a time range return the current snapshot.

func AsStatsProvider added in v0.1.2 

func AsStatsProvider(s Store) StorageStatsProvider

AsStatsProvider returns the given store as a StorageStatsProvider, or nil if the store does not implement the statistics interface.

type Store 

type Store interface {
	// Read returns an io.ReadCloser for the object at key, plus its size.
	Read(key string) (io.ReadCloser, int64, error)
	// Write stores the contents of r under key.
	Write(key string, r io.Reader) error
	// Delete removes the object at key. No error is returned if the key
	// does not exist.
	Delete(key string) error
	// Exists reports whether an object exists at key.
	Exists(key string) (bool, error)
	// PublicURL returns a publicly accessible URL for key, or "" if the
	// backend cannot construct one.
	PublicURL(key string) string
}

Store is the unified object storage interface. All backends implement it.

type StoreError 

type StoreError struct {
	Code    int    // HTTP status code or 0
	Message string // human-readable message
}

StoreError is a structured error returned by store backends.

func (*StoreError) Error 

func (e *StoreError) Error() string

type TimeRange added in v0.1.2 

type TimeRange struct {
	Start time.Time `json:"start"` // inclusive
	End   time.Time `json:"end"`   // inclusive
}

TimeRange specifies a query interval for time-series statistics.

type UploadPartRequest added in v0.1.1 

type UploadPartRequest struct {
	UploadID   string    `json:"uploadId"`
	PartNumber int       `json:"partNumber"`
	Body       io.Reader `json:"-"`
}

UploadPartRequest holds parameters for uploading a single part.

type UploadPartResponse added in v0.1.1 

type UploadPartResponse struct {
	ETag string `json:"etag"`
}

UploadPartResponse holds the result of uploading a part.

Source Files

View all Source files

Directories

Path Synopsis
cos module
kodo module
ks3 module
local module
minio module
obs module
oss module
s3 module
tos module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.