Documentation
¶
Overview ¶
AWS S3 driver (provider: aws). Uses aws-sdk-go-v2/service/s3 under the hood.
AWS client construction (provider: aws).
Microsoft Azure Blob driver (provider: azure).
Tencent COS / WeChat COS driver (providers: tencent, wechat).
Package upload provides a thin, testable object storage client for major cloud providers.
Design constraints (for use in application frameworks):
- No viper, no global singletons, no log.Fatal inside the library.
- Storage interface covers upload, download, list, delete, and multipart transfers.
- One Storage per process; reuse it across goroutines.
- Call Close on shutdown.
Supported providers: Aliyun OSS, Tencent COS, WeChat (COS config), AWS S3, Azure Blob.
Aliyun OSS driver (provider: aliyun).
Index ¶
- Constants
- Variables
- func PublicURLFromHost(host, key string) string
- func SetMemObject(s Storage, key string, data []byte) error
- type AWSConfig
- type AliyunConfig
- type AzureConfig
- type COSConfig
- type Config
- type GetOption
- type ListOption
- type ListPage
- type MultipartConfig
- type ObjectInfo
- type ProviderSettings
- type PutOption
- type RetryConfig
- type Storage
- type Uploader
Examples ¶
Constants ¶
const ( ProviderAliyun = "aliyun" ProviderTencent = "tencent" ProviderWechat = "wechat" ProviderAWS = "aws" ProviderAzure = "azure" )
Variables ¶
var ( ErrInvalidConfig = errors.New("upload: invalid config") ErrNotInitialized = errors.New("upload: not initialized") ErrNotFound = errors.New("upload: object not found") ErrUnsupported = errors.New("upload: unsupported operation") ErrObjectTooLarge = errors.New("upload: object too large for simple upload") // ErrInvalidRange is returned for negative offsets, non-positive lengths, or offset+length overflow. ErrInvalidRange = errors.New("upload: invalid range") )
Functions ¶
func PublicURLFromHost ¶
PublicURLFromHost joins CDN host and object key.
Types ¶
type AWSConfig ¶
type AWSConfig struct {
// Region is the AWS region (e.g. us-east-1).
Region string
// AccessKeyID is the AWS access key ID.
AccessKeyID string
// SecretAccessKey is the AWS secret access key.
SecretAccessKey string
// SessionToken is the optional STS session token.
SessionToken string
// Bucket is the S3 bucket name.
Bucket string
// Endpoint is an optional custom S3-compatible endpoint.
Endpoint string
// CDNHost is the optional CDN base URL for public object URLs.
CDNHost string
}
AWSConfig is Amazon S3 settings.
type AliyunConfig ¶
type AliyunConfig struct {
// Endpoint is the OSS endpoint URL.
Endpoint string
// AccessKeyID is the Aliyun access key ID.
AccessKeyID string
// AccessKeySecret is the Aliyun access key secret.
AccessKeySecret string
// Bucket is the target OSS bucket name.
Bucket string
// CDNHost is the optional CDN base URL for public object URLs.
CDNHost string
}
AliyunConfig is Aliyun OSS settings.
type AzureConfig ¶
type AzureConfig struct {
// AccountName is the storage account name.
AccountName string
// AccountKey is the storage account access key.
AccountKey string
// Container is the blob container name.
Container string
// Endpoint is an optional custom blob endpoint.
Endpoint string
// CDNHost is the optional CDN base URL for public object URLs.
CDNHost string
}
AzureConfig is Azure Blob settings.
type COSConfig ¶
type COSConfig struct {
// Region is the COS region (e.g. ap-shanghai).
Region string
// SecretID is the Tencent Cloud secret ID.
SecretID string
// SecretKey is the Tencent Cloud secret key.
SecretKey string
// Bucket is the bucket name without app ID suffix.
Bucket string
// AppID is the Tencent Cloud app ID appended to the bucket.
AppID string
// CDNHost is the optional CDN base URL for public object URLs.
CDNHost string
// BaseURL is an optional custom API base URL.
BaseURL string
}
COSConfig is Tencent/WeChat COS settings.
type Config ¶
type Config struct {
// Provider selects the active storage backend.
Provider ProviderSettings
// Aliyun holds Aliyun OSS credentials when Provider is aliyun.
Aliyun AliyunConfig
// Tencent holds Tencent COS credentials when Provider is tencent.
Tencent COSConfig
// Wechat holds WeChat COS credentials when Provider is wechat.
Wechat COSConfig
// AWS holds Amazon S3 credentials when Provider is aws.
AWS AWSConfig
// Azure holds Azure Blob credentials when Provider is azure.
Azure AzureConfig
// Multipart tunes chunked upload/download behavior.
Multipart MultipartConfig
// Retry controls startup connectivity retry.
Retry RetryConfig
}
Config holds provider credentials and tuning.
func ConfigDefault ¶
func ConfigDefault() Config
ConfigDefault returns tuning defaults (provider block must still be set).
type GetOption ¶
type GetOption func(*getOptions)
GetOption configures downloads.
func GetWithMultipart ¶
func GetWithMultipart() GetOption
GetWithMultipart forces ranged/multipart download.
type ListOption ¶
type ListOption func(*listOptions)
ListOption configures list operations.
func WithContinuationToken ¶
func WithContinuationToken(token string) ListOption
WithContinuationToken continues a previous list page.
func WithDelimiter ¶
func WithDelimiter(d string) ListOption
WithDelimiter sets folder delimiter (e.g. "/").
type MultipartConfig ¶
type MultipartConfig struct {
// PartSize is the upload part size in bytes.
PartSize int64
// Concurrency is the number of parallel part uploads.
Concurrency int
// Threshold is the file size above which multipart upload is used.
Threshold int64
// DownloadPartSize is the chunk size for ranged downloads.
DownloadPartSize int64
}
MultipartConfig controls chunked transfers.
type ObjectInfo ¶
type ObjectInfo struct {
Key string
Size int64
LastModified int64 // unix seconds when unknown
ETag string
}
ObjectInfo describes one listed object.
type ProviderSettings ¶
type ProviderSettings struct {
// Name is one of ProviderAliyun, ProviderTencent, ProviderWechat, ProviderAWS, ProviderAzure.
Name string
}
ProviderSettings selects active backend.
type PutOption ¶
type PutOption func(*putOptions)
PutOption configures uploads.
func WithContentLength ¶
WithContentLength sets known object size (enables streaming upload optimizations).
func WithMultipart ¶
func WithMultipart() PutOption
WithMultipart forces multipart upload regardless of threshold.
type RetryConfig ¶
type RetryConfig struct {
// MaxAttempts is the total number of connection attempts.
MaxAttempts int
// InitialDelay is the base delay before the first retry.
InitialDelay time.Duration
// MaxDelay caps exponential backoff delay.
MaxDelay time.Duration
// Multiplier scales delay after each failed attempt.
Multiplier float64
}
RetryConfig controls startup connectivity retry.
func DefaultRetryConfig ¶
func DefaultRetryConfig() RetryConfig
DefaultRetryConfig returns startup retry defaults.
type Storage ¶
type Storage interface {
Upload(ctx context.Context, key string, r io.Reader, opts ...PutOption) error
Download(ctx context.Context, key string, w io.Writer, opts ...GetOption) error
UploadLarge(ctx context.Context, key string, r io.Reader, size int64, opts ...PutOption) error
DownloadLarge(ctx context.Context, key string, w io.Writer, size int64, opts ...GetOption) error
List(ctx context.Context, prefix string, opts ...ListOption) ([]ObjectInfo, *ListPage, error)
Delete(ctx context.Context, key string) error
PublicURL(key string) string
Close() error
}
Storage is the primary object storage abstraction.
func NewStorage ¶
NewStorage creates a storage client with optional startup ping retry. ctx bounds the startup ping and its retries.
Example ¶
package main
import (
"context"
"github.com/andywang2011/go-upload"
)
func main() {
cfg := upload.ConfigDefault()
cfg.Provider.Name = upload.ProviderAliyun
cfg.Aliyun = upload.AliyunConfig{
Endpoint: "oss-cn-hangzhou.aliyuncs.com",
AccessKeyID: "...",
AccessKeySecret: "...",
Bucket: "my-bucket",
}
store, err := upload.NewStorage(context.Background(), cfg)
if err != nil {
return
}
defer store.Close()
}
Output:
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package coscred issues temporary Tencent Cloud COS credentials for client-side uploads.
|
Package coscred issues temporary Tencent Cloud COS credentials for client-side uploads. |
|
Package osscred issues Aliyun STS temporary credentials and OSS post policies for client-side uploads.
|
Package osscred issues Aliyun STS temporary credentials and OSS post policies for client-side uploads. |