gouploader

package module
v1.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 12, 2026 License: MIT Imports: 16 Imported by: 0

README

gouploader

gouploader 是框架无关的 Go 上传服务与云存储适配包。它只提供 Service/API、对象存储抽象、分片能力、STS 直传原语和云厂商 adapter；HTTP 路由、鉴权、业务任务、数据库回写、操作日志和 CDN/Edge 刷新由业务项目负责。

安装

go get github.com/gtkit/gouploader

# 按需引入云厂商 adapter
go get github.com/gtkit/gouploader/aliyun
go get github.com/gtkit/gouploader/tencent
go get github.com/gtkit/gouploader/huawei
go get github.com/gtkit/gouploader/aws
go get github.com/gtkit/gouploader/minio

核心能力

  • 对象 API：ObjectService.SaveObject/OpenObject/DeleteObject/DeleteObjects/StatObject/CopyObject/MoveObject/SignedURL/PublicURL
  • 动态存储解析：StorageResolver 支持按请求选择 provider、bucket、region、endpoint、credential
  • 分片上传：ChunkUploadService 提供初始化、分片上传、状态查询、合并、中止和云原生 Multipart 直传原语
  • STS 直传：DirectUploadService 提供临时凭证签发与完成确认原语
  • 对象存储抽象：Storage、ObjectStorage、ObjectStorageWithResult、ObjectStatter、MultipartStorage、SignedURLStorage
  • 生命周期钩子：上传初始化、失败、中止、秒传命中等事件可由业务接入日志、指标和 trace

不包含的能力

  • 不提供 RegisterRoutes 或任何 Gin handler
  • 不提供 CORS 中间件
  • 不引入 CDN/EdgeOne SDK
  • 不读取业务数据库，不判断业务权限，不管理业务任务状态

对象 Service 示例

objects, err := gouploader.NewObjectService(
    gouploader.WithObjectStorageResolver(gouploader.StorageResolverFunc(
        func(ctx context.Context, scope gouploader.ObjectScope) (gouploader.Storage, error) {
            // 业务方在这里按 scope.Provider / scope.Bucket 读取 DB 配置、解密凭证、构造 adapter。
            return storage, nil
        },
    )),
    gouploader.WithObjectPublicURLBuilder(func(key string) string {
        return "https://static.example.com/" + key
    }),
)
if err != nil {
    return err
}

result, err := objects.SaveObject(ctx, gouploader.SaveObjectRequest{
    Scope: gouploader.ObjectScope{Provider: "tencent", Bucket: "packages"},
    Key:   "business/app.apk",
    Body:  file,
    Size:  fileSize,
    Metadata: gouploader.ObjectMetadata{
        ContentType:        "application/vnd.android.package-archive",
        ContentDisposition: "attachment; filename=app.apk",
        CacheControl:       "public, max-age=31536000",
        UserMetadata:       map[string]string{"business": "demo"},
    },
})
if err != nil {
    return err
}
fmt.Println(result.URL)
fmt.Println(result.ETag)

实现 ObjectStorageWithResult 的 adapter 会在 SaveObject 后回填 provider 返回的 ETag、VersionID、StorageClass 和元数据；旧的 ObjectStorage 实现仍保持兼容，未返回的字段为空。 StatObject 统一返回 Size、ETag、VersionID、LastModified、ContentType、ContentDisposition、CacheControl、StorageClass 和用户元数据；对象不存在会归一为 ErrObjectNotFound。

预签名下载 URL

私有 bucket 场景下，业务方可通过 ObjectService.SignedURL 直接生成临时 GET 下载 URL；该方法复用 StorageResolver，并在底层 storage 不支持 SignedURLStorage 时返回 ErrUnsupportedOperation。

url, err := objects.SignedURL(ctx, gouploader.SignedURLRequest{
    Scope: gouploader.ObjectScope{Provider: "tencent", Bucket: "packages"},
    Key:   "business/app.apk",
    TTL:   15 * time.Minute,
})
if err != nil {
    return err
}
fmt.Println(url)

SignedURL 不会在签名前额外调用 Exists 或 StatObject；如业务需要先展示对象大小、ETag 或 Content-Type，请显式调用 ObjectService.StatObject。

Uploader Service 容器

New 创建的是纯 Go service 容器，可按需取得内部 service：

uploader, err := gouploader.New(
    gouploader.WithStorage(storage),
    gouploader.WithHashStore(hashStore),
    gouploader.WithUserIDFn(func(ctx context.Context) string {
        uid, _ := ctx.Value(userIDKey{}).(string)
        return uid
    }),
)
if err != nil {
    return err
}
defer uploader.Close()

chunkSvc := uploader.ChunkService()
initResp, err := chunkSvc.InitUpload(ctx, &gouploader.ChunkInitRequest{
    FileName: "app.apk",
    FileSize: fileSize,
    FileHash: sha256Hex,
})

云厂商 adapter 状态

当前已有官方 adapter 子模块：

  • github.com/gtkit/gouploader/aliyun：阿里云 OSS，支持 Storage、对象扩展接口、云原生 Multipart、预签名 URL 和 RawClient()
  • github.com/gtkit/gouploader/tencent：腾讯云 COS，支持 Storage、对象扩展接口、云原生 Multipart、预签名 URL 和 RawClient()
  • github.com/gtkit/gouploader/huawei：华为云 OBS，支持 Storage、对象扩展接口、云原生 Multipart、预签名 URL 和 RawClient()
  • github.com/gtkit/gouploader/aws：AWS S3，支持 Storage、对象扩展接口、云原生 Multipart、预签名 URL、path/virtual-host 自动寻址和 RawClient()
  • github.com/gtkit/gouploader/minio：MinIO，默认 path-style，支持 Storage、对象扩展接口、云原生 Multipart、预签名 URL 和 RawClient()
Adapter 能力矩阵
Adapter SaveObject StatObject CopyObject DeleteObjects Multipart SignedURL ACL RawClient()
aliyun ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
tencent ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
huawei ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
aws ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
minio ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅

普通对象上传会传递 Content-Type、Content-Disposition、Cache-Control 和 UserMetadata；StatObject 会回填 size、etag、lastModified、contentType 与 metadata。DeleteObject、DeleteObjects、AbortMultipart 对“不存在”按幂等成功处理。

RawClient() 返回 adapter 内部持有的官方 SDK client，不创建新 client、不触发网络请求。业务遇到本库尚未抽象的厂商高级能力时，可通过该方法直接调用官方 SDK；常规上传、对象操作和错误判断仍建议优先依赖 gouploader 的稳定接口。

启动期能力检测

业务服务可在启动时用 DetectCapabilities 校验当前 adapter 是否满足上传链路要求，避免运行时才发现能力缺失：

caps := gouploader.DetectCapabilities(storage)
err := caps.Require(gouploader.Capabilities{
    SaveObject:    true,
    StatObject:    true,
    Multipart:     true,
    SignedURL:     true,
    DeleteObjects: true,
})
if err != nil {
    return fmt.Errorf("storage capability check failed: %w", err)
}

DetectCapabilities 只基于 Go 接口做本地检测，不会访问云厂商网络。Capabilities.Require 缺失能力时返回可用 errors.Is(err, gouploader.ErrUnsupportedOperation) 判断的错误，并在错误消息中列出缺失能力名。

稳定 API 边界

推荐业务代码依赖以下稳定边界：

  • Storage、ObjectStorage、ObjectStatter、ObjectCopier、ObjectBatchDeleter、MultipartStorage、SignedURLStorage
  • ObjectService、UploadWorkflow、ChunkUploadService、DirectUploadService
  • StorageResolver、CredentialResolver、STSProviderResolver、UploadInfo 和 observer hooks
  • 云 adapter 子模块的 Config、NewStorage、Storage 与 RawClient()

internal/* 包、未导出的类型和测试辅助实现不属于兼容性承诺；业务项目不要直接依赖。新增 provider 能力会优先通过扩展接口暴露，调用方可用类型断言判断是否支持，例如 ObjectCopier、ObjectBatchDeleter 或 MultipartStorage。

错误处理

可判断错误使用包内 sentinel error，例如：

if errors.Is(err, gouploader.ErrObjectStatUnsupported) {
    // 当前 storage 不支持 StatObject
}
if errors.Is(err, gouploader.ErrObjectCopyUnsupported) {
    // 当前 storage 不支持服务端 CopyObject
}

adapter 会逐步把认证失败、桶不存在、权限不足、对象不存在、签名过期、限流、网络超时和服务端错误映射到统一错误分类；当前调用方可先使用 errors.Is 判断 gouploader sentinel error，并保留底层 provider 错误链用于日志排查。

云错误与重试语义

gouploader 提供 provider-neutral 错误分类：ErrorKindAuthFailed、ErrorKindBucketNotFound、ErrorKindPermissionDenied、ErrorKindObjectNotFound、ErrorKindSignatureExpired、ErrorKindRateLimited、ErrorKindTimeout、ErrorKindNetwork、ErrorKindServer、ErrorKindUnsupported、ErrorKindInvalidArgument 和 ErrorKindUnknown。

adapter 返回的 ProviderError 可通过 errors.Is 判断稳定 sentinel，也可通过 errors.As 提取 provider、operation、status code、provider code 和 request id：

var providerErr *gouploader.ProviderError
if errors.As(err, &providerErr) {
    log.Printf("provider=%s operation=%s code=%s request_id=%s", providerErr.Provider, providerErr.Operation, providerErr.Code, providerErr.RequestID)
}
if errors.Is(err, gouploader.ErrObjectNotFound) {
    // 对象不存在
}

RetryPolicy 只会对 retryable error kind 且默认幂等的操作重试。默认幂等操作包括 OpenObject、DeleteObject、DeleteObjects、StatObject、CopyObject、PresignPart、ListParts、AbortMultipart 和 ListOrphans。SaveObject 默认不重试，因为请求体通常不可重放；CompleteMultipart 默认不盲目重试，业务需要确认 provider 重入语义后再自行包裹策略。

policy := gouploader.RetryPolicy{
    MaxAttempts: 3,
    Backoff:     gouploader.ExponentialBackoff(100*time.Millisecond, time.Second),
}
err := policy.Do(ctx, gouploader.OperationDeleteObject, func() error {
    return objects.DeleteObject(ctx, gouploader.DeleteObjectRequest{Key: key})
})

幂等语义：DeleteObject、DeleteObjects 和 AbortMultipart 对“不存在”按成功处理；CopyObject 可安全重试到同一目标 key，但调用方需要接受覆盖语义；MoveObject 是 copy 后 delete，copy 成功但 delete 失败时需要业务层根据返回错误做补偿。

业务集成 API

UploadInfo 可把 user_id/business_id/bucket_id/task_id/temp_file_id 和自定义 labels 绑定到 context.Context，所有 observer 可从事件中读取这些业务标识：

ctx = gouploader.ContextWithUploadInfo(ctx, gouploader.UploadInfo{
    UserID:     "u-1",
    BusinessID: "biz-1",
    BucketID:   "bucket-1",
    TaskID:     "task-1",
    TempFileID: "temp-1",
    Labels:     map[string]string{"scene": "release"},
})

UploadWorkflow 提供“临时对象 → 正式对象”的业务语义原语：

workflow := gouploader.NewUploadWorkflow(objects)
_, _ = workflow.SaveTemp(ctx, gouploader.SaveTempRequest{Key: "tmp/a", Body: file, Size: size})
_, _ = workflow.CompleteUpload(ctx, gouploader.CompleteUploadRequest{TempKey: "tmp/a", FinalKey: "final/a"})
_ = workflow.AbortUpload(ctx, gouploader.AbortUploadRequest{Keys: []string{"tmp/a"}})

主链路动态解析可直接挂到 Uploader：WithStorageResolver 会被 ChunkUploadService 和 DirectUploadService.Complete 使用；WithCredentialResolver 会把短期凭证转换为 STS 直传响应；需要完全自定义签发逻辑时使用 WithSTSProviderResolver。

uploader, err := gouploader.New(
    gouploader.WithStorageResolver(gouploader.StorageResolverFunc(
        func(ctx context.Context, scope gouploader.ObjectScope) (gouploader.Storage, error) {
            // 按 scope.Provider / scope.Bucket / scope.Values 选择 adapter。
            return storage, nil
        },
    )),
    gouploader.WithCredentialResolver(gouploader.CredentialResolverFunc(
        func(ctx context.Context, scope gouploader.CredentialScope) (gouploader.Credential, error) {
            // 按 scope.BusinessID / scope.BucketID 查询并返回短期凭证。
            return gouploader.Credential{AccessKeyID: "AK", SecretAccessKey: "SK", SessionToken: "TOKEN"}, nil
        },
    )),
)

CredentialResolver 用于按请求解析短期凭证；业务方可在 resolver 内查数据库、解密、短期缓存和支持轮换。日志输出密钥时使用 MaskSecret 脱敏。

Resolver 错误语义
  • StorageResolver 返回错误时，ObjectService 返回 gouploader: 解析 Storage 失败: ...，分片/直传 service 返回解析失败错误；主链路不会继续执行存储操作。
  • StorageResolver 返回 nil storage 时，ObjectService 返回 gouploader: StorageResolver 返回 nil Storage；分片 service 返回 invalid param 错误；主链路不会回退到默认 storage。
  • StorageResolver 因 context.Context 超时或取消返回错误时，错误会被原样 wrap，调用方可继续用 errors.Is(err, context.DeadlineExceeded) 或 errors.Is(err, context.Canceled) 判断。
  • 权限失败应由 resolver 返回业务自定义错误或 wrap ErrPermissionDenied；库不会吞掉该错误，也不会替调用方降级到其他 bucket/provider。
  • CredentialResolver 返回错误时，STS 签发返回 gouploader: resolve credential: ... 包装错误；通过 Uploader 接入的直传签发会转换为 STS 签发失败，不会发放临时凭证。
  • CredentialResolver 当前返回值是结构体，无法返回 nil；如果凭证字段为空，后续 provider 签发会按其校验规则返回错误。

BuildObjectURL 支持 CDN 前缀、bucket virtual-host-style、path-style URL 拼接；ACL 提供 provider-neutral ACL 枚举，adapter 可映射到各厂商实际取值。

License

MIT / Apache-2.0（以仓库 LICENSE 为准）。

Documentation

Overview

Package gouploader provides production-grade upload services and cloud storage adapters.

The package is framework-neutral. It exposes Go services and storage interfaces only; HTTP routing, authentication, authorization, database writes, and CDN refresh belong in the consuming application.

Example (ObjectService)

Example_objectService demonstrates service-level object operations without HTTP routes. 

package main

import (
	"context"
	"fmt"
	"io"
	"log"
	"strings"
	"time"

	"github.com/gtkit/gouploader"
)

type memoryObjectStorage struct {
	data map[string]string
}

func newMemoryObjectStorage() *memoryObjectStorage {
	return &memoryObjectStorage{data: make(map[string]string)}
}

func (s *memoryObjectStorage) Save(_ context.Context, key string, reader io.Reader, _ int64) error {
	body, err := io.ReadAll(reader)
	if err != nil {
		return err
	}
	s.data[key] = string(body)
	return nil
}

func (s *memoryObjectStorage) Get(_ context.Context, key string) (io.ReadCloser, error) {
	return io.NopCloser(strings.NewReader(s.data[key])), nil
}

func (s *memoryObjectStorage) Delete(_ context.Context, key string) error {
	delete(s.data, key)
	return nil
}

func (s *memoryObjectStorage) Exists(_ context.Context, key string) (bool, error) {
	_, ok := s.data[key]
	return ok, nil
}

func (s *memoryObjectStorage) GetSignedGetURL(_ context.Context, key string, ttl time.Duration) (string, error) {
	return fmt.Sprintf("https://signed.example.com/%s?ttl=%s", key, ttl), nil
}

func main() {
	storage := newMemoryObjectStorage()
	objects, err := gouploader.NewObjectService(
		gouploader.WithObjectStorage(storage),
		gouploader.WithObjectPublicURLBuilder(func(key string) string {
			return "https://static.example.com/" + key
		}),
	)
	if err != nil {
		log.Fatal(err)
	}

	result, err := objects.SaveObject(context.Background(), gouploader.SaveObjectRequest{
		Key:  "business/app.apk",
		Body: strings.NewReader("package"),
		Size: 7,
		Metadata: gouploader.ObjectMetadata{
			ContentType:  "application/vnd.android.package-archive",
			CacheControl: "public, max-age=31536000",
		},
	})
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(result.URL)
}

Output:
https://static.example.com/business/app.apk

Index

Examples

Constants

View Source 
const (
	CodeFileTooLarge     = types.CodeFileTooLarge     // 文件超过大小限制
	CodeFileTypeNotAllow = types.CodeFileTypeNotAllow // 文件类型不允许
	CodeChecksumMismatch = types.CodeChecksumMismatch // 校验和不匹配
	CodeFileReadError    = types.CodeFileReadError    // 文件读取错误
	CodeInvalidParam     = types.CodeInvalidParam     // 参数无效
	CodeMultipartError   = types.CodeMultipartError   // multipart 解析错误

	CodeSessionNotFound    = types.CodeSessionNotFound    // 上传会话不存在
	CodeSessionExpired     = types.CodeSessionExpired     // 上传会话已过期
	CodeChunkIndexOutRange = types.CodeChunkIndexOutRange // 分片索引越界
	CodeChunkDuplicate     = types.CodeChunkDuplicate     // 分片重复上传
	CodeChunkHashMismatch  = types.CodeChunkHashMismatch  // 分片校验和不匹配
	CodeMergeIncomplete    = types.CodeMergeIncomplete    // 分片未全部上传
	CodeMergeHashFailed    = types.CodeMergeHashFailed    // 合并后校验失败

	CodeStorageWriteFail      = types.CodeStorageWriteFail      // 存储写入失败
	CodeStorageReadFail       = types.CodeStorageReadFail       // 存储读取失败
	CodeStorageDeleteFail     = types.CodeStorageDeleteFail     // 存储删除失败
	CodeObjectStatUnsupported = types.CodeObjectStatUnsupported // 当前存储不支持对象 Stat
	CodeObjectCopyUnsupported = types.CodeObjectCopyUnsupported // 当前存储不支持对象 Copy

	CodeSTSNotConfigured   = types.CodeSTSNotConfigured   // STS 未配置
	CodeSTSIssueFailed     = types.CodeSTSIssueFailed     // STS 凭证签发失败
	CodeCallbackVerifyFail = types.CodeCallbackVerifyFail // 回调验签失败
	CodeDirectFileNotFound = types.CodeDirectFileNotFound // 直传文件不存在

	CodeMultipartNotSupported = types.CodeMultipartNotSupported // 当前存储后端不支持 Multipart 直传
	CodeMultipartInitFailed   = types.CodeMultipartInitFailed   // 云端 Multipart 初始化失败
	CodeMultipartPresignFail  = types.CodeMultipartPresignFail  // 预签名 URL 生成失败
	CodeMultipartCompleteFail = types.CodeMultipartCompleteFail // 云端 Multipart 合并失败
	CodeMultipartAbortFail    = types.CodeMultipartAbortFail    // 云端 Multipart 中止失败
	CodePartVerifyFail        = types.CodePartVerifyFail        // 服务端二次校验 parts 失败
	CodePartETagMismatch      = types.CodePartETagMismatch      // 客户端上报 ETag 与云端不一致
	CodeUnauthorized          = types.CodeUnauthorized          // 无权访问该会话

)

错误码常量。

View Source 
const (
	StageInit     = types.StageInit     // 会话初始化阶段
	StagePart     = types.StagePart     // 分片上传阶段
	StageMerge    = types.StageMerge    // 代理路径合并阶段
	StageComplete = types.StageComplete // Multipart 直传 Complete 阶段
	StageAbort    = types.StageAbort    // 中止阶段
)

Stage 常量：UploadErrorEvent.Stage 字段的合法取值。 这些字符串值可直接作为 Prometheus label 使用。

View Source 
const (
	LifecyclePartSign        = types.LifecyclePartSign
	LifecyclePartComplete    = types.LifecyclePartComplete
	LifecycleCompleteSuccess = types.LifecycleCompleteSuccess
)

成功生命周期事件阶段常量。

View Source 
const (
	AbortReasonClientRequest = types.AbortReasonClientRequest
)

中止原因常量：UploadAbortEvent.Reason 字段的取值。

View Source 
const Version = "v1.7.0"

Variables

View Source 
var (
	ErrFileTooLarge     = types.ErrFileTooLarge
	ErrFileTypeNotAllow = types.ErrFileTypeNotAllow
	ErrChecksumMismatch = types.ErrChecksumMismatch
	ErrInvalidParam     = types.ErrInvalidParam
	ErrMultipartError   = types.ErrMultipartError

	ErrSessionNotFound    = types.ErrSessionNotFound
	ErrSessionExpired     = types.ErrSessionExpired
	ErrChunkIndexOutRange = types.ErrChunkIndexOutRange
	ErrChunkHashMismatch  = types.ErrChunkHashMismatch
	ErrMergeIncomplete    = types.ErrMergeIncomplete
	ErrMergeHashFailed    = types.ErrMergeHashFailed

	ErrStorageWrite          = types.ErrStorageWrite
	ErrStorageRead           = types.ErrStorageRead
	ErrStorageDelete         = types.ErrStorageDelete
	ErrObjectStatUnsupported = types.ErrObjectStatUnsupported
	ErrObjectCopyUnsupported = types.ErrObjectCopyUnsupported

	ErrSTSNotConfigured   = types.ErrSTSNotConfigured
	ErrSTSIssueFailed     = types.ErrSTSIssueFailed
	ErrCallbackVerifyFail = types.ErrCallbackVerifyFail
	ErrDirectFileNotFound = types.ErrDirectFileNotFound

	ErrMultipartNotSupported = types.ErrMultipartNotSupported
	ErrMultipartInitFailed   = types.ErrMultipartInitFailed
	ErrMultipartPresignFail  = types.ErrMultipartPresignFail
	ErrMultipartCompleteFail = types.ErrMultipartCompleteFail
	ErrMultipartAbortFail    = types.ErrMultipartAbortFail
	ErrPartVerifyFail        = types.ErrPartVerifyFail
	ErrPartETagMismatch      = types.ErrPartETagMismatch
	ErrUnauthorized          = types.ErrUnauthorized
)

预定义错误。

View Source 
var (
	// ErrProviderAuthFailed indicates provider authentication failure.
	ErrProviderAuthFailed = types.NewErr(31001, http.StatusUnauthorized, "云存储认证失败")
	// ErrBucketNotFound indicates provider bucket does not exist.
	ErrBucketNotFound = types.NewErr(31002, http.StatusNotFound, "云存储 bucket 不存在")
	// ErrPermissionDenied indicates provider permission denied.
	ErrPermissionDenied = types.NewErr(31003, http.StatusForbidden, "云存储权限不足")
	// ErrObjectNotFound indicates provider object does not exist.
	ErrObjectNotFound = types.NewErr(31004, http.StatusNotFound, "云存储对象不存在")
	// ErrSignatureExpired indicates request signature has expired.
	ErrSignatureExpired = types.NewErr(31005, http.StatusUnauthorized, "云存储签名已过期")
	// ErrRateLimited indicates provider rate limiting.
	ErrRateLimited = types.NewErr(31006, http.StatusTooManyRequests, "云存储请求被限流")
	// ErrTimeout indicates provider timeout.
	ErrTimeout = types.NewErr(31007, http.StatusGatewayTimeout, "云存储请求超时")
	// ErrNetwork indicates provider network error.
	ErrNetwork = types.NewErr(31008, http.StatusBadGateway, "云存储网络错误")
	// ErrProviderServer indicates provider server-side error.
	ErrProviderServer = types.NewErr(31009, http.StatusBadGateway, "云存储服务端错误")
	// ErrUnsupportedOperation indicates unsupported provider operation.
	ErrUnsupportedOperation = types.NewErr(31010, http.StatusNotImplemented, "云存储不支持该操作")
	// ErrProviderUnknown indicates unknown provider error.
	ErrProviderUnknown = types.NewErr(31011, http.StatusBadGateway, "云存储未知错误")
)

Functions

func BuildObjectURL added in v1.5.0 

func BuildObjectURL(cfg ObjectURLConfig, key string) (string, error)

BuildObjectURL builds a public object URL using CDN, virtual-host, or path-style rules.

func ContextWithUploadInfo added in v1.5.0 

func ContextWithUploadInfo(ctx context.Context, info UploadInfo) context.Context

ContextWithUploadInfo returns a context carrying upload business identifiers.

func MaskSecret added in v1.5.0 

func MaskSecret(secret string) string

MaskSecret returns a log-safe masked representation of a secret.

Types

type ACL added in v1.5.0 

type ACL string

ACL is a provider-neutral object ACL value. 

const (
	ACLDefault           ACL = ""
	ACLPrivate           ACL = "private"
	ACLPublicRead        ACL = "public_read"
	ACLPublicReadWrite   ACL = "public_read_write"
	ACLAuthenticatedRead ACL = "authenticated_read"
)

type AWSSTSConfig 

type AWSSTSConfig = sts.AWSConfig

AWSSTSConfig AWS S3 STS 配置。

type AbortUploadRequest added in v1.5.0 

type AbortUploadRequest struct {
	Scope ObjectScope
	Keys  []string
}

AbortUploadRequest is the input for UploadWorkflow.AbortUpload.

type AddressingStyle added in v1.5.0 

type AddressingStyle string

AddressingStyle controls bucket addressing in object URLs. 

const (
	// AddressingStyleAuto lets builders choose the provider default.
	AddressingStyleAuto AddressingStyle = "auto"
	// AddressingStyleVirtualHost builds https://bucket.endpoint/key URLs.
	AddressingStyleVirtualHost AddressingStyle = "virtual_host"
	// AddressingStylePath builds https://endpoint/bucket/key URLs.
	AddressingStylePath AddressingStyle = "path"
)

type AliyunSTSConfig 

type AliyunSTSConfig = sts.AliyunConfig

AliyunSTSConfig 阿里云 OSS STS 配置。

type BackoffPolicy added in v1.5.0 

type BackoffPolicy func(attempt int) time.Duration

BackoffPolicy returns the delay before retrying attempt. attempt is 1-based.

func ExponentialBackoff added in v1.5.0 

func ExponentialBackoff(base, maxDelay time.Duration) BackoffPolicy

ExponentialBackoff returns a capped exponential backoff policy.

func FixedBackoff added in v1.5.0 

func FixedBackoff(delay time.Duration) BackoffPolicy

FixedBackoff returns a backoff policy with a constant delay.

type Capabilities added in v1.5.0 

type Capabilities struct {
	// SaveObject reports support for metadata-aware object saving through ObjectStorage.
	SaveObject bool
	// StatObject reports support for provider-neutral object stat through ObjectStatter.
	StatObject bool
	// CopyObject reports support for server-side object copy through ObjectCopier.
	CopyObject bool
	// DeleteObjects reports support for provider batch delete through ObjectBatchDeleter.
	DeleteObjects bool
	// Multipart reports support for cloud-native multipart upload through MultipartStorage.
	Multipart bool
	// SignedURL reports support for presigned GET URLs through SignedURLStorage.
	SignedURL bool
	// RawClient reports that the storage exposes a RawClient method for provider SDK access.
	RawClient bool
}

Capabilities describes optional upload and object capabilities supported by a Storage. It is a local capability check based on Go interfaces only; it does not perform network I/O.

func DetectCapabilities added in v1.5.0 

func DetectCapabilities(storage Storage) Capabilities

DetectCapabilities returns optional capabilities implemented by storage. A nil storage returns the zero capability set.

func (Capabilities) Require added in v1.5.0 

func (c Capabilities) Require(required Capabilities) error

Require verifies that all capabilities marked true in required are present. It returns an error compatible with ErrUnsupportedOperation when any required capability is missing.

type ChunkCompleteRequest added in v1.2.0 

type ChunkCompleteRequest = types.ChunkCompleteRequest

ChunkCompleteRequest Multipart 直传路径的合并请求体。

type ChunkInitRequest 

type ChunkInitRequest = types.ChunkInitRequest

ChunkInitRequest 初始化分片上传的 JSON 请求体。

type ChunkInitResponse 

type ChunkInitResponse = types.ChunkInitResponse

ChunkInitResponse 分片上传会话创建后的响应。

type ChunkStatusResponse 

type ChunkStatusResponse = types.ChunkStatusResponse

ChunkStatusResponse 查询分片上传状态的响应。

type ChunkUploadResponse 

type ChunkUploadResponse = types.ChunkUploadResponse

ChunkUploadResponse 单个分片上传后的响应。

type CompleteResult added in v1.2.0 

type CompleteResult = types.CompleteResult

CompleteResult Multipart 合并结果。

type CompleteUploadRequest added in v1.5.0 

type CompleteUploadRequest struct {
	Scope    ObjectScope
	TempKey  string
	FinalKey string
	Metadata ObjectMetadata
}

CompleteUploadRequest is the input for UploadWorkflow.CompleteUpload.

type CopyObjectRequest added in v1.5.0 

type CopyObjectRequest struct {
	Scope          ObjectScope
	SourceKey      string
	DestinationKey string
	Metadata       ObjectMetadata
}

CopyObjectRequest is the input for ObjectService.CopyObject.

type Credential added in v1.5.0 

type Credential struct {
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string
	ExpiresAt       time.Time
}

Credential contains short-lived provider credentials.

type CredentialResolver added in v1.5.0 

type CredentialResolver interface {
	ResolveCredential(ctx context.Context, scope CredentialScope) (Credential, error)
}

CredentialResolver resolves short-lived provider credentials for one request.

type CredentialResolverFunc added in v1.5.0 

type CredentialResolverFunc func(context.Context, CredentialScope) (Credential, error)

CredentialResolverFunc adapts a function to CredentialResolver.

func (CredentialResolverFunc) ResolveCredential added in v1.5.0 

func (fn CredentialResolverFunc) ResolveCredential(ctx context.Context, scope CredentialScope) (Credential, error)

ResolveCredential resolves credentials for one request.

type CredentialScope added in v1.5.0 

type CredentialScope struct {
	Provider   string
	Bucket     string
	Region     string
	Endpoint   string
	BusinessID string
	BucketID   string
	Values     map[string]string
}

CredentialScope identifies credential lookup input.

type DeleteObjectRequest added in v1.5.0 

type DeleteObjectRequest struct {
	Scope ObjectScope
	Key   string
}

DeleteObjectRequest is the input for ObjectService.DeleteObject.

type DeleteObjectsRequest added in v1.5.0 

type DeleteObjectsRequest struct {
	Scope ObjectScope
	Keys  []string
}

DeleteObjectsRequest is the input for ObjectService.DeleteObjects.

type DirectCompleteRequest 

type DirectCompleteRequest = types.DirectCompleteRequest

DirectCompleteRequest 前端确认上传完成的请求体。

type DirectTokenRequest 

type DirectTokenRequest = types.DirectTokenRequest

DirectTokenRequest 请求 STS 凭证的请求体。

type DirectTokenResponse 

type DirectTokenResponse = types.DirectTokenResponse

DirectTokenResponse STS 凭证响应。

type DirectUploadResult 

type DirectUploadResult = types.DirectUploadResult

DirectUploadResult 前端直传完成后的结果。

type Error 

type Error = types.Error

Error 统一错误类型。

type ErrorKind added in v1.5.0 

type ErrorKind string

ErrorKind is a provider-neutral cloud storage error category. 

const (
	ErrorKindAuthFailed       ErrorKind = "auth_failed"
	ErrorKindBucketNotFound   ErrorKind = "bucket_not_found"
	ErrorKindPermissionDenied ErrorKind = "permission_denied"
	ErrorKindObjectNotFound   ErrorKind = "object_not_found"
	ErrorKindSignatureExpired ErrorKind = "signature_expired"
	ErrorKindRateLimited      ErrorKind = "rate_limited"
	ErrorKindTimeout          ErrorKind = "timeout"
	ErrorKindNetwork          ErrorKind = "network"
	ErrorKindServer           ErrorKind = "server"
	ErrorKindUnsupported      ErrorKind = "unsupported"
	ErrorKindInvalidArgument  ErrorKind = "invalid_argument"
	ErrorKindUnknown          ErrorKind = "unknown"
)

Provider-neutral cloud error kinds.

func (ErrorKind) Retryable added in v1.5.0 

func (k ErrorKind) Retryable() bool

Retryable reports whether errors in this kind are safe retry candidates.

type HashStore added in v1.2.0 

type HashStore = types.HashStore

HashStore 是秒传索引接口，由业务方实现（MySQL/Redis/Mongo 等）。

⚠️ 多租户私有 bucket 场景必须实现 UserScopedHashStore（v1.4.0+）， 否则用户 B 凭知道的 hash 可命中 A 的私有文件构成越权访问。

type HashedFile added in v1.2.0 

type HashedFile = types.HashedFile

HashedFile 秒传索引条目。

type HuaweiSTSConfig 

type HuaweiSTSConfig = sts.HuaweiConfig

HuaweiSTSConfig 华为云 OBS STS 配置。

type InstantHitEvent added in v1.2.0 

type InstantHitEvent = types.InstantHitEvent

InstantHitEvent 秒传命中时触发的生命周期钩子 payload。

type MoveObjectRequest added in v1.5.0 

type MoveObjectRequest struct {
	Scope          ObjectScope
	SourceKey      string
	DestinationKey string
	Metadata       ObjectMetadata
}

MoveObjectRequest is the input for ObjectService.MoveObject.

type MultipartMeta added in v1.2.0 

type MultipartMeta = types.MultipartMeta

MultipartMeta 创建 Multipart 时可设置的对象元数据。

type MultipartStorage added in v1.2.0 

type MultipartStorage = types.MultipartStorage

MultipartStorage 是支持云原生 Multipart 直传的存储扩展接口。 由各云厂商 adapter 子 module 实现（aliyun / tencent / huawei）。

type ObjectBatchDeleter added in v1.5.0 

type ObjectBatchDeleter interface {
	Storage
	DeleteObjects(ctx context.Context, keys []string) error
}

ObjectBatchDeleter is an optional Storage extension for batch delete.

type ObjectCopier added in v1.5.0 

type ObjectCopier interface {
	Storage
	CopyObject(ctx context.Context, req CopyObjectRequest) (*ObjectStat, error)
}

ObjectCopier is an optional Storage extension for server-side object copy.

type ObjectEvent added in v1.5.0 

type ObjectEvent struct {
	Operation      ObjectOperation
	Key            string
	SourceKey      string
	DestinationKey string
	Scope          ObjectScope
	Stat           *ObjectStat
	UploadInfo
}

ObjectEvent describes an object lifecycle event.

type ObjectMetadata added in v1.5.0 

type ObjectMetadata struct {
	// ContentType is the object's MIME type.
	ContentType string
	// ContentDisposition controls browser download/display behavior.
	ContentDisposition string
	// CacheControl controls downstream cache behavior.
	CacheControl string
	// UserMetadata stores provider-specific custom metadata key/value pairs.
	UserMetadata map[string]string
}

ObjectMetadata contains HTTP object metadata used when saving an object.

type ObjectOperation added in v1.5.0 

type ObjectOperation string

ObjectOperation identifies object lifecycle observer operation. 

const (
	// ObjectOperationSave is emitted after SaveObject succeeds.
	ObjectOperationSave ObjectOperation = "save"
	// ObjectOperationDelete is emitted after DeleteObject or DeleteObjects succeeds.
	ObjectOperationDelete ObjectOperation = "delete"
	// ObjectOperationCopy is emitted after CopyObject succeeds.
	ObjectOperationCopy ObjectOperation = "copy"
	// ObjectOperationMove is emitted after MoveObject succeeds.
	ObjectOperationMove ObjectOperation = "move"
)

type ObjectScope added in v1.5.0 

type ObjectScope struct {
	Provider     string
	Bucket       string
	Region       string
	Endpoint     string
	AccessDomain string
	Values       map[string]string
}

ObjectScope identifies the target storage selected for one object operation. It is safe to pass by value. Business permissions and credential loading stay outside gouploader and should be handled by the caller's StorageResolver.

type ObjectService added in v1.5.0 

type ObjectService struct {
	// contains filtered or unexported fields
}

ObjectService provides service-level object operations without Gin handlers. It is safe for concurrent use by multiple goroutines when the configured StorageResolver and returned Storage implementations are safe for concurrent use.

func NewObjectService added in v1.5.0 

func NewObjectService(opts ...ObjectServiceOption) (*ObjectService, error)

NewObjectService creates an ObjectService with the given options.

func (*ObjectService) CopyObject added in v1.5.0 

func (s *ObjectService) CopyObject(ctx context.Context, req CopyObjectRequest) (*ObjectStat, error)

CopyObject copies an object when the resolved storage supports ObjectCopier.

func (*ObjectService) DeleteObject added in v1.5.0 

func (s *ObjectService) DeleteObject(ctx context.Context, req DeleteObjectRequest) error

DeleteObject deletes an object by key.

func (*ObjectService) DeleteObjects added in v1.5.0 

func (s *ObjectService) DeleteObjects(ctx context.Context, req DeleteObjectsRequest) error

DeleteObjects deletes multiple objects. Missing objects are treated as success by adapters.

func (*ObjectService) MoveObject added in v1.5.0 

func (s *ObjectService) MoveObject(ctx context.Context, req MoveObjectRequest) (*ObjectStat, error)

MoveObject copies an object to a new key and deletes the source after copy succeeds.

func (*ObjectService) OpenObject added in v1.5.0 

func (s *ObjectService) OpenObject(ctx context.Context, req OpenObjectRequest) (io.ReadCloser, error)

OpenObject opens an object for reading. The caller must close the returned reader.

func (*ObjectService) PublicURL added in v1.5.0 

func (s *ObjectService) PublicURL(key string) string

PublicURL returns a public URL for key when a builder is configured.

func (*ObjectService) SaveObject added in v1.5.0 

func (s *ObjectService) SaveObject(ctx context.Context, req SaveObjectRequest) (*SaveObjectResult, error)

SaveObject saves an object using the resolved storage.

func (*ObjectService) SignedURL added in v1.6.0 

func (s *ObjectService) SignedURL(ctx context.Context, req SignedURLRequest) (string, error)

SignedURL returns a presigned GET URL when the resolved storage supports SignedURLStorage.

Example

ExampleObjectService_SignedURL demonstrates service-level presigned download URLs. 

package main

import (
	"context"
	"fmt"
	"io"
	"log"
	"strings"
	"time"

	"github.com/gtkit/gouploader"
)

type memoryObjectStorage struct {
	data map[string]string
}

func newMemoryObjectStorage() *memoryObjectStorage {
	return &memoryObjectStorage{data: make(map[string]string)}
}

func (s *memoryObjectStorage) Save(_ context.Context, key string, reader io.Reader, _ int64) error {
	body, err := io.ReadAll(reader)
	if err != nil {
		return err
	}
	s.data[key] = string(body)
	return nil
}

func (s *memoryObjectStorage) Get(_ context.Context, key string) (io.ReadCloser, error) {
	return io.NopCloser(strings.NewReader(s.data[key])), nil
}

func (s *memoryObjectStorage) Delete(_ context.Context, key string) error {
	delete(s.data, key)
	return nil
}

func (s *memoryObjectStorage) Exists(_ context.Context, key string) (bool, error) {
	_, ok := s.data[key]
	return ok, nil
}

func (s *memoryObjectStorage) GetSignedGetURL(_ context.Context, key string, ttl time.Duration) (string, error) {
	return fmt.Sprintf("https://signed.example.com/%s?ttl=%s", key, ttl), nil
}

func main() {
	storage := newMemoryObjectStorage()
	objects, err := gouploader.NewObjectService(gouploader.WithObjectStorage(storage))
	if err != nil {
		log.Fatal(err)
	}

	url, err := objects.SignedURL(context.Background(), gouploader.SignedURLRequest{
		Key: "business/app.apk",
		TTL: 15 * time.Minute,
	})
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(url)
}

Output:
https://signed.example.com/business/app.apk?ttl=15m0s

func (*ObjectService) StatObject added in v1.5.0 

func (s *ObjectService) StatObject(ctx context.Context, req StatObjectRequest) (*ObjectStat, error)

StatObject returns object attributes if the resolved storage supports ObjectStatter.

type ObjectServiceOption added in v1.5.0 

type ObjectServiceOption func(*objectServiceConfig)

ObjectServiceOption configures ObjectService.

func WithObjectObserver added in v1.5.0 

func WithObjectObserver(observer func(context.Context, ObjectEvent)) ObjectServiceOption

WithObjectObserver sets an object lifecycle observer.

func WithObjectPublicURLBuilder added in v1.5.0 

func WithObjectPublicURLBuilder(builder func(key string) string) ObjectServiceOption

WithObjectPublicURLBuilder sets the key-to-public-URL builder used by ObjectService.

func WithObjectStorage added in v1.5.0 

func WithObjectStorage(storage Storage) ObjectServiceOption

WithObjectStorage sets a static Storage used by ObjectService.

func WithObjectStorageResolver added in v1.5.0 

func WithObjectStorageResolver(resolver StorageResolver) ObjectServiceOption

WithObjectStorageResolver sets a dynamic resolver used for each object operation.

type ObjectStat added in v1.5.0 

type ObjectStat struct {
	Key                string
	Size               int64
	ETag               string
	VersionID          string
	LastModified       time.Time
	ContentType        string
	ContentDisposition string
	CacheControl       string
	StorageClass       string
	Metadata           map[string]string
}

ObjectStat contains provider-neutral object attributes returned by StatObject.

type ObjectStatter added in v1.5.0 

type ObjectStatter interface {
	Storage
	StatObject(ctx context.Context, key string) (*ObjectStat, error)
}

ObjectStatter is an optional Storage extension for object Stat.

type ObjectStorage added in v1.5.0 

type ObjectStorage interface {
	Storage
	SaveObject(ctx context.Context, req SaveObjectRequest) error
}

ObjectStorage is an optional Storage extension for metadata-aware object save.

type ObjectStorageWithResult added in v1.7.0 

type ObjectStorageWithResult interface {
	Storage
	SaveObjectWithResult(ctx context.Context, req SaveObjectRequest) (*SaveObjectResult, error)
}

ObjectStorageWithResult is an optional Storage extension for metadata-aware object save operations that return provider-neutral saved object attributes.

type ObjectURLConfig added in v1.5.0 

type ObjectURLConfig struct {
	Scheme          string
	Endpoint        string
	Bucket          string
	CDNHost         string
	AddressingStyle AddressingStyle
}

ObjectURLConfig configures provider-neutral object URL building.

type OpenObjectRequest added in v1.5.0 

type OpenObjectRequest struct {
	Scope ObjectScope
	Key   string
}

OpenObjectRequest is the input for ObjectService.OpenObject.

type Operation added in v1.5.0 

type Operation string

Operation identifies a storage operation for idempotency-aware retry decisions. 

const (
	OperationSaveObject        Operation = "SaveObject"
	OperationOpenObject        Operation = "OpenObject"
	OperationDeleteObject      Operation = "DeleteObject"
	OperationDeleteObjects     Operation = "DeleteObjects"
	OperationStatObject        Operation = "StatObject"
	OperationCopyObject        Operation = "CopyObject"
	OperationMoveObject        Operation = "MoveObject"
	OperationInitMultipart     Operation = "InitMultipart"
	OperationPresignPart       Operation = "PresignPart"
	OperationListParts         Operation = "ListParts"
	OperationCompleteMultipart Operation = "CompleteMultipart"
	OperationAbortMultipart    Operation = "AbortMultipart"
	OperationListOrphans       Operation = "ListOrphans"
)

Storage operation names used by RetryPolicy.

func (Operation) Idempotent added in v1.5.0 

func (op Operation) Idempotent() bool

Idempotent reports whether the operation is safe to retry by default.

type Option 

type Option func(*config)

Option 用于配置 Uploader。

func WithAWSSTS 

func WithAWSSTS(cfg AWSSTSConfig) Option

WithAWSSTS 配置 AWS S3 STS 直传。

func WithAliyunSTS 

func WithAliyunSTS(cfg AliyunSTSConfig) Option

WithAliyunSTS 配置阿里云 OSS STS 直传。

func WithAllowedTypes 

func WithAllowedTypes(exts ...string) Option

WithAllowedTypes 设置允许的文件扩展名（如 ".jpg", ".png", ".mp4"）。 空列表表示允许所有类型。

func WithChunkCleanupInterval 

func WithChunkCleanupInterval(interval time.Duration) Option

WithChunkCleanupInterval 设置过期会话的清理间隔。

interval 必须为正数；非法值会让 New() 返回错误（v1.4.0 之前会 panic）。

func WithChunkSessionTTL 

func WithChunkSessionTTL(ttl time.Duration) Option

WithChunkSessionTTL 设置分片上传会话的过期时间。

func WithChunkSize 

func WithChunkSize(defaultSize, minSize, maxSize int64) Option

WithChunkSize 设置默认、最小和最大分片大小。

func WithChunkTempDir 

func WithChunkTempDir(dir string) Option

WithChunkTempDir 设置分片数据的临时目录。

func WithCredentialResolver added in v1.5.0 

func WithCredentialResolver(resolver CredentialResolver) Option

WithCredentialResolver sets a dynamic short-lived credential resolver for STS direct-upload token issuing.

func WithHashAlgorithm 

func WithHashAlgorithm(algo string) Option

WithHashAlgorithm 设置校验和算法（"md5" 或 "sha256"）。

func WithHashStore added in v1.2.0 

func WithHashStore(hs HashStore) Option

WithHashStore 注入秒传索引存储。nil 时秒传功能关闭。

典型用法： 

type dbHashStore struct{ db *gorm.DB }
func (s *dbHashStore) LookupByHash(ctx, algo, hash) (*HashedFile, error) { ... }
func (s *dbHashStore) Register(ctx, entry *HashedFile) error { ... }

gouploader.WithHashStore(&dbHashStore{db: db})

func WithHashStoreRegisterTimeout added in v1.3.0 

func WithHashStoreRegisterTimeout(d time.Duration) Option

WithHashStoreRegisterTimeout 设置 HashStore.Register 异步注册的最大等待时长。 默认 30 秒。HashStore 实现卡死时，此超时防止后台 goroutine 无限累积。 业务方根据下游存储 SLA 自行调整（如 Redis 一般 100ms，MySQL 写入一般 1-2s）。

func WithHuaweiSTS 

func WithHuaweiSTS(cfg HuaweiSTSConfig) Option

WithHuaweiSTS 配置华为云 OBS STS 直传。

func WithKeyPrefixSigner added in v1.4.0 

func WithKeyPrefixSigner(secret []byte) Option

WithKeyPrefixSigner 启用 STS 直传 KeyPrefix HMAC 签名（v1.4.0+，强烈建议配置）。

配置后：

  • IssueToken 返回的 STSCredential 增加 KeyPrefixSig 字段（HMAC-SHA256 base64）
  • Complete 请求必须传 KeyPrefixSig，服务端验签通过才能 Complete

防御场景：v1.3.0 / v1.4.0-rc1 中，攻击者凭知道的 KeyPrefix（如默认日期格式公开可见） 可调 Complete 越权"确认"他人文件，触发 onComplete 钩子被记账到攻击者名下， 或拿到他人 storage_key + URL（PublicURLBuilder 拼出）。HMAC 签名让 KeyPrefix 必须由本服务进程签发才能通过 Complete。

secret 长度建议 ≥ 32 字节随机值；多实例部署时所有实例必须使用相同 secret （否则跨实例 IssueToken/Complete 会失败）。secret 不可对外泄漏。

推荐用法： 

secret := []byte(os.Getenv("UPLOAD_KEY_PREFIX_SECRET")) // ≥ 32 字节
uploader, _ := gouploader.New(
    gouploader.WithKeyPrefixSigner(secret),
    gouploader.WithAliyunSTS(...),
)

不配置时退化到不签名行为（向后兼容），但 README 红色警告"多租户场景必须启用"。

func WithLocalStorage 

func WithLocalStorage(basePath string) Option

WithLocalStorage 配置本地磁盘存储。

func WithMaxFileSize 

func WithMaxFileSize(size int64) Option

WithMaxFileSize 设置允许的最大文件大小（字节）。

func WithMultipartDisabled added in v1.2.0 

func WithMultipartDisabled() Option

WithMultipartDisabled 强制关闭 Multipart 直传路径，所有大文件都走代理合并。 用于灰度回滚或调试。默认启用（前提是 Storage 实现了 MultipartStorage 接口）。

func WithMultipartFeatureFlag added in v1.2.0 

func WithMultipartFeatureFlag(fn func(ctx context.Context, req *ChunkInitRequest) bool) Option

WithMultipartFeatureFlag 注入 Feature Flag 钩子，按请求动态决定是否走 Multipart 路径。 典型用法：灰度发布 / 按用户 ID hash / 按 bucket 分流。 

gouploader.WithMultipartFeatureFlag(func(ctx, ctx, req) bool {
    uid := ctx.GetString("user_id")
    return hashMod(uid) < 10  // 10% 灰度
})

func WithMultipartKeyPrefix added in v1.2.0 

func WithMultipartKeyPrefix(fn func(ctx context.Context, userID string) string) Option

WithMultipartKeyPrefix 生成 Multipart 路径下的对象键前缀。 默认按日期生成 "{year}/{month}/{day}/"。

典型用法（按用户隔离）： 

gouploader.WithMultipartKeyPrefix(func(c, userID) string {
    return fmt.Sprintf("user-%s/%s/", userID, time.Now().Format("2006-01-02"))
})

func WithMultipartThreshold added in v1.2.0 

func WithMultipartThreshold(bytes int64) Option

WithMultipartThreshold 设置走 Multipart 直传的最小文件大小（字节）。 小于此阈值的文件强制走代理路径——省掉签名开销。默认 100MB。

func WithOnDirectUploadComplete 

func WithOnDirectUploadComplete(fn func(result *DirectUploadResult)) Option

WithOnDirectUploadComplete 注册直传完成后的回调。 在 Complete 或 Callback 确认文件上传成功后触发。

func WithOnInstantHit added in v1.2.0 

func WithOnInstantHit(fn func(ctx context.Context, ev *InstantHitEvent)) Option

WithOnInstantHit 注册秒传命中钩子。 用于统计命中率、节省流量等业务指标。 仅在 HashStore 配置且 LookupByHash 返回非 nil 时触发。

func WithOnUploadAbort added in v1.2.0 

func WithOnUploadAbort(fn func(ctx context.Context, ev *UploadAbortEvent)) Option

WithOnUploadAbort 注册上传中止钩子。 触发时机：客户端 DELETE /chunk/:id 清理完 session 后（云端 Multipart 也已 Abort）。

func WithOnUploadComplete 

func WithOnUploadComplete(fn func(result *UploadResult)) Option

WithOnUploadComplete 注册上传完成后的回调函数。 每次上传成功后触发（包括普通上传和分片合并）。 可用于保存数据库、发送通知等业务逻辑。

func WithOnUploadError added in v1.2.0 

func WithOnUploadError(fn func(ctx context.Context, ev *UploadErrorEvent)) Option

WithOnUploadError 注册上传阶段失败钩子。 触发阶段见 Stage 常量：StageInit / StagePart / StageMerge / StageComplete / StageAbort。

func WithOnUploadInit added in v1.2.0 

func WithOnUploadInit(fn func(ctx context.Context, ev *UploadInitEvent)) Option

WithOnUploadInit 注册上传会话初始化成功钩子。 触发时机：InitiateMultipart（云端）或本地 session 创建成功后。 秒传命中时不触发此钩子，改为触发 WithOnInstantHit。

func WithOnUploadLifecycle added in v1.5.0 

func WithOnUploadLifecycle(fn func(ctx context.Context, ev *UploadLifecycleEvent)) Option

WithOnUploadLifecycle 注册上传成功生命周期钩子。 覆盖 part sign、part complete、complete success 等事件。

func WithOrphanCleanup added in v1.2.0 

func WithOrphanCleanup(ttl, interval time.Duration) Option

WithOrphanCleanup 启用悬挂 Multipart 清理 worker。 ttl: 超过此时长未完成的 Multipart 视为悬挂，会被 Abort。默认 24h。 interval: 清理 worker 扫描间隔。默认 1h。

注意：此功能需要 Storage 实现 MultipartStorage 接口（包含 ListOrphans / AbortMultipart）。 建议同时在 bucket 层面配置 AbortIncompleteMultipartUpload 生命周期规则做双保险。

func WithPartSignTTL added in v1.2.0 

func WithPartSignTTL(ttl time.Duration) Option

WithPartSignTTL 设置单个 part 预签名 URL 的有效期。默认 15 分钟。 太短会导致网络慢的客户端 URL 过期，太长会扩大 URL 泄漏的爆炸半径。

func WithPublicURLBuilder added in v1.3.0 

func WithPublicURLBuilder(fn func(storageKey string) string) Option

WithPublicURLBuilder 注入"对外可见 URL"拼装策略。

配置后，所有上传完成路径返回的 result（UploadResult / DirectUploadResult）的 URL 字段会被填充为 builder(storageKey) 的输出。覆盖路径：

  • 普通上传 POST /
  • 代理分片合并 POST /chunk/merge/:upload_id
  • Multipart 直传合并 POST /chunk/complete
  • 秒传命中 POST /chunk/init
  • STS 直传 POST /direct/complete
  • STS 直传回调 POST /direct/callback

builder 应保持轻量（纯字符串拼接），避免阻塞主上传流程。 builder 内 panic 被 recover 隔离，不影响上传；但 URL 字段会为空字符串。

典型用法（CDN 前缀拼接）： 

gouploader.WithPublicURLBuilder(func(key string) string {
    return "https://static.example.com/" + key
})

私有 bucket 临时访问 URL 示例（业务方自行调云 SDK 签名）： 

gouploader.WithPublicURLBuilder(func(key string) string {
    url, _ := storage.GetSignedGetURL(context.Background(), key, 15*time.Minute)
    return url
})

不配置 / 设为 nil 时，所有 result.URL 字段为空，JSON 因 omitempty 不输出， 行为等价于未引入此特性的旧版本。

func WithRequireUserID added in v1.4.0 

func WithRequireUserID() Option

WithRequireUserID 强制要求 UserIDFn 返回非空字符串，否则禁用秒传查询。

默认行为（不启用）：UserIDFn 返回空字符串时退化为 "anon"，匿名用户共享秒传索引。 多租户私有 bucket 场景应启用此 Option，避免匿名场景下用户 B 凭 hash 命中 用户 A 的私有文件 storage_key（即使 HashStore 实现了 UserScopedHashStore， "anon" 用户仍是一个共享的虚拟身份）。

启用后，对于 UserIDFn 返回空字符串的请求：

  • InitUpload 秒传查询路径直接跳过（返回未命中，走正常上传流程）
  • Resume 秒传查询路径同上

推荐多租户业务方一律启用： 

gouploader.WithUserIDFn(func(ctx context.Context) string {
    if user, ok := c.Get("user_id"); ok { return user.(string) }
    return ""  // 未登录返回空
}),
gouploader.WithRequireUserID(),

func WithS3Storage 

func WithS3Storage(s3 S3Config) Option

WithS3Storage 配置 S3 兼容对象存储。

func WithSTSKeyPrefix 

func WithSTSKeyPrefix(fn func(ctx context.Context) string) Option

WithSTSKeyPrefix 设置生成 keyPrefix 的函数。 默认按日期生成，如 "direct/2026/04/09/"。 可用于按用户 ID 隔离：func(ctx context.Context) string { return "user-" + getUserID(c) + "/" }.

func WithSTSProvider 

func WithSTSProvider(p STSProvider) Option

WithSTSProvider 设置自定义 STS Provider 实现。

func WithSTSProviderResolver added in v1.5.0 

func WithSTSProviderResolver(resolver STSProviderResolver) Option

WithSTSProviderResolver sets a dynamic STS provider resolver for direct-upload token issuing.

func WithStorage 

func WithStorage(s Storage) Option

WithStorage 设置自定义 Storage 实现。 会覆盖 WithLocalStorage 和 WithS3Storage。

func WithStorageResolver added in v1.5.0 

func WithStorageResolver(resolver StorageResolver) Option

WithStorageResolver sets a dynamic StorageResolver for chunk/direct main flows.

func WithStrictPartsVerify added in v1.4.0 

func WithStrictPartsVerify(strict bool) Option

WithStrictPartsVerify 设置 Multipart 直传 Complete 前的服务端二次校验严格度。

strict=true（默认）：调用云端 ListParts 失败时拒绝 Complete，返回 

ErrPartVerifyFail.WithDetails("云端校验不可用")

这是默认行为，安全优先——避免客户端伪造 ETag 在云 API 抖动时通过校验。

strict=false：ListParts 失败时放行 Complete，但日志级别从 Warn 升级为 Error 便于业务方监控降级。仅在云 API 抖动率高且业务可接受弱校验时启用。

v1.4.0 之前的行为等价于 strict=false（但日志为 Warn），新版默认 strict=true。

func WithTencentSTS 

func WithTencentSTS(cfg TencentSTSConfig) Option

WithTencentSTS 配置腾讯云 COS STS 直传。

func WithUserIDFn added in v1.2.0 

func WithUserIDFn(fn func(ctx context.Context) string) Option

WithUserIDFn 从 Gin 上下文提取用户标识，用于 keyPrefix 隔离和 hash 索引二级键。 未配置时默认返回 "anon"。

type OrphanMultipart added in v1.2.0 

type OrphanMultipart = types.OrphanMultipart

OrphanMultipart 悬挂 Multipart 元信息。

type PartETag added in v1.2.0 

type PartETag = types.PartETag

PartETag 分片号 + ETag 的键值对，用于 Multipart 合并。

type PartInfo added in v1.2.0 

type PartInfo = types.PartInfo

PartInfo 云端分片元信息。

type PartSignRequest added in v1.2.0 

type PartSignRequest = types.PartSignRequest

PartSignRequest 按需签名请求体。

type PartSignResponse added in v1.2.0 

type PartSignResponse = types.PartSignResponse

PartSignResponse 按需签名响应。

type PresignPartRequest added in v1.2.0 

type PresignPartRequest = types.PresignPartRequest

PresignPartRequest 预签名单片请求。

type ProviderError added in v1.5.0 

type ProviderError struct {
	Kind       ErrorKind
	Provider   string
	Operation  string
	StatusCode int
	Code       string
	RequestID  string
	Err        error
}

ProviderError preserves provider details while exposing a stable ErrorKind.

func NewProviderError added in v1.5.0 

func NewProviderError(info ProviderErrorInfo) *ProviderError

NewProviderError creates a normalized provider error.

func (*ProviderError) Error added in v1.5.0 

func (e *ProviderError) Error() string

func (*ProviderError) Is added in v1.5.0 

func (e *ProviderError) Is(target error) bool

Is supports errors.Is against provider-neutral sentinel errors.

func (*ProviderError) Unwrap added in v1.5.0 

func (e *ProviderError) Unwrap() error

Unwrap returns the original provider error.

type ProviderErrorInfo added in v1.5.0 

type ProviderErrorInfo struct {
	Kind       ErrorKind
	Provider   string
	Operation  string
	StatusCode int
	Code       string
	RequestID  string
	Err        error
}

ProviderErrorInfo contains normalized provider error details.

type ResumeRequest added in v1.2.0 

type ResumeRequest = types.ResumeRequest

ResumeRequest 秒传 + 断点续传查询请求体。

type ResumeResponse added in v1.2.0 

type ResumeResponse = types.ResumeResponse

ResumeResponse 秒传 + 断点续传查询响应。

type RetryPolicy added in v1.5.0 

type RetryPolicy struct {
	MaxAttempts int
	Backoff     BackoffPolicy
}

RetryPolicy controls retries for retryable, idempotent operations.

func (RetryPolicy) Do added in v1.5.0 

func (p RetryPolicy) Do(ctx context.Context, operation Operation, fn func() error) error

Do executes fn with idempotency-aware retry behavior.

type S3Config 

type S3Config struct {
	Endpoint  string
	AccessKey string
	SecretKey string
	Bucket    string
	Region    string
	UseSSL    bool

	// CDNHost 配置了 CDN 加速域名时，GetSignedGetURL 返回的预签名 URL 会用此域名替换原始 endpoint。
	//
	// 接受三种输入格式（自动 normalize）：纯 host / 带 scheme / 带 scheme + path。
	// 注意：CDN 必须在云厂商控制台绑定为对应 bucket 加速域名，否则签名校验失败 403。
	CDNHost string
}

S3Config 包含 S3 兼容对象存储的连接参数。

Endpoint 示例： 

MinIO:      "127.0.0.1:9000"
阿里云 OSS:  "oss-cn-hangzhou.aliyuncs.com"
腾讯云 COS:  "cos.ap-guangzhou.myqcloud.com"
华为云 OBS:  "obs.cn-north-4.myhuaweicloud.com"
AWS S3:     "s3.us-east-1.amazonaws.com"

type STSCredential 

type STSCredential = types.STSCredential

STSCredential STS 临时凭证。

type STSProvider 

type STSProvider = types.STSProvider

STSProvider STS 凭证签发接口。

type STSProviderResolver added in v1.5.0 

type STSProviderResolver interface {
	ResolveSTSProvider(ctx context.Context) (STSProvider, error)
}

STSProviderResolver resolves the STS provider used by one direct-upload request. It is safe for concurrent use when the implementation is safe for concurrent use.

type STSProviderResolverFunc added in v1.5.0 

type STSProviderResolverFunc func(context.Context) (STSProvider, error)

STSProviderResolverFunc adapts a function to STSProviderResolver.

func (STSProviderResolverFunc) ResolveSTSProvider added in v1.5.0 

func (fn STSProviderResolverFunc) ResolveSTSProvider(ctx context.Context) (STSProvider, error)

ResolveSTSProvider resolves the STS provider used by one direct-upload request.

type SaveObjectRequest added in v1.5.0 

type SaveObjectRequest struct {
	Scope    ObjectScope
	Key      string
	Body     io.Reader
	Size     int64
	Metadata ObjectMetadata
}

SaveObjectRequest is the input for ObjectService.SaveObject.

type SaveObjectResult added in v1.5.0 

type SaveObjectResult struct {
	Key          string
	Size         int64
	ContentType  string
	URL          string
	ETag         string
	VersionID    string
	Metadata     map[string]string
	StorageClass string
}

SaveObjectResult describes a saved object.

type SaveTempRequest added in v1.5.0 

type SaveTempRequest struct {
	Scope    ObjectScope
	Key      string
	Body     io.Reader
	Size     int64
	Metadata ObjectMetadata
}

SaveTempRequest is the input for UploadWorkflow.SaveTemp.

type SignedURLRequest added in v1.6.0 

type SignedURLRequest struct {
	Scope ObjectScope
	Key   string
	TTL   time.Duration
}

SignedURLRequest is the input for ObjectService.SignedURL.

type SignedURLStorage added in v1.4.0 

type SignedURLStorage = types.SignedURLStorage

SignedURLStorage 是 Storage 的可选扩展接口，提供"生成预签名 GET URL"能力。 业务方通过 type assertion 检测；详见 types.SignedURLStorage 文档。

内置实现：aliyun / tencent / huawei adapter + S3Storage。LocalStorage 不实现。

type StatObjectRequest added in v1.5.0 

type StatObjectRequest struct {
	Scope ObjectScope
	Key   string
}

StatObjectRequest is the input for ObjectService.StatObject.

type Storage 

type Storage = types.Storage

Storage 抽象文件持久化层。 所有方法均使用 io.Reader 流式传输，不会将完整文件缓冲到内存。

内置实现：本地磁盘、S3 兼容（MinIO/阿里云/腾讯云/华为云/AWS）。 也可通过 WithStorage 选项提供自定义实现。

type StorageResolver added in v1.5.0 

type StorageResolver interface {
	ResolveStorage(ctx context.Context, scope ObjectScope) (Storage, error)
}

StorageResolver resolves the storage used by a single object operation.

type StorageResolverFunc added in v1.5.0 

type StorageResolverFunc func(context.Context, ObjectScope) (Storage, error)

StorageResolverFunc adapts a function to StorageResolver.

func (StorageResolverFunc) ResolveStorage added in v1.5.0 

func (fn StorageResolverFunc) ResolveStorage(ctx context.Context, scope ObjectScope) (Storage, error)

ResolveStorage resolves a storage for one object operation.

type TencentSTSConfig 

type TencentSTSConfig = sts.TencentConfig

TencentSTSConfig 腾讯云 COS STS 配置。

type UploadAbortEvent added in v1.2.0 

type UploadAbortEvent = types.UploadAbortEvent

UploadAbortEvent 会话被中止时触发的生命周期钩子 payload。

type UploadErrorEvent added in v1.2.0 

type UploadErrorEvent = types.UploadErrorEvent

UploadErrorEvent 任一上传阶段失败时触发的生命周期钩子 payload。

type UploadInfo added in v1.5.0 

type UploadInfo struct {
	UserID     string
	BusinessID string
	BucketID   string
	TaskID     string
	TempFileID string
	Provider   string
	Bucket     string
	Labels     map[string]string
}

UploadInfo carries business identifiers supplied by the caller.

func UploadInfoFromContext added in v1.5.0 

func UploadInfoFromContext(ctx context.Context) (UploadInfo, bool)

UploadInfoFromContext extracts business identifiers from context.

type UploadInitEvent added in v1.2.0 

type UploadInitEvent = types.UploadInitEvent

UploadInitEvent 会话初始化成功时触发的生命周期钩子 payload。

type UploadLifecycleEvent added in v1.5.0 

type UploadLifecycleEvent = types.UploadLifecycleEvent

UploadLifecycleEvent 上传成功生命周期事件 payload。

type UploadResult 

type UploadResult = types.UploadResult

UploadResult 文件上传完成后返回的结果（普通上传和分片合并）。

type UploadWorkflow added in v1.5.0 

type UploadWorkflow struct {
	// contains filtered or unexported fields
}

UploadWorkflow provides business-oriented temp-to-final upload primitives.

func NewUploadWorkflow added in v1.5.0 

func NewUploadWorkflow(objects *ObjectService) *UploadWorkflow

NewUploadWorkflow creates a business-oriented workflow facade over ObjectService.

func (*UploadWorkflow) AbortUpload added in v1.5.0 

func (w *UploadWorkflow) AbortUpload(ctx context.Context, req AbortUploadRequest) error

AbortUpload deletes temporary objects after a failed or canceled business task.

func (*UploadWorkflow) CompleteUpload added in v1.5.0 

func (w *UploadWorkflow) CompleteUpload(ctx context.Context, req CompleteUploadRequest) (*ObjectStat, error)

CompleteUpload moves a temporary object to its final object key.

func (*UploadWorkflow) SaveTemp added in v1.5.0 

func (w *UploadWorkflow) SaveTemp(ctx context.Context, req SaveTempRequest) (*SaveObjectResult, error)

SaveTemp saves a temporary object.

type Uploader 

type Uploader struct {
	// contains filtered or unexported fields
}

Uploader is the service container for upload operations. It is safe for concurrent use when the configured storage and stores are safe for concurrent use.

func New 

func New(opts ...Option) (*Uploader, error)

New creates an Uploader service container with the given options.

func (*Uploader) ChunkService added in v1.5.0 

func (u *Uploader) ChunkService() *service.ChunkUploadService

ChunkService returns the multipart/chunk upload service.

func (*Uploader) Close 

func (u *Uploader) Close()

Close releases background resources owned by Uploader.

func (*Uploader) DirectService added in v1.5.0 

func (u *Uploader) DirectService() *service.DirectUploadService

DirectService returns the STS direct-upload service, or nil when STS is disabled.

func (*Uploader) Storage added in v1.5.0 

func (u *Uploader) Storage() Storage

Storage returns the configured storage backend.

func (*Uploader) UploadService added in v1.5.0 

func (u *Uploader) UploadService() *service.UploadService

UploadService returns the direct service for proxy uploads.

type UserScopedHashStore added in v1.4.0 

type UserScopedHashStore = types.UserScopedHashStore

UserScopedHashStore 是 HashStore 的可选扩展接口，提供按用户隔离的秒传索引。 多租户场景必须使用。详见 types.UserScopedHashStore 文档。

Source Files

View all Source files

Directories

Path Synopsis
aliyun module
aws module
huawei module
internal
chunk
hash
logx
s3adapter
service
store
sts
types
minio module
tencent module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.