s3

package module
v2.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2021 License: Apache-2.0 Imports: 25 Imported by: 3

README

Build Status Integration Tests License

go-services-s3

AWS S3 service support for go-storage.

Install

go get github.com/beyondstorage/go-service-s3/v2

Usage

import (
	"log"

	_ "github.com/beyondstorage/go-service-s3/v2"
	"github.com/beyondstorage/go-storage/v4/services"
)

func main() {
	store, err := services.NewStoragerFromString("s3://bucket_name/path/to/workdir")
	if err != nil {
		log.Fatal(err)
	}

	// Write data from io.Reader into hello.txt
	n, err := store.Write("hello.txt", r, length)
}

Compatible Services

We can use go-service-s3 for the following services:

Documentation

Overview

Package s3 provided support for AWS s3 (https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html)

Code generated by go generate via cmd/definitions; DO NOT EDIT.

Index

Constants

View Source 
const (
	StorageClassStandard           = s3.ObjectStorageClassStandard
	StorageClassReducedRedundancy  = s3.ObjectStorageClassReducedRedundancy
	StorageClassGlacier            = s3.ObjectStorageClassGlacier
	StorageClassStandardIa         = s3.ObjectStorageClassStandardIa
	StorageClassOnezoneIa          = s3.ObjectStorageClassOnezoneIa
	StorageClassIntelligentTiering = s3.ObjectStorageClassIntelligentTiering
	StorageClassDeepArchive        = s3.ObjectStorageClassDeepArchive
)

All available storage classes are listed here.

View Source 
const (
	ServerSideEncryptionAes256 = s3.ServerSideEncryptionAes256
	ServerSideEncryptionAwsKms = s3.ServerSideEncryptionAwsKms
)

All available server side algorithm are listed here.

View Source 
const Type = "s3"

Type is the type for s3

Variables

View Source 
var (
	// ErrServerSideEncryptionCustomerKeyInvalid will be returned while server-side encryption customer key is invalid.
	ErrServerSideEncryptionCustomerKeyInvalid = services.NewErrorCode("invalid server-side encryption customer key")
)

Functions

func New 

func New(pairs ...typ.Pair) (typ.Servicer, typ.Storager, error)

New will create both Servicer and Storager.

func NewServicer 

func NewServicer(pairs ...typ.Pair) (typ.Servicer, error)

NewServicer will create Servicer only.

func NewStorager 

func NewStorager(pairs ...typ.Pair) (typ.Storager, error)

NewStorager will create Storager only.

func WithDefaultServicePairs 

func WithDefaultServicePairs(v DefaultServicePairs) Pair

WithDefaultServicePairs will apply default_service_pairs value to Options.

func WithDefaultStorageClass added in v2.4.0 

func WithDefaultStorageClass(v string) Pair

WithDefaultStorageClass will apply default_storage_class value to Options.

func WithDefaultStoragePairs 

func WithDefaultStoragePairs(v DefaultStoragePairs) Pair

WithDefaultStoragePairs will apply default_storage_pairs value to Options.

func WithDisable100Continue 

func WithDisable100Continue() Pair

WithDisable100Continue will apply disable_100_continue value to Options.

set this to `true` to disable the SDK adding the `Expect: 100-Continue` header to PUT requests over 2MB of content

func WithEnableVirtualDir added in v2.4.0 

func WithEnableVirtualDir() Pair

WithEnableVirtualDir will apply enable_virtual_dir value to Options.

virtual_dir feature is designed for a service that doesn't have native dir support but wants to provide simulated operations.

- If this feature is disabled (the default behavior), the service will behave like it doesn't have any dir support. - If this feature is enabled, the service will support simulated dir behavior in create_dir, create, list, delete, and so on.

This feature was introduced in GSP-109. 

func WithEnableVirtualLink() Pair

WithEnableVirtualLink will apply enable_virtual_link value to Options.

virtual_link feature is designed for a service that doesn't have native support for link.

- If this feature is enabled, the service will run compatible mode: create link via native methods, but allow read link from old-style link object. - If this feature is not enabled, the service will run in native as other service.

This feature was introduced in GSP-86.

func WithExceptedBucketOwner 

func WithExceptedBucketOwner(v string) Pair

WithExceptedBucketOwner will apply excepted_bucket_owner value to Options.

the account ID of the excepted bucket owner

func WithForcePathStyle 

func WithForcePathStyle() Pair

WithForcePathStyle will apply force_path_style value to Options.

see http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html for Amazon S3: Virtual Hosting of Buckets

func WithServerSideEncryption 

func WithServerSideEncryption(v string) Pair

WithServerSideEncryption will apply server_side_encryption value to Options.

the server-side encryption algorithm used when storing this object in Amazon

func WithServerSideEncryptionAwsKmsKeyID 

func WithServerSideEncryptionAwsKmsKeyID(v string) Pair

WithServerSideEncryptionAwsKmsKeyID will apply server_side_encryption_aws_kms_key_id value to Options.

specifies the AWS KMS key ID to use for object encryption

func WithServerSideEncryptionBucketKeyEnabled 

func WithServerSideEncryptionBucketKeyEnabled() Pair

WithServerSideEncryptionBucketKeyEnabled will apply server_side_encryption_bucket_key_enabled value to Options.

specifies whether Amazon S3 should use an S3 Bucket Key for object encryption with server-side encryption using AWS KMS (SSE-KMS)

func WithServerSideEncryptionContext 

func WithServerSideEncryptionContext(v string) Pair

WithServerSideEncryptionContext will apply server_side_encryption_context value to Options.

specifies the AWS KMS Encryption Context to use for object encryption. The value of this header is a base64-encoded UTF-8 string holding JSON with the encryption context key-value pairs.

func WithServerSideEncryptionCustomerAlgorithm 

func WithServerSideEncryptionCustomerAlgorithm(v string) Pair

WithServerSideEncryptionCustomerAlgorithm will apply server_side_encryption_customer_algorithm value to Options.

specifies the algorithm to use to when encrypting the object. The header value must be `AES256`.

func WithServerSideEncryptionCustomerKey 

func WithServerSideEncryptionCustomerKey(v []byte) Pair

WithServerSideEncryptionCustomerKey will apply server_side_encryption_customer_key value to Options.

specifies the customer-provided encryption key for Amazon S3 to use to encrypt/decrypt the source object. It must be 32-byte AES-256 key.

func WithServiceFeatures added in v2.1.0 

func WithServiceFeatures(v ServiceFeatures) Pair

WithServiceFeatures will apply service_features value to Options.

func WithStorageClass 

func WithStorageClass(v string) Pair

WithStorageClass will apply storage_class value to Options.

func WithStorageFeatures added in v2.1.0 

func WithStorageFeatures(v StorageFeatures) Pair

WithStorageFeatures will apply storage_features value to Options.

func WithUseAccelerate 

func WithUseAccelerate() Pair

WithUseAccelerate will apply use_accelerate value to Options.

set this to `true` to enable S3 Accelerate feature

func WithUseArnRegion 

func WithUseArnRegion() Pair

WithUseArnRegion will apply use_arn_region value to Options.

set this to `true` to have the S3 service client to use the region specified in the ARN, when an ARN is provided as an argument to a bucket parameter

Types

type DefaultServicePairs 

type DefaultServicePairs struct {
	Create []Pair
	Delete []Pair
	Get    []Pair
	List   []Pair
}

DefaultServicePairs is default pairs for specific action

type DefaultStoragePairs 

type DefaultStoragePairs struct {
	CompleteMultipart              []Pair
	Create                         []Pair
	CreateDir                      []Pair
	CreateLink                     []Pair
	CreateMultipart                []Pair
	Delete                         []Pair
	List                           []Pair
	ListMultipart                  []Pair
	Metadata                       []Pair
	QuerySignHTTPCompleteMultipart []Pair
	QuerySignHTTPCreateMultipart   []Pair
	QuerySignHTTPDelete            []Pair
	QuerySignHTTPListMultipart     []Pair
	QuerySignHTTPRead              []Pair
	QuerySignHTTPWrite             []Pair
	QuerySignHTTPWriteMultipart    []Pair
	Read                           []Pair
	Stat                           []Pair
	Write                          []Pair
	WriteMultipart                 []Pair
}

DefaultStoragePairs is default pairs for specific action

type ObjectSystemMetadata added in v2.2.0 

type ObjectSystemMetadata struct {
	ServerSideEncryption                  string
	ServerSideEncryptionAwsKmsKeyID       string
	ServerSideEncryptionBucketKeyEnabled  bool
	ServerSideEncryptionContext           string
	ServerSideEncryptionCustomerAlgorithm string
	ServerSideEncryptionCustomerKeyMd5    string
	StorageClass                          string
}

ObjectSystemMetadata stores system metadata for object.

func GetObjectSystemMetadata added in v2.2.0 

func GetObjectSystemMetadata(o *Object) ObjectSystemMetadata

GetObjectSystemMetadata will get ObjectSystemMetadata from Object.

- This function should not be called by service implementer. - The returning ObjectServiceMetadata is read only and should not be modified.

type Service 

type Service struct {
	typ.UnimplementedServicer
	// contains filtered or unexported fields
}

Service is the s3 service config.

func (*Service) Create 

func (s *Service) Create(name string, pairs ...Pair) (store Storager, err error)

func (*Service) CreateWithContext 

func (s *Service) CreateWithContext(ctx context.Context, name string, pairs ...Pair) (store Storager, err error)

func (*Service) Delete 

func (s *Service) Delete(name string, pairs ...Pair) (err error)

func (*Service) DeleteWithContext 

func (s *Service) DeleteWithContext(ctx context.Context, name string, pairs ...Pair) (err error)

func (*Service) Get 

func (s *Service) Get(name string, pairs ...Pair) (store Storager, err error)

func (*Service) GetWithContext 

func (s *Service) GetWithContext(ctx context.Context, name string, pairs ...Pair) (store Storager, err error)

func (*Service) List 

func (s *Service) List(pairs ...Pair) (sti *StoragerIterator, err error)

func (*Service) ListWithContext 

func (s *Service) ListWithContext(ctx context.Context, pairs ...Pair) (sti *StoragerIterator, err error)

func (*Service) String 

func (s *Service) String() string

String implements Servicer.String

type ServiceFeatures added in v2.1.0 

type ServiceFeatures struct {
}

type Storage 

type Storage struct {
	typ.UnimplementedStorager
	typ.UnimplementedDirer
	typ.UnimplementedMultiparter
	typ.UnimplementedLinker
	typ.UnimplementedStorageHTTPSigner
	typ.UnimplementedMultipartHTTPSigner
	// contains filtered or unexported fields
}

Storage is the s3 object storage service.

func (*Storage) CompleteMultipart 

func (s *Storage) CompleteMultipart(o *Object, parts []*Part, pairs ...Pair) (err error)

func (*Storage) CompleteMultipartWithContext 

func (s *Storage) CompleteMultipartWithContext(ctx context.Context, o *Object, parts []*Part, pairs ...Pair) (err error)

func (*Storage) Create 

func (s *Storage) Create(path string, pairs ...Pair) (o *Object)

func (*Storage) CreateDir added in v2.2.0 

func (s *Storage) CreateDir(path string, pairs ...Pair) (o *Object, err error)

func (*Storage) CreateDirWithContext added in v2.2.0 

func (s *Storage) CreateDirWithContext(ctx context.Context, path string, pairs ...Pair) (o *Object, err error)
func (s *Storage) CreateLink(path string, target string, pairs ...Pair) (o *Object, err error)

func (*Storage) CreateLinkWithContext added in v2.4.0 

func (s *Storage) CreateLinkWithContext(ctx context.Context, path string, target string, pairs ...Pair) (o *Object, err error)

func (*Storage) CreateMultipart 

func (s *Storage) CreateMultipart(path string, pairs ...Pair) (o *Object, err error)

func (*Storage) CreateMultipartWithContext 

func (s *Storage) CreateMultipartWithContext(ctx context.Context, path string, pairs ...Pair) (o *Object, err error)

func (*Storage) Delete 

func (s *Storage) Delete(path string, pairs ...Pair) (err error)

func (*Storage) DeleteWithContext 

func (s *Storage) DeleteWithContext(ctx context.Context, path string, pairs ...Pair) (err error)

func (*Storage) List 

func (s *Storage) List(path string, pairs ...Pair) (oi *ObjectIterator, err error)

func (*Storage) ListMultipart 

func (s *Storage) ListMultipart(o *Object, pairs ...Pair) (pi *PartIterator, err error)

func (*Storage) ListMultipartWithContext 

func (s *Storage) ListMultipartWithContext(ctx context.Context, o *Object, pairs ...Pair) (pi *PartIterator, err error)

func (*Storage) ListWithContext 

func (s *Storage) ListWithContext(ctx context.Context, path string, pairs ...Pair) (oi *ObjectIterator, err error)

func (*Storage) Metadata 

func (s *Storage) Metadata(pairs ...Pair) (meta *StorageMeta)

func (*Storage) QuerySignHTTPCompleteMultipart added in v2.5.0 

func (s *Storage) QuerySignHTTPCompleteMultipart(o *Object, parts []*Part, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPCompleteMultipartWithContext added in v2.5.0 

func (s *Storage) QuerySignHTTPCompleteMultipartWithContext(ctx context.Context, o *Object, parts []*Part, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPCreateMultipart added in v2.5.0 

func (s *Storage) QuerySignHTTPCreateMultipart(path string, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPCreateMultipartWithContext added in v2.5.0 

func (s *Storage) QuerySignHTTPCreateMultipartWithContext(ctx context.Context, path string, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPDelete added in v2.5.0 

func (s *Storage) QuerySignHTTPDelete(path string, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPDeleteWithContext added in v2.5.0 

func (s *Storage) QuerySignHTTPDeleteWithContext(ctx context.Context, path string, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPListMultipart added in v2.5.0 

func (s *Storage) QuerySignHTTPListMultipart(o *Object, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPListMultipartWithContext added in v2.5.0 

func (s *Storage) QuerySignHTTPListMultipartWithContext(ctx context.Context, o *Object, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPRead added in v2.4.0 

func (s *Storage) QuerySignHTTPRead(path string, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPReadWithContext added in v2.4.0 

func (s *Storage) QuerySignHTTPReadWithContext(ctx context.Context, path string, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWrite added in v2.4.0 

func (s *Storage) QuerySignHTTPWrite(path string, size int64, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWriteMultipart added in v2.5.0 

func (s *Storage) QuerySignHTTPWriteMultipart(o *Object, size int64, index int, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWriteMultipartWithContext added in v2.5.0 

func (s *Storage) QuerySignHTTPWriteMultipartWithContext(ctx context.Context, o *Object, size int64, index int, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWriteWithContext added in v2.4.0 

func (s *Storage) QuerySignHTTPWriteWithContext(ctx context.Context, path string, size int64, expire time.Duration, pairs ...Pair) (req *http.Request, err error)

func (*Storage) Read 

func (s *Storage) Read(path string, w io.Writer, pairs ...Pair) (n int64, err error)

func (*Storage) ReadWithContext 

func (s *Storage) ReadWithContext(ctx context.Context, path string, w io.Writer, pairs ...Pair) (n int64, err error)

func (*Storage) Stat 

func (s *Storage) Stat(path string, pairs ...Pair) (o *Object, err error)

func (*Storage) StatWithContext 

func (s *Storage) StatWithContext(ctx context.Context, path string, pairs ...Pair) (o *Object, err error)

func (*Storage) String 

func (s *Storage) String() string

String implements Storager.String

func (*Storage) Write 

func (s *Storage) Write(path string, r io.Reader, size int64, pairs ...Pair) (n int64, err error)

func (*Storage) WriteMultipart 

func (s *Storage) WriteMultipart(o *Object, r io.Reader, size int64, index int, pairs ...Pair) (n int64, part *Part, err error)

func (*Storage) WriteMultipartWithContext 

func (s *Storage) WriteMultipartWithContext(ctx context.Context, o *Object, r io.Reader, size int64, index int, pairs ...Pair) (n int64, part *Part, err error)

func (*Storage) WriteWithContext 

func (s *Storage) WriteWithContext(ctx context.Context, path string, r io.Reader, size int64, pairs ...Pair) (n int64, err error)

type StorageFeatures added in v2.1.0 

type StorageFeatures struct {
	// provide simulated operations.
	//
	// - If this feature is disabled (the default behavior), the service will behave like it doesn't have
	// any dir support.
	// - If this feature is enabled, the service will support simulated dir behavior in create_dir, create,
	// list, delete, and so on.
	//
	// This feature was introduced in GSP-109.
	VirtualDir bool
	// virtual_link feature is designed for a service that doesn't have native support for link.
	//
	// - If this feature is enabled, the service will run compatible mode: create link via native methods,
	// but allow read link from old-style link object.
	// - If this feature is not enabled, the service will run in native as other service.
	//
	// This feature was introduced in GSP-86.
	VirtualLink bool
}

type StorageSystemMetadata added in v2.2.0 

type StorageSystemMetadata struct {
	ServerSideEncryption                  string
	ServerSideEncryptionAwsKmsKeyID       string
	ServerSideEncryptionBucketKeyEnabled  bool
	ServerSideEncryptionContext           string
	ServerSideEncryptionCustomerAlgorithm string
	ServerSideEncryptionCustomerKeyMd5    string
	StorageClass                          string
}

StorageSystemMetadata stores system metadata for object.

func GetStorageSystemMetadata added in v2.2.0 

func GetStorageSystemMetadata(s *StorageMeta) StorageSystemMetadata

GetStorageSystemMetadata will get StorageSystemMetadata from Storage.

- This function should not be called by service implementer. - The returning StorageServiceMetadata is read only and should not be modified.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.