s3

package module
v0.0.0-...-49b6d02 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 4, 2022 License: Apache-2.0 Imports: 25 Imported by: 0

README

Services Test S3

s3

AWS S3 service support for go-storage.

Install

go get go.beyondstorage.io/services/s3/v3

Usage

import (
	"log"

	_ "go.beyondstorage.io/services/s3/v3"
	"go.beyondstorage.io/v5/services"
)

func main() {
	store, err := services.NewStoragerFromString("s3://bucket_name/path/to/workdir")
	if err != nil {
		log.Fatal(err)
	}

	// Write data from io.Reader into hello.txt
	n, err := store.Write("hello.txt", r, length)
}

Compatible Services

We can use go-service-s3 for the following services:

Documentation

Overview

Package s3 provided support for AWS s3 (https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html)

Code generated by go generate via cmd/definitions; DO NOT EDIT.

Index

Constants

View Source 
const (
	StorageClassStandard           = string(s3types.ObjectStorageClassStandard)
	StorageClassReducedRedundancy  = string(s3types.ObjectStorageClassReducedRedundancy)
	StorageClassGlacier            = string(s3types.ObjectStorageClassGlacier)
	StorageClassStandardIa         = string(s3types.ObjectStorageClassStandardIa)
	StorageClassOnezoneIa          = string(s3types.ObjectStorageClassOnezoneIa)
	StorageClassIntelligentTiering = string(s3types.ObjectStorageClassIntelligentTiering)
	StorageClassDeepArchive        = string(s3types.ObjectStorageClassDeepArchive)
)

All available storage classes are listed here.

View Source 
const (
	ServerSideEncryptionAes256 = string(s3types.ServerSideEncryptionAes256)
	ServerSideEncryptionAwsKms = string(s3types.ServerSideEncryptionAwsKms)
)

All available server side algorithm are listed here.

View Source 
const Type = "s3"

Type is the type for s3

Variables

View Source 
var (
	// ErrServerSideEncryptionCustomerKeyInvalid will be returned while server-side encryption customer key is invalid.
	ErrServerSideEncryptionCustomerKeyInvalid = services.NewErrorCode("invalid server-side encryption customer key")
)

Functions

func New 

func New(pairs ...typ.Pair) (typ.Servicer, typ.Storager, error)

New will create both Servicer and Storager.

func NewServicer 

func NewServicer(pairs ...typ.Pair) (typ.Servicer, error)

NewServicer will create Servicer only.

func NewStorager 

func NewStorager(pairs ...typ.Pair) (typ.Storager, error)

NewStorager will create Storager only.

func WithDefaultStorageClass 

func WithDefaultStorageClass(v string) types.Pair

WithDefaultStorageClass will apply default_storage_class value to Options.

default value for storage_class

func WithDisable100Continue 

func WithDisable100Continue() types.Pair

WithDisable100Continue will apply disable_100_continue value to Options.

set this to `true` to disable the SDK adding the `Expect: 100-Continue` header to PUT requests over 2MB of content

func WithExpectedBucketOwner 

func WithExpectedBucketOwner(v string) types.Pair

WithExpectedBucketOwner will apply expected_bucket_owner value to Options.

the account ID of the expected bucket owner

func WithForcePathStyle 

func WithForcePathStyle() types.Pair

WithForcePathStyle will apply force_path_style value to Options.

see http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html for Amazon S3: Virtual Hosting of Buckets

func WithServerSideEncryption 

func WithServerSideEncryption(v string) types.Pair

WithServerSideEncryption will apply server_side_encryption value to Options.

the server-side encryption algorithm used when storing this object in Amazon

func WithServerSideEncryptionAwsKmsKeyID 

func WithServerSideEncryptionAwsKmsKeyID(v string) types.Pair

WithServerSideEncryptionAwsKmsKeyID will apply server_side_encryption_aws_kms_key_id value to Options.

specifies the AWS KMS key ID to use for object encryption

func WithServerSideEncryptionBucketKeyEnabled 

func WithServerSideEncryptionBucketKeyEnabled() types.Pair

WithServerSideEncryptionBucketKeyEnabled will apply server_side_encryption_bucket_key_enabled value to Options.

specifies whether Amazon S3 should use an S3 Bucket Key for object encryption with server-side encryption using AWS KMS (SSE-KMS)

func WithServerSideEncryptionContext 

func WithServerSideEncryptionContext(v string) types.Pair

WithServerSideEncryptionContext will apply server_side_encryption_context value to Options.

specifies the AWS KMS Encryption Context to use for object encryption. The value of this header is a base64-encoded UTF-8 string holding JSON with the encryption context key-value pairs.

func WithServerSideEncryptionCustomerAlgorithm 

func WithServerSideEncryptionCustomerAlgorithm(v string) types.Pair

WithServerSideEncryptionCustomerAlgorithm will apply server_side_encryption_customer_algorithm value to Options.

specifies the algorithm to use to when encrypting the object. The header value must be `AES256`.

func WithServerSideEncryptionCustomerKey 

func WithServerSideEncryptionCustomerKey(v []byte) types.Pair

WithServerSideEncryptionCustomerKey will apply server_side_encryption_customer_key value to Options.

specifies the customer-provided encryption key for Amazon S3 to use to encrypt/decrypt the source object. It must be 32-byte AES-256 key.

func WithStorageClass 

func WithStorageClass(v string) types.Pair

WithStorageClass will apply storage_class value to Options.

func WithUseAccelerate 

func WithUseAccelerate() types.Pair

WithUseAccelerate will apply use_accelerate value to Options.

set this to `true` to enable S3 Accelerate feature

func WithUseArnRegion 

func WithUseArnRegion() types.Pair

WithUseArnRegion will apply use_arn_region value to Options.

set this to `true` to have the S3 service client to use the region specified in the ARN, when an ARN is provided as an argument to a bucket parameter

Types

type DefaultServicePairs deprecated 

type DefaultServicePairs = types.DefaultServicePairs

Deprecated: Use types.DefaultServicePairs instead.

type DefaultStoragePairs deprecated 

type DefaultStoragePairs = types.DefaultStoragePairs

Deprecated: Use types.DefaultStoragePairs instead.

type Factory 

type Factory struct {
	Credential          string
	DefaultStorageClass string
	EnableVirtualDir    bool
	EnableVirtualLink   bool
	Endpoint            string
	ForcePathStyle      bool
	Location            string
	Name                string
	UseAccelerate       bool
	UseArnRegion        bool
	WorkDir             string
}

func (*Factory) FromMap 

func (f *Factory) FromMap(m map[string]interface{}) (err error)

func (*Factory) FromString 

func (f *Factory) FromString(conn string) (err error)

func (*Factory) NewServicer 

func (f *Factory) NewServicer() (srv types.Servicer, err error)

func (*Factory) NewStorager 

func (f *Factory) NewStorager() (sto types.Storager, err error)

func (*Factory) WithPairs 

func (f *Factory) WithPairs(ps ...types.Pair) (err error)

type ObjectSystemMetadata 

type ObjectSystemMetadata struct {
	ServerSideEncryption                  string
	ServerSideEncryptionAwsKmsKeyID       string
	ServerSideEncryptionBucketKeyEnabled  bool
	ServerSideEncryptionContext           string
	ServerSideEncryptionCustomerAlgorithm string
	ServerSideEncryptionCustomerKeyMd5    string
	StorageClass                          string
}

ObjectSystemMetadata stores system metadata for object.

func GetObjectSystemMetadata 

func GetObjectSystemMetadata(o *types.Object) ObjectSystemMetadata

GetObjectSystemMetadata will get ObjectSystemMetadata from Object.

- This function should not be called by service implementer. - The returning ObjectServiceMetadata is read only and should not be modified.

type Service 

type Service struct {
	typ.UnimplementedServicer
	// contains filtered or unexported fields
}

Service is the s3 service config.

func (*Service) Create 

func (s *Service) Create(name string, pairs ...types.Pair) (store types.Storager, err error)

func (*Service) CreateWithContext 

func (s *Service) CreateWithContext(ctx context.Context, name string, pairs ...types.Pair) (store types.Storager, err error)

func (*Service) Delete 

func (s *Service) Delete(name string, pairs ...types.Pair) (err error)

func (*Service) DeleteWithContext 

func (s *Service) DeleteWithContext(ctx context.Context, name string, pairs ...types.Pair) (err error)

func (*Service) Features 

func (s *Service) Features() types.ServiceFeatures

func (*Service) Get 

func (s *Service) Get(name string, pairs ...types.Pair) (store types.Storager, err error)

func (*Service) GetWithContext 

func (s *Service) GetWithContext(ctx context.Context, name string, pairs ...types.Pair) (store types.Storager, err error)

func (*Service) List 

func (s *Service) List(pairs ...types.Pair) (sti *types.StoragerIterator, err error)

func (*Service) ListWithContext 

func (s *Service) ListWithContext(ctx context.Context, pairs ...types.Pair) (sti *types.StoragerIterator, err error)

func (*Service) String 

func (s *Service) String() string

String implements Servicer.String

type ServiceFeatures deprecated 

type ServiceFeatures = types.ServiceFeatures

Deprecated: Use types.ServiceFeatures instead.

type Storage 

type Storage struct {
	typ.UnimplementedStorager
	// contains filtered or unexported fields
}

Storage is the s3 object storage service.

func (*Storage) CombineBlock 

func (s *Storage) CombineBlock(o *types.Object, bids []string, pairs ...types.Pair) (err error)

func (*Storage) CombineBlockWithContext 

func (s *Storage) CombineBlockWithContext(ctx context.Context, o *types.Object, bids []string, pairs ...types.Pair) (err error)

func (*Storage) CommitAppend 

func (s *Storage) CommitAppend(o *types.Object, pairs ...types.Pair) (err error)

func (*Storage) CommitAppendWithContext 

func (s *Storage) CommitAppendWithContext(ctx context.Context, o *types.Object, pairs ...types.Pair) (err error)

func (*Storage) CompleteMultipart 

func (s *Storage) CompleteMultipart(o *types.Object, parts []*types.Part, pairs ...types.Pair) (err error)

func (*Storage) CompleteMultipartWithContext 

func (s *Storage) CompleteMultipartWithContext(ctx context.Context, o *types.Object, parts []*types.Part, pairs ...types.Pair) (err error)

func (*Storage) Copy 

func (s *Storage) Copy(src string, dst string, pairs ...types.Pair) (err error)

func (*Storage) CopyWithContext 

func (s *Storage) CopyWithContext(ctx context.Context, src string, dst string, pairs ...types.Pair) (err error)

func (*Storage) Create 

func (s *Storage) Create(path string, pairs ...types.Pair) (o *types.Object)

func (*Storage) CreateAppend 

func (s *Storage) CreateAppend(path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateAppendWithContext 

func (s *Storage) CreateAppendWithContext(ctx context.Context, path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateBlock 

func (s *Storage) CreateBlock(path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateBlockWithContext 

func (s *Storage) CreateBlockWithContext(ctx context.Context, path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateDir 

func (s *Storage) CreateDir(path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateDirWithContext 

func (s *Storage) CreateDirWithContext(ctx context.Context, path string, pairs ...types.Pair) (o *types.Object, err error)
func (s *Storage) CreateLink(path string, target string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateLinkWithContext 

func (s *Storage) CreateLinkWithContext(ctx context.Context, path string, target string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateMultipart 

func (s *Storage) CreateMultipart(path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreateMultipartWithContext 

func (s *Storage) CreateMultipartWithContext(ctx context.Context, path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreatePage 

func (s *Storage) CreatePage(path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) CreatePageWithContext 

func (s *Storage) CreatePageWithContext(ctx context.Context, path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) Delete 

func (s *Storage) Delete(path string, pairs ...types.Pair) (err error)

func (*Storage) DeleteWithContext 

func (s *Storage) DeleteWithContext(ctx context.Context, path string, pairs ...types.Pair) (err error)

func (*Storage) Features 

func (s *Storage) Features() types.StorageFeatures

func (*Storage) Fetch 

func (s *Storage) Fetch(path string, url string, pairs ...types.Pair) (err error)

func (*Storage) FetchWithContext 

func (s *Storage) FetchWithContext(ctx context.Context, path string, url string, pairs ...types.Pair) (err error)

func (*Storage) List 

func (s *Storage) List(path string, pairs ...types.Pair) (oi *types.ObjectIterator, err error)

func (*Storage) ListBlock 

func (s *Storage) ListBlock(o *types.Object, pairs ...types.Pair) (bi *types.BlockIterator, err error)

func (*Storage) ListBlockWithContext 

func (s *Storage) ListBlockWithContext(ctx context.Context, o *types.Object, pairs ...types.Pair) (bi *types.BlockIterator, err error)

func (*Storage) ListMultipart 

func (s *Storage) ListMultipart(o *types.Object, pairs ...types.Pair) (pi *types.PartIterator, err error)

func (*Storage) ListMultipartWithContext 

func (s *Storage) ListMultipartWithContext(ctx context.Context, o *types.Object, pairs ...types.Pair) (pi *types.PartIterator, err error)

func (*Storage) ListWithContext 

func (s *Storage) ListWithContext(ctx context.Context, path string, pairs ...types.Pair) (oi *types.ObjectIterator, err error)

func (*Storage) Metadata 

func (s *Storage) Metadata(pairs ...types.Pair) (meta *types.StorageMeta)

func (*Storage) Move 

func (s *Storage) Move(src string, dst string, pairs ...types.Pair) (err error)

func (*Storage) MoveWithContext 

func (s *Storage) MoveWithContext(ctx context.Context, src string, dst string, pairs ...types.Pair) (err error)

func (*Storage) QuerySignHTTPCompleteMultipart 

func (s *Storage) QuerySignHTTPCompleteMultipart(o *types.Object, parts []*types.Part, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPCompleteMultipartWithContext 

func (s *Storage) QuerySignHTTPCompleteMultipartWithContext(ctx context.Context, o *types.Object, parts []*types.Part, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPCreateMultipart 

func (s *Storage) QuerySignHTTPCreateMultipart(path string, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPCreateMultipartWithContext 

func (s *Storage) QuerySignHTTPCreateMultipartWithContext(ctx context.Context, path string, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPDelete 

func (s *Storage) QuerySignHTTPDelete(path string, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPDeleteWithContext 

func (s *Storage) QuerySignHTTPDeleteWithContext(ctx context.Context, path string, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPListMultipart 

func (s *Storage) QuerySignHTTPListMultipart(o *types.Object, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPListMultipartWithContext 

func (s *Storage) QuerySignHTTPListMultipartWithContext(ctx context.Context, o *types.Object, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPRead 

func (s *Storage) QuerySignHTTPRead(path string, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPReadWithContext 

func (s *Storage) QuerySignHTTPReadWithContext(ctx context.Context, path string, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWrite 

func (s *Storage) QuerySignHTTPWrite(path string, size int64, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWriteMultipart 

func (s *Storage) QuerySignHTTPWriteMultipart(o *types.Object, size int64, index int, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWriteMultipartWithContext 

func (s *Storage) QuerySignHTTPWriteMultipartWithContext(ctx context.Context, o *types.Object, size int64, index int, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) QuerySignHTTPWriteWithContext 

func (s *Storage) QuerySignHTTPWriteWithContext(ctx context.Context, path string, size int64, expire time.Duration, pairs ...types.Pair) (req *http.Request, err error)

func (*Storage) Read 

func (s *Storage) Read(path string, w io.Writer, pairs ...types.Pair) (n int64, err error)

func (*Storage) ReadWithContext 

func (s *Storage) ReadWithContext(ctx context.Context, path string, w io.Writer, pairs ...types.Pair) (n int64, err error)

func (*Storage) Stat 

func (s *Storage) Stat(path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) StatWithContext 

func (s *Storage) StatWithContext(ctx context.Context, path string, pairs ...types.Pair) (o *types.Object, err error)

func (*Storage) String 

func (s *Storage) String() string

String implements Storager.String

func (*Storage) Write 

func (s *Storage) Write(path string, r io.Reader, size int64, pairs ...types.Pair) (n int64, err error)

func (*Storage) WriteAppend 

func (s *Storage) WriteAppend(o *types.Object, r io.Reader, size int64, pairs ...types.Pair) (n int64, err error)

func (*Storage) WriteAppendWithContext 

func (s *Storage) WriteAppendWithContext(ctx context.Context, o *types.Object, r io.Reader, size int64, pairs ...types.Pair) (n int64, err error)

func (*Storage) WriteBlock 

func (s *Storage) WriteBlock(o *types.Object, r io.Reader, size int64, bid string, pairs ...types.Pair) (n int64, err error)

func (*Storage) WriteBlockWithContext 

func (s *Storage) WriteBlockWithContext(ctx context.Context, o *types.Object, r io.Reader, size int64, bid string, pairs ...types.Pair) (n int64, err error)

func (*Storage) WriteMultipart 

func (s *Storage) WriteMultipart(o *types.Object, r io.Reader, size int64, index int, pairs ...types.Pair) (n int64, part *types.Part, err error)

func (*Storage) WriteMultipartWithContext 

func (s *Storage) WriteMultipartWithContext(ctx context.Context, o *types.Object, r io.Reader, size int64, index int, pairs ...types.Pair) (n int64, part *types.Part, err error)

func (*Storage) WritePage 

func (s *Storage) WritePage(o *types.Object, r io.Reader, size int64, offset int64, pairs ...types.Pair) (n int64, err error)

func (*Storage) WritePageWithContext 

func (s *Storage) WritePageWithContext(ctx context.Context, o *types.Object, r io.Reader, size int64, offset int64, pairs ...types.Pair) (n int64, err error)

func (*Storage) WriteWithContext 

func (s *Storage) WriteWithContext(ctx context.Context, path string, r io.Reader, size int64, pairs ...types.Pair) (n int64, err error)

type StorageFeatures deprecated 

type StorageFeatures = types.StorageFeatures

Deprecated: Use types.StorageFeatures instead.

type StorageSystemMetadata 

type StorageSystemMetadata struct {
}

StorageSystemMetadata stores system metadata for object.

func GetStorageSystemMetadata 

func GetStorageSystemMetadata(s *types.StorageMeta) StorageSystemMetadata

GetStorageSystemMetadata will get StorageSystemMetadata from Storage.

- This function should not be called by service implementer. - The returning StorageServiceMetadata is read only and should not be modified.

Source Files

View all Source files

Directories

Path Synopsis
internal
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.