minio

package module
v2.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 7, 2017 License: Apache-2.0 Imports: 36 Imported by: 0

README

Minio Go Client SDK for Amazon S3 Compatible Cloud Storage Slack Sourcegraph

The Minio Go Client SDK provides simple APIs to access any Amazon S3 compatible object storage.

Supported cloud storage providers:

  • AWS Signature Version 4

    • Amazon S3
    • Minio

  • AWS Signature Version 2

    • Google Cloud Storage (Compatibility Mode)
    • Openstack Swift + Swift3 middleware
    • Ceph Object Gateway
    • Riak CS

This quickstart guide will show you how to install the Minio client SDK, connect to Minio, and provide a walkthrough for a simple file uploader. For a complete list of APIs and examples, please take a look at the Go Client API Reference.

This document assumes that you have a working Go development environment.

Download from Github

go get -u github.com/minio/minio-go

Initialize Minio Client

Minio client requires the following four parameters specified to connect to an Amazon S3 compatible object storage.

Parameter Description
endpoint URL to object storage service.
accessKeyID Access key is the user ID that uniquely identifies your account.
secretAccessKey Secret key is the password to your account.
secure Set this value to 'true' to enable secure (HTTPS) access. 
package main

import (
	"github.com/minio/minio-go"
	"log"
)

func main() {
	endpoint := "play.minio.io:9000"
	accessKeyID := "Q3AM3UQ867SPQQA43P2F"
	secretAccessKey := "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
	useSSL := true

	// Initialize minio client object.
	minioClient, err := minio.New(endpoint, accessKeyID, secretAccessKey, useSSL)
	if err != nil {
		log.Fatalln(err)
	}

	log.Println("%v", minioClient) // minioClient is now setup

Quick Start Example - File Uploader

This example program connects to an object storage server, creates a bucket and uploads a file to the bucket.

We will use the Minio server running at https://play.minio.io:9000 in this example. Feel free to use this service for testing and development. Access credentials shown in this example are open to the public.

FileUploader.go
package main

import (
	"github.com/minio/minio-go"
	"log"
)

func main() {
	endpoint := "play.minio.io:9000"
	accessKeyID := "Q3AM3UQ867SPQQA43P2F"
	secretAccessKey := "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
	useSSL := true

	// Initialize minio client object.
	minioClient, err := minio.New(endpoint, accessKeyID, secretAccessKey, useSSL)
	if err != nil {
		log.Fatalln(err)
	}

	// Make a new bucket called mymusic.
	bucketName := "mymusic"
	location := "us-east-1"

	err = minioClient.MakeBucket(bucketName, location)
	if err != nil {
		// Check to see if we already own this bucket (which happens if you run this twice)
		exists, err := minioClient.BucketExists(bucketName)
		if err == nil && exists {
			log.Printf("We already own %s\n", bucketName)
		} else {
			log.Fatalln(err)
		}
	}
	log.Printf("Successfully created %s\n", bucketName)

	// Upload the zip file
	objectName := "golden-oldies.zip"
	filePath := "/tmp/golden-oldies.zip"
	contentType := "application/zip"

	// Upload the zip file with FPutObject
	n, err := minioClient.FPutObject(bucketName, objectName, filePath, contentType)
	if err != nil {
		log.Fatalln(err)
	}

	log.Printf("Successfully uploaded %s of size %d\n", objectName, n)
}
Run FileUploader
go run file-uploader.go
2016/08/13 17:03:28 Successfully created mymusic 
2016/08/13 17:03:40 Successfully uploaded golden-oldies.zip of size 16253413

mc ls play/mymusic/
[2016-05-27 16:02:16 PDT]  17MiB golden-oldies.zip

API Reference

The full API Reference is available here.

API Reference : Bucket Operations
API Reference : Bucket policy Operations
API Reference : Bucket notification Operations
API Reference : File Object Operations
API Reference : Object Operations
API Reference: Encrypted Object Operations
API Reference : Presigned Operations
API Reference : Client custom settings

Full Examples

Full Examples : Bucket Operations
Full Examples : Bucket policy Operations
Full Examples : Bucket notification Operations
Full Examples : File Object Operations
Full Examples : Object Operations
Full Examples : Encrypted Object Operations
Full Examples : Presigned Operations

Explore Further

Contribute

Contributors Guide

Build Status Build status

Documentation

Index

Constants

View Source 
const (
	ObjectCreatedAll                     NotificationEventType = "s3:ObjectCreated:*"
	ObjectCreatedPut                                           = "s3:ObjectCreated:Put"
	ObjectCreatedPost                                          = "s3:ObjectCreated:Post"
	ObjectCreatedCopy                                          = "s3:ObjectCreated:Copy"
	ObjectCreatedCompleteMultipartUpload                       = "s3:ObjectCreated:CompleteMultipartUpload"
	ObjectAccessedGet                                          = "s3:ObjectAccessed:Get"
	ObjectAccessedHead                                         = "s3:ObjectAccessed:Head"
	ObjectAccessedAll                                          = "s3:ObjectAccessed:*"
	ObjectRemovedAll                                           = "s3:ObjectRemoved:*"
	ObjectRemovedDelete                                        = "s3:ObjectRemoved:Delete"
	ObjectRemovedDeleteMarkerCreated                           = "s3:ObjectRemoved:DeleteMarkerCreated"
	ObjectReducedRedundancyLostObject                          = "s3:ReducedRedundancyLostObject"
)

The role of all event types are described in : 

http://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html#notification-how-to-event-types-and-destinations
View Source 
const DefaultRetryCap = time.Second * 30

DefaultRetryCap - Each retry attempt never waits no longer than this maximum time duration.

View Source 
const DefaultRetryUnit = time.Second

DefaultRetryUnit - default unit multiplicative per retry. defaults to 1 second.

View Source 
const MaxJitter = 1.0

MaxJitter will randomize over the full exponential backoff time

View Source 
const NoJitter = 0.0

NoJitter disables the use of jitter for randomizing the exponential backoff time

Variables

View Source 
var ErrInvalidObjectPrefix = ErrInvalidObjectName

ErrInvalidObjectPrefix - Invalid object prefix response is similar to object name response.

View Source 
var MaxRetry = 5

MaxRetry is the maximum number of retries before stopping.

Functions

func ErrAPINotSupported added in v2.0.2 

func ErrAPINotSupported(message string) error

ErrAPINotSupported - API not supported response The specified API call is not supported

func ErrEntityTooLarge 

func ErrEntityTooLarge(totalSize, maxObjectSize int64, bucketName, objectName string) error

ErrEntityTooLarge - Input size is larger than supported maximum.

func ErrEntityTooSmall 

func ErrEntityTooSmall(totalSize int64, bucketName, objectName string) error

ErrEntityTooSmall - Input size is smaller than supported minimum.

func ErrInvalidArgument 

func ErrInvalidArgument(message string) error

ErrInvalidArgument - Invalid argument response.

func ErrInvalidBucketName 

func ErrInvalidBucketName(message string) error

ErrInvalidBucketName - Invalid bucket name response.

func ErrInvalidObjectName 

func ErrInvalidObjectName(message string) error

ErrInvalidObjectName - Invalid object name response.

func ErrNoSuchBucketPolicy 

func ErrNoSuchBucketPolicy(message string) error

ErrNoSuchBucketPolicy - No Such Bucket Policy response The specified bucket does not have a bucket policy.

func ErrTransferAccelerationBucket added in v2.0.4 

func ErrTransferAccelerationBucket(bucketName string) error

ErrTransferAccelerationBucket - bucket name is invalid to be used with transfer acceleration.

func ErrUnexpectedEOF 

func ErrUnexpectedEOF(totalRead, totalSize int64, bucketName, objectName string) error

ErrUnexpectedEOF - Unexpected end of file reached.

Types

type Arn added in v2.0.2 

type Arn struct {
	Partition string
	Service   string
	Region    string
	AccountID string
	Resource  string
}

Arn - holds ARN information that will be sent to the web service, ARN desciption can be found in http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html

func NewArn added in v2.0.2 

func NewArn(partition, service, region, accountID, resource string) Arn

NewArn creates new ARN based on the given partition, service, region, account id and resource

func (Arn) String added in v2.0.2 

func (arn Arn) String() string

Return the string format of the ARN

type BucketInfo 

type BucketInfo struct {
	// The name of the bucket.
	Name string `json:"name"`
	// Date the bucket was created.
	CreationDate time.Time `json:"creationDate"`
}

BucketInfo container for bucket metadata.

type BucketNotification added in v2.0.2 

type BucketNotification struct {
	XMLName       xml.Name       `xml:"NotificationConfiguration"`
	LambdaConfigs []LambdaConfig `xml:"CloudFunctionConfiguration"`
	TopicConfigs  []TopicConfig  `xml:"TopicConfiguration"`
	QueueConfigs  []QueueConfig  `xml:"QueueConfiguration"`
}

BucketNotification - the struct that represents the whole XML to be sent to the web service

func (*BucketNotification) AddLambda added in v2.0.2 

func (b *BucketNotification) AddLambda(lambdaConfig NotificationConfig)

AddLambda adds a given lambda config to the general bucket notification config

func (*BucketNotification) AddQueue added in v2.0.2 

func (b *BucketNotification) AddQueue(queueConfig NotificationConfig)

AddQueue adds a given queue config to the general bucket notification config

func (*BucketNotification) AddTopic added in v2.0.2 

func (b *BucketNotification) AddTopic(topicConfig NotificationConfig)

AddTopic adds a given topic config to the general bucket notification config

func (*BucketNotification) RemoveLambdaByArn added in v2.0.2 

func (b *BucketNotification) RemoveLambdaByArn(arn Arn)

RemoveLambdaByArn removes all lambda configurations that match the exact specified ARN

func (*BucketNotification) RemoveQueueByArn added in v2.0.2 

func (b *BucketNotification) RemoveQueueByArn(arn Arn)

RemoveQueueByArn removes all queue configurations that match the exact specified ARN

func (*BucketNotification) RemoveTopicByArn added in v2.0.2 

func (b *BucketNotification) RemoveTopicByArn(arn Arn)

RemoveTopicByArn removes all topic configurations that match the exact specified ARN

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client implements Amazon S3 compatible methods.

func New 

func New(endpoint, accessKeyID, secretAccessKey string, secure bool) (*Client, error)

New - instantiate minio client Client, adds automatic verification of signature.

func NewV2 

func NewV2(endpoint string, accessKeyID, secretAccessKey string, secure bool) (*Client, error)

NewV2 - instantiate minio client with Amazon S3 signature version '2' compatibility.

func NewV4 

func NewV4(endpoint string, accessKeyID, secretAccessKey string, secure bool) (*Client, error)

NewV4 - instantiate minio client with Amazon S3 signature version '4' compatibility.

func NewWithRegion added in v2.1.0 

func NewWithRegion(endpoint, accessKeyID, secretAccessKey string, secure bool, region string) (*Client, error)

NewWithRegion - instantiate minio client, with region configured. Unlike New(), NewWithRegion avoids bucket-location lookup operations and it is slightly faster. Use this function when if your application deals with single region.

func (Client) BucketExists 

func (c Client) BucketExists(bucketName string) (bool, error)

BucketExists verify if bucket exists and you have permission to access it.

func (Client) CopyObject 

func (c Client) CopyObject(bucketName string, objectName string, objectSource string, cpCond CopyConditions) error

CopyObject - copy a source object into a new object with the provided name in the provided bucket

func (Client) FGetObject 

func (c Client) FGetObject(bucketName, objectName, filePath string) error

FGetObject - download contents of an object to a local file.

func (Client) FPutObject 

func (c Client) FPutObject(bucketName, objectName, filePath, contentType string) (n int64, err error)

FPutObject - Create an object in a bucket, with contents from file at filePath.

func (Client) GetBucketLocation added in v2.0.2 

func (c Client) GetBucketLocation(bucketName string) (string, error)

GetBucketLocation - get location for the bucket name from location cache, if not fetch freshly by making a new request.

func (Client) GetBucketNotification added in v2.0.2 

func (c Client) GetBucketNotification(bucketName string) (bucketNotification BucketNotification, err error)

GetBucketNotification - get bucket notification at a given path.

func (Client) GetBucketPolicy 

func (c Client) GetBucketPolicy(bucketName, objectPrefix string) (bucketPolicy policy.BucketPolicy, err error)

GetBucketPolicy - get bucket policy at a given path.

func (Client) GetEncryptedObject added in v2.1.0 

func (c Client) GetEncryptedObject(bucketName, objectName string, encryptMaterials encrypt.Materials) (io.Reader, error)

GetEncryptedObject deciphers and streams data stored in the server after applying a specifed encryption materiels

func (Client) GetObject 

func (c Client) GetObject(bucketName, objectName string) (*Object, error)

GetObject - returns an seekable, readable object.

func (Client) ListBucketPolicies added in v2.0.2 

func (c Client) ListBucketPolicies(bucketName, objectPrefix string) (bucketPolicies map[string]policy.BucketPolicy, err error)

ListBucketPolicies - list all policies for a given prefix and all its children.

func (Client) ListBuckets 

func (c Client) ListBuckets() ([]BucketInfo, error)

ListBuckets list all buckets owned by this authenticated user.

This call requires explicit authentication, no anonymous requests are allowed for listing buckets. 

api := client.New(....)
for message := range api.ListBuckets() {
    fmt.Println(message)
}

func (Client) ListIncompleteUploads 

func (c Client) ListIncompleteUploads(bucketName, objectPrefix string, recursive bool, doneCh <-chan struct{}) <-chan ObjectMultipartInfo

ListIncompleteUploads - List incompletely uploaded multipart objects.

ListIncompleteUploads lists all incompleted objects matching the objectPrefix from the specified bucket. If recursion is enabled it would list all subdirectories and all its contents.

Your input parameters are just bucketName, objectPrefix, recursive and a done channel to pro-actively close the internal go routine. If you enable recursive as 'true' this function will return back all the multipart objects in a given bucket name. 

api := client.New(....)
// Create a done channel.
doneCh := make(chan struct{})
defer close(doneCh)
// Recurively list all objects in 'mytestbucket'
recursive := true
for message := range api.ListIncompleteUploads("mytestbucket", "starthere", recursive) {
    fmt.Println(message)
}

func (Client) ListObjects 

func (c Client) ListObjects(bucketName, objectPrefix string, recursive bool, doneCh <-chan struct{}) <-chan ObjectInfo

ListObjects - (List Objects) - List some objects or all recursively.

ListObjects lists all objects matching the objectPrefix from the specified bucket. If recursion is enabled it would list all subdirectories and all its contents.

Your input parameters are just bucketName, objectPrefix, recursive and a done channel for pro-actively closing the internal go routine. If you enable recursive as 'true' this function will return back all the objects in a given bucket name and object prefix. 

api := client.New(....)
// Create a done channel.
doneCh := make(chan struct{})
defer close(doneCh)
// Recurively list all objects in 'mytestbucket'
recursive := true
for message := range api.ListObjects("mytestbucket", "starthere", recursive, doneCh) {
    fmt.Println(message)
}

func (Client) ListObjectsV2 

func (c Client) ListObjectsV2(bucketName, objectPrefix string, recursive bool, doneCh <-chan struct{}) <-chan ObjectInfo

ListObjectsV2 lists all objects matching the objectPrefix from the specified bucket. If recursion is enabled it would list all subdirectories and all its contents.

Your input parameters are just bucketName, objectPrefix, recursive and a done channel for pro-actively closing the internal go routine. If you enable recursive as 'true' this function will return back all the objects in a given bucket name and object prefix. 

api := client.New(....)
// Create a done channel.
doneCh := make(chan struct{})
defer close(doneCh)
// Recurively list all objects in 'mytestbucket'
recursive := true
for message := range api.ListObjectsV2("mytestbucket", "starthere", recursive, doneCh) {
    fmt.Println(message)
}

func (Client) ListenBucketNotification added in v2.0.2 

func (c Client) ListenBucketNotification(bucketName, prefix, suffix string, events []string, doneCh <-chan struct{}) <-chan NotificationInfo

ListenBucketNotification - listen on bucket notifications.

func (Client) MakeBucket 

func (c Client) MakeBucket(bucketName string, location string) (err error)

MakeBucket creates a new bucket with bucketName.

Location is an optional argument, by default all buckets are created in US Standard Region.

For Amazon S3 for more supported regions - http://docs.aws.amazon.com/general/latest/gr/rande.html For Google Cloud Storage for more supported regions - https://cloud.google.com/storage/docs/bucket-locations

func (Client) PresignedGetObject 

func (c Client) PresignedGetObject(bucketName string, objectName string, expires time.Duration, reqParams url.Values) (u *url.URL, err error)

PresignedGetObject - Returns a presigned URL to access an object without credentials. Expires maximum is 7days - ie. 604800 and minimum is 1. Additionally you can override a set of response headers using the query parameters.

func (Client) PresignedPostPolicy 

func (c Client) PresignedPostPolicy(p *PostPolicy) (u *url.URL, formData map[string]string, err error)

PresignedPostPolicy - Returns POST urlString, form data to upload an object.

func (Client) PresignedPutObject 

func (c Client) PresignedPutObject(bucketName string, objectName string, expires time.Duration) (u *url.URL, err error)

PresignedPutObject - Returns a presigned URL to upload an object without credentials. Expires maximum is 7days - ie. 604800 and minimum is 1.

func (Client) PutEncryptedObject added in v2.1.0 

func (c Client) PutEncryptedObject(bucketName, objectName string, reader io.Reader, encryptMaterials encrypt.Materials, metaData map[string][]string, progress io.Reader) (n int64, err error)

PutEncryptedObject - Encrypt and store object.

func (Client) PutObject 

func (c Client) PutObject(bucketName, objectName string, reader io.Reader, contentType string) (n int64, err error)

PutObject creates an object in a bucket.

You must have WRITE permissions on a bucket to create an object.

  • For size smaller than 5MiB PutObject automatically does a single atomic Put operation.
  • For size larger than 5MiB PutObject automatically does a resumable multipart Put operation.
  • For size input as -1 PutObject does a multipart Put operation until input stream reaches EOF. Maximum object size that can be uploaded through this operation will be 5TiB.

NOTE: Google Cloud Storage does not implement Amazon S3 Compatible multipart PUT. So we fall back to single PUT operation with the maximum limit of 5GiB.

func (Client) PutObjectStreaming added in v2.1.0 

func (c Client) PutObjectStreaming(bucketName, objectName string, reader io.Reader) (n int64, err error)

PutObjectStreaming using AWS streaming signature V4

func (Client) PutObjectStreamingWithMetadata added in v2.1.0 

func (c Client) PutObjectStreamingWithMetadata(bucketName, objectName string, reader io.Reader, metadata map[string][]string) (n int64, err error)

PutObjectStreamingWithMetadata using AWS streaming signature V4

func (Client) PutObjectStreamingWithProgress added in v2.1.0 

func (c Client) PutObjectStreamingWithProgress(bucketName, objectName string, reader io.Reader, metadata map[string][]string, progress io.Reader) (n int64, err error)

PutObjectStreamingWithProgress using AWS streaming signature V4

func (Client) PutObjectWithMetadata added in v2.0.3 

func (c Client) PutObjectWithMetadata(bucketName, objectName string, reader io.Reader, metaData map[string][]string, progress io.Reader) (n int64, err error)

PutObjectWithMetadata - with metadata.

func (Client) PutObjectWithProgress 

func (c Client) PutObjectWithProgress(bucketName, objectName string, reader io.Reader, contentType string, progress io.Reader) (n int64, err error)

PutObjectWithProgress - with progress.

func (Client) RemoveAllBucketNotification added in v2.0.2 

func (c Client) RemoveAllBucketNotification(bucketName string) error

RemoveAllBucketNotification - Remove bucket notification clears all previously specified config

func (Client) RemoveBucket 

func (c Client) RemoveBucket(bucketName string) error

RemoveBucket deletes the bucket name. 

All objects (including all object versions and delete markers).
in the bucket must be deleted before successfully attempting this request.

func (Client) RemoveIncompleteUpload 

func (c Client) RemoveIncompleteUpload(bucketName, objectName string) error

RemoveIncompleteUpload aborts an partially uploaded object.

func (Client) RemoveObject 

func (c Client) RemoveObject(bucketName, objectName string) error

RemoveObject remove an object from a bucket.

func (Client) RemoveObjects added in v2.0.2 

func (c Client) RemoveObjects(bucketName string, objectsCh <-chan string) <-chan RemoveObjectError

RemoveObjects remove multiples objects from a bucket. The list of objects to remove are received from objectsCh. Remove failures are sent back via error channel.

func (*Client) SetAppInfo 

func (c *Client) SetAppInfo(appName string, appVersion string)

SetAppInfo - add application details to user agent.

func (Client) SetBucketNotification added in v2.0.2 

func (c Client) SetBucketNotification(bucketName string, bucketNotification BucketNotification) error

SetBucketNotification saves a new bucket notification.

func (Client) SetBucketPolicy 

func (c Client) SetBucketPolicy(bucketName string, objectPrefix string, bucketPolicy policy.BucketPolicy) error

SetBucketPolicy set the access permissions on an existing bucket.

For example 

none - owner gets full access [default].
readonly - anonymous get access for everyone at a given object prefix.
readwrite - anonymous list/put/delete access to a given object prefix.
writeonly - anonymous put/delete access to a given object prefix.

func (*Client) SetCustomTransport 

func (c *Client) SetCustomTransport(customHTTPTransport http.RoundTripper)

SetCustomTransport - set new custom transport.

func (*Client) SetS3TransferAccelerate added in v2.0.4 

func (c *Client) SetS3TransferAccelerate(accelerateEndpoint string)

SetS3TransferAccelerate - turns s3 accelerated endpoint on or off for all your requests. This feature is only specific to S3 for all other endpoints this function does nothing. To read further details on s3 transfer acceleration please vist - http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html

func (Client) StatObject 

func (c Client) StatObject(bucketName, objectName string) (ObjectInfo, error)

StatObject verifies if object exists and you have permission to access.

func (*Client) TraceOff 

func (c *Client) TraceOff()

TraceOff - disable HTTP tracing.

func (*Client) TraceOn 

func (c *Client) TraceOn(outputStream io.Writer)

TraceOn - enable HTTP tracing.

type CompletePart added in v2.1.0 

type CompletePart struct {
	XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ Part" json:"-"`

	// Part number identifies the part.
	PartNumber int
	ETag       string
}

CompletePart sub container lists individual part numbers and their md5sum, part of completeMultipartUpload.

type CopyConditions 

type CopyConditions struct {
	// contains filtered or unexported fields
}

CopyConditions - copy conditions.

func NewCopyConditions 

func NewCopyConditions() CopyConditions

NewCopyConditions - Instantiate new list of conditions. This function is left behind for backward compatibility. The idiomatic way to set an empty set of copy conditions is, 

``copyConditions := CopyConditions{}``.

func (*CopyConditions) SetMatchETag 

func (c *CopyConditions) SetMatchETag(etag string) error

SetMatchETag - set match etag.

func (*CopyConditions) SetMatchETagExcept 

func (c *CopyConditions) SetMatchETagExcept(etag string) error

SetMatchETagExcept - set match etag except.

func (*CopyConditions) SetModified 

func (c *CopyConditions) SetModified(modTime time.Time) error

SetModified - set modified time since.

func (*CopyConditions) SetUnmodified 

func (c *CopyConditions) SetUnmodified(modTime time.Time) error

SetUnmodified - set unmodified time since.

type Core added in v2.1.0 

type Core struct {
	*Client
}

Core - Inherits Client and adds new methods to expose the low level S3 APIs.

func NewCore added in v2.1.0 

func NewCore(endpoint string, accessKeyID, secretAccessKey string, secure bool) (*Core, error)

NewCore - Returns new initialized a Core client, this CoreClient should be only used under special conditions such as need to access lower primitives and being able to use them to write your own wrappers.

func (Core) AbortMultipartUpload added in v2.1.0 

func (c Core) AbortMultipartUpload(bucket, object, uploadID string) error

AbortMultipartUpload - Abort an incomplete upload.

func (Core) CompleteMultipartUpload added in v2.1.0 

func (c Core) CompleteMultipartUpload(bucket, object, uploadID string, parts []CompletePart) error

CompleteMultipartUpload - Concatenate uploaded parts and commit to an object.

func (Core) GetBucketPolicy added in v2.1.0 

func (c Core) GetBucketPolicy(bucket string) (policy.BucketAccessPolicy, error)

GetBucketPolicy - fetches bucket access policy for a given bucket.

func (Core) GetObject added in v2.1.0 

func (c Core) GetObject(bucketName, objectName string, reqHeaders RequestHeaders) (io.ReadCloser, ObjectInfo, error)

GetObject is a lower level API implemented to support reading partial objects and also downloading objects with special conditions matching etag, modtime etc.

func (Core) ListMultipartUploads added in v2.1.0 

func (c Core) ListMultipartUploads(bucket, prefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (result ListMultipartUploadsResult, err error)

ListMultipartUploads - List incomplete uploads.

func (Core) ListObjectParts added in v2.1.0 

func (c Core) ListObjectParts(bucket, object, uploadID string, partNumberMarker int, maxParts int) (result ListObjectPartsResult, err error)

ListObjectParts - List uploaded parts of an incomplete upload.x

func (Core) ListObjects added in v2.1.0 

func (c Core) ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (result ListBucketResult, err error)

ListObjects - List all the objects at a prefix, optionally with marker and delimiter you can further filter the results.

func (Core) ListObjectsV2 added in v2.1.0 

func (c Core) ListObjectsV2(bucketName, objectPrefix, continuationToken string, fetchOwner bool, delimiter string, maxkeys int) (ListBucketV2Result, error)

ListObjectsV2 - Lists all the objects at a prefix, similar to ListObjects() but uses continuationToken instead of marker to further filter the results.

func (Core) NewMultipartUpload added in v2.1.0 

func (c Core) NewMultipartUpload(bucket, object string, metadata map[string][]string) (uploadID string, err error)

NewMultipartUpload - Initiates new multipart upload and returns the new uploaID.

func (Core) PutBucketPolicy added in v2.1.0 

func (c Core) PutBucketPolicy(bucket string, bucketPolicy policy.BucketAccessPolicy) error

PutBucketPolicy - applies a new bucket access policy for a given bucket.

func (Core) PutObject added in v2.1.0 

func (c Core) PutObject(bucket, object string, size int64, data io.Reader, md5Sum, sha256Sum []byte, metadata map[string][]string) (ObjectInfo, error)

PutObject - Upload object. Uploads using single PUT call.

func (Core) PutObjectPart added in v2.1.0 

func (c Core) PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Sum, sha256Sum []byte) (ObjectPart, error)

PutObjectPart - Upload an object part.

func (Core) StatObject added in v2.1.0 

func (c Core) StatObject(bucketName, objectName string, reqHeaders RequestHeaders) (ObjectInfo, error)

StatObject is a lower level API implemented to support special conditions matching etag, modtime on a request.

type ErrorResponse 

type ErrorResponse struct {
	XMLName    xml.Name `xml:"Error" json:"-"`
	Code       string
	Message    string
	BucketName string
	Key        string
	RequestID  string `xml:"RequestId"`
	HostID     string `xml:"HostId"`

	// Region where the bucket is located. This header is returned
	// only in HEAD bucket and ListObjects response.
	Region string

	// Headers of the returned S3 XML error
	Headers http.Header `xml:"-" json:"-"`
}

ErrorResponse - Is the typed error returned by all API operations.

func ToErrorResponse 

func ToErrorResponse(err error) ErrorResponse

ToErrorResponse - Returns parsed ErrorResponse struct from body and http headers.

For example: 

import s3 "github.com/minio/minio-go"
...
...
reader, stat, err := s3.GetObject(...)
if err != nil {
   resp := s3.ToErrorResponse(err)
}
...

func (ErrorResponse) Error 

func (e ErrorResponse) Error() string

Error - Returns S3 error string.

type Filter added in v2.0.2 

type Filter struct {
	S3Key S3Key `xml:"S3Key,omitempty"`
}

Filter - a tag in the notification xml structure which carries suffix/prefix filters

type FilterRule added in v2.0.2 

type FilterRule struct {
	Name  string `xml:"Name"`
	Value string `xml:"Value"`
}

FilterRule - child of S3Key, a tag in the notification xml which carries suffix/prefix filters

type LambdaConfig added in v2.0.2 

type LambdaConfig struct {
	NotificationConfig
	Lambda string `xml:"CloudFunction"`
}

LambdaConfig carries one single cloudfunction notification configuration

type ListBucketResult added in v2.1.0 

type ListBucketResult struct {
	// A response can contain CommonPrefixes only if you have
	// specified a delimiter.
	CommonPrefixes []commonPrefix
	// Metadata about each object returned.
	Contents  []ObjectInfo
	Delimiter string

	// Encoding type used to encode object keys in the response.
	EncodingType string

	// A flag that indicates whether or not ListObjects returned all of the results
	// that satisfied the search criteria.
	IsTruncated bool
	Marker      string
	MaxKeys     int64
	Name        string

	// When response is truncated (the IsTruncated element value in
	// the response is true), you can use the key name in this field
	// as marker in the subsequent request to get next set of objects.
	// Object storage lists objects in alphabetical order Note: This
	// element is returned only if you have delimiter request
	// parameter specified. If response does not include the NextMaker
	// and it is truncated, you can use the value of the last Key in
	// the response as the marker in the subsequent request to get the
	// next set of object keys.
	NextMarker string
	Prefix     string
}

ListBucketResult container for listObjects response.

type ListBucketV2Result added in v2.1.0 

type ListBucketV2Result struct {
	// A response can contain CommonPrefixes only if you have
	// specified a delimiter.
	CommonPrefixes []commonPrefix
	// Metadata about each object returned.
	Contents  []ObjectInfo
	Delimiter string

	// Encoding type used to encode object keys in the response.
	EncodingType string

	// A flag that indicates whether or not ListObjects returned all of the results
	// that satisfied the search criteria.
	IsTruncated bool
	MaxKeys     int64
	Name        string

	// Hold the token that will be sent in the next request to fetch the next group of keys
	NextContinuationToken string

	ContinuationToken string
	Prefix            string

	// FetchOwner and StartAfter are currently not used
	FetchOwner string
	StartAfter string
}

ListBucketV2Result container for listObjects response version 2.

type ListMultipartUploadsResult added in v2.1.0 

type ListMultipartUploadsResult struct {
	Bucket             string
	KeyMarker          string
	UploadIDMarker     string `xml:"UploadIdMarker"`
	NextKeyMarker      string
	NextUploadIDMarker string `xml:"NextUploadIdMarker"`
	EncodingType       string
	MaxUploads         int64
	IsTruncated        bool
	Uploads            []ObjectMultipartInfo `xml:"Upload"`
	Prefix             string
	Delimiter          string
	// A response can contain CommonPrefixes only if you specify a delimiter.
	CommonPrefixes []commonPrefix
}

ListMultipartUploadsResult container for ListMultipartUploads response

type ListObjectPartsResult added in v2.1.0 

type ListObjectPartsResult struct {
	Bucket   string
	Key      string
	UploadID string `xml:"UploadId"`

	Initiator initiator
	Owner     owner

	StorageClass         string
	PartNumberMarker     int
	NextPartNumberMarker int
	MaxParts             int

	// Indicates whether the returned list of parts is truncated.
	IsTruncated bool
	ObjectParts []ObjectPart `xml:"Part"`

	EncodingType string
}

ListObjectPartsResult container for ListObjectParts response.

type NotificationConfig added in v2.0.2 

type NotificationConfig struct {
	ID     string                  `xml:"Id,omitempty"`
	Arn    Arn                     `xml:"-"`
	Events []NotificationEventType `xml:"Event"`
	Filter *Filter                 `xml:"Filter,omitempty"`
}

NotificationConfig - represents one single notification configuration such as topic, queue or lambda configuration.

func NewNotificationConfig added in v2.0.2 

func NewNotificationConfig(arn Arn) NotificationConfig

NewNotificationConfig creates one notification config and sets the given ARN

func (*NotificationConfig) AddEvents added in v2.0.2 

func (t *NotificationConfig) AddEvents(events ...NotificationEventType)

AddEvents adds one event to the current notification config

func (*NotificationConfig) AddFilterPrefix added in v2.0.2 

func (t *NotificationConfig) AddFilterPrefix(prefix string)

AddFilterPrefix sets the prefix configuration to the current notification config

func (*NotificationConfig) AddFilterSuffix added in v2.0.2 

func (t *NotificationConfig) AddFilterSuffix(suffix string)

AddFilterSuffix sets the suffix configuration to the current notification config

type NotificationEvent added in v2.0.2 

type NotificationEvent struct {
	EventVersion      string            `json:"eventVersion"`
	EventSource       string            `json:"eventSource"`
	AwsRegion         string            `json:"awsRegion"`
	EventTime         string            `json:"eventTime"`
	EventName         string            `json:"eventName"`
	UserIdentity      identity          `json:"userIdentity"`
	RequestParameters map[string]string `json:"requestParameters"`
	ResponseElements  map[string]string `json:"responseElements"`
	S3                eventMeta         `json:"s3"`
	Source            sourceInfo        `json:"source"`
}

NotificationEvent represents an Amazon an S3 bucket notification event.

type NotificationEventType added in v2.0.2 

type NotificationEventType string

NotificationEventType is a S3 notification event associated to the bucket notification configuration

type NotificationInfo added in v2.0.2 

type NotificationInfo struct {
	Records []NotificationEvent
	Err     error
}

NotificationInfo - represents the collection of notification events, additionally also reports errors if any while listening on bucket notifications.

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object represents an open object. It implements Reader, ReaderAt, Seeker, Closer for a HTTP stream.

func (*Object) Close 

func (o *Object) Close() (err error)

Close - The behavior of Close after the first call returns error for subsequent Close() calls.

func (*Object) Read 

func (o *Object) Read(b []byte) (n int, err error)

Read reads up to len(b) bytes into b. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Returns io.EOF upon end of file.

func (*Object) ReadAt 

func (o *Object) ReadAt(b []byte, offset int64) (n int, err error)

ReadAt reads len(b) bytes from the File starting at byte offset off. It returns the number of bytes read and the error, if any. ReadAt always returns a non-nil error when n < len(b). At end of file, that error is io.EOF.

func (*Object) Seek 

func (o *Object) Seek(offset int64, whence int) (n int64, err error)

Seek sets the offset for the next Read or Write to offset, interpreted according to whence: 0 means relative to the origin of the file, 1 means relative to the current offset, and 2 means relative to the end. Seek returns the new offset and an error, if any.

Seeking to a negative offset is an error. Seeking to any positive offset is legal, subsequent io operations succeed until the underlying object is not closed.

func (*Object) Stat 

func (o *Object) Stat() (ObjectInfo, error)

Stat returns the ObjectInfo structure describing Object.

type ObjectInfo 

type ObjectInfo struct {
	// An ETag is optionally set to md5sum of an object.  In case of multipart objects,
	// ETag is of the form MD5SUM-N where MD5SUM is md5sum of all individual md5sums of
	// each parts concatenated into one string.
	ETag string `json:"etag"`

	Key          string    `json:"name"`         // Name of the object
	LastModified time.Time `json:"lastModified"` // Date and time the object was last modified.
	Size         int64     `json:"size"`         // Size in bytes of the object.
	ContentType  string    `json:"contentType"`  // A standard MIME type describing the format of the object data.

	// Collection of additional metadata on the object.
	// eg: x-amz-meta-*, content-encoding etc.
	Metadata http.Header `json:"metadata"`

	// Owner name.
	Owner struct {
		DisplayName string `json:"name"`
		ID          string `json:"id"`
	} `json:"owner"`

	// The class of storage used to store the object.
	StorageClass string `json:"storageClass"`

	// Error
	Err error `json:"-"`
}

ObjectInfo container for object metadata.

type ObjectMultipartInfo 

type ObjectMultipartInfo struct {
	// Date and time at which the multipart upload was initiated.
	Initiated time.Time `type:"timestamp" timestampFormat:"iso8601"`

	Initiator initiator
	Owner     owner

	// The type of storage to use for the object. Defaults to 'STANDARD'.
	StorageClass string

	// Key of the object for which the multipart upload was initiated.
	Key string

	// Size in bytes of the object.
	Size int64

	// Upload ID that identifies the multipart upload.
	UploadID string `xml:"UploadId"`

	// Error
	Err error
}

ObjectMultipartInfo container for multipart object metadata.

type ObjectPart added in v2.1.0 

type ObjectPart struct {
	// Part number identifies the part.
	PartNumber int

	// Date and time the part was uploaded.
	LastModified time.Time

	// Entity tag returned when the part was uploaded, usually md5sum
	// of the part.
	ETag string

	// Size of the uploaded part data.
	Size int64
}

ObjectPart container for particular part of an object.

type PostPolicy 

type PostPolicy struct {
	// contains filtered or unexported fields
}

PostPolicy - Provides strict static type conversion and validation for Amazon S3's POST policy JSON string.

func NewPostPolicy 

func NewPostPolicy() *PostPolicy

NewPostPolicy - Instantiate new post policy.

func (*PostPolicy) SetBucket 

func (p *PostPolicy) SetBucket(bucketName string) error

SetBucket - Sets bucket at which objects will be uploaded to.

func (*PostPolicy) SetContentLengthRange 

func (p *PostPolicy) SetContentLengthRange(min, max int64) error

SetContentLengthRange - Set new min and max content length condition for all incoming uploads.

func (*PostPolicy) SetContentType 

func (p *PostPolicy) SetContentType(contentType string) error

SetContentType - Sets content-type of the object for this policy based upload.

func (*PostPolicy) SetExpires 

func (p *PostPolicy) SetExpires(t time.Time) error

SetExpires - Sets expiration time for the new policy.

func (*PostPolicy) SetKey 

func (p *PostPolicy) SetKey(key string) error

SetKey - Sets an object name for the policy based upload.

func (*PostPolicy) SetKeyStartsWith 

func (p *PostPolicy) SetKeyStartsWith(keyStartsWith string) error

SetKeyStartsWith - Sets an object name that an policy based upload can start with.

func (*PostPolicy) SetSuccessStatusAction added in v2.0.3 

func (p *PostPolicy) SetSuccessStatusAction(status string) error

SetSuccessStatusAction - Sets the status success code of the object for this policy based upload.

func (PostPolicy) String 

func (p PostPolicy) String() string

Stringer interface for printing policy in json formatted string.

type QueueConfig added in v2.0.2 

type QueueConfig struct {
	NotificationConfig
	Queue string `xml:"Queue"`
}

QueueConfig carries one single queue notification configuration

type RemoveObjectError added in v2.0.2 

type RemoveObjectError struct {
	ObjectName string
	Err        error
}

RemoveObjectError - container of Multi Delete S3 API error

type RequestHeaders added in v2.1.0 

type RequestHeaders struct {
	http.Header
}

RequestHeaders - implement methods for setting special request headers for GET, HEAD object operations. http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html

func NewGetReqHeaders added in v2.1.0 

func NewGetReqHeaders() RequestHeaders

NewGetReqHeaders - initializes a new request headers for GET request.

func NewHeadReqHeaders added in v2.1.0 

func NewHeadReqHeaders() RequestHeaders

NewHeadReqHeaders - initializes a new request headers for HEAD request.

func (RequestHeaders) SetMatchETag added in v2.1.0 

func (c RequestHeaders) SetMatchETag(etag string) error

SetMatchETag - set match etag.

func (RequestHeaders) SetMatchETagExcept added in v2.1.0 

func (c RequestHeaders) SetMatchETagExcept(etag string) error

SetMatchETagExcept - set match etag except.

func (RequestHeaders) SetModified added in v2.1.0 

func (c RequestHeaders) SetModified(modTime time.Time) error

SetModified - set modified time since.

func (RequestHeaders) SetRange added in v2.1.0 

func (c RequestHeaders) SetRange(start, end int64) error

SetRange - set the start and end offset of the object to be read. See https://tools.ietf.org/html/rfc7233#section-3.1 for reference.

func (RequestHeaders) SetUnmodified added in v2.1.0 

func (c RequestHeaders) SetUnmodified(modTime time.Time) error

SetUnmodified - set unmodified time since.

type S3Key added in v2.0.2 

type S3Key struct {
	FilterRules []FilterRule `xml:"FilterRule,omitempty"`
}

S3Key - child of Filter, a tag in the notification xml which carries suffix/prefix filters

type SignatureType 

type SignatureType int

SignatureType is type of Authorization requested for a given HTTP request. 

const (
	Latest SignatureType = iota
	SignatureV4
	SignatureV2
	SignatureV4Streaming
)

Different types of supported signatures - default is Latest i.e SignatureV4.

type TopicConfig added in v2.0.2 

type TopicConfig struct {
	NotificationConfig
	Topic string `xml:"Topic"`
}

TopicConfig carries one single topic notification configuration

Source Files

View all Source files

Directories

Path Synopsis
pkg
encrypt
Package encrypt implements a generic interface to encrypt any stream of data.
 Package encrypt implements a generic interface to encrypt any stream of data.
policy
s3signer
s3utils
set

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.