minio

package
v0.0.0-...-ed3a749 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 9, 2016 License: Apache-2.0, Apache-2.0 Imports: 29 Imported by: 0

README

Minio Go Library for Amazon S3 Compatible Cloud Storage Gitter

Description

Minio Go library is a simple client library for S3 compatible cloud storage servers. Supports AWS Signature Version 4 and 2. AWS Signature Version 4 is chosen as default.

List of supported cloud storage providers.

  • AWS Signature Version 4

    • Amazon S3
    • Minio

  • AWS Signature Version 2

    • Google Cloud Storage (Compatibility Mode)
    • Openstack Swift + Swift3 middleware
    • Ceph Object Gateway
    • Riak CS

Install

If you do not have a working Golang environment, please follow Install Golang.

$ go get github.com/minio/minio-go

Example

ListBuckets()

This example shows how to List your buckets.

package main

import (
	"log"

	"github.com/minio/minio-go"
)

func main() {
	// Requests are always secure (HTTPS) by default. Set insecure=true to enable insecure (HTTP) access.
	// This boolean value is the last argument for New().

	// New returns an Amazon S3 compatible client object. API copatibality (v2 or v4) is automatically
	// determined based on the Endpoint value.
	s3Client, err := minio.New("s3.amazonaws.com", "YOUR-ACCESS-KEY-HERE", "YOUR-SECRET-KEY-HERE", false)
	if err != nil {
	    log.Fatalln(err)
	}
	buckets, err := s3Client.ListBuckets()
	if err != nil {
		log.Fatalln(err)
	}
	for _, bucket := range buckets {
		log.Println(bucket)
	}
}

Documentation

Bucket Operations.
Object Operations.
File Object Operations.
Presigned Operations.
API Reference

GoDoc

Contribute

Contributors Guide

Build Status Build status

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrInvalidObjectPrefix = ErrInvalidObjectName

ErrInvalidObjectPrefix - Invalid object prefix response is similar to object name response.

Functions

func ErrEntityTooLarge 

func ErrEntityTooLarge(totalSize, maxObjectSize int64, bucketName, objectName string) error

ErrEntityTooLarge - Input size is larger than supported maximum.

func ErrEntityTooSmall 

func ErrEntityTooSmall(totalSize int64, bucketName, objectName string) error

ErrEntityTooSmall - Input size is smaller than supported minimum.

func ErrInvalidArgument 

func ErrInvalidArgument(message string) error

ErrInvalidArgument - Invalid argument response.

func ErrInvalidBucketName 

func ErrInvalidBucketName(message string) error

ErrInvalidBucketName - Invalid bucket name response.

func ErrInvalidObjectName 

func ErrInvalidObjectName(message string) error

ErrInvalidObjectName - Invalid object name response.

func ErrInvalidParts 

func ErrInvalidParts(expectedParts, uploadedParts int) error

ErrInvalidParts - Invalid number of parts.

func ErrUnexpectedEOF 

func ErrUnexpectedEOF(totalRead, totalSize int64, bucketName, objectName string) error

ErrUnexpectedEOF - Unexpected end of file reached.

Types

type BucketACL 

type BucketACL string

BucketACL - Bucket level access control.

func (BucketACL) String 

func (b BucketACL) String() string

Stringify acl.

type BucketInfo 

type BucketInfo struct {
	// The name of the bucket.
	Name string `json:"name"`
	// Date the bucket was created.
	CreationDate time.Time `json:"creationDate"`
}

BucketInfo container for bucket metadata.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client implements Amazon S3 compatible methods.

func (Client) BucketExists 

func (c Client) BucketExists(bucketName string) error

BucketExists verify if bucket exists and you have permission to access it.

func (Client) FGetObject 

func (c Client) FGetObject(bucketName, objectName, filePath string) error

FGetObject - download contents of an object to a local file.

func (Client) FPutObject 

func (c Client) FPutObject(bucketName, objectName, filePath, contentType string) (n int64, err error)

FPutObject - Create an object in a bucket, with contents from file at filePath.

func (Client) GetBucketACL 

func (c Client) GetBucketACL(bucketName string) (BucketACL, error)

GetBucketACL - Get the permissions on an existing bucket.

Returned values are: 

private - Owner gets full access.
public-read - Owner gets full access, others get read access.
public-read-write - Owner gets full access, others get full access
too.
authenticated-read - Owner gets full access, authenticated users
get read access.

func (Client) GetObject 

func (c Client) GetObject(bucketName, objectName string) (*Object, error)

GetObject - returns an seekable, readable object.

func (Client) ListBuckets 

func (c Client) ListBuckets() ([]BucketInfo, error)

ListBuckets list all buckets owned by this authenticated user.

This call requires explicit authentication, no anonymous requests are allowed for listing buckets. 

api := client.New(....)
for message := range api.ListBuckets() {
    fmt.Println(message)
}

func (Client) ListIncompleteUploads 

func (c Client) ListIncompleteUploads(bucketName, objectPrefix string, recursive bool, doneCh <-chan struct{}) <-chan ObjectMultipartInfo

ListIncompleteUploads - List incompletely uploaded multipart objects.

ListIncompleteUploads lists all incompleted objects matching the objectPrefix from the specified bucket. If recursion is enabled it would list all subdirectories and all its contents.

Your input parameters are just bucketName, objectPrefix, recursive and a done channel to pro-actively close the internal go routine. If you enable recursive as 'true' this function will return back all the multipart objects in a given bucket name. 

api := client.New(....)
// Create a done channel.
doneCh := make(chan struct{})
defer close(doneCh)
// Recurively list all objects in 'mytestbucket'
recursive := true
for message := range api.ListIncompleteUploads("mytestbucket", "starthere", recursive) {
    fmt.Println(message)
}

func (Client) ListObjects 

func (c Client) ListObjects(bucketName, objectPrefix string, recursive bool, doneCh <-chan struct{}) <-chan ObjectInfo

ListObjects - (List Objects) - List some objects or all recursively.

ListObjects lists all objects matching the objectPrefix from the specified bucket. If recursion is enabled it would list all subdirectories and all its contents.

Your input parameters are just bucketName, objectPrefix, recursive and a done channel for pro-actively closing the internal go routine. If you enable recursive as 'true' this function will return back all the objects in a given bucket name and object prefix. 

api := client.New(....)
// Create a done channel.
doneCh := make(chan struct{})
defer close(doneCh)
// Recurively list all objects in 'mytestbucket'
recursive := true
for message := range api.ListObjects("mytestbucket", "starthere", recursive, doneCh) {
    fmt.Println(message)
}

func (Client) MakeBucket 

func (c Client) MakeBucket(bucketName string, acl BucketACL, location string) error

MakeBucket makes a new bucket.

Optional arguments are acl and location - by default all buckets are created with “private“ acl and in US Standard region.

ACL valid values - http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html 

private - owner gets full access [default].
public-read - owner gets full access, all others get read access.
public-read-write - owner gets full access, all others get full access too.
authenticated-read - owner gets full access, authenticated users get read access.

For Amazon S3 for more supported regions - http://docs.aws.amazon.com/general/latest/gr/rande.html For Google Cloud Storage for more supported regions - https://cloud.google.com/storage/docs/bucket-locations

func (Client) PresignedGetObject 

func (c Client) PresignedGetObject(bucketName string, objectName string, expires time.Duration) (url string, err error)

PresignedGetObject - Returns a presigned URL to access an object without credentials. Expires maximum is 7days - ie. 604800 and minimum is 1.

func (Client) PresignedPostPolicy 

func (c Client) PresignedPostPolicy(p *PostPolicy) (map[string]string, error)

PresignedPostPolicy - Returns POST form data to upload an object at a location.

func (Client) PresignedPutObject 

func (c Client) PresignedPutObject(bucketName string, objectName string, expires time.Duration) (url string, err error)

PresignedPutObject - Returns a presigned URL to upload an object without credentials. Expires maximum is 7days - ie. 604800 and minimum is 1.

func (Client) PutObject 

func (c Client) PutObject(bucketName, objectName string, reader io.Reader, contentType string) (n int64, err error)

PutObject creates an object in a bucket.

You must have WRITE permissions on a bucket to create an object.

  • For size smaller than 5MiB PutObject automatically does a single atomic Put operation.
  • For size larger than 5MiB PutObject automatically does a resumable multipart Put operation.
  • For size input as -1 PutObject does a multipart Put operation until input stream reaches EOF. Maximum object size that can be uploaded through this operation will be 5TiB.

NOTE: Google Cloud Storage does not implement Amazon S3 Compatible multipart PUT. So we fall back to single PUT operation with the maximum limit of 5GiB.

NOTE: For anonymous requests Amazon S3 doesn't allow multipart upload. So we fall back to single PUT operation.

func (Client) PutObjectWithProgress 

func (c Client) PutObjectWithProgress(bucketName, objectName string, reader io.Reader, contentType string, progress io.Reader) (n int64, err error)

PutObjectWithProgress - With progress.

func (Client) RemoveBucket 

func (c Client) RemoveBucket(bucketName string) error

RemoveBucket deletes the bucket name. 

All objects (including all object versions and delete markers).
in the bucket must be deleted before successfully attempting this request.

func (Client) RemoveIncompleteUpload 

func (c Client) RemoveIncompleteUpload(bucketName, objectName string) error

RemoveIncompleteUpload aborts an partially uploaded object. Requires explicit authentication, no anonymous requests are allowed for multipart API.

func (Client) RemoveObject 

func (c Client) RemoveObject(bucketName, objectName string) error

RemoveObject remove an object from a bucket.

func (*Client) SetAppInfo 

func (c *Client) SetAppInfo(appName string, appVersion string)

SetAppInfo - add application details to user agent.

func (Client) SetBucketACL 

func (c Client) SetBucketACL(bucketName string, acl BucketACL) error

SetBucketACL set the permissions on an existing bucket using access control lists (ACL).

For example 

private - owner gets full access [default].
public-read - owner gets full access, all others get read access.
public-read-write - owner gets full access, all others get full access too.
authenticated-read - owner gets full access, authenticated users get read access.

func (*Client) SetCustomTransport 

func (c *Client) SetCustomTransport(customHTTPTransport http.RoundTripper)

SetCustomTransport - set new custom transport.

func (Client) StatObject 

func (c Client) StatObject(bucketName, objectName string) (ObjectInfo, error)

StatObject verifies if object exists and you have permission to access.

func (*Client) TraceOff 

func (c *Client) TraceOff()

TraceOff - disable HTTP tracing.

func (*Client) TraceOn 

func (c *Client) TraceOn(outputStream io.Writer)

TraceOn - enable HTTP tracing.

type CloudStorageClient 

type CloudStorageClient interface {
	// Bucket Read/Write/Stat operations.
	MakeBucket(bucketName string, cannedACL BucketACL, location string) error
	BucketExists(bucketName string) error
	RemoveBucket(bucketName string) error
	SetBucketACL(bucketName string, cannedACL BucketACL) error
	GetBucketACL(bucketName string) (BucketACL, error)

	ListBuckets() ([]BucketInfo, error)
	ListObjects(bucket, prefix string, recursive bool, doneCh <-chan struct{}) <-chan ObjectInfo
	ListIncompleteUploads(bucket, prefix string, recursive bool, doneCh <-chan struct{}) <-chan ObjectMultipartInfo

	// Object Read/Write/Stat operations.
	GetObject(bucketName, objectName string) (reader *Object, err error)
	PutObject(bucketName, objectName string, reader io.Reader, contentType string) (n int64, err error)
	StatObject(bucketName, objectName string) (ObjectInfo, error)
	RemoveObject(bucketName, objectName string) error
	RemoveIncompleteUpload(bucketName, objectName string) error

	// File to Object API.
	FPutObject(bucketName, objectName, filePath, contentType string) (n int64, err error)
	FGetObject(bucketName, objectName, filePath string) error

	// PutObjectWithProgress for progress.
	PutObjectWithProgress(bucketName, objectName string, reader io.Reader, contentType string, progress io.Reader) (n int64, err error)

	// Presigned operations.
	PresignedGetObject(bucketName, objectName string, expires time.Duration) (presignedURL string, err error)
	PresignedPutObject(bucketName, objectName string, expires time.Duration) (presignedURL string, err error)
	PresignedPostPolicy(*PostPolicy) (formData map[string]string, err error)

	// Application info.
	SetAppInfo(appName, appVersion string)

	// Set custom transport.
	SetCustomTransport(customTransport http.RoundTripper)

	// HTTP tracing methods.
	TraceOn(traceOutput io.Writer)
	TraceOff()
}

CloudStorageClient - Cloud Storage Client interface.

func New 

func New(endpoint string, accessKeyID, secretAccessKey string, insecure bool) (CloudStorageClient, error)

New - instantiate minio client Client, adds automatic verification of signature.

func NewV2 

func NewV2(endpoint string, accessKeyID, secretAccessKey string, insecure bool) (CloudStorageClient, error)

NewV2 - instantiate minio client with Amazon S3 signature version '2' compatibility.

func NewV4 

func NewV4(endpoint string, accessKeyID, secretAccessKey string, insecure bool) (CloudStorageClient, error)

NewV4 - instantiate minio client with Amazon S3 signature version '4' compatibility.

type ErrorResponse 

type ErrorResponse struct {
	XMLName    xml.Name `xml:"Error" json:"-"`
	Code       string
	Message    string
	BucketName string
	Key        string
	RequestID  string `xml:"RequestId"`
	HostID     string `xml:"HostId"`

	// Region where the bucket is located. This header is returned
	// only in HEAD bucket and ListObjects response.
	AmzBucketRegion string
}

ErrorResponse - Is the typed error returned by all API operations.

func ToErrorResponse 

func ToErrorResponse(err error) ErrorResponse

ToErrorResponse - Returns parsed ErrorResponse struct from body and http headers.

For example: 

import s3 "github.com/minio/minio-go"
...
...
reader, stat, err := s3.GetObject(...)
if err != nil {
   resp := s3.ToErrorResponse(err)
}
...

func (ErrorResponse) Error 

func (e ErrorResponse) Error() string

Error - Returns HTTP error string

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object represents an open object. It implements Read, ReadAt, Seeker, Close for a HTTP stream.

func (*Object) Close 

func (o *Object) Close() (err error)

Close - The behavior of Close after the first call returns error for subsequent Close() calls.

func (*Object) Read 

func (o *Object) Read(b []byte) (n int, err error)

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Returns io.EOF upon end of file.

func (*Object) ReadAt 

func (o *Object) ReadAt(b []byte, offset int64) (n int, err error)

ReadAt reads len(b) bytes from the File starting at byte offset off. It returns the number of bytes read and the error, if any. ReadAt always returns a non-nil error when n < len(b). At end of file, that error is io.EOF.

func (*Object) Seek 

func (o *Object) Seek(offset int64, whence int) (n int64, err error)

Seek sets the offset for the next Read or Write to offset, interpreted according to whence: 0 means relative to the origin of the file, 1 means relative to the current offset, and 2 means relative to the end. Seek returns the new offset and an error, if any.

Seeking to a negative offset is an error. Seeking to any positive offset is legal, subsequent io operations succeed until the underlying object is not closed.

func (*Object) Stat 

func (o *Object) Stat() (ObjectInfo, error)

Stat returns the ObjectInfo structure describing object.

type ObjectInfo 

type ObjectInfo struct {
	// An ETag is optionally set to md5sum of an object.  In case of multipart objects,
	// ETag is of the form MD5SUM-N where MD5SUM is md5sum of all individual md5sums of
	// each parts concatenated into one string.
	ETag string `json:"etag"`

	Key          string    `json:"name"`         // Name of the object
	LastModified time.Time `json:"lastModified"` // Date and time the object was last modified.
	Size         int64     `json:"size"`         // Size in bytes of the object.
	ContentType  string    `json:"contentType"`  // A standard MIME type describing the format of the object data.

	// Owner name.
	Owner struct {
		DisplayName string `json:"name"`
		ID          string `json:"id"`
	} `json:"owner"`

	// The class of storage used to store the object.
	StorageClass string `json:"storageClass"`

	// Error
	Err error `json:"-"`
}

ObjectInfo container for object metadata.

type ObjectMultipartInfo 

type ObjectMultipartInfo struct {
	// Date and time at which the multipart upload was initiated.
	Initiated time.Time `type:"timestamp" timestampFormat:"iso8601"`

	Initiator initiator
	Owner     owner

	// The type of storage to use for the object. Defaults to 'STANDARD'.
	StorageClass string

	// Key of the object for which the multipart upload was initiated.
	Key string

	// Size in bytes of the object.
	Size int64

	// Upload ID that identifies the multipart upload.
	UploadID string `xml:"UploadId"`

	// Error
	Err error
}

ObjectMultipartInfo container for multipart object metadata.

type PostPolicy 

type PostPolicy struct {
	// contains filtered or unexported fields
}

PostPolicy - Provides strict static type conversion and validation for Amazon S3's POST policy JSON string.

func NewPostPolicy 

func NewPostPolicy() *PostPolicy

NewPostPolicy - Instantiate new post policy.

func (*PostPolicy) SetBucket 

func (p *PostPolicy) SetBucket(bucketName string) error

SetBucket - Sets bucket at which objects will be uploaded to.

func (*PostPolicy) SetContentLengthRange 

func (p *PostPolicy) SetContentLengthRange(min, max int64) error

SetContentLengthRange - Set new min and max content length condition for all incoming uploads.

func (*PostPolicy) SetContentType 

func (p *PostPolicy) SetContentType(contentType string) error

SetContentType - Sets content-type of the object for this policy based upload.

func (*PostPolicy) SetExpires 

func (p *PostPolicy) SetExpires(t time.Time) error

SetExpires - Sets expiration time for the new policy.

func (*PostPolicy) SetKey 

func (p *PostPolicy) SetKey(key string) error

SetKey - Sets an object name for the policy based upload.

func (*PostPolicy) SetKeyStartsWith 

func (p *PostPolicy) SetKeyStartsWith(keyStartsWith string) error

SetKeyStartsWith - Sets an object name that an policy based upload can start with.

func (PostPolicy) String 

func (p PostPolicy) String() string

Stringer interface for printing policy in json formatted string.

type SignatureType 

type SignatureType int

SignatureType is type of Authorization requested for a given HTTP request. 

const (
	Latest SignatureType = iota
	SignatureV4
	SignatureV2
)

Different types of supported signatures - default is Latest i.e SignatureV4.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.