Documentation

Overview

    Package gs implements a versatile Google Storage client on top of the standard Google Storage Go API. It adds:

    - The ability to read from specific byte offsets.
    - Exponential backoff retries on transient errors.
    - Logging
    - The ability to easily stub a Google Storage interface.
    

    Index

    Constants

    This section is empty.

    Variables

    View Source
    var (
    	// ReadWriteScopes is the set of scopes needed for read/write Google Storage
    	// access.
    	ReadWriteScopes = []string{gs.ScopeReadWrite}
    
    	// ReadOnlyScopes is the set of scopes needed for read/write Google Storage
    	// read-only access.
    	ReadOnlyScopes = []string{gs.ScopeReadOnly}
    )

    Functions

    This section is empty.

    Types

    type Client

    type Client interface {
    	io.Closer
    
    	// Attrs retrieves Object attributes for a given path.
    	Attrs(p Path) (*gs.ObjectAttrs, error)
    
    	// NewReader instantiates a new Reader instance for the named bucket/path.
    	//
    	// The supplied offset must be >= 0, or else this function will panic.
    	//
    	// If the supplied length is <0, no upper byte bound will be set.
    	NewReader(p Path, offset, length int64) (io.ReadCloser, error)
    
    	// NewWriter instantiates a new Writer instance for the named bucket/path.
    	NewWriter(p Path) (Writer, error)
    
    	// Delete deletes the object at the specified path.
    	//
    	// If the object does not exist, it is considered a success.
    	Delete(p Path) error
    
    	// Rename renames an object from one path to another.
    	//
    	// NOTE: The object should be removed from its original path, but current
    	// implementation uses two operations (Copy + Delete), so it may
    	// occasionally fail.
    	Rename(src, dst Path) error
    }

      Client abstracts functionality to connect with and use Google Storage from the actual Google Storage client.

      Non-production implementations are used primarily for testing.

      func NewProdClient

      func NewProdClient(ctx context.Context, rt http.RoundTripper) (Client, error)

        NewProdClient creates a new Client instance that uses production Cloud Storage.

        The supplied RoundTripper will be used to make connections. If nil, the default HTTP client will be used.

        type LimitedClient

        type LimitedClient struct {
        	// Base is the base Client instance.
        	Client
        
        	// MaxReadBytes, if >0, is the maximum number of bytes that can be read at a
        	// time. If more bytes are required, additional read calls will be made.
        	MaxReadBytes int64
        }

          LimitedClient wraps a base Client, allowing additional limits to be applied to its calls.

          func (*LimitedClient) NewReader

          func (lc *LimitedClient) NewReader(p Path, offset, length int64) (io.ReadCloser, error)

            NewReader implements Client.

            type Path

            type Path string

              Path is a Google Storage path. A full path consists of a Google storage bucket and a series of path components.

              An example of a Path is:

              gs://test-bucket/path/to/thing.txt
              

              func MakePath

              func MakePath(bucket string, parts ...string) Path

                MakePath constructs a Google Storage path from optional bucket and filename components.

                Trailing forward slashes will be removed from the bucket name, if present.

                func (Path) Bucket

                func (p Path) Bucket() string

                  Bucket returns the Google Storage bucket component of the Path. If there is no bucket, an empty string will be returned.

                  func (Path) Concat

                  func (p Path) Concat(v string, parts ...string) Path

                    Concat concatenates a filename component to the end of Path.

                    Multiple components may be specified. In this case, each will be added as a "/"-delimited component, and will have any present trailing slashes stripped.

                    func (Path) Filename

                    func (p Path) Filename() string

                      Filename returns the filename component of the Path. If there is no filename component, an empty string will be returned.

                      Leading and trailing slashes will be truncated.

                      func (Path) IsFullPath

                      func (p Path) IsFullPath() bool

                        IsFullPath returns true if the Path contains both a bucket and file name.

                        func (Path) Split

                        func (p Path) Split() (bucket string, filename string)

                          Split returns the bucket and filename components of the Path.

                          If a bucket is not defined (doesn't begin with "gs://"), the remainder will be considered to be the filename component. If a filename is not defined, an empty string will be returned.

                          type Writer

                          type Writer interface {
                          	io.WriteCloser
                          
                          	// Count returns the number of bytes written by the object.
                          	Count() int64
                          }

                            Writer is an augmented io.WriteCloser instance.