Documentation

Overview

Package gcsblob provides a blob implementation that uses GCS. Use OpenBucket to construct a *blob.Bucket.

URLs

For blob.OpenBucket, gcsblob registers for the scheme "gs". The default URL opener will set up a connection using default credentials from the environment, as described in https://cloud.google.com/docs/authentication/production. Some environments, such as GCE, come without a private key. In such cases the IAM Credentials API will be configured for use in Options.MakeSignBytes, which will introduce latency to any and all calls to bucket.SignedURL that you can avoid by installing a service account credentials file or obtaining and configuring a private key: https://cloud.google.com/iam/docs/creating-managing-service-account-keys

To customize the URL opener, or for more details on the URL format, see URLOpener. See https://gocloud.dev/concepts/urls/ for background information.

Escaping

Go CDK supports all UTF-8 strings; to make this work with services lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for gcsblob:

- Blob keys: ASCII characters 10 and 13 are escaped to "__0x<hex>__".
  Additionally, the "/" in "../" is escaped in the same way.

As

gcsblob exposes the following types for As:

- Bucket: *storage.Client
- Error: *googleapi.Error
- ListObject: storage.ObjectAttrs
- ListOptions.BeforeList: *storage.Query
- Reader: *storage.Reader
- ReaderOptions.BeforeRead: **storage.ObjectHandle, *storage.Reader (if accessing both, must be in that order)
- Attributes: storage.ObjectAttrs
- CopyOptions.BeforeCopy: *CopyObjectHandles, *storage.Copier (if accessing both, must be in that order)
- WriterOptions.BeforeWrite: **storage.ObjectHandle, *storage.Writer (if accessing both, must be in that order)
- SignedURLOptions.BeforeSign: *storage.SignedURLOptions
Example (OpenBucketFromURL)
Output:

Index

Examples

Constants

View Source
const Scheme = "gs"

    Scheme is the URL scheme gcsblob registers its URLOpener under on blob.DefaultMux.

    Variables

    View Source
    var Set = wire.NewSet(
    	wire.Struct(new(URLOpener), "Client"),
    )

      Set holds Wire providers for this package.

      Functions

      func OpenBucket

      func OpenBucket(ctx context.Context, client *gcp.HTTPClient, bucketName string, opts *Options) (*blob.Bucket, error)

        OpenBucket returns a *blob.Bucket backed by an existing GCS bucket. See the package documentation for an example.

        Example
        Output:
        
        

        Types

        type CopyObjectHandles

        type CopyObjectHandles struct {
        	Dst, Src *storage.ObjectHandle
        }

          CopyObjectHandles holds the ObjectHandles for the destination and source of a Copy. It is used by the BeforeCopy As hook.

          type Options

          type Options struct {
          	// GoogleAccessID represents the authorizer for SignedURL.
          	// Required to use SignedURL.
          	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
          	GoogleAccessID string
          
          	// PrivateKey is the Google service account private key.
          	// Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
          	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
          	// Deprecated: Use MakeSignBytes instead.
          	PrivateKey []byte
          
          	// SignBytes is a function for implementing custom signing.
          	// Exactly one of PrivateKey, SignBytes, or MakeSignBytes must be non-nil to use SignedURL.
          	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
          	// Deprecated: Use MakeSignBytes instead.
          	SignBytes func([]byte) ([]byte, error)
          
          	// MakeSignBytes is a factory for functions that are being used in place of an empty SignBytes.
          	// If your implementation of 'SignBytes' needs a request context, set this instead.
          	MakeSignBytes func(requestCtx context.Context) SignBytesFunc
          }

            Options sets options for constructing a *blob.Bucket backed by GCS.

            type SignBytesFunc

            type SignBytesFunc func([]byte) ([]byte, error)

              SignBytesFunc is shorthand for the signature of Options.SignBytes.

              type URLOpener

              type URLOpener struct {
              	// Client must be set to a non-nil HTTP client authenticated with
              	// Cloud Storage scope or equivalent.
              	Client *gcp.HTTPClient
              
              	// Options specifies the default options to pass to OpenBucket.
              	Options Options
              }

                URLOpener opens GCS URLs like "gs://mybucket".

                The URL host is used as the bucket name.

                The following query parameters are supported:

                - access_id: sets Options.GoogleAccessID
                - private_key_path: path to read for Options.PrivateKey
                

                Currently their use is limited to SignedURL.

                func (*URLOpener) OpenBucketURL

                func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

                  OpenBucketURL opens the GCS bucket with the same name as the URL's host.

                  Source Files