gcs

README

gcs

gcs implements Google Cloud Storage object operations through the JSON API. Its direct runtime dependencies are cloud.google.com/go/compute/metadata and golang.org/x/oauth2.

Install

go get github.com/git-pkgs/gcs

Write and read objects

ctx := context.Background()
bucket, err := gcs.OpenBucket(ctx, "gs://my-bucket")
if err != nil {
	return err
}

_, err = bucket.Write(ctx, "packages/example.txt", strings.NewReader("content"))
if err != nil {
	return err
}

reader, err := bucket.Open(ctx, "packages/example.txt")
if err != nil {
	return err
}
defer reader.Close()

Write returns the number of bytes uploaded and sends each object as one non-resumable media upload. Retrying after a failed request starts the upload again.

Open, Exists, Delete, and Size operate on individual object names. Open and Size return ErrNotFound for a missing object; Delete accepts one. ListPrefix reads object metadata by prefix, while UsedSpace reads every object page with only the size field and takes time proportional to the object count.

Authentication

OpenBucket reads Application Default Credentials from attached service accounts on GKE, GCE, and Cloud Run. It also reads a file selected by GOOGLE_APPLICATION_CREDENTIALS or local credentials created by gcloud auth application-default login.

Signed URLs

SignedURL generates a V2 signed URL with a service-account private key when the credential file contains one. For Workload Identity and impersonated credentials, signing calls the IAM Credentials signBlob API and requires permission to sign blobs. The method returns ErrSignedURLUnsupported for the emulator and plain user credentials that do not impersonate a service account.

Emulator

Set STORAGE_EMULATOR_HOST to the address of a Cloud Storage emulator. The value may include an http:// or https:// scheme.

License

MIT

Documentation

Overview

Package gcs provides a small Google Cloud Storage client built on the JSON API.

Index

• Variables
• type Bucket
  • func OpenBucket(ctx context.Context, urlStr string) (*Bucket, error)
  • func (g *Bucket) Delete(ctx context.Context, path string) error
  • func (g *Bucket) Exists(ctx context.Context, path string) (bool, error)
  • func (g *Bucket) ListPrefix(ctx context.Context, prefix string) ([]ObjectInfo, error)
  • func (g *Bucket) Open(ctx context.Context, path string) (io.ReadCloser, error)
  • func (g *Bucket) SignedURL(ctx context.Context, path string, expiry time.Duration) (string, error)
  • func (g *Bucket) Size(ctx context.Context, path string) (int64, error)
  • func (g *Bucket) UsedSpace(ctx context.Context) (int64, error)
  • func (g *Bucket) Write(ctx context.Context, path string, r io.Reader) (int64, error)
• type ObjectInfo

Variables

var (
	// ErrNotFound indicates that an object does not exist.
	ErrNotFound = errors.New("gcs object not found")

	// ErrSignedURLUnsupported indicates that the current credentials cannot sign URLs.
	ErrSignedURLUnsupported = errors.New("signed gcs URLs not supported by current credentials")
)

Types

type Bucket

type Bucket struct {
	// contains filtered or unexported fields
}

Bucket provides access to a Google Cloud Storage bucket through the JSON API.

func OpenBucket

func OpenBucket(ctx context.Context, urlStr string) (*Bucket, error)

OpenBucket opens a Google Cloud Storage bucket from a gs:// URL.

func (*Bucket) Delete

func (g *Bucket) Delete(ctx context.Context, path string) error

Delete removes an object. A missing object is not an error.

func (*Bucket) Exists

func (g *Bucket) Exists(ctx context.Context, path string) (bool, error)

Exists reports whether an object exists.

func (*Bucket) ListPrefix

func (g *Bucket) ListPrefix(ctx context.Context, prefix string) ([]ObjectInfo, error)

ListPrefix returns metadata for objects whose names start with prefix.

func (*Bucket) Open

func (g *Bucket) Open(ctx context.Context, path string) (io.ReadCloser, error)

Open reads an object. The caller must close the returned reader.

func (*Bucket) SignedURL

func (g *Bucket) SignedURL(ctx context.Context, path string, expiry time.Duration) (string, error)

SignedURL returns a time-limited URL for reading an object.

func (*Bucket) Size

func (g *Bucket) Size(ctx context.Context, path string) (int64, error)

Size returns an object's size in bytes.

func (*Bucket) UsedSpace

func (g *Bucket) UsedSpace(ctx context.Context) (int64, error)

UsedSpace returns the total size of every object in the bucket.

func (*Bucket) Write

func (g *Bucket) Write(ctx context.Context, path string, r io.Reader) (int64, error)

Write stores an object and returns the number of bytes read from r.

type ObjectInfo

type ObjectInfo struct {
	Name    string
	Size    int64
	ModTime time.Time
}

ObjectInfo contains metadata for an object.