cloud

package module
v0.19.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 27, 2018 License: Apache-2.0 Imports: 0 Imported by: 0

README

Google Cloud Client Libraries for Go

GoDoc

Go packages for Google Cloud Platform services.

import "cloud.google.com/go"

To install the packages on your system,

$ go get -u cloud.google.com/go/...

NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.

NOTE: Github repo is a mirror of https://code.googlesource.com/gocloud.

News

February 26, 2018

v0.19.0

  • bigquery:

    • Support customer-managed encryption keys.
  • bigtable:

    • Improved emulator support.
    • Support GetCluster.
  • datastore:

    • Add general mutations.
    • Support pointer struct fields.
    • Support transaction options.
  • firestore:

    • Add Transaction.GetAll.
    • Support document cursors.
  • logging:

    • Support concurrent RPCs to the service.
    • Support per-entry resources.
  • profiler:

    • Add config options to disable heap and thread profiling.
    • Read the project ID from $GOOGLE_CLOUD_PROJECT when it's set.
  • pubsub:

    • BEHAVIOR CHANGE: Release flow control after ack/nack (instead of after the callback returns).
    • Add SubscriptionInProject.
    • Add OpenCensus instrumentation for streaming pull.
  • storage:

    • Support CORS.

January 18, 2018

v0.18.0

  • bigquery:

    • Marked stable.
    • Schema inference of nullable fields supported.
    • Added TimePartitioning to QueryConfig.
  • firestore: Data provided to DocumentRef.Set with a Merge option can contain Delete sentinels.

  • logging: Clients can accept parent resources other than projects.

  • pubsub:

    • pubsub/pstest: A lighweight fake for pubsub. Experimental; feedback welcome.
    • Support updating more subscription metadata: AckDeadline, RetainAckedMessages and RetentionDuration.
  • oslogin/apiv1beta: New client for the Cloud OS Login API.

  • rpcreplay: A package for recording and replaying gRPC traffic.

  • spanner:

    • Add a ReadWithOptions that supports a row limit, as well as an index.
    • Support query plan and execution statistics.
    • Added OpenCensus support.
  • storage: Clarify checksum validation for gzipped files (it is not validated when the file is served uncompressed).

December 11, 2017

v0.17.0

  • firestore BREAKING CHANGES:
    • Remove UpdateMap and UpdateStruct; rename UpdatePaths to Update. Change docref.UpdateMap(ctx, map[string]interface{}{"a.b", 1}) to docref.Update(ctx, []firestore.Update{{Path: "a.b", Value: 1}})

      Change docref.UpdateStruct(ctx, []string{"Field"}, aStruct) to docref.Update(ctx, []firestore.Update{{Path: "Field", Value: aStruct.Field}})

    • Rename MergePaths to Merge; require args to be FieldPaths

    • A value stored as an integer can be read into a floating-point field, and vice versa.

  • bigtable/cmd/cbt:
    • Support deleting a column.
    • Add regex option for row read.
  • spanner: Mark stable.
  • storage:
    • Add Reader.ContentEncoding method.
    • Fix handling of SignedURL headers.
  • bigquery:
    • If Uploader.Put is called with no rows, it returns nil without making a call.
    • Schema inference supports the "nullable" option in struct tags for non-required fields.
    • TimePartitioning supports "Field".

Older news

Supported APIs

Google API Status Package
BigQuery stable cloud.google.com/go/bigquery
Bigtable stable cloud.google.com/go/bigtable
Container alpha cloud.google.com/go/container/apiv1
Data Loss Prevention alpha cloud.google.com/go/dlp/apiv2beta1
Datastore stable cloud.google.com/go/datastore
Debugger alpha cloud.google.com/go/debugger/apiv2
ErrorReporting alpha cloud.google.com/go/errorreporting
Firestore beta cloud.google.com/go/firestore
Language stable cloud.google.com/go/language/apiv1
Logging stable cloud.google.com/go/logging
Monitoring beta cloud.google.com/go/monitoring/apiv3
OS Login alpha cloud.google.com/compute/docs/oslogin/rest
Pub/Sub beta cloud.google.com/go/pubsub
Spanner stable cloud.google.com/go/spanner
Speech stable cloud.google.com/go/speech/apiv1
Storage stable cloud.google.com/go/storage
Translation stable cloud.google.com/go/translate
Video Intelligence beta cloud.google.com/go/videointelligence/apiv1beta1
Vision stable cloud.google.com/go/vision/apiv1

Alpha status: the API is still being actively developed. As a result, it might change in backward-incompatible ways and is not recommended for production use.

Beta status: the API is largely complete, but still has outstanding features and bugs to be addressed. There may be minor backwards-incompatible changes where necessary.

Stable status: the API is mature and ready for production use. We will continue addressing bugs and feature requests.

Documentation and examples are available at https://godoc.org/cloud.google.com/go

Visit or join the google-api-go-announce group for updates on these packages.

Go Versions Supported

We support the two most recent major versions of Go. If Google App Engine uses an older version, we support that as well. You can see which versions are currently supported by looking at the lines following go: in .travis.yml.

Authorization

By default, each API will use Google Application Default Credentials for authorization credentials used in calling the API endpoints. This will allow your application to run in many environments without requiring explicit configuration.

client, err := storage.NewClient(ctx)

To authorize using a JSON key file, pass option.WithServiceAccountFile to the NewClient function of the desired package. For example:

client, err := storage.NewClient(ctx, option.WithServiceAccountFile("path/to/keyfile.json"))

You can exert more control over authorization by using the golang.org/x/oauth2 package to create an oauth2.TokenSource. Then pass option.WithTokenSource to the NewClient function: snip:# (auth-ts)

tokenSource := ...
client, err := storage.NewClient(ctx, option.WithTokenSource(tokenSource))

Cloud Datastore GoDoc

Example Usage

First create a datastore.Client to use throughout your application:

client, err := datastore.NewClient(ctx, "my-project-id")
if err != nil {
	log.Fatal(err)
}

Then use that client to interact with the API:

type Post struct {
	Title       string
	Body        string `datastore:",noindex"`
	PublishedAt time.Time
}
keys := []*datastore.Key{
	datastore.NameKey("Post", "post1", nil),
	datastore.NameKey("Post", "post2", nil),
}
posts := []*Post{
	{Title: "Post 1", Body: "...", PublishedAt: time.Now()},
	{Title: "Post 2", Body: "...", PublishedAt: time.Now()},
}
if _, err := client.PutMulti(ctx, keys, posts); err != nil {
	log.Fatal(err)
}

Cloud Storage GoDoc

Example Usage

First create a storage.Client to use throughout your application:

client, err := storage.NewClient(ctx)
if err != nil {
	log.Fatal(err)
}
// Read the object1 from bucket.
rc, err := client.Bucket("bucket").Object("object1").NewReader(ctx)
if err != nil {
	log.Fatal(err)
}
defer rc.Close()
body, err := ioutil.ReadAll(rc)
if err != nil {
	log.Fatal(err)
}

Cloud Pub/Sub GoDoc

Example Usage

First create a pubsub.Client to use throughout your application:

client, err := pubsub.NewClient(ctx, "project-id")
if err != nil {
	log.Fatal(err)
}

Then use the client to publish and subscribe:

// Publish "hello world" on topic1.
topic := client.Topic("topic1")
res := topic.Publish(ctx, &pubsub.Message{
	Data: []byte("hello world"),
})
// The publish happens asynchronously.
// Later, you can get the result from res:
...
msgID, err := res.Get(ctx)
if err != nil {
	log.Fatal(err)
}

// Use a callback to receive messages via subscription1.
sub := client.Subscription("subscription1")
err = sub.Receive(ctx, func(ctx context.Context, m *pubsub.Message) {
	fmt.Println(m.Data)
	m.Ack() // Acknowledge that we've consumed the message.
})
if err != nil {
	log.Println(err)
}

Cloud BigQuery GoDoc

Example Usage

First create a bigquery.Client to use throughout your application: snip:# (bq-1)

c, err := bigquery.NewClient(ctx, "my-project-ID")
if err != nil {
	// TODO: Handle error.
}

Then use that client to interact with the API: snip:# (bq-2)

// Construct a query.
q := c.Query(`
    SELECT year, SUM(number)
    FROM [bigquery-public-data:usa_names.usa_1910_2013]
    WHERE name = "William"
    GROUP BY year
    ORDER BY year
`)
// Execute the query.
it, err := q.Read(ctx)
if err != nil {
	// TODO: Handle error.
}
// Iterate through the results.
for {
	var values []bigquery.Value
	err := it.Next(&values)
	if err == iterator.Done {
		break
	}
	if err != nil {
		// TODO: Handle error.
	}
	fmt.Println(values)
}

Stackdriver Logging GoDoc

Example Usage

First create a logging.Client to use throughout your application: snip:# (logging-1)

ctx := context.Background()
client, err := logging.NewClient(ctx, "my-project")
if err != nil {
	// TODO: Handle error.
}

Usually, you'll want to add log entries to a buffer to be periodically flushed (automatically and asynchronously) to the Stackdriver Logging service. snip:# (logging-2)

logger := client.Logger("my-log")
logger.Log(logging.Entry{Payload: "something happened!"})

Close your client before your program exits, to flush any buffered log entries. snip:# (logging-3)

err = client.Close()
if err != nil {
	// TODO: Handle error.
}

Cloud Spanner GoDoc

Example Usage

First create a spanner.Client to use throughout your application:

client, err := spanner.NewClient(ctx, "projects/P/instances/I/databases/D")
if err != nil {
	log.Fatal(err)
}
// Simple Reads And Writes
_, err = client.Apply(ctx, []*spanner.Mutation{
	spanner.Insert("Users",
		[]string{"name", "email"},
		[]interface{}{"alice", "a@example.com"})})
if err != nil {
	log.Fatal(err)
}
row, err := client.Single().ReadRow(ctx, "Users",
	spanner.Key{"alice"}, []string{"email"})
if err != nil {
	log.Fatal(err)
}

Contributing

Contributions are welcome. Please, see the CONTRIBUTING document for details. We're using Gerrit for our code reviews. Please don't open pull requests against this repo, new pull requests will be automatically closed.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.

Documentation

Overview

Package cloud is the root of the packages used to access Google Cloud Services. See https://godoc.org/cloud.google.com/go for a full list of sub-packages.

Examples in this package show ways to authorize and authenticate the sub packages.

Connection Pooling

Connection pooling differs in clients based on their transport. Cloud clients either rely on HTTP or gRPC transports to communicate with Google Cloud.

Cloud clients that use HTTP (bigquery, compute, storage, and translate) rely on the underlying HTTP transport to cache connections for later re-use. These are cached to the default http.MaxIdleConns and http.MaxIdleConnsPerHost settings in http.DefaultTransport.

For gPRC clients (all others in this repo), connection pooling is configurable. Users of cloud client libraries may specify option.WithGRPCConnectionPool(n) as a client option to NewClient calls. This configures the underlying gRPC connections to be pooled and addressed in a round robin fashion.

Example (ApplicationDefaultCredentials)
package main

import (
	"cloud.google.com/go/datastore"
	"golang.org/x/net/context"
)

func main() {
	// Google Application Default Credentials is the recommended way to authorize
	// and authenticate clients.
	//
	// See the following link on how to create and obtain Application Default Credentials:
	// https://developers.google.com/identity/protocols/application-default-credentials.
	client, err := datastore.NewClient(context.Background(), "project-id")
	if err != nil {
		// TODO: handle error.
	}
	_ = client // Use the client.
}
Output:

Example (ServiceAccountFile)
package main

import (
	"cloud.google.com/go/datastore"
	"golang.org/x/net/context"
	"google.golang.org/api/option"
)

func main() {
	// Use a JSON key file associated with a Google service account to
	// authenticate and authorize. Service Account keys can be created and
	// downloaded from https://console.developers.google.com/permissions/serviceaccounts.
	//
	// Note: This example uses the datastore client, but the same steps apply to
	// the other client libraries underneath this package.
	client, err := datastore.NewClient(context.Background(),
		"project-id", option.WithServiceAccountFile("/path/to/service-account-key.json"))
	if err != nil {
		// TODO: handle error.
	}
	_ = client // Use the client.
}
Output:

Directories

Path Synopsis
Package bigquery provides a client for the BigQuery service.
Package bigquery provides a client for the BigQuery service.
datatransfer/apiv1
Package datatransfer is an auto-generated package for the BigQuery Data Transfer API.
Package datatransfer is an auto-generated package for the BigQuery Data Transfer API.
Package bigtable is an API to Google Cloud Bigtable.
Package bigtable is an API to Google Cloud Bigtable.
bttest
Package bttest contains test helpers for working with the bigtable package.
Package bttest contains test helpers for working with the bigtable package.
cmd/cbt
Cbt is a tool for doing basic interactions with Cloud Bigtable.
Cbt is a tool for doing basic interactions with Cloud Bigtable.
cmd/emulator
cbtemulator launches the in-memory Cloud Bigtable server on the given address.
cbtemulator launches the in-memory Cloud Bigtable server on the given address.
cmd/loadtest
Loadtest does some load testing through the Go client library for Cloud Bigtable.
Loadtest does some load testing through the Go client library for Cloud Bigtable.
cmd/scantest
Scantest does scan-related load testing against Cloud Bigtable.
Scantest does scan-related load testing against Cloud Bigtable.
internal/cbtconfig
Package cbtconfig encapsulates common code for reading configuration from .cbtrc and gcloud.
Package cbtconfig encapsulates common code for reading configuration from .cbtrc and gcloud.
internal/gax
This is ia snapshot from github.com/googleapis/gax-go with minor modifications.
This is ia snapshot from github.com/googleapis/gax-go with minor modifications.
internal/option
Package option contains common code for dealing with client options.
Package option contains common code for dealing with client options.
Package civil implements types for civil time, a time-zone-independent representation of time that follows the rules of the proleptic Gregorian calendar with exactly 24-hour days, 60-minute hours, and 60-second minutes.
Package civil implements types for civil time, a time-zone-independent representation of time that follows the rules of the proleptic Gregorian calendar with exactly 24-hour days, 60-minute hours, and 60-second minutes.
cmd
go-cloud-debug-agent/internal/breakpoints
Package breakpoints handles breakpoint requests we get from the user through the Debuglet Controller, and manages corresponding breakpoints set in the code.
Package breakpoints handles breakpoint requests we get from the user through the Debuglet Controller, and manages corresponding breakpoints set in the code.
go-cloud-debug-agent/internal/controller
Package controller is a library for interacting with the Google Cloud Debugger's Debuglet Controller service.
Package controller is a library for interacting with the Google Cloud Debugger's Debuglet Controller service.
go-cloud-debug-agent/internal/valuecollector
Package valuecollector is used to collect the values of variables in a program.
Package valuecollector is used to collect the values of variables in a program.
compute
metadata
Package metadata provides access to Google Compute Engine (GCE) metadata and API service accounts.
Package metadata provides access to Google Compute Engine (GCE) metadata and API service accounts.
Package container contains a deprecated Google Container Engine client.
Package container contains a deprecated Google Container Engine client.
apiv1
Package container is an auto-generated package for the Google Container Engine API.
Package container is an auto-generated package for the Google Container Engine API.
dataproc
apiv1
Package dataproc is an auto-generated package for the Google Cloud Dataproc API.
Package dataproc is an auto-generated package for the Google Cloud Dataproc API.
Package datastore provides a client for Google Cloud Datastore.
Package datastore provides a client for Google Cloud Datastore.
debugger
apiv2
Package debugger is an auto-generated package for the Stackdriver Debugger API.
Package debugger is an auto-generated package for the Stackdriver Debugger API.
dlp
apiv2beta1
Package dlp is an auto-generated package for the DLP API.
Package dlp is an auto-generated package for the DLP API.
Package errorreporting is a Google Stackdriver Error Reporting library.
Package errorreporting is a Google Stackdriver Error Reporting library.
apiv1beta1
Package errorreporting is an auto-generated package for the Stackdriver Error Reporting API.
Package errorreporting is an auto-generated package for the Stackdriver Error Reporting API.
Package firestore provides a client for reading and writing to a Cloud Firestore database.
Package firestore provides a client for reading and writing to a Cloud Firestore database.
apiv1beta1
Package firestore is an auto-generated package for the Google Cloud Firestore API.
Package firestore is an auto-generated package for the Google Cloud Firestore API.
genproto
Package tests is a generated protocol buffer package.
Package tests is a generated protocol buffer package.
iam
Package iam supports the resource-specific operations of Google Cloud IAM (Identity and Access Management) for the Google Cloud Libraries.
Package iam supports the resource-specific operations of Google Cloud IAM (Identity and Access Management) for the Google Cloud Libraries.
admin/apiv1
Package admin is an experimental, auto-generated package for the Google Identity and Access Management (IAM) API.
Package admin is an experimental, auto-generated package for the Google Identity and Access Management (IAM) API.
atomiccache
Package atomiccache provides a map-based cache that supports very fast reads.
Package atomiccache provides a map-based cache that supports very fast reads.
fields
Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity.
Package fields provides a view of the fields of a struct that follows the Go rules, amended to consider tags and case insensitivity.
optional
Package optional provides versions of primitive types that can be nil.
Package optional provides versions of primitive types that can be nil.
pretty
Package pretty implements a simple pretty-printer.
Package pretty implements a simple pretty-printer.
protostruct
Package protostruct supports operations on the protocol buffer Struct message.
Package protostruct supports operations on the protocol buffer Struct message.
testutil
Package testutil contains helper functions for writing tests.
Package testutil contains helper functions for writing tests.
tracecontext
Package tracecontext provides encoders and decoders for Stackdriver Trace contexts.
Package tracecontext provides encoders and decoders for Stackdriver Trace contexts.
version
Package version contains version information for Google Cloud Client Libraries for Go, as reported in request headers.
Package version contains version information for Google Cloud Client Libraries for Go, as reported in request headers.
language
apiv1
Google Cloud Natural Language API provides natural language understanding technologies to developers.
Google Cloud Natural Language API provides natural language understanding technologies to developers.
apiv1beta2
Package language is an auto-generated package for the Google Cloud Natural Language API.
Package language is an auto-generated package for the Google Cloud Natural Language API.
Package logging contains a Stackdriver Logging client suitable for writing logs.
Package logging contains a Stackdriver Logging client suitable for writing logs.
apiv2
Package logging is an auto-generated package for the Stackdriver Logging API.
Package logging is an auto-generated package for the Stackdriver Logging API.
internal/testing
Package testing provides support for testing the logging client.
Package testing provides support for testing the logging client.
logadmin
Package logadmin contains a Stackdriver Logging client that can be used for reading logs and working with sinks, metrics and monitored resources.
Package logadmin contains a Stackdriver Logging client that can be used for reading logs and working with sinks, metrics and monitored resources.
Package longrunning supports Long Running Operations for the Google Cloud Libraries.
Package longrunning supports Long Running Operations for the Google Cloud Libraries.
autogen
Package longrunning is an auto-generated package for the Google Long Running Operations API.
Package longrunning is an auto-generated package for the Google Long Running Operations API.
monitoring
apiv3
Package monitoring is an auto-generated package for the Stackdriver Monitoring API.
Package monitoring is an auto-generated package for the Stackdriver Monitoring API.
oslogin
apiv1beta
Package oslogin is an auto-generated package for the Google Cloud OS Login API.
Package oslogin is an auto-generated package for the Google Cloud OS Login API.
privacy
dlp/apiv2beta2
Package dlp is an auto-generated package for the DLP API.
Package dlp is an auto-generated package for the DLP API.
Package profiler is a client for the Stackdriver Profiler service.
Package profiler is a client for the Stackdriver Profiler service.
Package pubsub provides an easy way to publish and receive Google Cloud Pub/Sub messages, hiding the the details of the underlying server RPCs.
Package pubsub provides an easy way to publish and receive Google Cloud Pub/Sub messages, hiding the the details of the underlying server RPCs.
apiv1
Package pubsub is an auto-generated package for the Google Cloud Pub/Sub API.
Package pubsub is an auto-generated package for the Google Cloud Pub/Sub API.
loadtest
Package loadtest implements load testing for pubsub, following the interface defined in https://github.com/GoogleCloudPlatform/pubsub/tree/master/load-test-framework/ .
Package loadtest implements load testing for pubsub, following the interface defined in https://github.com/GoogleCloudPlatform/pubsub/tree/master/load-test-framework/ .
loadtest/pb
Package google_pubsub_loadtest is a generated protocol buffer package.
Package google_pubsub_loadtest is a generated protocol buffer package.
pstest
Package pstest provides a fake Cloud PubSub service for testing.
Package pstest provides a fake Cloud PubSub service for testing.
Package rpcreplay supports the capture and replay of gRPC calls.
Package rpcreplay supports the capture and replay of gRPC calls.
proto/intstore
Package intstore is a generated protocol buffer package.
Package intstore is a generated protocol buffer package.
proto/rpcreplay
Package rpcreplay is a generated protocol buffer package.
Package rpcreplay is a generated protocol buffer package.
Package spanner provides a client for reading and writing to Cloud Spanner databases.
Package spanner provides a client for reading and writing to Cloud Spanner databases.
admin/database/apiv1
Package database is an auto-generated package for the Cloud Spanner Database Admin API.
Package database is an auto-generated package for the Cloud Spanner Database Admin API.
admin/instance/apiv1
Package instance is an auto-generated package for the Cloud Spanner Instance Admin API.
Package instance is an auto-generated package for the Cloud Spanner Instance Admin API.
apiv1
Package spanner is an auto-generated package for the Cloud Spanner API.
Package spanner is an auto-generated package for the Cloud Spanner API.
speech
apiv1
Google Cloud Speech API.
Google Cloud Speech API.
apiv1beta1
Package speech is an auto-generated package for the Google Cloud Speech API.
Package speech is an auto-generated package for the Google Cloud Speech API.
Package storage provides an easy way to work with Google Cloud Storage.
Package storage provides an easy way to work with Google Cloud Storage.
This package is OBSOLETE.
This package is OBSOLETE.
apiv1
Package trace is an auto-generated package for the Stackdriver Trace API.
Package trace is an auto-generated package for the Stackdriver Trace API.
apiv2
Package trace is an auto-generated package for the Stackdriver Trace API.
Package trace is an auto-generated package for the Stackdriver Trace API.
Package translate is a client for the Google Translation API.
Package translate is a client for the Google Translation API.
internal/translate/v2
Package translate provides access to the Translate API.
Package translate provides access to the Translate API.
videointelligence
apiv1
Package videointelligence is an auto-generated package for the Cloud Video Intelligence API.
Package videointelligence is an auto-generated package for the Cloud Video Intelligence API.
apiv1beta1
Package videointelligence is an auto-generated package for the Google Cloud Video Intelligence API.
Package videointelligence is an auto-generated package for the Google Cloud Video Intelligence API.
apiv1beta2
Package videointelligence is an auto-generated package for the Google Cloud Video Intelligence API.
Package videointelligence is an auto-generated package for the Google Cloud Video Intelligence API.
vision
apiv1
Integrates Google Vision features, including image labeling, face, logo, and landmark detection, optical character recognition (OCR), and detection of explicit content, into applications.
Integrates Google Vision features, including image labeling, face, logo, and landmark detection, optical character recognition (OCR), and detection of explicit content, into applications.
apiv1p1beta1
Integrates Google Vision features, including image labeling, face, logo, and landmark detection, optical character recognition (OCR), and detection of explicit content, into applications.
Integrates Google Vision features, including image labeling, face, logo, and landmark detection, optical character recognition (OCR), and detection of explicit content, into applications.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL