Documentation
¶
Overview ¶
Package bigquery is a database/sql driver for Google BigQuery, which runs GoogleSQL over its REST API, POST /bigquery/v2/projects/{project}/queries, with HTTPS and JSON. It registers the one name "bigquery", and takes a DSN of the form bigquery://project/dataset?credential_file=/path/key.json (D189).
db, err := sql.Open("bigquery", "bigquery://my-project/my_dataset?credential_file=/path/key.json")
The host of the DSN is the project, and the path is the dataset of each statement, with an optional location before it. The secret is the path of a service account key file, in the key credential_file, and never the text of the key. The driver reads the file, signs a token request with RS256, and exchanges it at the token endpoint that the file names. It keeps the access token and signs a new one before the old one ends. It sends the token only to the host of the endpoint. A caller can set a ready access token in the Config of a connector.
An emulator has no login. A DSN for the emulator names its address in the key endpoint and sets disable_auth=true.
The driver always sends useLegacySql as false, and asks for timestamps as ISO8601_STRING. If a query is still running after a short wait, the driver polls the job with jobs.getQueryResults, and it cancels the job with jobs.cancel when the context ends. It reads each page of a result with the page token of the service, one token at a time. The server binds each argument with an explicit type. The driver has no transactions in the first release, because a transaction needs a session across requests (D189). docs/BIGQUERY.md holds what the driver knows about the service.
Index ¶
Constants ¶
const ( // ErrCut is the error of a result that ends before the count of rows that // the server named in totalRows. The service runs the whole statement before // it sends the first page, so only a later page can be cut (D21 and D189). // After a row, it comes wrapped with dbimp.ErrIncomplete (D107). ErrCut dbimp.Error = "the answer was cut short" // ErrCanceled is the error of a job that the service ended with the reason // stopped, when the context of the statement did not end. Someone else // canceled it (recorded: bigquery-212). ErrCanceled dbimp.Error = "the job was canceled" // ErrNoCredential is the error of a connector that has no way to log in: it // has no key file, no access token, and no disable_auth. ErrNoCredential dbimp.Error = "no credential" //nolint:gosec // The text is a message, not a secret. )
The sentinel errors of the driver. Each one wraps into a failure with errors.Is (D6).
const Name = "bigquery"
Name is the name that the driver registers with database/sql, and the scheme of its DSN.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
// Project is the id of the project that runs and pays for each statement.
// It is the host of the DSN, and it is a part of every request path.
Project string
// Dataset is the default dataset of each statement, which lets a statement
// name a table with no dataset. It is optional.
Dataset string
// Location is the location of each job, such as US, EU or asia-southeast1.
// Empty leaves the choice to the service, which uses the location of the
// dataset of the statement. A job that is not in US or EU needs it on every
// call that follows, and the driver sends the location that the service
// answers.
Location string
// Endpoint is the address of the service, such as http://127.0.0.1:9050 for
// an emulator. Empty is https://bigquery.googleapis.com.
Endpoint string
// CredentialFile is the path of the key file of a service account, and the
// secret of the DSN (D189). The driver reads the file when it makes the
// connector. The key text never sits in a URL.
CredentialFile string
// AccessToken is a ready access token, such as the output of gcloud auth
// print-access-token. The driver sends it as it is and never renews it. A
// DSN cannot hold it.
AccessToken string
// DisableAuth sends no Authorization header, for an emulator, which has no
// login.
DisableAuth bool
// Scopes are the scopes of the token that the driver asks for. Empty is
// https://www.googleapis.com/auth/bigquery.
Scopes []string
// Timeout is the time that the service gives each job, as jobTimeoutMs. Zero
// sends none.
Timeout time.Duration
// MaxResults is the most rows in a page of a result, as maxResults. Zero
// leaves the page size to the service, which cuts a page at 10 MB.
MaxResults int
}
Config is the configuration of a connector, which the DSN holds. The caller owns it (D7).
func ParseDSN ¶
ParseDSN parses a DSN of the form bigquery://project/dataset?credential_file=/path/key.json (D27, D35 and D189). The host is the project. The path holds the dataset, with an optional location before it as /location/dataset, and each is optional. The keys are endpoint, credential_file, disable_auth, scopes, location, timeout and max_results. The user of the URL is ignored, because dbrun writes one for the emulator, and a password is refused, because the secret is a path in credential_file. The timeout is a duration with a unit, such as 60s or 1m.
type Connector ¶
type Connector struct {
// contains filtered or unexported fields
}
Connector opens connections to one project. It owns its transport, and every connection shares it, with the access token. A caller can build one from a Config and open it with sql.OpenDB.
func NewConnector ¶
NewConnector returns a Connector for cfg. The connector keeps a copy of cfg. It reads the key file of cfg, if there is one, and a key file that is not valid fails the first Connect.
type Driver ¶
type Driver struct{}
Driver is the database/sql driver for BigQuery.
type Error ¶
type Error struct {
// HTTPStatus is the status code of the response.
HTTPStatus int
// Reason is the reason of the first entry of errors, such as invalidQuery,
// notFound or accessDenied, and empty when the answer has none.
Reason string
// Status is status, such as INVALID_ARGUMENT, and empty when the answer has
// none.
Status string
// Message is message, or the text of the answer when it has none.
Message string
// Location is the location of the first entry of errors, such as q for the
// text of a query, and empty when the answer has none.
Location string
// JobID is the id of the job, and empty when the driver did not know it.
JobID string
// contains filtered or unexported fields
}
Error is an error that BigQuery reported. Every answer that is not a result has a JSON object with an error member, which holds the HTTP code, the message, a list of errors with a reason, and a status name (recorded: bigquery-181 and bigquery-182). An error of a query comes before any row, because the service runs the whole statement before it answers.
type Option ¶
Option sets an option of one statement (D109). An option comes from the DSN, then from the context through WithOptions, then from an argument of the statement, and a later one wins.
func WithDatabase ¶
WithDatabase sets the default dataset of one statement, as the path of the DSN does. BigQuery calls the database a dataset.
func WithLocation ¶
WithLocation sets the location of the job of one statement, as the key location of the DSN does.
func WithMaxResults ¶
WithMaxResults sets the most rows in a page of the result of one statement, as the key max_results of the DSN does. Zero leaves the size to the service.
func WithParameter ¶
WithParameter sets a member of the request of one statement by its name, such as "labels", "maximumBytesBilled", "dryRun", "useQueryCache", "requestId" or "createSession". The value is written as JSON. It replaces the member of the same name that the driver writes, as dbimp.MarshalParams does (D109).
A member that changes how the driver binds or reads a value is refused with dbimp.ErrNotSupported: query, queryParameters, parameterMode, useLegacySql, formatOptions and queryResultsFormat. The members defaultDataset, location, maxResults and jobTimeoutMs have an option of their own, and WithParameter replaces them too. A value of nil fails the statement with dbimp.ErrInvalidValue.
func WithReadonly ¶
WithReadonly asks that the statement write nothing. BigQuery has no such setting for one statement, so WithReadonly(true) fails the statement with dbimp.ErrNotSupported (D109).
func WithTimeout ¶
WithTimeout sets the time that the service gives the job, as jobTimeoutMs, as the key timeout of the DSN does. The service tries to stop a job that runs longer, and it can fail to. The driver rounds d up to the next millisecond. Zero sends none, and the service uses its own limit.
Source Files