Documentation
¶
Overview ¶
Package databricks is a database/sql driver for Databricks, which runs SQL over its SQL Statement Execution API, /api/2.0/sql/statements, with HTTPS and JSON. It registers the one name "databricks", and takes a DSN of the form databricks://token:<personal access token>@<workspace host>/<warehouse id> (D193).
db, err := sql.Open("databricks", "databricks://token:"+pat+"@dbc-1234.cloud.databricks.com/1111222233334444?catalog=main&schema=default")
The password of the DSN is the personal access token of a user or of a service principal. The driver sends it as a Bearer token, and only to the host of the DSN. It reads the result in the format JSON_ARRAY with the disposition INLINE, one token at a time. It sends each statement with a wait of 50 seconds, and it polls a statement that is still running. The server has no session across two requests, so the driver has no transactions, and a SET, a USE, a temporary view or a variable does not reach the next statement. docs/DATABRICKS.md holds what the driver knows about the server.
Index ¶
Constants ¶
const ( // ErrCanceled is the error of a statement that the server ended in the // state CANCELED, when the context of the statement did not end. Someone // else canceled it, such as an administrator (measured). ErrCanceled dbimp.Error = "the statement was canceled" // ErrClosed is the error of a statement whose state is CLOSED. The // documentation names the state, and no recording shows it (D193). ErrClosed dbimp.Error = "the result of the statement is closed" // ErrTruncated is the error of a result that the server cut at a limit, // which the manifest names with truncated of true (D193 item 2). The // driver sends no limit, so the server cuts a result only when the // result is larger than the limit of the disposition INLINE. ErrTruncated dbimp.Error = "the server cut the result" // ErrCut is the error of a result that ends before the count of rows that // the server named for it. After a row, it comes wrapped with // dbimp.ErrIncomplete (D21 and D107). ErrCut dbimp.Error = "the answer was cut short" )
The sentinel errors of the driver. Each one wraps into a failure with errors.Is (D6).
const Name = "databricks"
Name is the name that the driver registers with database/sql, and the scheme of its DSN.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
// Host is the host of the workspace, such as
// dbc-1234.cloud.databricks.com.
Host string
// Port is the port of the server, 443 by default.
Port int
// Token is the personal access token, which the driver sends as a Bearer
// token (D193 item 4 and D94).
Token string
// Warehouse is the id of the SQL warehouse, which the path of the DSN
// holds. It is 16 hexadecimal digits for a warehouse of Databricks.
Warehouse string
// Catalog and Schema are the catalog and the schema of each statement,
// which the driver sends in the body. Each one is optional.
Catalog string
Schema string
// Timeout is the longest time that the driver waits for one statement,
// from the request until the end of its result. Zero waits as long as the
// context of the caller allows.
Timeout time.Duration
// Insecure makes the driver use HTTP and not HTTPS. A test of the driver
// sets it, for a fake server on the local host (D193 item 11). The DSN
// writes it as tls=false.
Insecure bool
}
Config is the configuration of a connector, which the DSN holds. The caller owns it (D7).
func ParseDSN ¶
ParseDSN parses a DSN of the form databricks://token:<personal access token>@<host>/<warehouse id>?catalog=c&schema=s (D27, D35 and D193). The password is the token. The path is the id of the SQL warehouse, and the catalog and the schema are keys of the query. The keys are catalog, schema, timeout and tls. The timeout is a duration with a unit, such as 5m, and a bare number is refused. The key tls is true by default, and tls=false makes the driver use HTTP, for a fake server of a test.
type Connector ¶
type Connector struct {
// contains filtered or unexported fields
}
Connector opens connections to one workspace. It owns its transport, and every connection shares it. A caller can build one from a Config and open it with sql.OpenDB.
func NewConnector ¶
NewConnector returns a Connector for cfg. The connector keeps a copy of cfg, and fills each field that is zero with its default.
type Driver ¶
type Driver struct{}
Driver is the database/sql driver for Databricks.
type Error ¶
type Error struct {
// HTTPStatus is the status code of the response.
HTTPStatus int
// Code is error_code, such as BAD_REQUEST or PERMISSION_DENIED. A wrong
// token answers with a JSON number, 403, which Code holds as text.
Code string
// SQLState is sql_state of a statement that failed, such as 42601, and
// empty when the answer has none.
SQLState string
// Message is the message of the error, with no white space at its ends.
Message string
// StatementID is statement_id, and empty when the answer has none.
StatementID string
// contains filtered or unexported fields
}
Error is an error that Databricks reported. A statement that fails in the engine is HTTP 200 with the state FAILED, and the error is in the member status of the answer (measured). A request that the server refuses before it runs a statement is an HTTP status of 400, 403 or 404 with a JSON object of the code and the message (measured).
type Option ¶
Option sets an option of one statement (D109). An option comes from the DSN, then from the context through WithOptions, then from an argument of the statement, and a later one wins.
func WithCatalog ¶
WithCatalog sets the catalog of one statement, as the key catalog of the DSN does.
func WithDatabase ¶
WithDatabase sets the schema of one statement, because Spark calls a schema a database: current_database() answers the schema (measured). It is the same as WithSchema, and as the key schema of the DSN.
func WithParameter ¶
WithParameter sets a member of the body of the request by its name, such as "query_tags". It replaces the member of the driver with the same name, as dbimp.MarshalParams does (D109). The driver refuses a member that changes how it sends the statement or how it reads the result: warehouse_id, statement, parameters, wait_timeout, on_wait_timeout, row_limit, byte_limit, disposition and format. A value of nil fails the statement with dbimp.ErrInvalidValue.
func WithReadonly ¶
WithReadonly asks that the statement write nothing. The Statement Execution API has no such setting for one statement, so WithReadonly(true) fails the statement with dbimp.ErrNotSupported (D109).
func WithSchema ¶
WithSchema sets the schema of one statement, as the key schema of the DSN does.
func WithTimeout ¶
WithTimeout sets the longest time that the driver waits for the statement, as the key timeout of the DSN does. The time runs from the request to the end of the result. When it ends, the driver cancels the statement on the server if it knows its id, and the error is context.DeadlineExceeded. Zero waits as long as the context allows.
Source Files