dbimp

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 27, 2026 License: MIT Imports: 24 Imported by: 0

README

dbimp

dbimp holds Go database/sql drivers for databases that have no idiomatic Go driver, and for groups of databases that can share one implementation. usql and dbtpl use them. The first drivers are for databases that take queries over HTTP. The drivers use the Go standard library, with apd for decimals, and need no cgo.

The repository is new. It holds two drivers: couchbase, which was first released in v0.1.0, and surrealdb, which was first released in v0.2.0. A driver for Neo4j is in progress. docs/TARGETS.md names the databases it aims to support.

Use

Each driver is its own package, and there is no package that imports every driver. Import the driver for your database. Open it with the name of the database, and with a URL whose scheme is that name:

import (
	"database/sql"

	_ "github.com/xo/dbimp/couchbase"
)

db, err := sql.Open("couchbase", "couchbase://user:pass@localhost:8093/")

A driver registers one name and knows no alias. dburl turns an alias, such as n1ql, into the URL that the driver reads, so every alias works in usql.

Documents

Document Holds
CONTRIBUTING.md How to change this repository
AGENTS.md The rules, written for a coding agent. They apply to a person too
CLAUDE.md One line that imports AGENTS.md for Claude Code
docs/PLAN.md The purpose of the project, and the open questions
docs/decisions/README.md Every decision, one file each, and their index
docs/TARGETS.md Every target database, its priority, and the review of the list
docs/DRIVER.md Every step to add a driver, in order
docs/DESIGN.md The design of the code that every driver shares
docs/COUCHBASE.md What is measured about the Couchbase query service
docs/SURREALDB.md What is known about the HTTP interface of SurrealDB
docs/NEO4J.md What is known about the HTTP interface of Neo4j
docs/BACKLOG.md The planned work, in order

Documentation

Overview

Package dbimp holds the code that the dbimp database/sql drivers share, such as the HTTP client and the adapters that encode and decode values.

Each driver is its own package under this module, such as github.com/xo/dbimp/<driver>. A driver registers itself from init under the name that github.com/xo/dburl gives its scheme, and a consumer imports the driver package directly. There is no registry of drivers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Any 

func Any(v jsontext.Value) (any, error)

Any returns v as a Go value: nil, bool, string, a number as Number returns it, []any, or map[string]any. It is for a whole document, by rule 3 of D18. A map has no order, so a driver never returns the columns of a row through Any.

func Assign 

func Assign(scanCtx driver.ScanContext, dest, src any) error

Assign stores src in dest, for the ScanColumn method of a driver that implements driver.RowsColumnScanner. It stores src itself in a *any, and an *apd.Decimal in an *apd.Decimal. It hands every other pair to sql.ConvertAssign, and it hands a decimal to it as text, which is the form that a *string, a *float64 and the Scan method of apd.Decimal take.

func Bool 

func Bool(v jsontext.Value) (bool, error)

Bool returns the JSON literal v as a bool.

func CheckStatus 

func CheckStatus(res *http.Response) error

CheckStatus returns nil if the status of res is 2xx. Otherwise it reads at most 64 KiB of the body, closes the body, and returns a *StatusError.

func Decimal 

func Decimal(v jsontext.Value) (*apd.Decimal, error)

Decimal returns v as a new *apd.Decimal (D33). It takes a JSON number, and a JSON string that holds a number, because some servers send a decimal as a string to keep its digits.

func Float64 

func Float64(v jsontext.Value) (float64, error)

Float64 returns the JSON number v as a float64.

func Int64 

func Int64(v jsontext.Value) (int64, error)

Int64 returns the JSON number v as an int64. It returns an error for a number with a fraction or an exponent, and for a number out of range.

func IsNull 

func IsNull(v jsontext.Value) bool

IsNull reports whether v is a missing value or a JSON null.

func NewClient 

func NewClient(rt http.RoundTripper, redirects bool) *http.Client

NewClient returns a client for rt with no timeout, because the context of each request sets its deadline. If redirects is false, the client returns a redirect as the response and does not follow it. A driver decides in step 9 of docs/DRIVER.md whether it follows a redirect. When a client follows one to another host, net/http drops the Authorization header.

func NewTransport 

func NewTransport(cfg *tls.Config) *http.Transport

NewTransport returns the transport for one connector. It takes a proxy from the environment, and it bounds the dial and the TLS handshake. It sets no timeout for the headers of a response, because a server can send them only after a long query ends. The context of each request sets its deadline. The transport asks for gzip and decompresses it, because no driver sets Accept-Encoding itself.

func Number 

func Number(v jsontext.Value) (any, error)

Number returns the JSON number v as the Go value that holds it exactly, for a server that sends no type with a number. An integer that fits is an int64. A larger integer is an *apd.Decimal. A number with a fraction or an exponent is a float64, because a JSON number that is not an integer is a double in every product that sends no type.

func ParseURL 

func ParseURL(name, dsn string) (*url.URL, error)

ParseURL parses dsn with net/url, and returns an error if its scheme is not name (D27 and D35). A driver registers one name, so it takes one scheme. An error never holds the DSN, because the DSN can hold a password.

func Send 

func Send(c *http.Client, req *http.Request) (*http.Response, error)

Send sends req with c. It sends each request once, and never again after it can have reached the server (D8).

If c made no connection for the request, such as when the dial or the TLS handshake failed, the error wraps driver.ErrBadConn, so that database/sql can try a new connection. If the context of req ended, the error wraps the error of the context, and never driver.ErrBadConn. The error never holds a password, because net/http removes it from the URL of the error.

func String 

func String(v jsontext.Value) (string, error)

String returns the JSON string v, unquoted.

Types

type ArrayRows 

type ArrayRows struct {
	// contains filtered or unexported fields
}

ArrayRows reads a JSON array of arrays as rows, by rule 1 of D18. The server names the columns elsewhere in the response, and each row is an array with one value for each column.

func NewArrayRows 

func NewArrayRows(dec *jsontext.Decoder, n int) (*ArrayRows, error)

NewArrayRows reads the start of an array of rows of n columns from dec.

func (*ArrayRows) Next 

func (r *ArrayRows) Next(vals []jsontext.Value) error

Next reads the next row into vals, which has one entry for each column. It returns io.EOF after the last row, and reads the end of the array.

type CBORDecoder added in v0.2.0 

type CBORDecoder struct {
	// contains filtered or unexported fields
}

CBORDecoder reads CBOR items from a reader, one at a time.

func NewCBORDecoder added in v0.2.0 

func NewCBORDecoder(r io.Reader) *CBORDecoder

NewCBORDecoder returns a decoder that reads from r.

func (*CBORDecoder) More added in v0.2.0 

func (d *CBORDecoder) More(h CBORHead, n uint64) (bool, error)

More reports whether the container whose head is h has another item, when n items of it were read. For an item of indefinite length, it reads the break that ends it.

func (*CBORDecoder) PeekHead added in v0.2.0 

func (d *CBORDecoder) PeekHead() (CBORHead, error)

PeekHead returns the head of the next item, and reads nothing.

func (*CBORDecoder) ReadHead added in v0.2.0 

func (d *CBORDecoder) ReadHead() (CBORHead, error)

ReadHead reads the head of the next item. For a string, an array, a map or a tag, the content follows, and the caller reads it.

func (*CBORDecoder) ReadRaw added in v0.2.0 

func (d *CBORDecoder) ReadRaw() ([]byte, error)

ReadRaw reads the next item whole, and returns its bytes, which the caller owns.

func (*CBORDecoder) ReadString added in v0.2.0 

func (d *CBORDecoder) ReadString(h CBORHead) ([]byte, error)

ReadString reads the content of the byte string or the text string whose head is h, including each chunk of one of indefinite length.

func (*CBORDecoder) ReadText added in v0.2.0 

func (d *CBORDecoder) ReadText() (string, error)

ReadText reads the next item, which must be a text string.

func (*CBORDecoder) Skip added in v0.2.0 

func (d *CBORDecoder) Skip() error

Skip reads the next item whole, and keeps none of it.

type CBOREncoder added in v0.2.0 

type CBOREncoder struct {
	// contains filtered or unexported fields
}

CBOREncoder appends CBOR items to a buffer.

func (*CBOREncoder) Array added in v0.2.0 

func (e *CBOREncoder) Array(n int)

Array appends the head of an array of n items, which the caller appends.

func (*CBOREncoder) Bool added in v0.2.0 

func (e *CBOREncoder) Bool(b bool)

Bool appends true or false.

func (*CBOREncoder) ByteString added in v0.2.0 

func (e *CBOREncoder) ByteString(b []byte)

ByteString appends a byte string.

func (*CBOREncoder) Bytes added in v0.2.0 

func (e *CBOREncoder) Bytes() []byte

Bytes returns what the encoder holds.

func (*CBOREncoder) Float added in v0.2.0 

func (e *CBOREncoder) Float(f float64)

Float appends a float64.

func (*CBOREncoder) Head added in v0.2.0 

func (e *CBOREncoder) Head(m CBORMajor, arg uint64)

Head appends the head of an item of major type m with the argument arg, in its shortest form.

func (*CBOREncoder) Int added in v0.2.0 

func (e *CBOREncoder) Int(i int64)

Int appends an integer.

func (*CBOREncoder) Map added in v0.2.0 

func (e *CBOREncoder) Map(n int)

Map appends the head of a map of n pairs, which the caller appends.

func (*CBOREncoder) Null added in v0.2.0 

func (e *CBOREncoder) Null()

Null appends null.

func (*CBOREncoder) Raw added in v0.2.0 

func (e *CBOREncoder) Raw(b []byte)

Raw appends an item that is already encoded.

func (*CBOREncoder) Tag added in v0.2.0 

func (e *CBOREncoder) Tag(num uint64)

Tag appends the tag num, whose item the caller appends.

func (*CBOREncoder) Text added in v0.2.0 

func (e *CBOREncoder) Text(s string)

Text appends a text string.

func (*CBOREncoder) Uint added in v0.2.0 

func (e *CBOREncoder) Uint(u uint64)

Uint appends an unsigned integer.

type CBORHead added in v0.2.0 

type CBORHead struct {
	// Major is the major type.
	Major CBORMajor
	// Info is the additional information of the first byte, from 0 to 31.
	Info byte
	// Arg is the argument. It is the value of an integer, the length of a
	// string, an array or a map, the number of a tag, the bits of a float,
	// or the number of a simple value. It is 0 for an item of indefinite
	// length.
	Arg uint64
}

CBORHead is the head of one CBOR item: its major type and its argument.

func (CBORHead) Bool added in v0.2.0 

func (h CBORHead) Bool() (bool, bool)

Bool reports whether the item is true or false, and returns it.

func (CBORHead) Break added in v0.2.0 

func (h CBORHead) Break() bool

Break reports whether the head is the break that ends an item of indefinite length.

func (CBORHead) Float added in v0.2.0 

func (h CBORHead) Float() (float64, bool)

Float reports whether the item is a float, and returns it.

func (CBORHead) Indefinite added in v0.2.0 

func (h CBORHead) Indefinite() bool

Indefinite reports whether the item has an indefinite length, which a break ends.

func (CBORHead) Null added in v0.2.0 

func (h CBORHead) Null() bool

Null reports whether the item is null or undefined.

type CBORMajor added in v0.2.0 

type CBORMajor byte

CBORMajor is the major type of a CBOR item. 

const (
	CBORUint   CBORMajor = 0
	CBORNegInt CBORMajor = 1
	CBORBytes  CBORMajor = 2
	CBORText   CBORMajor = 3
	CBORArray  CBORMajor = 4
	CBORMap    CBORMajor = 5
	CBORTag    CBORMajor = 6
	// CBORSimple holds false, true, null, undefined, the floats and the
	// break that ends an item of indefinite length.
	CBORSimple CBORMajor = 7
)

The major types of CBOR, from section 3.1 of RFC 8949.

type Error 

type Error string

Error is an error of this package. 

const (
	// ErrNotSupported is the error for a feature that the database does not
	// have, such as a transaction (D20).
	ErrNotSupported Error = "not supported"
	// ErrScheme is the error for a DSN whose scheme is not the name of the
	// driver (D35).
	ErrScheme Error = "wrong scheme"
	// ErrUnknownKey is the error for a key in the query of a DSN that the
	// driver does not know (D27).
	ErrUnknownKey Error = "unknown key"
	// ErrRepeatedKey is the error for a key that appears more than once in
	// the query of a DSN (D27).
	ErrRepeatedKey Error = "repeated key"
	// ErrInvalidValue is the error for a value that has the wrong form.
	ErrInvalidValue Error = "invalid value"
	// ErrExtraColumn is the error for a row that has a column that the first
	// row does not have (D18).
	ErrExtraColumn Error = "column not in the first row"
	// ErrColumnCount is the error for a row that has the wrong number of
	// columns.
	ErrColumnCount Error = "wrong number of columns"
	// ErrIncomplete is the error for a result that the server cut short
	// (D21).
	ErrIncomplete Error = "result is incomplete"
	// ErrUnterminated is the error for a statement that ends inside a
	// literal, a quoted identifier or a comment.
	ErrUnterminated Error = "unterminated literal or comment"
	// ErrArguments is the error for arguments that do not match the
	// placeholders of a statement (D34).
	ErrArguments Error = "arguments do not match the placeholders"
)

Error values.

func (Error) Error 

func (err Error) Error() string

Error satisfies the error interface.

type ObjectRows 

type ObjectRows struct {
	// contains filtered or unexported fields
}

ObjectRows reads a JSON array of objects as rows, by rule 2 of D18.

The columns are the keys of the first object, in the order that they arrive, or the columns that the server named. A key that a later object lacks is a nil value. A key that only a later object has is an error, and never a new column. A JSON null is the value "null", so a driver can tell a null from a missing key if its product has both.

func ContinueObjectRows 

func ContinueObjectRows(dec *jsontext.Decoder, cols []string) (*ObjectRows, error)

ContinueObjectRows is NewObjectRows for a caller that has read the start of the array, such as to peek at the kind of the first row.

func NewObjectRows 

func NewObjectRows(dec *jsontext.Decoder, cols []string) (*ObjectRows, error)

NewObjectRows reads the start of an array of objects from dec. If cols is nil, it reads the first object to learn the columns, and Next returns that object first.

func (*ObjectRows) Columns 

func (r *ObjectRows) Columns() []string

Columns returns the columns of the rows.

func (*ObjectRows) Next 

func (r *ObjectRows) Next(vals []jsontext.Value) error

Next reads the next row into vals, which has one entry for each column. It returns io.EOF after the last row, and reads the end of the array.

type Placeholder 

type Placeholder struct {
	// Offset is the byte offset of the placeholder in the statement.
	Offset int
	// Len is the length of the placeholder in bytes.
	Len int
	// Name is the name of an @name placeholder without the @, and "" for a ?.
	Name string
}

Placeholder is one placeholder in a statement.

type Query 

type Query struct {
	// contains filtered or unexported fields
}

Query holds the keys of the query of a DSN. NewQuery refuses a key that is unknown or repeated, so a value that Query returns is the only value of its key.

func NewQuery 

func NewQuery(u *url.URL, known ...string) (Query, error)

NewQuery reads the query of u. It returns an error for a key that is not in known, and for a key that appears more than once (D27).

func (Query) Bool 

func (q Query) Bool(key string, def bool) (bool, error)

Bool returns the value of key as a bool, or def if the query does not hold key. It takes the forms that strconv.ParseBool takes.

func (Query) Duration 

func (q Query) Duration(key string, def time.Duration) (time.Duration, error)

Duration returns the value of key as a time.Duration, or def if the query does not hold key. It takes the forms that time.ParseDuration takes.

func (Query) Has 

func (q Query) Has(key string) bool

Has reports whether the query holds key.

func (Query) Int 

func (q Query) Int(key string, def int) (int, error)

Int returns the value of key as an int, or def if the query does not hold key.

func (Query) String 

func (q Query) String(key, def string) string

String returns the value of key, or def if the query does not hold key.

type StatusError 

type StatusError struct {
	// Code is the status code of the response.
	Code int
	// Body is the start of the body of the response, at most 64 KiB.
	Body string
}

StatusError is the error for a response whose status is not 2xx. HTTP 429 and HTTP 503 are a StatusError too, and nothing retries them (D8).

func (*StatusError) Error 

func (err *StatusError) Error() string

Error satisfies the error interface.

type Stream 

type Stream struct {
	// contains filtered or unexported fields
}

Stream reads the body of a response one token at a time (D36). It holds the body and a jsontext.Decoder that reads from it, and no other buffer, because the decoder keeps its own.

A driver reads the rows through Decoder, and calls End after the last value of the response. Close before End closes the body and reads nothing more, so a large result is never drained. Close after End closes a body that is at EOF, so the connection goes back to the pool.

func NewStream 

func NewStream(body io.ReadCloser) *Stream

NewStream returns a Stream that reads body.

func (*Stream) Close 

func (s *Stream) Close() error

Close closes the body. It reads nothing more from it. It can run more than once, and returns the same error each time.

func (*Stream) Decoder 

func (s *Stream) Decoder() *jsontext.Decoder

Decoder returns the decoder that reads the body.

func (*Stream) End 

func (s *Stream) End() error

End reads to the end of the body after the last value. It returns an error if anything but white space follows that value.

type Syntax 

type Syntax struct {
	// Quotes holds each character that opens and closes a literal or a quoted
	// identifier, such as `'"` or "'\"`". Inside one, the character written
	// twice stands for itself.
	Quotes string
	// Backslash is true if a backslash inside a quote escapes the character
	// after it.
	Backslash bool
	// DashComments is true if -- starts a comment that ends at a new line.
	DashComments bool
	// HashComments is true if # starts a comment that ends at a new line.
	HashComments bool
	// BlockComments is true if /* starts a comment that ends at */.
	BlockComments bool
}

Syntax says how a product writes literals, quoted identifiers and comments, so that the parser for placeholders can skip them (D34). The zero value knows no quote and no comment.

func (Syntax) Bind 

func (s Syntax) Bind(query string, args []driver.NamedValue, lit func(v any) (string, error)) (string, error)

Bind writes each argument into query as a literal, for a server that binds no arguments (D34). A ? takes the next argument that has no name, in order, and an @name takes the argument with that name. lit writes one value as a literal of the product, and each driver supplies its own, because each product quotes in its own way. Bind returns an error if an argument is missing, or if an argument is left over.

func (Syntax) Placeholders 

func (s Syntax) Placeholders(query string) ([]Placeholder, error)

Placeholders returns each ? and each @name in query, in order. It skips literals, quoted identifiers and comments. An @ that another @ follows, as in @@version, is not a placeholder. It returns an error if query ends inside a literal, a quoted identifier or a comment.

Source Files

View all Source files

Directories

Path Synopsis
Package couchbase is a database/sql driver for the query service of Couchbase Server, which runs SQL++ (N1QL) over HTTP.
 Package couchbase is a database/sql driver for the query service of Couchbase Server, which runs SQL++ (N1QL) over HTTP.
Package dbimptest holds the test helpers that every driver in github.com/xo/dbimp shares.
 Package dbimptest holds the test helpers that every driver in github.com/xo/dbimp shares.
cmd/record command
Command record sends the requests of step 6 of docs/DRIVER.md to a real server, as the administrator and as the ordinary user, and writes each exchange and the manifest into testdata/<driver>/.
 Command record sends the requests of step 6 of docs/DRIVER.md to a real server, as the administrator and as the ordinary user, and writes each exchange and the manifest into testdata/<driver>/.
Package neo4j is a database/sql driver for Neo4j, which runs Cypher over the Query API of HTTP.
 Package neo4j is a database/sql driver for Neo4j, which runs Cypher over the Query API of HTTP.
Package surrealdb is a database/sql driver for SurrealDB, which runs SurrealQL over HTTP.
 Package surrealdb is a database/sql driver for SurrealDB, which runs SurrealQL over HTTP.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.