README
¶
Go SQL driver for DuckDB
The DuckDB driver conforms to the built-in database/sql interface.
Installation
go get github.com/marcboeker/go-duckdb
go-duckdb uses CGO to make calls to DuckDB. You must build your binaries with CGO_ENABLED=1.
Usage
go-duckdb hooks into the database/sql interface provided by the Go stdlib. To open a connection, simply specify the driver type as duckdb.
db, err := sql.Open("duckdb", "")
if err != nil {
...
}
defer db.Close()
This creates an in-memory instance of DuckDB. To open a persistent database, you need to specify a filepath to the database file. If the file does not exist, then DuckDB creates it.
db, err := sql.Open("duckdb", "/path/to/foo.db")
if err != nil {
...
}
defer db.Close()
If you want to set specific config options for DuckDB, you can add them as query style parameters in the form of name=value pairs to the DSN.
db, err := sql.Open("duckdb", "/path/to/foo.db?access_mode=read_only&threads=4")
if err != nil {
...
}
defer db.Close()
Alternatively, you can use sql.OpenDB. That way, you can perform initialization steps in a callback function before opening the database.
Here's an example that installs and loads the JSON extension when opening a database with sql.OpenDB(connector).
connector, err := duckdb.NewConnector("/path/to/foo.db?access_mode=read_only&threads=4", func(execer driver.Execer) error {
bootQueries := []string{
"INSTALL 'json'",
"LOAD 'json'",
}
for _, query := range bootQueries {
_, err = execer.Exec(query, nil)
if err != nil {
...
}
}
return nil
})
if err != nil {
...
}
db := sql.OpenDB(connector)
defer db.Close()
Please refer to the database/sql documentation for further usage instructions.
A Note on Memory Allocation
DuckDB lives in-process. Therefore, all its memory lives in the driver. All allocations live in the host process, which
is the Go application. Especially for long-running applications, it is crucial to call the corresponding Close-functions as specified
in database/sql. The following is a list of examples.
db, err := sql.Open("duckdb", "")
defer db.Close()
conn, err := db.Conn(context.Background())
defer conn.Close()
rows, err := conn.QueryContext(context.Background(), "SELECT 42")
// alternatively, rows.Next() has to return false
rows.Close()
appender, err := NewAppenderFromConn(conn, "", "test")
defer appender.Close()
// if not passed to sql.OpenDB
connector, err := NewConnector("", nil)
defer connector.Close()
DuckDB Appender API
If you want to use the DuckDB Appender API, you can obtain a new Appender by passing a DuckDB connection to NewAppenderFromConn().
connector, err := duckdb.NewConnector("test.db", nil)
if err != nil {
...
}
defer connector.Close()
conn, err := connector.Connect(context.Background())
if err != nil {
...
}
defer conn.Close()
// obtain an appender from the connection
// NOTE: the table 'test_tbl' must exist in test.db
appender, err := NewAppenderFromConn(conn, "", "test_tbl")
if err != nil {
...
}
defer appender.Close()
err = appender.AppendRow(...)
if err != nil {
...
}
DuckDB Apache Arrow Interface
If you want to use the DuckDB Arrow Interface, you can obtain a new Arrow by passing a DuckDB connection to NewArrowFromConn().
connector, err := duckdb.NewConnector("", nil)
if err != nil {
...
}
defer connector.Close()
conn, err := connector.Connect(context.Background())
if err != nil {
...
}
defer conn.Close()
// obtain the Arrow from the connection
arrow, err := duckdb.NewArrowFromConn(conn)
if err != nil w
...
}
rdr, err := arrow.QueryContext(context.Background(), "SELECT * FROM generate_series(1, 10)")
if err != nil {
...
}
defer rdr.Release()
for rdr.Next() {
// process records
}
Linking DuckDB
By default, go-duckdb statically links DuckDB into your binary. Statically linking DuckDB adds around 30 MB to your binary size. On Linux (Intel) and macOS (Intel and ARM), go-duckdb bundles pre-compiled static libraries for fast builds.
Alternatively, you can dynamically link DuckDB by passing -tags=duckdb_use_lib to go build. You must have a copy of libduckdb available on your system (.so on Linux or .dylib on macOS), which you can download from the DuckDB releases page. For example:
# On Linux
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/libs" go build -tags=duckdb_use_lib main.go
LD_LIBRARY_PATH=/path/to/libs ./main
# On macOS
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/libs" go build -tags=duckdb_use_lib main.go
DYLD_LIBRARY_PATH=/path/to/libs ./main
Documentation
¶
Overview ¶
Package duckdb implements a database/sql driver for the DuckDB database.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Appender ¶
type Appender struct {
// contains filtered or unexported fields
}
Appender holds the DuckDB appender. It allows efficient bulk loading into a DuckDB database.
func NewAppenderFromConn ¶
NewAppenderFromConn returns a new Appender from a DuckDB driver connection.
func (*Appender) AppendRow ¶
AppendRow loads a row of values into the appender. The values are provided as separate arguments.
func (*Appender) Close ¶
Close the appender. This will flush the appender to the underlying table. It is vital to call this when you are done with the appender to avoid leaking memory.
type Arrow ¶
type Arrow struct {
// contains filtered or unexported fields
}
Arrow exposes DuckDB Apache Arrow interface. https://duckdb.org/docs/api/c/api#arrow-interface
func NewArrowFromConn ¶
NewArrowFromConn returns a new Arrow from a DuckDB driver connection.
func (*Arrow) QueryContext ¶
func (a *Arrow) QueryContext(ctx context.Context, query string, args ...any) (array.RecordReader, error)
QueryContext prepares statements, executes them, returns Apache Arrow array.RecordReader as a result of the last executed statement. Arguments are bound to the last statement.
type Composite ¶
type Composite[T any] struct { // contains filtered or unexported fields }
Use as the `Scanner` type for any composite types (maps, lists, structs)
type Connector ¶
type Connector struct {
// contains filtered or unexported fields
}
func NewConnector ¶
func NewConnector(dsn string, connInitFn func(execer driver.ExecerContext) error) (*Connector, error)
NewConnector opens a new Connector for a DuckDB database. The user must close the returned Connector, if it is not passed to the sql.OpenDB function. Otherwise, sql.DB closes the Connector when calling sql.DB.Close().
type SetColValue ¶
SetColValue is the type definition for all column callback functions.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
deps
|
|
|
darwin_amd64
Package darwin_amd64 is required to provide support for vendoring modules DO NOT REMOVE
|
Package darwin_amd64 is required to provide support for vendoring modules DO NOT REMOVE |
|
darwin_arm64
Package darwin_arm64 is required to provide support for vendoring modules DO NOT REMOVE
|
Package darwin_arm64 is required to provide support for vendoring modules DO NOT REMOVE |
|
freebsd_amd64
Package freebsd_amd64 is required to provide support for vendoring modules DO NOT REMOVE
|
Package freebsd_amd64 is required to provide support for vendoring modules DO NOT REMOVE |
|
linux_amd64
Package linux_amd64 is required to provide support for vendoring modules DO NOT REMOVE
|
Package linux_amd64 is required to provide support for vendoring modules DO NOT REMOVE |
|
linux_arm64
Package linux_arm64 is required to provide support for vendoring modules DO NOT REMOVE
|
Package linux_arm64 is required to provide support for vendoring modules DO NOT REMOVE |