duckdb

README

Go SQL driver for DuckDB

The DuckDB driver conforms to the built-in database/sql interface.

Notice on v1.4.0

Version 1.4.0 changed the DuckDB decimal representation from float64 to a new Decimal type, which is much more precise. If you are upgrading to v1.4.0 and are using DuckDBs decimals, please make sure to update your code to make use of the new Decimal type.

Installation

go get github.com/marcboeker/go-duckdb

go-duckdb uses CGO to make calls to DuckDB. You must build your binaries with CGO_ENABLED=1.

Usage

go-duckdb hooks into the database/sql interface provided by the Go stdlib. To open a connection, simply specify the driver type as duckdb:

db, err := sql.Open("duckdb", "")

This creates an in-memory instance of DuckDB. If you would like to store the data on the filesystem, you need to specify the path where to store the database:

db, err := sql.Open("duckdb", "/path/to/foo.db")

If you want to set specific config options for DuckDB, you can add them as query style parameters in the form of name=value to the DSN, like:

db, err := sql.Open("duckdb", "/path/to/foo.db?access_mode=read_only&threads=4")

Alternatively, you can also use sql.OpenDB when you want to perform some initialization before the connection is created and returned from the connection pool on call to db.Conn. Here's an example that installs and loads the JSON extension for each connection:

connector, err := duckdb.NewConnector("/path/to/foo.db?access_mode=read_only&threads=4", func(execer driver.Execer) error {
  bootQueries := []string{
    "INSTALL 'json'",
    "LOAD 'json'",
  }

  for _, qry := range bootQueries {
    _, err = execer.Exec(qry, nil)
    if err != nil {
      return err
    }
  }
  return nil
})
if err != nil {
  return nil, err
}

db := sql.OpenDB(connector)
db.SetMaxOpenConns(poolsize)
...

Please refer to the database/sql GoDoc for further usage instructions.

DuckDB Appender API

If you want to use the DuckDB Appender API, you can obtain a new Appender by supplying a DuckDB connection to NewAppenderFromConn().

connector, err := NewConnector("test.db", nil)
if err != {
  ...
}
conn, err := connector.Connect(context.Background())
if err != {
  ...
}
defer conn.Close()

// Retrieve appender from connection (note that you have to create the table 'test' beforehand).
appender, err := NewAppenderFromConn(conn, "", "test")
if err != {
  ...
}
defer appender.Close()

err = appender.AppendRow(...)
if err != {
  ...
}

// Optional, if you want to access the appended rows immediately.
err = appender.Flush()
if err != {
  ...
}

Linking DuckDB

By default, go-duckdb statically links DuckDB into your binary. Statically linking DuckDB adds around 30 MB to your binary size. On Linux (Intel) and macOS (Intel and ARM), go-duckdb bundles pre-compiled static libraries for fast builds.

Alternatively, you can dynamically link DuckDB by passing -tags=duckdb_use_lib to go build. You must have a copy of libduckdb available on your system (.so on Linux or .dylib on macOS), which you can download from the DuckDB releases page. For example:

# On Linux
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/libs" go build -tags=duckdb_use_lib main.go
LD_LIBRARY_PATH=/path/to/libs ./main

# On macOS
CGO_ENABLED=1 CGO_LDFLAGS="-L/path/to/libs" go build -tags=duckdb_use_lib main.go
DYLD_LIBRARY_PATH=/path/to/libs ./main

Documentation

Overview

Package duckdb implements a database/sql driver for the DuckDB database.

Index

• func NewConnector(dsn string, connInitFn func(execer driver.ExecerContext) error) (driver.Connector, error)
• type Appender
  • func NewAppenderFromConn(driverConn driver.Conn, schema string, table string) (*Appender, error)
  • func (a *Appender) AppendRow(args ...driver.Value) error
  • func (a *Appender) AppendRowArray(args []driver.Value) error
  • func (a *Appender) Close() error
  • func (a *Appender) Error() error
  • func (a *Appender) Flush() error
• type Composite
  • func (s Composite[T]) Get() T
  • func (s *Composite[T]) Scan(v any) error
• type Conn
  • func (c *Conn) Begin() (driver.Tx, error)
  • func (c *Conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error)
  • func (c *Conn) CheckNamedValue(nv *driver.NamedValue) error
  • func (c *Conn) Close() error
  • func (c *Conn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error)
  • func (c *Conn) Prepare(cmd string) (driver.Stmt, error)
  • func (c *Conn) QueryArrow(ctx context.Context, query string, args ...any) (array.RecordReader, error)
  • func (c *Conn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error)
• type Decimal
  • func (d *Decimal) Float64() float64
• type Driver
  • func (d Driver) Open(dataSourceName string) (driver.Conn, error)
  • func (Driver) OpenConnector(dataSourceName string) (driver.Connector, error)
• type Interval
• type Map
  • func (m *Map) Scan(v any) error

Functions

func NewConnector

func NewConnector(dsn string, connInitFn func(execer driver.ExecerContext) error) (driver.Connector, error)

NewConnector creates a new Connector for the DuckDB database.

Types

type Appender

type Appender struct {
	// contains filtered or unexported fields
}

Appender holds the duckdb appender. It allows to load bulk data into a DuckDB database.

func NewAppenderFromConn

func NewAppenderFromConn(driverConn driver.Conn, schema string, table string) (*Appender, error)

NewAppenderFromConn returns a new Appender from a DuckDB driver connection.

func (*Appender) AppendRow

func (a *Appender) AppendRow(args ...driver.Value) error

AppendRow loads a row of values into the appender. The values are provided as separate arguments.

func (*Appender) AppendRowArray

func (a *Appender) AppendRowArray(args []driver.Value) error

AppendRowArray loads a row of values into the appender. The values are provided as an array.

func (*Appender) Close

func (a *Appender) Close() error

Close closes the appender.

func (*Appender) Error

func (a *Appender) Error() error

Error returns the last duckdb appender error.

func (*Appender) Flush

func (a *Appender) Flush() error

Flush the appender to the underlying table and clear the internal cache.

type Composite

type Composite[T any] struct {
	// contains filtered or unexported fields
}

Use as the `Scanner` type for any composite types (maps, lists, structs)

func (Composite[T]) Get

func (s Composite[T]) Get() T

func (*Composite[T]) Scan

func (s *Composite[T]) Scan(v any) error

type Conn

type Conn struct {
	// contains filtered or unexported fields
}

func (*Conn) Begin

func (c *Conn) Begin() (driver.Tx, error)

Deprecated: Use BeginTx instead.

func (*Conn) BeginTx

func (c *Conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error)

func (*Conn) CheckNamedValue

func (c *Conn) CheckNamedValue(nv *driver.NamedValue) error

func (*Conn) Close

func (c *Conn) Close() error

func (*Conn) ExecContext

func (c *Conn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error)

func (*Conn) Prepare

func (c *Conn) Prepare(cmd string) (driver.Stmt, error)

func (*Conn) QueryArrow

func (c *Conn) QueryArrow(ctx context.Context, query string, args ...any) (array.RecordReader, error)

QueryArrow prepares statements, executes them, returns Apache Arrow array.RecordReader as a result of the last executed statement. Arguments are bound to the last statement. https://duckdb.org/docs/api/c/api#arrow-interface NOTE: Experimental interface.

func (*Conn) QueryContext

func (c *Conn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error)

type Decimal

type Decimal struct {
	Width uint8
	Scale uint8
	Value *big.Int
}

func (*Decimal) Float64

func (d *Decimal) Float64() float64

type Driver

type Driver struct{}

func (Driver) Open

func (d Driver) Open(dataSourceName string) (driver.Conn, error)

func (Driver) OpenConnector

func (Driver) OpenConnector(dataSourceName string) (driver.Connector, error)

type Interval

type Interval struct {
	Days   int32 `json:"days"`
	Months int32 `json:"months"`
	Micros int64 `json:"micros"`
}

type Map

type Map map[any]any

func (*Map) Scan

func (m *Map) Scan(v any) error