duckdb

package module
v2.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 17, 2025 License: MIT Imports: 23 Imported by: 67

README

Go SQL Driver For DuckDB

Tests status GoDoc

The DuckDB driver conforms to the built-in database/sql interface.

Current DuckDB version: v1.4.0.

The first go-duckdb tag with that version is v2.4.0.

Previous DuckDB versions:

DuckDB go-duckdb
v1.4.0 v2.4.0
v1.3.2 v2.3.3
v1.3.1 v2.3.2
v1.3.0 v2.3.0
v1.2.2 v2.2.0
v1.2.1 v2.1.0
v1.2.0 v2.0.3
v1.1.3 v1.8.5
Breaking Changes

[!WARNING] Starting with v2.0.0, go-duckdb supports DuckDB v1.2.0 and upward. Moving to v2 includes the following list of breaking changes.

Dropping pre-built FreeBSD support

Starting with v2, go-duckdb drops pre-built FreeBSD support. This change is because DuckDB does not publish any bundled FreeBSD libraries. Thus, you must build your static library for FreeBSD using the steps below.

The Arrow dependency is now opt-in

The DuckDB Arrow Interface is a heavy dependency. Starting with v2, the DuckDB Arrow Interface is opt-in instead of opt-out. If you want to use it, you can enable it by passing -tags=duckdb_arrow to go build.

JSON type scanning changes

The pre-built libraries ship DuckDB's JSON extension containing the JSON type. Pre-v2, it was possible to scan a JSON type into []byte via Rows.Scan. However, scanning into any (driver.Value) would cause the JSON string to contain escape characters and other unexpected behavior.

It is now possible to scan into any, or directly into go-duckdb's Composite type, as shown in the JSON example. Scanning directly into string or []byte is no longer possible. A workaround is casting to ::VARCHAR or ::BLOB in DuckDB if you do not need to scan the result into a JSON interface.

Installation

go get github.com/marcboeker/go-duckdb/v2
Windows

You must have the correct version of gcc and the necessary runtime libraries installed on Windows. One method to do this is using msys64. To begin, install msys64 using their installer. Once you installed msys64, open a msys64 shell and run:

pacman -S mingw-w64-ucrt-x86_64-gcc

Select "yes" when necessary; it is okay if the shell closes. Then, add gcc to the path using whatever method you prefer. In powershell this is $env:PATH = "C:\msys64\ucrt64\bin:$env:PATH". After, you can compile this package in Windows.

Vendoring

You can use go mod vendor to make a copy of the third-party packages in this package, including the pre-built DuckDB libraries in duckdb-go-bindings.

Usage

Note: For readability, we omit error handling in most examples.

go-duckdb hooks into the database/sql interface provided by the Go stdlib. To open a connection, specify the driver type as duckdb.

db, err := sql.Open("duckdb", "")
defer db.Close()

The above lines create an in-memory instance of DuckDB. To open a persistent database, specify a file path to the database file. If the file does not exist, then DuckDB creates it.

db, err := sql.Open("duckdb", "/path/to/foo.db")
defer db.Close()

If you want to set specific config options for DuckDB, you can add them as query style parameters in the form of name=value pairs to the DSN.

db, err := sql.Open("duckdb", "/path/to/foo.db?access_mode=read_only&threads=4")
defer db.Close()

Alternatively, you can use sql.OpenDB. That way, you can perform initialization steps in a callback function before opening the database. Here's an example that configures some parameters when opening a database with sql.OpenDB(connector).

c, err := duckdb.NewConnector("/path/to/foo.db?access_mode=read_only&threads=4", func(execer driver.ExecerContext) error {
    bootQueries := []string{
        `SET schema=main`,
        `SET search_path=main`,
    }
    for _, query := range bootQueries {
        _, err = execer.ExecContext(context.Background(), query, nil)
        if err != nil {
            return err
        }
    }
    return nil
})
defer c.Close()
db := sql.OpenDB(c)
defer db.Close()

Please refer to the database/sql documentation for further instructions on usage.

Linking DuckDB

By default, go-duckdb statically links pre-built DuckDB libraries into your binary. Statically linking DuckDB increases your binary size.

go-duckdb bundles the following pre-compiled static libraries.

  • MacOS: amd64, arm64.
  • Linux: amd64, arm64.
  • Windows: amd64.
Linking a Static Library

If none of the pre-built libraries satisfy your needs, you can build a custom static library.

  1. Clone and build the DuckDB source code.
    • Use their bundle-library Makefile target (e.g., make bundle-library).
    • Common build flags are: DUCKDB_PLATFORM=any BUILD_EXTENSIONS="icu;json;parquet;autocomplete".
    • See DuckDB's development instructions for more details.
  2. Link against the resulting static library, which you can find in: duckdb/build/release/libduckdb_bundle.a.

For Darwin ARM64, you can then build your module like so:

CGO_ENABLED=1 CPPFLAGS="-DDUCKDB_STATIC_BUILD" CGO_LDFLAGS="-lduckdb_bundle -lc++ -L/path/to/libs" go build -tags=duckdb_use_static_lib

You can also find these steps in the Makefile and the tests.yaml.

The DuckDB team also publishes pre-built libraries as part of their releases. The published zipped archives contain libraries for DuckDB core, the third-party libraries, and the default extensions. When linking, you might want to bundle these libraries into a single archive first. You can use any archive tool (e.g., ar). DuckDB's bundle-library Makefile target contains an example of ar, or you can look at the Docker file here.

Note on FreeBSD

Starting with v2, go-duckdb drops pre-built FreeBSD support. This change is because DuckDB does not publish any bundled FreeBSD libraries. Thus, you must build your static library for FreeBSD using the steps above.

Linking a Dynamic Library

Alternatively, you can dynamically link DuckDB by passing -tags=duckdb_use_lib to go build. You must have a copy of libduckdb available on your system (.so on Linux or .dylib on macOS), which you can download from the DuckDB releases page.

For example:

# On Linux.
CGO_ENABLED=1 CGO_LDFLAGS="-lduckdb -L/path/to/libs" go build -tags=duckdb_use_lib main.go
LD_LIBRARY_PATH=/path/to/libs ./main

# On MacOS.
CGO_ENABLED=1 CGO_LDFLAGS="-lduckdb -L/path/to/libs" go build -tags=duckdb_use_lib main.go
DYLD_LIBRARY_PATH=/path/to/libs ./main

You can also find these steps in the Makefile and the tests.yaml.

Notes and FAQs

undefined: conn

Some people encounter an undefined: conn error when building this package. This error is due to the Go compiler determining that CGO is unavailable. This error can happen due to a few issues.

The first cause, as noted in the comment here, might be that the buildtools are not installed. To fix this for ubuntu, you can install them using:

sudo apt-get update && sudo apt-get install build-essential

Another cause can be cross-compilation since the Go compiler automatically disables CGO when cross-compiling. To enable CGO when cross-compiling, use CC={C cross compiler} CGO_ENABLED=1 {command} to force-enable CGO and set the right cross-compiler.

TIMESTAMP vs. TIMESTAMP_TZ

In the C API, DuckDB stores both TIMESTAMP and TIMESTAMP_TZ as duckdb_timestamp, which holds the number of microseconds elapsed since January 1, 1970, UTC (i.e., an instant without offset information). When passing a time.Time to go-duckdb, go-duckdb transforms it to an instant with UnixMicro(), even when using TIMESTAMP_TZ. Later, scanning either type of value returns an instant, as SQL types do not model time zone information for individual values.

Memory Allocation

DuckDB lives in process. Therefore, all its memory lives in the driver. All allocations live in the host process, which is the Go application. Especially for long-running applications, it is crucial to call the corresponding Close-functions as specified in database/sql.

Additionally, it is crucial to call Close() on the database and/or connector of a persistent DuckDB database. That way, DuckDB synchronizes all changes from the WAL to its persistent storage.

The following is a list of examples of Close()-functions.

db, err := sql.Open("duckdb", "")
defer db.Close()

conn, err := db.Conn(context.Background())
defer conn.Close()

rows, err := conn.QueryContext(context.Background(), "SELECT 42")
// Alternatively, rows.Next() has to return false.
rows.Close()

appender, err := duckdb.NewAppenderFromConn(conn, "", "test")
defer appender.Close()

c, err := duckdb.NewConnector("", nil)
// Optional, if passed to sql.OpenDB.
defer c.Close()

DuckDB Appender API

If you want to use the DuckDB Appender API, you can obtain a new Appender by passing a DuckDB connection to NewAppenderFromConn(). See examples/appender.go for a complete example.

c, err := duckdb.NewConnector("test.db", nil)
defer c.Close()

conn, err := c.Connect(context.Background())
defer conn.Close()

// Obtain an appender from the connection.
// NOTE: The table 'test_tbl' must exist in test.db.
appender, err := NewAppenderFromConn(conn, "", "test_tbl")
defer appender.Close()

err = appender.AppendRow(...)

DuckDB Profiling API

This section describes using the DuckDB Profiling API. DuckDB's profiling information is connection-local. The following example walks you through the necessary steps to obtain the ProfilingInfo type, which contains all available metrics. Please refer to the DuckDB documentation on configuring and collecting specific metrics.

  • First, you need to obtain a connection.
  • Then, you enable profiling for the connection.
  • Now, for each subsequent query on this connection, DuckDB will collect profiling information.
    • Optionally, you can turn off profiling at any point.
  • Next, you execute the query for which you want to obtain profiling information.
  • Finally, directly after executing the query, retrieve any available profiling information.
db, err := sql.Open("duckdb", "")
defer db.Close()

conn, err := db.Conn(context.Background())
defer conn.Close()

_, err = conn.ExecContext(context.Background(), `PRAGMA enable_profiling = 'no_output'`)
_, err = conn.ExecContext(context.Background(), `PRAGMA profiling_mode = 'detailed'`)

res, err := conn.QueryContext(context.Background(), `SELECT 42`)
defer res.Close()

info, err := GetProfilingInfo(conn)

_, err = conn.ExecContext(context.Background(), `PRAGMA disable_profiling`)

DuckDB Apache Arrow Interface

The DuckDB Arrow Interface is a heavy dependency. Starting with v2, the DuckDB Arrow Interface is opt-in instead of opt-out. If you want to use it, you can enable it by passing -tags=duckdb_arrow to go build.

go build -tags="duckdb_arrow"

You can obtain a new Arrow by passing a DuckDB connection to NewArrowFromConn().

c, err := duckdb.NewConnector("", nil)
defer c.Close()

conn, err := c.Connect(context.Background())
defer conn.Close()

// Obtain the Arrow from the connection.
arrow, err := duckdb.NewArrowFromConn(conn)

rdr, err := arrow.QueryContext(context.Background(), "SELECT * FROM generate_series(1, 10)")
defer rdr.Release()

for rdr.Next() {
  // Process each record.
}

[!WARNING]
Arrow connections are not safe for concurrent use, and do not benefit from database/sql connection pooling.

DuckDB Extensions

go-duckdb relies on the duckdb-go-bindings module. Any pre-built library in duckdb-go-bindings statically links the default extensions: ICU, JSON, Parquet, and Autocomplete. Additionally, automatic extension loading is enabled.

Releasing a New DuckDB Version

Update the Mappings
  1. Create a new branch.
  2. Update the duckdb-go-bindings dependencies in mapping/mod.go and arrowmapping/mod.go.
  3. Run go mod tidy inside mapping and inside arrowmapping.
  4. Commit and PR changes.
  5. Push two new tagged releases, mapping/vx.x.x and arrowmapping/vx.x.x.
Update the Main Module
  1. Create a new branch.
  2. Update the mapping and arrowmapping dependencies in mod.go.
  3. Run go mod tidy.
  4. Update VERSION= in tests.yaml.
  5. Update the latest version in README.md.
  6. Commit and PR changes.
  7. Push a new tagged release, vx.x.x.
git tag <tagname>
git push origin <tagname>

Example PRs:

Documentation

Overview

Package duckdb implements a database/sql driver for the DuckDB database.

Example (SimpleConnection) 
// Connect to DuckDB using '[database/sql.Open]'.
db, err := sql.Open("duckdb", "?access_mode=READ_WRITE")
defer func() {
	if err = db.Close(); err != nil {
		log.Fatalf("failed to close the database: %s", err)
	}
}()
if err != nil {
	log.Fatalf("failed to open connection to duckdb: %s", err)
}

ctx := context.Background()

createStmt := `CREATE table users(name VARCHAR, age INTEGER)`
_, err = db.ExecContext(ctx, createStmt)
if err != nil {
	log.Fatalf("failed to create table: %s", err)
}

insertStmt := `INSERT INTO users(name, age) VALUES (?, ?)`
res, err := db.ExecContext(ctx, insertStmt, "Marc", 30)
if err != nil {
	log.Fatalf("failed to insert users: %s", err)
}

rowsAffected, err := res.RowsAffected()
if err != nil {
	log.Fatal("failed to get number of rows affected")
}

fmt.Printf("Inserted %d row(s) into users table", rowsAffected)

Output:

Inserted 1 row(s) into users table

Index

Examples

Constants

View Source 
const (
	STATEMENT_TYPE_INVALID      = StmtType(mapping.StatementTypeInvalid)
	STATEMENT_TYPE_SELECT       = StmtType(mapping.StatementTypeSelect)
	STATEMENT_TYPE_INSERT       = StmtType(mapping.StatementTypeInsert)
	STATEMENT_TYPE_UPDATE       = StmtType(mapping.StatementTypeUpdate)
	STATEMENT_TYPE_EXPLAIN      = StmtType(mapping.StatementTypeExplain)
	STATEMENT_TYPE_DELETE       = StmtType(mapping.StatementTypeDelete)
	STATEMENT_TYPE_PREPARE      = StmtType(mapping.StatementTypePrepare)
	STATEMENT_TYPE_CREATE       = StmtType(mapping.StatementTypeCreate)
	STATEMENT_TYPE_EXECUTE      = StmtType(mapping.StatementTypeExecute)
	STATEMENT_TYPE_ALTER        = StmtType(mapping.StatementTypeAlter)
	STATEMENT_TYPE_TRANSACTION  = StmtType(mapping.StatementTypeTransaction)
	STATEMENT_TYPE_COPY         = StmtType(mapping.StatementTypeCopy)
	STATEMENT_TYPE_ANALYZE      = StmtType(mapping.StatementTypeAnalyze)
	STATEMENT_TYPE_VARIABLE_SET = StmtType(mapping.StatementTypeVariableSet)
	STATEMENT_TYPE_CREATE_FUNC  = StmtType(mapping.StatementTypeCreateFunc)
	STATEMENT_TYPE_DROP         = StmtType(mapping.StatementTypeDrop)
	STATEMENT_TYPE_EXPORT       = StmtType(mapping.StatementTypeExport)
	STATEMENT_TYPE_PRAGMA       = StmtType(mapping.StatementTypePragma)
	STATEMENT_TYPE_VACUUM       = StmtType(mapping.StatementTypeVacuum)
	STATEMENT_TYPE_CALL         = StmtType(mapping.StatementTypeCall)
	STATEMENT_TYPE_SET          = StmtType(mapping.StatementTypeSet)
	STATEMENT_TYPE_LOAD         = StmtType(mapping.StatementTypeLoad)
	STATEMENT_TYPE_RELATION     = StmtType(mapping.StatementTypeRelation)
	STATEMENT_TYPE_EXTENSION    = StmtType(mapping.StatementTypeExtension)
	STATEMENT_TYPE_LOGICAL_PLAN = StmtType(mapping.StatementTypeLogicalPlan)
	STATEMENT_TYPE_ATTACH       = StmtType(mapping.StatementTypeAttach)
	STATEMENT_TYPE_DETACH       = StmtType(mapping.StatementTypeDetach)
	STATEMENT_TYPE_MULTI        = StmtType(mapping.StatementTypeMulti)
)
View Source 
const (
	TYPE_INVALID      = mapping.TypeInvalid
	TYPE_BOOLEAN      = mapping.TypeBoolean
	TYPE_TINYINT      = mapping.TypeTinyInt
	TYPE_SMALLINT     = mapping.TypeSmallInt
	TYPE_INTEGER      = mapping.TypeInteger
	TYPE_BIGINT       = mapping.TypeBigInt
	TYPE_UTINYINT     = mapping.TypeUTinyInt
	TYPE_USMALLINT    = mapping.TypeUSmallInt
	TYPE_UINTEGER     = mapping.TypeUInteger
	TYPE_UBIGINT      = mapping.TypeUBigInt
	TYPE_FLOAT        = mapping.TypeFloat
	TYPE_DOUBLE       = mapping.TypeDouble
	TYPE_TIMESTAMP    = mapping.TypeTimestamp
	TYPE_DATE         = mapping.TypeDate
	TYPE_TIME         = mapping.TypeTime
	TYPE_INTERVAL     = mapping.TypeInterval
	TYPE_HUGEINT      = mapping.TypeHugeInt
	TYPE_UHUGEINT     = mapping.TypeUHugeInt
	TYPE_VARCHAR      = mapping.TypeVarchar
	TYPE_BLOB         = mapping.TypeBlob
	TYPE_DECIMAL      = mapping.TypeDecimal
	TYPE_TIMESTAMP_S  = mapping.TypeTimestampS
	TYPE_TIMESTAMP_MS = mapping.TypeTimestampMS
	TYPE_TIMESTAMP_NS = mapping.TypeTimestampNS
	TYPE_ENUM         = mapping.TypeEnum
	TYPE_LIST         = mapping.TypeList
	TYPE_STRUCT       = mapping.TypeStruct
	TYPE_MAP          = mapping.TypeMap
	TYPE_ARRAY        = mapping.TypeArray
	TYPE_UUID         = mapping.TypeUUID
	TYPE_UNION        = mapping.TypeUnion
	TYPE_BIT          = mapping.TypeBit
	TYPE_TIME_TZ      = mapping.TypeTimeTZ
	TYPE_TIMESTAMP_TZ = mapping.TypeTimestampTZ
	TYPE_ANY          = mapping.TypeAny
	TYPE_BIGNUM       = mapping.TypeBigNum
	TYPE_SQLNULL      = mapping.TypeSQLNull
)

Variables

View Source 
var GetInstanceCache = sync.OnceValue(
	func() mapping.InstanceCache {
		return mapping.CreateInstanceCache()
	})

Functions

func ConnId added in v2.3.1 

func ConnId(c *sql.Conn) (uint64, error)

ConnId returns the connection id of the internal DuckDB connection. It expects a *sql.Conn connection.

func GetDataChunkCapacity 

func GetDataChunkCapacity() int

GetDataChunkCapacity returns the capacity of a data chunk.

func GetTableNames added in v2.3.1 

func GetTableNames(c *sql.Conn, query string, qualified bool) ([]string, error)

GetTableNames returns the tables names of a query. It expects a *sql.Conn connection, and a query for which to extract the table names. If qualified is true, then it returns the fully qualified table names, else it returns only the table names.

func RegisterReplacementScan 

func RegisterReplacementScan(c *Connector, callback ReplacementScanCallback)

func RegisterScalarUDF 

func RegisterScalarUDF(c *sql.Conn, name string, f ScalarFunc) error

RegisterScalarUDF registers a user-defined scalar function. *sql.Conn is the SQL connection on which to register the scalar function. name is the function name, and f is the scalar function's interface ScalarFunc. RegisterScalarUDF takes ownership of f, so you must pass it as a pointer.

func RegisterScalarUDFSet 

func RegisterScalarUDFSet(c *sql.Conn, name string, functions ...ScalarFunc) error

RegisterScalarUDFSet registers a set of user-defined scalar functions with the same name. This enables overloading of scalar functions. E.g., the function my_length() can have implementations like my_length(LIST(ANY)) and my_length(VARCHAR). *sql.Conn is the SQL connection on which to register the scalar function set. name is the function name of each function in the set. functions contains all ScalarFunc functions of the scalar function set.

func RegisterTableUDF 

func RegisterTableUDF[TFT TableFunction](conn *sql.Conn, name string, f TFT) error

RegisterTableUDF registers a user-defined table function. Projection pushdown is enabled by default.

func SetChunkValue 

func SetChunkValue[T any](chunk DataChunk, colIdx, rowIdx int, val T) error

SetChunkValue writes a single value to a column in a data chunk. The difference with `chunk.SetValue` is that `SetChunkValue` does not require casting the value to `any` (implicitly). NOTE: Custom ENUM types must be passed as string.

func SetRowValue 

func SetRowValue[T any](row Row, colIdx int, val T) error

SetRowValue sets the value at colIdx to val. Returns an error on failure, and nil for non-projected columns.

Types

type Appender 

type Appender struct {
	// contains filtered or unexported fields
}

Appender holds the DuckDB appender. It allows efficient bulk loading into a DuckDB database.

func NewAppender added in v2.0.2 

func NewAppender(driverConn driver.Conn, catalog, schema, table string) (*Appender, error)

NewAppender returns a new Appender from a DuckDB driver connection.

func NewAppenderFromConn 

func NewAppenderFromConn(driverConn driver.Conn, schema, table string) (*Appender, error)

NewAppenderFromConn returns a new Appender for the default catalog from a DuckDB driver connection.

func (*Appender) AppendRow 

func (a *Appender) AppendRow(args ...driver.Value) error

AppendRow loads a row of values into the appender. The values are provided as separate arguments.

func (*Appender) Close 

func (a *Appender) Close() error

Close the appender. This will flush the appender to the underlying table. It is vital to call this when you are done with the appender to avoid leaking memory.

func (*Appender) Flush 

func (a *Appender) Flush() error

Flush the data chunks to the underlying table and clear the internal cache. Does not close the appender, even if it returns an error. Unless you have a good reason to call this, call Close when you are done with the appender.

type CardinalityInfo 

type CardinalityInfo struct {
	// The absolute Cardinality.
	Cardinality uint
	// IsExact indicates whether the cardinality is exact.
	Exact bool
}

CardinalityInfo contains the cardinality of a (table) function. If it is impossible or difficult to determine the exact cardinality, an approximate cardinality may be used.

type ChunkTableFunction 

type ChunkTableFunction = tableFunction[ChunkTableSource]

A ChunkTableFunction is a type which can be bound to return a ChunkTableSource.

type ChunkTableSource 

type ChunkTableSource interface {

	// FillChunk takes a Chunk and fills it with values.
	// Set the chunk size to 0 to end the function.
	FillChunk(DataChunk) error
	// contains filtered or unexported methods
}

A ChunkTableSource represents anything that produces rows in a vectorised way. The cardinality is requested before function initialization. After initializing the ChunkTableSource, go-duckdb requests the rows. It sequentially calls the FillChunk method with a single thread.

type ColumnInfo 

type ColumnInfo struct {
	// The column Name.
	Name string
	// The column type T.
	T TypeInfo
}

ColumnInfo contains the metadata of a column.

type Composite 

type Composite[T any] struct {
	// contains filtered or unexported fields
}

Use as the `Scanner` type for any composite types (maps, lists, structs)

func (Composite[T]) Get 

func (s Composite[T]) Get() T

func (*Composite[T]) Scan 

func (s *Composite[T]) Scan(v any) error

type Conn 

type Conn struct {
	// contains filtered or unexported fields
}

Conn holds a connection to a DuckDB database. It implements the driver.Conn interface.

func (*Conn) Begin 

func (conn *Conn) Begin() (driver.Tx, error)

Begin is deprecated: Use BeginTx instead.

func (*Conn) BeginTx 

func (conn *Conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error)

BeginTx starts and returns a new transaction. It implements the driver.ConnBeginTx interface.

func (*Conn) CheckNamedValue 

func (conn *Conn) CheckNamedValue(nv *driver.NamedValue) error

CheckNamedValue implements the driver.NamedValueChecker interface.

func (*Conn) Close 

func (conn *Conn) Close() error

Close closes the connection to the database. It implements the driver.Conn interface.

func (*Conn) ExecContext 

func (conn *Conn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error)

ExecContext executes a query that doesn't return rows, such as an INSERT or UPDATE. It implements the driver.ExecerContext interface.

func (*Conn) Prepare 

func (conn *Conn) Prepare(query string) (driver.Stmt, error)

Prepare returns a prepared statement, bound to this connection. It implements the driver.Conn interface.

func (*Conn) PrepareContext 

func (conn *Conn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error)

PrepareContext returns a prepared statement, bound to this connection. It implements the driver.ConnPrepareContext interface.

func (*Conn) QueryContext 

func (conn *Conn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error)

QueryContext executes a query that may return rows, such as a SELECT. It implements the driver.QueryerContext interface.

type Connector 

type Connector struct {
	// contains filtered or unexported fields
}

func NewConnector 

func NewConnector(dsn string, connInitFn func(execer driver.ExecerContext) error) (*Connector, error)

NewConnector opens a new Connector for a DuckDB database. The user must close the Connector, if it is not passed to the sql.OpenDB function. Otherwise, sql.DB closes the Connector when calling sql.DB.Close().

Example 
c, err := NewConnector(`duck.db?access_mode=READ_WRITE`, func(execer driver.ExecerContext) error {
	initQueries := []string{
		`SET memory_limit = '10GB'`,
		`SET threads TO 1`,
	}

	ctx := context.Background()
	for _, query := range initQueries {
		_, err := execer.ExecContext(ctx, query, nil)
		if err != nil {
			return err
		}
	}
	return nil
})
if err != nil {
	log.Fatalf("failed to create a new duckdb connector: %s", err)
}
defer func() {
	if err = c.Close(); err != nil {
		log.Fatalf("failed to close the connector: %s", err)
	}
}()

db := sql.OpenDB(c)
defer func() {
	if err = db.Close(); err != nil {
		log.Fatalf("failed to close the database: %s", err)
	}
	if err = os.Remove("duck.db"); err != nil {
		log.Fatalf("failed to remove the database file: %s", err)
	}
}()

var value string
row := db.QueryRow(`SELECT value FROM duckdb_settings() WHERE name = 'max_memory'`)
if row.Scan(&value) != nil {
	log.Fatalf("failed to scan row: %s", err)
}

fmt.Printf("The memory_limit is %s.", value)

Output:

The memory_limit is 9.3 GiB.

func (*Connector) Close 

func (c *Connector) Close() error

func (*Connector) Connect 

func (c *Connector) Connect(ctx context.Context) (driver.Conn, error)

func (*Connector) Driver 

func (*Connector) Driver() driver.Driver

type DataChunk 

type DataChunk struct {
	// contains filtered or unexported fields
}

DataChunk storage of a DuckDB table.

func (*DataChunk) GetSize 

func (chunk *DataChunk) GetSize() int

GetSize returns the internal size of the data chunk.

func (*DataChunk) GetValue 

func (chunk *DataChunk) GetValue(colIdx, rowIdx int) (any, error)

GetValue returns a single value of a column.

func (*DataChunk) SetSize 

func (chunk *DataChunk) SetSize(size int) error

SetSize sets the internal size of the data chunk. Cannot exceed GetCapacity().

func (*DataChunk) SetValue 

func (chunk *DataChunk) SetValue(colIdx, rowIdx int, val any) error

SetValue writes a single value to a column in a data chunk. Note that this requires casting the type for each invocation. NOTE: Custom ENUM types must be passed as string.

type Decimal 

type Decimal struct {
	Width uint8
	Scale uint8
	Value *big.Int
}

func (Decimal) Float64 

func (d Decimal) Float64() float64

func (Decimal) String 

func (d Decimal) String() string

type Driver 

type Driver struct{}

func (Driver) Open 

func (d Driver) Open(dsn string) (driver.Conn, error)

func (Driver) OpenConnector 

func (Driver) OpenConnector(dsn string) (driver.Connector, error)

type Error 

type Error struct {
	Type ErrorType
	Msg  string
}

func (*Error) Error 

func (e *Error) Error() string

func (*Error) Is 

func (e *Error) Is(err error) bool

type ErrorType 

type ErrorType int
const (
	ErrorTypeInvalid              ErrorType = iota // invalid type
	ErrorTypeOutOfRange                            // value out of range error
	ErrorTypeConversion                            // conversion/casting error
	ErrorTypeUnknownType                           // unknown type error
	ErrorTypeDecimal                               // decimal related
	ErrorTypeMismatchType                          // type mismatch
	ErrorTypeDivideByZero                          // divide by 0
	ErrorTypeObjectSize                            // object size exceeded
	ErrorTypeInvalidType                           // incompatible for operation
	ErrorTypeSerialization                         // serialization
	ErrorTypeTransaction                           // transaction management
	ErrorTypeNotImplemented                        // method not implemented
	ErrorTypeExpression                            // expression parsing
	ErrorTypeCatalog                               // catalog related
	ErrorTypeParser                                // parser related
	ErrorTypePlanner                               // planner related
	ErrorTypeScheduler                             // scheduler related
	ErrorTypeExecutor                              // executor related
	ErrorTypeConstraint                            // constraint related
	ErrorTypeIndex                                 // index related
	ErrorTypeStat                                  // stat related
	ErrorTypeConnection                            // connection related
	ErrorTypeSyntax                                // syntax related
	ErrorTypeSettings                              // settings related
	ErrorTypeBinder                                // binder related
	ErrorTypeNetwork                               // network related
	ErrorTypeOptimizer                             // optimizer related
	ErrorTypeNullPointer                           // nullptr exception
	ErrorTypeIO                                    // IO exception
	ErrorTypeInterrupt                             // interrupt
	ErrorTypeFatal                                 // Fatal exceptions are non-recoverable, and render the entire DB in an unusable state
	ErrorTypeInternal                              // Internal exceptions indicate something went wrong internally (i.e. bug in the code base)
	ErrorTypeInvalidInput                          // Input or arguments error
	ErrorTypeOutOfMemory                           // out of memory
	ErrorTypePermission                            // insufficient permissions
	ErrorTypeParameterNotResolved                  // parameter types could not be resolved
	ErrorTypeParameterNotAllowed                   // parameter types not allowed
	ErrorTypeDependency                            // dependency
	ErrorTypeHTTP
	ErrorTypeMissingExtension // Thrown when an extension is used but not loaded
	ErrorTypeAutoLoad         // Thrown when an extension is used but not loaded
	ErrorTypeSequence
	ErrorTypeInvalidConfiguration // An invalid configuration was detected (e.g. a Secret param was missing, or a required setting not found)
)

type Interval 

type Interval struct {
	Days   int32 `json:"days"`
	Months int32 `json:"months"`
	Micros int64 `json:"micros"`
}

type Map 

type Map map[any]any

func (*Map) Scan 

func (m *Map) Scan(v any) error

type ParallelChunkTableFunction 

type ParallelChunkTableFunction = tableFunction[ParallelChunkTableSource]

A ParallelChunkTableFunction is a type which can be bound to return a ParallelChunkTableSource.

type ParallelChunkTableSource 

type ParallelChunkTableSource interface {

	// FillChunk takes a Chunk and fills it with values.
	// Set the chunk size to 0 to end the function
	FillChunk(any, DataChunk) error
	// contains filtered or unexported methods
}

A ParallelChunkTableSource represents anything that produces rows in a vectorised way. The cardinality is requested before function initialization. After initializing the ParallelChunkTableSource, go-duckdb requests the rows. It simultaneously calls the FillChunk method with multiple threads. If ParallelTableSourceInfo.MaxThreads is greater than one, FillChunk must use synchronization primitives to avoid race conditions.

type ParallelRowTableFunction 

type ParallelRowTableFunction = tableFunction[ParallelRowTableSource]

A ParallelRowTableFunction is a type which can be bound to return a ParallelRowTableSource.

type ParallelRowTableSource 

type ParallelRowTableSource interface {

	// FillRow takes a Row and fills it with values.
	// Returns true, if there are more rows to fill.
	FillRow(any, Row) (bool, error)
	// contains filtered or unexported methods
}

A ParallelRowTableSource represents anything that produces rows in a non-vectorised way. The cardinality is requested before function initialization. After initializing the ParallelRowTableSource, go-duckdb requests the rows. It simultaneously calls the FillRow method with multiple threads. If ParallelTableSourceInfo.MaxThreads is greater than one, FillRow must use synchronisation primitives to avoid race conditions.

type ParallelTableSourceInfo 

type ParallelTableSourceInfo struct {
	// MaxThreads is the maximum number of threads on which to run the table source function.
	// If set to 0, it uses DuckDB's default thread configuration.
	MaxThreads int
}

ParallelTableSourceInfo contains information for initializing a parallelism-aware table source.

type ProfilingInfo 

type ProfilingInfo struct {
	// Metrics contains all key-value pairs of the current node.
	// The key represents the name and corresponds to the measured value.
	Metrics map[string]string
	// Children contains all children of the node and their respective metrics.
	Children []ProfilingInfo
}

ProfilingInfo is a recursive type containing metrics for each node in DuckDB's query plan. There are two types of nodes: the QUERY_ROOT and OPERATOR nodes. The QUERY_ROOT refers exclusively to the top-level node; its metrics are measured over the entire query. The OPERATOR nodes refer to the individual operators in the query plan.

func GetProfilingInfo 

func GetProfilingInfo(c *sql.Conn) (ProfilingInfo, error)

GetProfilingInfo obtains all available metrics set by the current connection.

type ReplacementScanCallback 

type ReplacementScanCallback func(tableName string) (string, []any, error)

type Row 

type Row struct {
	// contains filtered or unexported fields
}

Row represents one row in duckdb. It references the internal vectors.

func (Row) IsProjected 

func (r Row) IsProjected(colIdx int) bool

IsProjected returns whether the column is projected.

func (Row) SetRowValue 

func (r Row) SetRowValue(colIdx int, val any) error

SetRowValue sets the value at colIdx to val. Returns an error on failure, and nil for non-projected columns.

type RowContextExecutorFn added in v2.3.1 

type RowContextExecutorFn func(ctx context.Context, values []driver.Value) (any, error)

RowContextExecutorFn accepts a row-based execution function using a context. It takes a context and the row values, and returns the row execution result, or error.

type RowExecutorFn added in v2.3.1 

type RowExecutorFn func(values []driver.Value) (any, error)

RowExecutorFn is the type for any row-based execution function. It takes the row values and returns the row execution result, or error.

type RowTableFunction 

type RowTableFunction = tableFunction[RowTableSource]

A RowTableFunction is a type which can be bound to return a RowTableSource.

type RowTableSource 

type RowTableSource interface {

	// FillRow takes a Row and fills it with values.
	// Returns true, if there are more rows to fill.
	FillRow(Row) (bool, error)
	// contains filtered or unexported methods
}

A RowTableSource represents anything that produces rows in a non-vectorised way. The cardinality is requested before function initialization. After initializing the RowTableSource, go-duckdb requests the rows. It sequentially calls the FillRow method with a single thread.

type ScalarFunc 

type ScalarFunc interface {
	// Config returns ScalarFuncConfig to configure the scalar function.
	Config() ScalarFuncConfig
	// Executor returns ScalarFuncExecutor to execute the scalar function.
	Executor() ScalarFuncExecutor
}

ScalarFunc is the user-defined scalar function interface. Any scalar function must implement a Config function, and an Executor function.

type ScalarFuncConfig 

type ScalarFuncConfig struct {
	// InputTypeInfos contains Type information for each input parameter of the scalar function.
	InputTypeInfos []TypeInfo
	// ResultTypeInfo holds the Type information of the scalar function's result type.
	ResultTypeInfo TypeInfo

	// VariadicTypeInfo configures the number of input parameters.
	// If this field is nil, then the input parameters match InputTypeInfos.
	// Otherwise, the scalar function's input parameters are set to variadic, allowing any number of input parameters.
	// The Type of the first len(InputTypeInfos) parameters is configured by InputTypeInfos, and all
	// remaining parameters must match the variadic Type. To configure different variadic parameter types,
	// you must set the VariadicTypeInfo's Type to TYPE_ANY.
	VariadicTypeInfo TypeInfo
	// Volatile sets the stability of the scalar function to volatile, if true.
	// Volatile scalar functions might create a different result per row.
	// E.g., random() is a volatile scalar function.
	Volatile bool
	// SpecialNullHandling disables the default NULL handling of scalar functions, if true.
	// The default NULL handling is: NULL in, NULL out. I.e., if any input parameter is NULL, then the result is NULL.
	SpecialNullHandling bool
}

ScalarFuncConfig contains the fields to configure a user-defined scalar function.

type ScalarFuncExecutor 

type ScalarFuncExecutor struct {
	// RowExecutor accepts a row-based execution function of type RowExecutorFn.
	RowExecutor RowExecutorFn
	// RowContextExecutor accepts a row-based execution function of type RowContextExecutorFn.
	RowContextExecutor RowContextExecutorFn
}

ScalarFuncExecutor contains the functions to execute a user-defined scalar function. It invokes its first non-nil member.

type Stmt 

type Stmt struct {
	// contains filtered or unexported fields
}

Stmt implements the driver.Stmt interface.

func (*Stmt) Bind 

func (s *Stmt) Bind(args []driver.NamedValue) error

Bind the parameters to the statement. WARNING: This is a low-level API and should be used with caution.

func (*Stmt) Close 

func (s *Stmt) Close() error

Close the statement. Implements the driver.Stmt interface.

func (*Stmt) Exec deprecated 

func (s *Stmt) Exec(args []driver.Value) (driver.Result, error)

Deprecated: Use ExecContext instead.

func (*Stmt) ExecBound 

func (s *Stmt) ExecBound(ctx context.Context) (driver.Result, error)

ExecBound executes a bound query that doesn't return rows, such as an INSERT or UPDATE. It can only be used after Bind has been called. WARNING: This is a low-level API and should be used with caution.

func (*Stmt) ExecContext 

func (s *Stmt) ExecContext(ctx context.Context, nargs []driver.NamedValue) (driver.Result, error)

ExecContext executes a query that doesn't return rows, such as an INSERT or UPDATE. It implements the driver.StmtExecContext interface.

func (*Stmt) NumInput 

func (s *Stmt) NumInput() int

NumInput returns the number of placeholder parameters. Implements the driver.Stmt interface.

func (*Stmt) ParamName 

func (s *Stmt) ParamName(n int) (string, error)

ParamName returns the name of the parameter at the given index (1-based).

func (*Stmt) ParamType 

func (s *Stmt) ParamType(n int) (Type, error)

ParamType returns the expected type of the parameter at the given index (1-based).

func (*Stmt) Query deprecated 

func (s *Stmt) Query(args []driver.Value) (driver.Rows, error)

Deprecated: Use QueryContext instead.

func (*Stmt) QueryBound 

func (s *Stmt) QueryBound(ctx context.Context) (driver.Rows, error)

QueryBound executes a bound query that may return rows, such as a SELECT. It can only be used after Bind has been called. WARNING: This is a low-level API and should be used with caution.

func (*Stmt) QueryContext 

func (s *Stmt) QueryContext(ctx context.Context, nargs []driver.NamedValue) (driver.Rows, error)

QueryContext executes a query that may return rows, such as a SELECT. It implements the driver.StmtQueryContext interface.

func (*Stmt) StatementType 

func (s *Stmt) StatementType() (StmtType, error)

StatementType returns the type of the statement.

type StmtType 

type StmtType mapping.StatementType

type StructEntry 

type StructEntry interface {
	// Info returns a STRUCT entry's type information.
	Info() TypeInfo
	// Name returns a STRUCT entry's name.
	Name() string
}

StructEntry is an interface to provide STRUCT entry information.

func NewStructEntry 

func NewStructEntry(info TypeInfo, name string) (StructEntry, error)

NewStructEntry returns a STRUCT entry. info contains information about the entry's type, and name holds the entry's name.

type TableFunction 

type TableFunction interface {
	RowTableFunction | ParallelRowTableFunction | ChunkTableFunction | ParallelChunkTableFunction
}

TableFunction implements different table function types: RowTableFunction, ParallelRowTableFunction, ChunkTableFunction, and ParallelChunkTableFunction.

type TableFunctionConfig 

type TableFunctionConfig struct {
	// The Arguments of the table function.
	Arguments []TypeInfo
	// The NamedArguments of the table function.
	NamedArguments map[string]TypeInfo
}

TableFunctionConfig contains any information passed to DuckDB when registering the table function.

type Type 

type Type = mapping.Type

Type wraps the corresponding DuckDB type enumapping.

type TypeInfo 

type TypeInfo interface {
	// InternalType returns the Type.
	InternalType() Type
	// contains filtered or unexported methods
}

TypeInfo is an interface for a DuckDB type.

func NewArrayInfo 

func NewArrayInfo(childInfo TypeInfo, size uint64) (TypeInfo, error)

NewArrayInfo returns ARRAY type information. childInfo contains the type information of the ARRAY's elements. size is the ARRAY's fixed size.

func NewDecimalInfo 

func NewDecimalInfo(width, scale uint8) (TypeInfo, error)

NewDecimalInfo returns DECIMAL type information. Its input parameters are the width and scale of the DECIMAL type.

func NewEnumInfo 

func NewEnumInfo(first string, others ...string) (TypeInfo, error)

NewEnumInfo returns ENUM type information. Its input parameters are the dictionary values.

func NewListInfo 

func NewListInfo(childInfo TypeInfo) (TypeInfo, error)

NewListInfo returns LIST type information. childInfo contains the type information of the LIST's elements.

func NewMapInfo 

func NewMapInfo(keyInfo, valueInfo TypeInfo) (TypeInfo, error)

NewMapInfo returns MAP type information. keyInfo contains the type information of the MAP keys. valueInfo contains the type information of the MAP values.

func NewStructInfo 

func NewStructInfo(firstEntry StructEntry, others ...StructEntry) (TypeInfo, error)

NewStructInfo returns STRUCT type information. Its input parameters are the STRUCT entries.

func NewTypeInfo 

func NewTypeInfo(t Type) (TypeInfo, error)

NewTypeInfo returns type information for DuckDB's primitive types. It returns the TypeInfo, if the Type parameter is a valid primitive type. Else, it returns nil, and an error. Valid types are: TYPE_[BOOLEAN, TINYINT, SMALLINT, INTEGER, BIGINT, UTINYINT, USMALLINT, UINTEGER, UBIGINT, FLOAT, DOUBLE, TIMESTAMP, DATE, TIME, INTERVAL, HUGEINT, VARCHAR, BLOB, TIMESTAMP_S, TIMESTAMP_MS, TIMESTAMP_NS, UUID, TIMESTAMP_TZ, ANY].

func NewUnionInfo added in v2.2.1 

func NewUnionInfo(memberTypes []TypeInfo, memberNames []string) (TypeInfo, error)

NewUnionInfo returns UNION type information. memberTypes contains the type information of the union members. memberNames contains the names of the union members.

type UUID 

type UUID [uuidLength]byte

func (*UUID) Scan 

func (u *UUID) Scan(v any) error

Scan implements the sql.Scanner interface.

func (*UUID) String 

func (u *UUID) String() string

String implements the fmt.Stringer interface.

func (*UUID) Value added in v2.3.4 

func (u *UUID) Value() (driver.Value, error)

Value implements the driver.Valuer interface.

type Union added in v2.2.1 

type Union struct {
	Value driver.Value `json:"value"`
	Tag   string       `json:"tag"`
}

Source Files

View all Source files

Directories

Path Synopsis
examples
appender command
json command
scalar_udf command
simple command
table_udf command
table_udf_parallel command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.