sqlite

README

The repository you are currently viewing might be a mirror. Please review the guidelines below based on where you are viewing this:

Platform	Role	Contributing Guidelines
GitLab	Primary Source	This is the canonical repository (cznic/sqlite). CI pipelines and main development happen here.
GitHub	Mirror	This is a mirror (modernc-org/sqlite). We do accept Issues and Pull Requests here for your convenience! Note: PRs submitted here will be manually merged into the GitLab source, so please allow extra time for processing.

---

Github Sponsors Account / j-modernc-org

Enterprise Infrastructure Tier Sponsor

Tailscale

Startup / Small Business Tier Sponsor

exe.dev

October Swimmer

---

The SQLite Drivers Benchmarks Game

---

Virtual Tables (vtab)

The driver exposes a Go API to implement SQLite virtual table modules in pure Go via the modernc.org/sqlite/vtab package. This lets you back SQL tables with arbitrary data sources (e.g., vector indexes, CSV files, remote APIs) and integrate with SQLite’s planner.

• Register: vtab.RegisterModule(db, name, module). Registration applies to new connections only.
• Schema declaration: Call ctx.Declare("CREATE TABLE <name>(<cols...>)") within Create or Connect. The driver does not auto-declare schemas, enabling dynamic schemas.
• Module arguments: args []string passed to Create/Connect are configuration parsed from USING module(...). They are not treated as columns unless your module chooses to.
• Planning (BestIndex):
  • Inspect info.Constraints (with Column, Op, Usable, 0-based ArgIndex, and Omit), info.OrderBy, and info.ColUsed (bitmask of referenced columns).
  • Set ArgIndex (0-based) to populate Filter’s vals in the chosen order; set Omit to ask SQLite not to re-check a constraint you fully handle.
• Execution: Cursor.Filter(idxNum, idxStr, vals) receives arguments in the order implied by ArgIndex.
• Operators: Common SQLite operators map to ConstraintOp (EQ/NE/GT/GE/LT/LE/MATCH/IS/ISNOT/ISNULL/ISNOTNULL/LIKE/GLOB/REGEXP/FUNCTION/LIMIT/OFFSET). Unknown operators map to OpUnknown.
• Errors: Returning an error from vtab methods surfaces a descriptive message to SQLite (e.g., zErrMsg for xCreate/xConnect/xBestIndex/xFilter; sqlite3_result_error for xColumn).

Examples

• Vector search (sqlite-vec style):

  • CREATE VIRTUAL TABLE vec_docs USING vec(dim=128, metric="cosine")
  • Module reads args (e.g., dim, metric), calls ctx.Declare("CREATE TABLE vec_docs(id, embedding, content HIDDEN)"), and implements search via BestIndex/Filter.

• CSV loader:

  • CREATE VIRTUAL TABLE csv_users USING csv(filename="/tmp/users.csv", delimiter=",", header=true)
  • Module reads the file header to compute columns, declares them via ctx.Declare("CREATE TABLE csv_users(name, email, ...)"), and streams rows via a cursor.

See vtab package docs for full API details.

Generated sources (deduplication)

The transpiled SQLite C amalgamation in lib/ and the sqlite-vec extension in vec/ ship one generated Go file per GOOS/GOARCH. Declarations that are byte-identical across targets are folded into build-tagged shared files — lib/sqlite.go plus lib/sqlite_g_<hex>.go (and vec/vec.go plus vec/vec_g_<hex>.go) — by modernc.org/undup, wired into make vendor. Go's build constraints make every target compile exactly the same set of declarations as before, so this is purely a packaging change: it keeps each tag's module download well under Go's 500 MB cap and does not affect the public API or behavior.

To read or debug a single target's full, self-contained generated source, expand the tree back to one complete file per target:

go run modernc.org/undup@v0.0.5 -expand -dir lib   # writes full lib/sqlite_<goos>_<goarch>.go
go run modernc.org/undup@v0.0.5 -expand -dir vec   # writes full vec/vec_<goos>_<goarch>.go

This removes the shared *_g_*.go files and rewrites each *_<goos>_<goarch>.go as a standalone file — convenient for grepping, reading, or stepping through one platform's code. Restore the committed, deduplicated form with git checkout -- lib vec, or re-fold in place with go run modernc.org/undup@v0.0.5 -dir lib (and -dir vec). Hand-written platform files (libsqlite3_*.go, hooks_*.go, …) carry no generated-code marker and are never touched by either step.

Documentation

Overview

Package sqlite is a sql/database driver using a CGo-free port of the C SQLite3 library.

SQLite is an in-process implementation of a self-contained, serverless, zero-configuration, transactional SQL database engine.

Pluggable page cache

The package exposes a Go-facing wrapper for SQLite's SQLITE_CONFIG_PCACHE2 mechanism. Applications can supply their own page cache implementation by registering a PageCache before the first sql.Open via RegisterPageCache. See the docstrings on PageCache, Cache, and Page for the contract; the binding owns the sqlite3_pcache_page stub on behalf of the implementation and re-consults Cache.Fetch on every SQLite request, so a bounded and evicting purgeable cache works as the C contract intends.

Fragile modernc.org/libc dependency

When you import this package you should use in your go.mod file the exact same version of modernc.org/libc as seen in the go.mod file of this repository.

See the discussion at https://gitlab.com/cznic/sqlite/-/issues/177 for more details.

Thanks

This project is sponsored by Schleibinger Geräte Teubert u. Greim GmbH by allowing one of the maintainers to work on it also in office hours.

Supported platforms and architectures

These combinations of GOOS and GOARCH are currently supported

OS      Arch    SQLite version
------------------------------
darwin	amd64   3.53.2
darwin	arm64   3.53.2
freebsd	amd64   3.53.2
freebsd	arm64   3.53.2
linux	386     3.53.2
linux	amd64   3.53.2
linux	arm     3.53.2
linux	arm64   3.53.2
linux	loong64 3.53.2
linux	ppc64le 3.53.2
linux	riscv64 3.53.2
linux	s390x   3.53.2
windows	386     3.53.2
windows	amd64   3.53.2
windows	arm64   3.53.2

Benchmarks

The SQLite Drivers Benchmarks Game

Builders

Builder results available at:

https://modern-c.appspot.com/-/builder/?importpath=modernc.org%2fsqlite

Connecting to a database

To access a Sqlite database do something like

import (
	"database/sql"

	_ "modernc.org/sqlite"
)

...

db, err := sql.Open("sqlite", dsnURI)

...

Debug and development versions

A comma separated list of options can be passed to `go generate` via the environment variable GO_GENERATE. Some useful options include for example:

-DSQLITE_DEBUG
-DSQLITE_MEM_DEBUG
-ccgo-verify-structs

To create a debug/development version, issue for example:

$ GO_GENERATE=-DSQLITE_DEBUG,-DSQLITE_MEM_DEBUG go generate

Note: To run `go generate` you need to have modernc.org/ccgo/v3 installed.

Hacking

This is an example of how to use the debug logs in modernc.org/libc when hunting a bug.

0:jnml@e5-1650:~/src/modernc.org/sqlite$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
0:jnml@e5-1650:~/src/modernc.org/sqlite$ git log -1
commit df33b8d15107f3cc777799c0fe105f74ef499e62 (HEAD -> master, tag: v1.21.1, origin/master, origin/HEAD, wips, ok)
Author: Jan Mercl <0xjnml@gmail.com>
Date:   Mon Mar 27 16:18:28 2023 +0200

    upgrade to SQLite 3.41.2
0:jnml@e5-1650:~/src/modernc.org/sqlite$ rm -f /tmp/libc.log ; go test -v -tags=libc.dmesg -run TestScalar ; ls -l /tmp/libc.log
test binary compiled for linux/amd64
=== RUN   TestScalar
--- PASS: TestScalar (0.09s)
PASS
ok  modernc.org/sqlite 0.128s
-rw-r--r-- 1 jnml jnml 76 Apr  6 11:22 /tmp/libc.log
0:jnml@e5-1650:~/src/modernc.org/sqlite$ cat /tmp/libc.log
[10723 sqlite.test] 2023-04-06 11:22:48.288066057 +0200 CEST m=+0.000707150
0:jnml@e5-1650:~/src/modernc.org/sqlite$

The /tmp/libc.log file is created as requested. No useful messages there because none are enabled in libc. Let's try to enable Xwrite as an example.

0:jnml@e5-1650:~/src/modernc.org/libc$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
modified:   libc_linux.go

no changes added to commit (use "git add" and/or "git commit -a")
0:jnml@e5-1650:~/src/modernc.org/libc$ git log -1
commit 1e22c18cf2de8aa86d5b19b165f354f99c70479c (HEAD -> master, tag: v1.22.3, origin/master, origin/HEAD)
Author: Jan Mercl <0xjnml@gmail.com>
Date:   Wed Feb 22 20:27:45 2023 +0100

    support sqlite 3.41 on linux targets
0:jnml@e5-1650:~/src/modernc.org/libc$ git diff
diff --git a/libc_linux.go b/libc_linux.go
index 1c2f482..ac1f08d 100644
--- a/libc_linux.go
+++ b/libc_linux.go
@@ -332,19 +332,19 @@ func Xwrite(t *TLS, fd int32, buf uintptr, count types.Size_t) types.Ssize_t {
                var n uintptr
                switch n, _, err = unix.Syscall(unix.SYS_WRITE, uintptr(fd), buf, uintptr(count)); err {
                case 0:
-                       // if dmesgs {
-                       //      // dmesg("%v: %d %#x: %#x\n%s", origin(1), fd, count, n, hex.Dump(GoBytes(buf, int(n))))
-                       //      dmesg("%v: %d %#x: %#x", origin(1), fd, count, n)
-                       // }
+                       if dmesgs {
+                               // dmesg("%v: %d %#x: %#x\n%s", origin(1), fd, count, n, hex.Dump(GoBytes(buf, int(n))))
+                               dmesg("%v: %d %#x: %#x", origin(1), fd, count, n)
+                       }
                        return types.Ssize_t(n)
                case errno.EAGAIN:
                        // nop
                }
        }

-       // if dmesgs {
-       //      dmesg("%v: fd %v, count %#x: %v", origin(1), fd, count, err)
-       // }
+       if dmesgs {
+               dmesg("%v: fd %v, count %#x: %v", origin(1), fd, count, err)
+       }
        t.setErrno(err)
        return -1
 }
0:jnml@e5-1650:~/src/modernc.org/libc$

We need to tell the Go build system to use our local, patched/debug libc:

0:jnml@e5-1650:~/src/modernc.org/sqlite$ go work use $(go env GOPATH)/src/modernc.org/libc
0:jnml@e5-1650:~/src/modernc.org/sqlite$ go work use .

And run the test again:

0:jnml@e5-1650:~/src/modernc.org/sqlite$ rm -f /tmp/libc.log ; go test -v -tags=libc.dmesg -run TestScalar ; ls -l /tmp/libc.log
test binary compiled for linux/amd64
=== RUN   TestScalar
--- PASS: TestScalar (0.26s)
PASS
ok   modernc.org/sqlite 0.285s
-rw-r--r-- 1 jnml jnml 918 Apr  6 11:29 /tmp/libc.log
0:jnml@e5-1650:~/src/modernc.org/sqlite$ cat /tmp/libc.log
[11910 sqlite.test] 2023-04-06 11:29:13.143589542 +0200 CEST m=+0.000689270
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x200: 0x200
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0xc: 0xc
[11910 sqlite.test] libc_linux.go:337:Xwrite: 7 0x1000: 0x1000
[11910 sqlite.test] libc_linux.go:337:Xwrite: 7 0x1000: 0x1000
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x200: 0x200
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x4: 0x4
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x1000: 0x1000
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x4: 0x4
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x4: 0x4
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x1000: 0x1000
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0x4: 0x4
[11910 sqlite.test] libc_linux.go:337:Xwrite: 8 0xc: 0xc
[11910 sqlite.test] libc_linux.go:337:Xwrite: 7 0x1000: 0x1000
[11910 sqlite.test] libc_linux.go:337:Xwrite: 7 0x1000: 0x1000
0:jnml@e5-1650:~/src/modernc.org/sqlite$

Sqlite documentation

See https://sqlite.org/docs.html

Index

• Constants
• Variables
• func Limit(c *sql.Conn, id int, newVal int) (r int, err error)
• func MustRegisterCollationUtf8(zName string, impl func(left, right string) int)
• func MustRegisterDeterministicScalarFunction(zFuncName string, nArg int32, ...)
• func MustRegisterFunction(zFuncName string, impl *FunctionImpl)
• func MustRegisterPageCache(m PageCache)
• func MustRegisterScalarFunction(zFuncName string, nArg int32, ...)
• func RegisterCollationUtf8(zName string, impl func(left, right string) int) error
• func RegisterConnectionHook(fn ConnectionHookFn)
• func RegisterDeterministicScalarFunction(zFuncName string, nArg int32, ...) (err error)
• func RegisterFunction(zFuncName string, impl *FunctionImpl) error
• func RegisterPageCache(m PageCache) error
• func RegisterScalarFunction(zFuncName string, nArg int32, ...) (err error)
• type AggregateFunction
• type Backup
  • func (b *Backup) Commit() (driver.Conn, error)
  • func (b *Backup) Finish() error
  • func (b *Backup) PageCount() int
  • func (b *Backup) Remaining() int
  • func (b *Backup) Step(n int32) (bool, error)
• type Cache
• type ColumnInfo
• type CommitHookFn
• type ConnectionHookFn
• type DBStatus
• type DBStatusOp
• type Driver
  • func (d *Driver) Open(name string) (conn driver.Conn, err error)
  • func (d *Driver) RegisterConnectionHook(fn ConnectionHookFn)
• type Error
  • func (e *Error) Code() int
  • func (e *Error) Error() string
• type ExecQuerierContext
• type FetchMode
• type FileControl
• type FunctionContext
• type FunctionImpl
• type HookRegisterer
• type Page
• type PageCache
• type PreUpdateHookFn
• type RollbackHookFn
• type SQLitePreUpdateData
  • func (d *SQLitePreUpdateData) Count() int
  • func (d *SQLitePreUpdateData) Depth() int
  • func (d *SQLitePreUpdateData) New(dest ...any) error
  • func (d *SQLitePreUpdateData) Old(dest ...any) error

Constants

const (
	DBStatusLookasideUsed     = DBStatusOp(sqlite3.SQLITE_DBSTATUS_LOOKASIDE_USED)
	DBStatusCacheUsed         = DBStatusOp(sqlite3.SQLITE_DBSTATUS_CACHE_USED)
	DBStatusSchemaUsed        = DBStatusOp(sqlite3.SQLITE_DBSTATUS_SCHEMA_USED)
	DBStatusStmtUsed          = DBStatusOp(sqlite3.SQLITE_DBSTATUS_STMT_USED)
	DBStatusLookasideHit      = DBStatusOp(sqlite3.SQLITE_DBSTATUS_LOOKASIDE_HIT)
	DBStatusLookasideMissSize = DBStatusOp(sqlite3.SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE)
	DBStatusLookasideMissFull = DBStatusOp(sqlite3.SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL)
	DBStatusCacheHit          = DBStatusOp(sqlite3.SQLITE_DBSTATUS_CACHE_HIT)
	DBStatusCacheMiss         = DBStatusOp(sqlite3.SQLITE_DBSTATUS_CACHE_MISS)
	DBStatusCacheWrite        = DBStatusOp(sqlite3.SQLITE_DBSTATUS_CACHE_WRITE)
	DBStatusDeferredFKs       = DBStatusOp(sqlite3.SQLITE_DBSTATUS_DEFERRED_FKS)
	DBStatusCacheUsedShared   = DBStatusOp(sqlite3.SQLITE_DBSTATUS_CACHE_USED_SHARED)
	DBStatusCacheSpill        = DBStatusOp(sqlite3.SQLITE_DBSTATUS_CACHE_SPILL)
	DBStatusTempbufSpill      = DBStatusOp(sqlite3.SQLITE_DBSTATUS_TEMPBUF_SPILL)
)

DBStatus* are the operations accepted by DBStatus.Status. They report their value differently depending on the op:

• DBStatusLookasideUsed: current is the lookaside memory in use now; high is its high-water mark. The reset flag rebases the high-water mark to current. This is the only op that maintains a high-water mark.
• Memory-usage ops (DBStatusCacheUsed, DBStatusSchemaUsed, DBStatusStmtUsed, DBStatusCacheUsedShared): current is the bytes in use now; high is always 0; the reset flag is ignored.
• Running-counter ops (DBStatusCacheHit, DBStatusCacheMiss, DBStatusCacheWrite, DBStatusCacheSpill, DBStatusTempbufSpill): current is the cumulative count (bytes spilled, for DBStatusTempbufSpill); high is always 0. The reset flag zeroes current.
• Lookaside event ops (DBStatusLookasideHit, DBStatusLookasideMissSize, DBStatusLookasideMissFull): the count is reported in high, not current (current is always 0). The reset flag zeroes high.
• DBStatusDeferredFKs: current is 1 if the connection has unresolved deferred foreign-key constraints, else 0; high is always 0; the reset flag is ignored.

Variables

var ErrPageCacheConflict = errors.New(
	"sqlite: a different page cache module is already registered")

ErrPageCacheConflict is returned when a different PageCache has already been registered in this process. The same module value may be re-registered without error, which lets multiple library imports share a singleton without coordination.

var ErrPageCacheTooLate = errors.New(
	"sqlite: RegisterPageCache called after first Open; " +
		"SQLITE_CONFIG_PCACHE2 must be installed before any connection is opened")

ErrPageCacheTooLate is returned by RegisterPageCache when a SQLite connection has already been opened in this process. SQLITE_CONFIG_PCACHE2 must be installed before sqlite3_initialize, which is called implicitly by the first sqlite3_open_v2. After that point SQLite returns SQLITE_MISUSE and the engine cannot switch its page cache backend.

var (
	// ErrorCodeString maps Error.Code() to its string representation.
	ErrorCodeString = map[int]string{
		sqlite3.SQLITE_ABORT:             "Callback routine requested an abort (SQLITE_ABORT)",
		sqlite3.SQLITE_AUTH:              "Authorization denied (SQLITE_AUTH)",
		sqlite3.SQLITE_BUSY:              "The database file is locked (SQLITE_BUSY)",
		sqlite3.SQLITE_CANTOPEN:          "Unable to open the database file (SQLITE_CANTOPEN)",
		sqlite3.SQLITE_CONSTRAINT:        "Abort due to constraint violation (SQLITE_CONSTRAINT)",
		sqlite3.SQLITE_CORRUPT:           "The database disk image is malformed (SQLITE_CORRUPT)",
		sqlite3.SQLITE_DONE:              "sqlite3_step() has finished executing (SQLITE_DONE)",
		sqlite3.SQLITE_EMPTY:             "Internal use only (SQLITE_EMPTY)",
		sqlite3.SQLITE_ERROR:             "Generic error (SQLITE_ERROR)",
		sqlite3.SQLITE_FORMAT:            "Not used (SQLITE_FORMAT)",
		sqlite3.SQLITE_FULL:              "Insertion failed because database is full (SQLITE_FULL)",
		sqlite3.SQLITE_INTERNAL:          "Internal logic error in SQLite (SQLITE_INTERNAL)",
		sqlite3.SQLITE_INTERRUPT:         "Operation terminated by sqlite3_interrupt()(SQLITE_INTERRUPT)",
		sqlite3.SQLITE_IOERR | (1 << 8):  "(SQLITE_IOERR_READ)",
		sqlite3.SQLITE_IOERR | (10 << 8): "(SQLITE_IOERR_DELETE)",
		sqlite3.SQLITE_IOERR | (11 << 8): "(SQLITE_IOERR_BLOCKED)",
		sqlite3.SQLITE_IOERR | (12 << 8): "(SQLITE_IOERR_NOMEM)",
		sqlite3.SQLITE_IOERR | (13 << 8): "(SQLITE_IOERR_ACCESS)",
		sqlite3.SQLITE_IOERR | (14 << 8): "(SQLITE_IOERR_CHECKRESERVEDLOCK)",
		sqlite3.SQLITE_IOERR | (15 << 8): "(SQLITE_IOERR_LOCK)",
		sqlite3.SQLITE_IOERR | (16 << 8): "(SQLITE_IOERR_CLOSE)",
		sqlite3.SQLITE_IOERR | (17 << 8): "(SQLITE_IOERR_DIR_CLOSE)",
		sqlite3.SQLITE_IOERR | (2 << 8):  "(SQLITE_IOERR_SHORT_READ)",
		sqlite3.SQLITE_IOERR | (3 << 8):  "(SQLITE_IOERR_WRITE)",
		sqlite3.SQLITE_IOERR | (4 << 8):  "(SQLITE_IOERR_FSYNC)",
		sqlite3.SQLITE_IOERR | (5 << 8):  "(SQLITE_IOERR_DIR_FSYNC)",
		sqlite3.SQLITE_IOERR | (6 << 8):  "(SQLITE_IOERR_TRUNCATE)",
		sqlite3.SQLITE_IOERR | (7 << 8):  "(SQLITE_IOERR_FSTAT)",
		sqlite3.SQLITE_IOERR | (8 << 8):  "(SQLITE_IOERR_UNLOCK)",
		sqlite3.SQLITE_IOERR | (9 << 8):  "(SQLITE_IOERR_RDLOCK)",
		sqlite3.SQLITE_IOERR:             "Some kind of disk I/O error occurred (SQLITE_IOERR)",
		sqlite3.SQLITE_LOCKED | (1 << 8): "(SQLITE_LOCKED_SHAREDCACHE)",
		sqlite3.SQLITE_LOCKED:            "A table in the database is locked (SQLITE_LOCKED)",
		sqlite3.SQLITE_MISMATCH:          "Data type mismatch (SQLITE_MISMATCH)",
		sqlite3.SQLITE_MISUSE:            "Library used incorrectly (SQLITE_MISUSE)",
		sqlite3.SQLITE_NOLFS:             "Uses OS features not supported on host (SQLITE_NOLFS)",
		sqlite3.SQLITE_NOMEM:             "A malloc() failed (SQLITE_NOMEM)",
		sqlite3.SQLITE_NOTADB:            "File opened that is not a database file (SQLITE_NOTADB)",
		sqlite3.SQLITE_NOTFOUND:          "Unknown opcode in sqlite3_file_control() (SQLITE_NOTFOUND)",
		sqlite3.SQLITE_NOTICE:            "Notifications from sqlite3_log() (SQLITE_NOTICE)",
		sqlite3.SQLITE_PERM:              "Access permission denied (SQLITE_PERM)",
		sqlite3.SQLITE_PROTOCOL:          "Database lock protocol error (SQLITE_PROTOCOL)",
		sqlite3.SQLITE_RANGE:             "2nd parameter to sqlite3_bind out of range (SQLITE_RANGE)",
		sqlite3.SQLITE_READONLY:          "Attempt to write a readonly database (SQLITE_READONLY)",
		sqlite3.SQLITE_ROW:               "sqlite3_step() has another row ready (SQLITE_ROW)",
		sqlite3.SQLITE_SCHEMA:            "The database schema changed (SQLITE_SCHEMA)",
		sqlite3.SQLITE_TOOBIG:            "String or BLOB exceeds size limit (SQLITE_TOOBIG)",
		sqlite3.SQLITE_WARNING:           "Warnings from sqlite3_log() (SQLITE_WARNING)",
	}
)

Functions

func Limit

func Limit(c *sql.Conn, id int, newVal int) (r int, err error)

Limit calls sqlite3_limit, see the docs at https://www.sqlite.org/c3ref/limit.html for details.

To get a sql.Conn from a *sql.DB, use (*sql.DB).Conn(). Limits are bound to the particular instance of 'c', so getting a new connection only to pass it to Limit is possibly not useful above querying what are the various configured default values.

func MustRegisterCollationUtf8

func MustRegisterCollationUtf8(
	zName string,
	impl func(left, right string) int,
)

MustRegisterCollationUtf8 is like RegisterCollationUtf8 but panics on error.

func MustRegisterDeterministicScalarFunction

func MustRegisterDeterministicScalarFunction(
	zFuncName string,
	nArg int32,
	xFunc func(ctx *FunctionContext, args []driver.Value) (driver.Value, error),
)

MustRegisterDeterministicScalarFunction is like RegisterDeterministicScalarFunction but panics on error.

func MustRegisterFunction

func MustRegisterFunction(
	zFuncName string,
	impl *FunctionImpl,
)

MustRegisterFunction is like RegisterFunction but panics on error.

func MustRegisterPageCache

func MustRegisterPageCache(m PageCache)

MustRegisterPageCache is like RegisterPageCache but panics on any error. Intended for init() use where a missing page cache is fatal. Mirrors the precedent set by MustRegisterDeterministicScalarFunction.

func MustRegisterScalarFunction

func MustRegisterScalarFunction(
	zFuncName string,
	nArg int32,
	xFunc func(ctx *FunctionContext, args []driver.Value) (driver.Value, error),
)

MustRegisterScalarFunction is like RegisterScalarFunction but panics on error.

func RegisterCollationUtf8

func RegisterCollationUtf8(
	zName string,
	impl func(left, right string) int,
) error

RegisterCollationUtf8 makes a Go function available as a collation named zName. impl receives two UTF-8 strings: left and right. The result needs to be:

- 0 if left == right - 1 if left < right - +1 if left > right

impl must always return the same result given the same inputs. Additionally, it must have the following properties for all strings A, B and C: - if A==B, then B==A - if A==B and B==C, then A==C - if A<B, then B>A - if A<B and B<C, then A<C.

The new collation will be available to all new connections opened after executing RegisterCollationUtf8.

func RegisterConnectionHook

func RegisterConnectionHook(fn ConnectionHookFn)

RegisterConnectionHook registers a function to be called after each connection is opened. This is called after all the connection has been set up.

func RegisterDeterministicScalarFunction

func RegisterDeterministicScalarFunction(
	zFuncName string,
	nArg int32,
	xFunc func(ctx *FunctionContext, args []driver.Value) (driver.Value, error),
) (err error)

RegisterDeterministicScalarFunction registers a deterministic scalar function named zFuncName with nArg arguments. Passing -1 for nArg indicates the function is variadic. A deterministic function means that the function always gives the same output when the input parameters are the same.

The new function will be available to all new connections opened after executing RegisterDeterministicScalarFunction.

func RegisterFunction

func RegisterFunction(
	zFuncName string,
	impl *FunctionImpl,
) error

RegisterFunction registers a function named zFuncName with nArg arguments. Passing -1 for nArg indicates the function is variadic. The FunctionImpl determines whether the function is deterministic or not, and whether it is a scalar function (when Scalar is defined) or an aggregate function (when Scalar is not defined and MakeAggregate is defined).

The new function will be available to all new connections opened after executing RegisterFunction.

func RegisterPageCache

func RegisterPageCache(m PageCache) error

RegisterPageCache installs m as the process-global SQLite page cache via SQLITE_CONFIG_PCACHE2. It MUST be called before the first sql.Open or driver.Open in the program.

Concurrency contract:

• Safe to call concurrently with itself and with other Register* entry points.
• Blocks until any sql.Open calls currently in progress complete. Trade-off: a Register call may block for the duration of an in-flight Open. WAL recovery or cold-file-lock contention can make that wait visible.
• Once any connection has been opened, returns ErrPageCacheTooLate without mutating the global module slot.
• Calling twice with the same module value is a no-op success. Calling twice with a different value returns ErrPageCacheConflict.
• A failed first install is sticky: every subsequent Register call returns the same error. Mutating the module fields after the first successful Register is silently ignored because SQLite has already copied the C methods table.

func RegisterScalarFunction

func RegisterScalarFunction(
	zFuncName string,
	nArg int32,
	xFunc func(ctx *FunctionContext, args []driver.Value) (driver.Value, error),
) (err error)

RegisterScalarFunction registers a scalar function named zFuncName with nArg arguments. Passing -1 for nArg indicates the function is variadic.

The new function will be available to all new connections opened after executing RegisterScalarFunction.

Types

type AggregateFunction

type AggregateFunction interface {
	// Step is called for each row of an aggregate function's SQL
	// invocation. The argument Values are not valid past the return of the
	// function. When the aggregate was registered with
	// [FunctionImpl.VolatileArgs] set to true, string and []byte arguments
	// in rowArgs are zero-copy views into SQLite-owned memory and retaining
	// them produces silent data corruption — see [FunctionImpl.VolatileArgs]
	// for the full safety contract.
	Step(ctx *FunctionContext, rowArgs []driver.Value) error

	// WindowInverse is called to remove the oldest presently aggregated
	// result of Step from the current window. The arguments are those
	// passed to Step for the row being removed. The argument Values are not
	// valid past the return of the function. The same
	// [FunctionImpl.VolatileArgs] caveat applies as for Step.
	WindowInverse(ctx *FunctionContext, rowArgs []driver.Value) error

	// WindowValue is called to get the current value of an aggregate
	// function. This is used to return the final value of the function,
	// whether it is used as a window function or not.
	WindowValue(ctx *FunctionContext) (driver.Value, error)

	// Final is called after all of the aggregate function's input rows have
	// been stepped through. No other methods will be called on the
	// AggregateFunction after calling Final. WindowValue returns the value
	// from the function.
	Final(ctx *FunctionContext)
}

An AggregateFunction is an invocation of an aggregate or window function. See the documentation for aggregate function callbacks and application-defined window functions for an overview.

type Backup

type Backup struct {
	// contains filtered or unexported fields
}

Backup object is used to manage progress and cleanup an online backup. It is returned by NewBackup or NewRestore.

func (*Backup) Commit

func (b *Backup) Commit() (driver.Conn, error)

Commit releases all resources associated with the Backup object but does not close the destination database connection.

The destination database connection is returned to the caller or an error if raised. It is the responsibility of the caller to handle the connection closure.

func (*Backup) Finish

func (b *Backup) Finish() error

Finish releases all resources associated with the Backup object. The Backup object is invalid and may not be used following a call to Finish.

func (*Backup) PageCount

func (b *Backup) PageCount() int

PageCount returns the total number of pages in the source database at the conclusion of the most recent Backup.Step call. Pair with Backup.Remaining to compute progress as a fraction (PageCount - Remaining) / PageCount.

See https://www.sqlite.org/c3ref/backup_finish.html.

func (*Backup) Remaining

func (b *Backup) Remaining() int

Remaining returns the number of source-database pages still to be backed up at the conclusion of the most recent Backup.Step call. The value is useful for driving progress UIs that need to estimate how much work is left.

If Step has not yet been called on this Backup, or if the most recent Step returned false (SQLITE_DONE), Remaining returns 0.

See https://www.sqlite.org/c3ref/backup_finish.html.

func (*Backup) Step

func (b *Backup) Step(n int32) (bool, error)

Step will copy up to n pages between the source and destination databases specified by the backup object. If n is negative, all remaining source pages are copied. If it successfully copies n pages and there are still more pages to be copied, then the function returns true with no error. If it successfully finishes copying all pages from source to destination, then it returns false with no error. If an error occurs while running, then an error is returned.

type Cache

type Cache interface {
	// SetSize advises the cache of the new target page count
	// (PRAGMA cache_size). The cache is free to ignore the hint.
	SetSize(n int)

	// PageCount returns the number of pages currently held (pinned and
	// unpinned combined).
	PageCount() int

	// Fetch returns the Page for key or nil per FetchMode. The binding
	// calls Fetch on every SQLite request and compares the returned
	// Page to the value held under key from the previous Fetch; when
	// they match (the cache retained the entry) the same
	// sqlite3_pcache_page stub is handed back to SQLite, and when they
	// differ the binding retires the stale stub and mints a new one.
	// The implementation is therefore free to evict and re-allocate
	// between Fetches without leaking a stale view to SQLite.
	Fetch(key uint32, mode FetchMode) Page

	// Unpin tells the cache that the engine is finished using the
	// Page for now. If discard is true SQLite has no further need
	// for the page and the cache may release its memory; if discard
	// is false the cache may retain the entry for reuse or release
	// it at its discretion (the normal way to honour a bounded
	// cache_size). SQLite never refcounts: one Unpin call is final
	// regardless of how many Fetches preceded.
	Unpin(p Page, discard bool)

	// Rekey changes the key under which p is filed from oldKey to
	// newKey. If an entry already exists at newKey it must be discarded
	// in the same call; SQLite guarantees the colliding entry is not
	// pinned at the moment of Rekey.
	Rekey(p Page, oldKey, newKey uint32)

	// Truncate discards every entry whose key is greater than or equal
	// to limit, including pinned entries. This is the only callback
	// permitted to evict a pinned page.
	Truncate(limit uint32)

	// Destroy releases every page and any resources owned by this
	// Cache. After Destroy returns the binding will not call any
	// other method on this Cache.
	Destroy()

	// Shrink hints the cache to release as much heap as possible. The
	// implementation is not obligated to free anything; this is purely
	// a memory-pressure advisory.
	Shrink()
}

Cache is one database's worth of cached pages. All callbacks for a single Cache are serialised by the SQLite engine: this driver opens every connection SQLITE_OPEN_FULLMUTEX without shared-cache mode, and database/sql never invokes one driver.Conn from two goroutines, so an implementation does not need to synchronise per-Cache state against concurrent calls.

Implementations should NOT call RegisterPageCache directly or transitively. Callbacks run under the openGate read lock that the Open path holds, and a re-entrant Register would deadlock on the gate's write lock.

type ColumnInfo

type ColumnInfo struct {
	// Name is the column's name, from sqlite3_column_name.
	Name string
	// DeclType is the declared column type, from sqlite3_column_decltype.
	DeclType string
	// DatabaseName is the name of the source database, from sqlite3_column_database_name.
	// Empty if the column does not resolve to an unambiguous reference to a single database column,
	// such as expressions, function calls, or constants.
	DatabaseName string
	// TableName is the name of the source table, from sqlite3_column_table_name.
	// Empty if the column does not resolve to an unambiguous reference to a single database column,
	// such as expressions, function calls, or constants.
	TableName string
	// OriginName is the name of the source column name, from sqlite3_column_origin_name.
	// Empty if the column does not resolve to an unambiguous reference to a single database column,
	// such as expressions, function calls, or constants.
	OriginName string
}

ColumnInfo describes one output column of a prepared SQL statement.

See https://www.sqlite.org/c3ref/column_database_name.html for more details.

type CommitHookFn

type CommitHookFn func() int32

type ConnectionHookFn

type ConnectionHookFn func(
	conn ExecQuerierContext,
	dsn string,
) error

ConnectionHookFn function type for a connection hook on the Driver. Connection hooks are called after the connection has been set up.

type DBStatus

type DBStatus interface {
	// Status returns the current and high-water values of the per-connection
	// counter identified by op. When reset is true the counter is reset after
	// the read; which value the reset affects depends on the op's family, see
	// the DBStatus* constant documentation. The returned error is non-nil only
	// when SQLite rejects op as out of range.
	Status(op DBStatusOp, reset bool) (current, high int, err error)
}

DBStatus exposes sqlite3_db_status, the per-connection runtime counters (cache hit/miss/write/spill rates, schema and prepared-statement memory, lookaside usage, deferred foreign keys). Reach it through the database/sql escape hatch, the same way as FileControl:

err := sqlConn.Raw(func(dc any) error {
	cur, _, err := dc.(sqlite.DBStatus).Status(sqlite.DBStatusCacheSpill, false)
	if err != nil {
		return err
	}
	// use cur
	return nil
})

type DBStatusOp

type DBStatusOp int32

DBStatusOp identifies a per-connection runtime counter readable through DBStatus.Status. The values mirror the SQLITE_DBSTATUS_* verbs of the C API; the distinct type keeps a counter from a different family (for example a file-control or db-config op) from compiling in its place.

See https://www.sqlite.org/c3ref/c_dbstatus_options.html for the per-op semantics.

type Driver

type Driver struct {
	// contains filtered or unexported fields
}

Driver implements database/sql/driver.Driver.

Registration functions and methods must be called before the first call to Open.

func (*Driver) Open

func (d *Driver) Open(name string) (conn driver.Conn, err error)

Open returns a new connection to the database. The name is a string in a driver-specific format.

Open may return a cached connection (one previously closed), but doing so is unnecessary; the sql package maintains a pool of idle connections for efficient re-use.

The returned connection is only used by one goroutine at a time.

The name may be a filename, e.g., "/tmp/mydata.sqlite", or a URI, in which case it may include a '?' followed by one or more query parameters. For example, "file:///tmp/mydata.sqlite?_pragma=foreign_keys(1)&_time_format=sqlite". The supported query parameters are:

_pragma: Each value will be run as a "PRAGMA ..." statement (with the PRAGMA keyword added for you). May be specified more than once, '&'-separated. For more information on supported PRAGMAs see: https://www.sqlite.org/pragma.html

_time_format: The name of a format to use when writing time values to the database. The currently supported values are (1) "sqlite" for YYYY-MM-DD HH:MM:SS.SSS[+-]HH:MM (format 4 from https://www.sqlite.org/lang_datefunc.html#time_values with sub-second precision and timezone specifier) and (2) "datetime" for YYYY-MM-DD HH:MM:SS (format 3, matching the output of SQLite's datetime() function). If this parameter is not specified, then the default String() format will be used.

_time_integer_format: The name of a integer format to use when writing time values. By default, the time is stored as string and the format can be set with _time_format parameter. If _time_integer_format is set, the time will be stored as an integer and the integer value will depend on the integer format. If you decide to set both _time_format and _time_integer_format, the time will be converted as integer and the _time_format value will be ignored. Currently the supported value are "unix","unix_milli", "unix_micro" and "unix_nano", which corresponds to seconds, milliseconds, microseconds or nanoseconds since unixepoch (1 January 1970 00:00:00 UTC).

_inttotime: Enable conversion of time column (DATE, DATETIME,TIMESTAMP) from integer to time if the field contain integer (int64).

_texttotime: Enable ColumnTypeScanType to report time.Time instead of string for TEXT columns declared as DATE, DATETIME, TIME, or TIMESTAMP.

_timezone: A timezone to use for all time reads and writes, such as "UTC". The value is parsed by time.LoadLocation. Writes will convert to the timezone before formatting as a string; it does not impact _inttotime integer values, as they always use UTC. Reads will interpret timezone-less strings as being in this timezone. Values that are in a known timezone, such as a string with a timezone specifier or an integer with _inttotime (specified to be in UTC), will be converted to this timezone.

_txlock: The locking behavior to use when beginning a transaction. May be "deferred" (the default), "immediate", or "exclusive" (case insensitive). See: https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions

_dqs: Opt-in toggle for SQLite's double-quoted string literal compatibility quirk on the connection. Accepts the values strconv.ParseBool understands ("0"/"1", "false"/"true", "f"/"t", case-insensitive). When absent or set to a true value, SQLite's built-in behavior is unchanged: a double-quoted identifier that fails to resolve is silently re-interpreted as a string literal. When set to a false value, SQLITE_DBCONFIG_DQS_DDL and SQLITE_DBCONFIG_DQS_DML are both turned off via sqlite3_db_config so that mistakes hidden by the legacy fallback surface as a parse error instead. See: https://www.sqlite.org/quirks.html#dblquote and https://gitlab.com/cznic/sqlite/-/issues/61

_error_rc: Opt-in error-string reporting mode for synthesised errors. Accepts the values strconv.ParseBool understands ("0"/"1", "false"/"true", "f"/"t", case-insensitive). When absent or set to a false value, the legacy "errstr: errmsg (rc)" form is preserved byte-for-byte: the canonical sqlite3_errstr(rc) and the connection's sqlite3_errmsg(db) are concatenated even when the latter belongs to a different operation, which can read as misleading on open-time failures such as SQLITE_CANTOPEN reporting "out of memory". When set to a true value, the appended errmsg is suppressed if sqlite3_extended_errcode(db) is inconsistent with the operation rc (full match first, primary code as fallback); in that case the canonical errstr(rc) is used alone. The Code() returned by the driver's *Error is unchanged in either mode. The parameter is parsed before sqlite3_open_v2 so open-time errors are covered. See https://gitlab.com/cznic/sqlite/-/issues/230.

func (*Driver) RegisterConnectionHook

func (d *Driver) RegisterConnectionHook(fn ConnectionHookFn)

RegisterConnectionHook registers a function to be called after each connection is opened. This is called after all the connection has been set up.

type Error

type Error struct {
	// contains filtered or unexported fields
}

Error represents sqlite library error code.

func (*Error) Code

func (e *Error) Code() int

Code returns the sqlite result code for this error.

func (*Error) Error

func (e *Error) Error() string

Error implements error.

type ExecQuerierContext

type ExecQuerierContext interface {
	driver.ExecerContext
	driver.QueryerContext
}

type FetchMode

type FetchMode int32

FetchMode tells Cache.Fetch how aggressively to allocate when the requested key is absent. It matches the createFlag of SQLite's xFetch (https://sqlite.org/c3ref/pcache_methods2.html).

const (
	// FetchLookup looks up an existing entry. Fetch returns nil if the
	// key is not in cache; no allocation is performed.
	FetchLookup FetchMode = 0
	// FetchCreateEasy allocates only if it can be done without effort
	// (no eviction, no memory pressure). Fetch may return nil; SQLite
	// will then spill dirty pages and retry with FetchCreateForce.
	FetchCreateEasy FetchMode = 1
	// FetchCreateForce allocates unconditionally and may evict to make
	// room. Fetch should return nil only on a genuine out-of-memory
	// condition.
	FetchCreateForce FetchMode = 2
)

type FileControl

type FileControl interface {
	// Set or query SQLITE_FCNTL_PERSIST_WAL, returns set mode or query result
	FileControlPersistWAL(dbName string, mode int) (int, error)
	// Query SQLITE_FCNTL_DATA_VERSION, returns the pager-cache data version
	// for dbName. The value changes whenever the contents of the database
	// file change, which makes it suitable for cache-invalidation use cases.
	// See
	// https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntldataversion.
	FileControlDataVersion(dbName string) (uint32, error)
}

Access to sqlite3_file_control

type FunctionContext

type FunctionContext struct {
	// contains filtered or unexported fields
}

FunctionContext represents the context user defined functions execute in. Fields and/or methods of this type may get addedd in the future.

type FunctionImpl

type FunctionImpl struct {
	// NArgs is the required number of arguments that the function accepts.
	// If NArgs is negative, then the function is variadic.
	NArgs int32

	// If Deterministic is true, the function must always give the same
	// output when the input parameters are the same. This enables functions
	// to be used in additional contexts like the WHERE clause of partial
	// indexes and enables additional optimizations.
	//
	// See https://sqlite.org/c3ref/c_deterministic.html#sqlitedeterministic
	// for more details.
	Deterministic bool

	// Scalar is called when a scalar function is invoked in SQL. The
	// argument Values are not valid past the return of the function.
	Scalar func(ctx *FunctionContext, args []driver.Value) (driver.Value, error)

	// MakeAggregate is called at the beginning of each evaluation of an
	// aggregate function.
	MakeAggregate func(ctx FunctionContext) (AggregateFunction, error)

	// VolatileArgs is an opt-in performance flag that eliminates the per-call
	// allocation of string and []byte argument bodies. When true, the driver
	// passes argument strings and byte slices as zero-copy views into
	// SQLite-owned memory instead of Go-allocated copies.
	//
	// Setting this is unsafe unless the user-provided Scalar / Step /
	// WindowInverse callbacks treat string and []byte arguments as strictly
	// transient: they must not be retained past the return of the call,
	// neither directly (storing the slice or string in a struct field, map,
	// channel, or outer-scope variable) nor indirectly (passing them to
	// something that captures them, including most concurrency primitives).
	//
	// Retaining a volatile argument produces silent data corruption: SQLite
	// reuses the underlying buffer for the next row, so on a later read every
	// retained value will appear to hold the contents of the most recent row.
	// The Go race detector cannot catch this because UDF execution is
	// sequential on a single goroutine; the corruption is deterministic and
	// invisible to -race.
	//
	// As a guard against accidental capture, callbacks that must retain
	// values across rows should copy:
	//
	//	saved := append([]byte(nil), args[0].([]byte)...) // []byte
	//	saved := string(append([]byte(nil), args[0].(string)...)) // string, no aliasing
	//
	// When in doubt, leave VolatileArgs at its default (false) — the driver
	// already pools the argument-slice header (issue #226), so the per-row
	// overhead with VolatileArgs=false is one make([]byte) per BLOB column
	// and one libc.GoString per TEXT column, not a fresh slice header.
	//
	// Similarly, do not re-enter SQLite on the same connection while a
	// volatile argument is in scope. A nested Query/Exec on the same conn
	// can cause SQLite to reuse the underlying value buffers, so a volatile
	// string or []byte read before the nested call may alias different
	// bytes after it returns.
	//
	// VolatileArgs has no effect on integer, float, time, or NULL arguments.
	VolatileArgs bool
}

FunctionImpl describes an application-defined SQL function. If Scalar is set, it is treated as a scalar function; otherwise, it is treated as an aggregate function using MakeAggregate.

type HookRegisterer

type HookRegisterer interface {
	RegisterPreUpdateHook(PreUpdateHookFn)
	RegisterCommitHook(CommitHookFn)
	RegisterRollbackHook(RollbackHookFn)
}

type Page

type Page interface {
	Buf() unsafe.Pointer
	Extra() unsafe.Pointer
}

Page is one cache entry. Buf and Extra return pointers into implementation-owned memory that MUST remain valid and at the same addresses for the duration of the pin: from the Fetch that returned the Page until the matching Unpin. While the page is unpinned the implementation is free to release the memory; the next Fetch for the same key will be consulted afresh and may return either the same Page (memory retained) or a different one.

The memory MUST be off-heap: libc.Xmalloc, sqlite3_malloc, mmap, or an equivalent allocator outside the Go heap. Go-heap memory is forbidden, including memory pinned with runtime.Pinner: SQLite stores Extra addresses inside its own C structures and performs interior pointer arithmetic on them (it overlays PgHdr at the head of Extra), which trips Go's checkptr enforcement under -race the moment _sqlite3PcacheFetchFinish runs. Pinned slices preserve the allocation but lose checkptr provenance through the binding's unsafe.Pointer round-trip, so the failure surfaces only under the race detector and not in normal test runs.

Page values are compared by the binding to detect whether the implementation retained or replaced the cached entry across a Fetch cycle, so Page MUST be a comparable type. Pointer-backed implementations (the natural shape) satisfy this automatically.

Buf must be at least pageSize bytes and is where SQLite stores the database page contents. Extra must be at least extraSize bytes (the extraSize passed to PageCache.Create, which already includes SQLite's PgHdr overhead) and is treated by SQLite as opaque scratch space. Implementations should zero Extra on a freshly-allocated Page so SQLite's PgHdr backpointer is read as null; the binding does not touch Extra contents.

type PageCache

type PageCache interface {
	Create(pageSize, extraSize int, purgeable bool) (Cache, error)
}

PageCache is the factory for per-database Cache instances. SQLite calls Create once per open database; each call must return a fresh Cache with the given pageSize and extraSize. The extraSize includes SQLite's private PgHdr overhead and must be honoured as the opaque size of every Page's Extra buffer.

purgeable is advisory: when false (in-memory databases), SQLite will only call Unpin with discard=true and the cache is permitted to free every page on Unpin. When true, the cache may retain unpinned pages for re-use.

type PreUpdateHookFn

type PreUpdateHookFn func(SQLitePreUpdateData)

type RollbackHookFn

type RollbackHookFn func()

type SQLitePreUpdateData

type SQLitePreUpdateData struct {
	Op           int32
	DatabaseName string
	TableName    string
	OldRowID     int64
	NewRowID     int64
	// contains filtered or unexported fields
}

func (*SQLitePreUpdateData) Count

func (d *SQLitePreUpdateData) Count() int

Count returns the number of columns in the row

func (*SQLitePreUpdateData) Depth

func (d *SQLitePreUpdateData) Depth() int

Depth returns the source path of the write, see sqlite3_preupdate_depth()

func (*SQLitePreUpdateData) New

func (d *SQLitePreUpdateData) New(dest ...any) error

New populates dest with the replacement row data. This works similar to database/sql's Rows.Scan()

func (*SQLitePreUpdateData) Old

func (d *SQLitePreUpdateData) Old(dest ...any) error

Old populates dest with the row data to be replaced. This works similar to database/sql's Rows.Scan()