lmdb

package
v1.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 11, 2017 License: BSD-3-Clause Imports: 10 Imported by: 92

Documentation

Overview

Package lmdb provides bindings to the lmdb C API. The package bindings are fairly low level and are designed to provide a minimal interface that prevents misuse to a reasonable extent. When in doubt refer to the C documentation as a reference. 

http://www.lmdb.tech/doc/
http://www.lmdb.tech/doc/starting.html
http://www.lmdb.tech/doc/modules.html

Environment

An LMDB environment holds named databases (key-value stores). An environment is represented as one file on the filesystem (though often a corresponding lock file exists).

LMDB recommends setting an environment's size as large as possible at the time of creation. On filesystems that support sparse files this should not adversely affect disk usage. Resizing an environment is possible but must be handled with care when concurrent access is involved.

Note that the package lmdb forces all Env objects to be opened with the NoTLS (MDB_NOTLS) flag. Without this flag LMDB would not be practically usable in Go (in the author's opinion). However, even for environments opened with this flag there are caveats regarding how transactions are used (see Caveats below).

Databases

A database in an LMDB environment is an ordered key-value store that holds arbitrary binary data. Typically the keys are unique but duplicate keys may be allowed (DupSort), in which case the values for each duplicate key are ordered.

A single LMDB environment can have multiple named databases. But there is also a 'root' (unnamed) database that can be used to store data. Use caution storing data in the root database when named databases are in use. The root database serves as an index for named databases.

A database is referenced by an opaque handle known as its DBI which must be opened inside a transaction with the OpenDBI or OpenRoot methods. DBIs may be closed but it is not required. Typically, applications acquire handles for all their databases immediately after opening an environment and retain them for the lifetime of the process.

Transactions

View (readonly) transactions in LMDB operate on a snapshot of the database at the time the transaction began. The number of simultaneously active view transactions is bounded and configured when the environment is initialized.

Update (read-write) transactions are serialized in LMDB. Attempts to create update transactions block until a lock may be obtained. Update transactions can create subtransactions which may be rolled back independently from their parent.

The lmdb package supplies managed and unmanaged transactions. Managed transactions do not require explicit calling of Abort/Commit and are provided through the Env methods Update, View, and RunTxn. The BeginTxn method on Env creates an unmanaged transaction but its use is not advised in most applications.

Caveats

Write transactions (those created without the Readonly flag) must be created in a goroutine that has been locked to its thread by calling the function runtime.LockOSThread. Futhermore, all methods on such transactions must be called from the goroutine which created them. This is a fundamental limitation of LMDB even when using the NoTLS flag (which the package always uses). The Env.Update method assists the programmer by calling runtime.LockOSThread automatically but it cannot sufficiently abstract write transactions to make them completely safe in Go.

A goroutine must never create a write transaction if the application programmer cannot determine whether the goroutine is locked to an OS thread. This is a consequence of goroutine restrictions on write transactions and limitations in the runtime's thread locking implementation. In such situations updates desired by the goroutine in question must be proxied by a goroutine with a known state (i.e. "locked" or "unlocked"). See the included examples for more details about dealing with such situations.

Example

This example demonstrates a complete workflow for an lmdb environment. The Env is first created. After being configured the Env is mapped to memory. Once mapped, database handles are opened and normal database operations begin. 

package main

import (
	"fmt"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

func main() {
	// create an environment and make sure it is eventually closed.
	env, err := lmdb.NewEnv()
	if err != nil {
		// ...
	}
	defer env.Close()

	// configure and open the environment.  most configuration must be done
	// before opening the environment.
	err = env.SetMaxDBs(1)
	if err != nil {
		// ..
	}
	err = env.SetMapSize(1 << 30)
	if err != nil {
		// ..
	}
	err = env.Open("/path/to/db/", 0, 0644)
	if err != nil {
		// ..
	}

	// open a database that can be used as long as the enviroment is mapped.
	var dbi lmdb.DBI
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		dbi, err = txn.CreateDBI("example")
		return err
	})
	if err != nil {
		// ...
	}

	// the database is now ready for use.  read the value for a key and print
	// it to standard output.
	err = env.View(func(txn *lmdb.Txn) (err error) {
		v, err := txn.Get(dbi, []byte("hello"))
		if err != nil {
			return err
		}
		fmt.Println(string(v))
		return nil
	})
	if err != nil {
		// ...
	}
}

Output:
Example (Threads)

This example demonstrates the simplest (and most naive) way to issue database updates from a goroutine for which it cannot be known ahead of time whether runtime.LockOSThread has been called. 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	// Create a function that wraps env.Update and sends the resulting error
	// over a channel.  Because env.Update is called our update function will
	// call runtime.LockOSThread to safely issue the update operation.
	update := func(res chan<- error, op lmdb.TxnOp) {
		res <- env.Update(op)
	}

	// ...

	// Now, in goroutine where we cannot determine if we are locked to a
	// thread, we can create a new goroutine to process the update(s) we want.
	res := make(chan error)
	go update(res, func(txn *lmdb.Txn) (err error) {
		return txn.Put(dbi, []byte("thisUpdate"), []byte("isSafe"), 0)
	})
	err = <-res
	if err != nil {
		panic(err)
	}
}

Output:
Example (Worker)

This example demonstrates a more sophisticated way to issue database updates from a goroutine for which it cannot be known ahead of time whether runtime.LockOSThread has been called. 

package main

import (
	"runtime"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	// Wrap operations in a struct that can be passed over a channel to a
	// worker goroutine.
	type lmdbop struct {
		op  lmdb.TxnOp
		res chan<- error
	}
	worker := make(chan *lmdbop)
	update := func(op lmdb.TxnOp) error {
		res := make(chan error)
		worker <- &lmdbop{op, res}
		return <-res
	}

	// Start issuing update operations in a goroutine in which we know
	// runtime.LockOSThread can be called and we can safely issue transactions.
	go func() {
		runtime.LockOSThread()
		defer runtime.LockOSThread()

		// Perform each operation as we get to it.  Because this goroutine is
		// already locked to a thread, env.UpdateLocked is called to avoid
		// premature unlocking of the goroutine from its thread.
		for op := range worker {
			op.res <- env.UpdateLocked(op.op)
		}
	}()

	// ...

	// In another goroutine, where we cannot determine if we are locked to a
	// thread already.
	err = update(func(txn *lmdb.Txn) (err error) {
		// This function will execute safely in the worker goroutine, which is
		// locked to its thread.
		return txn.Put(dbi, []byte("thisUpdate"), []byte("isSafe"), 0)
	})
}

Output:

Index

Examples

Constants

View Source 
const (
	First        = C.MDB_FIRST          // The first item.
	FirstDup     = C.MDB_FIRST_DUP      // The first value of current key (DupSort).
	GetBoth      = C.MDB_GET_BOTH       // Get the key as well as the value (DupSort).
	GetBothRange = C.MDB_GET_BOTH_RANGE // Get the key and the nearsest value (DupSort).
	GetCurrent   = C.MDB_GET_CURRENT    // Get the key and value at the current position.
	GetMultiple  = C.MDB_GET_MULTIPLE   // Get up to a page dup values for key at current position (DupFixed).
	Last         = C.MDB_LAST           // Last item.
	LastDup      = C.MDB_LAST_DUP       // Position at last value of current key (DupSort).
	Next         = C.MDB_NEXT           // Next value.
	NextDup      = C.MDB_NEXT_DUP       // Next value of the current key (DupSort).
	NextMultiple = C.MDB_NEXT_MULTIPLE  // Get key and up to a page of values from the next cursor position (DupFixed).
	NextNoDup    = C.MDB_NEXT_NODUP     // The first value of the next key (DupSort).
	Prev         = C.MDB_PREV           // The previous item.
	PrevDup      = C.MDB_PREV_DUP       // The previous item of the current key (DupSort).
	PrevNoDup    = C.MDB_PREV_NODUP     // The last data item of the previous key (DupSort).
	Set          = C.MDB_SET            // The specified key.
	SetKey       = C.MDB_SET_KEY        // Get key and data at the specified key.
	SetRange     = C.MDB_SET_RANGE      // The first key no less than the specified key.
)

These flags are used exclusively for Cursor.Get.

View Source 
const (
	Current     = C.MDB_CURRENT     // Replace the item at the current key position (Cursor only)
	NoDupData   = C.MDB_NODUPDATA   // Store the key-value pair only if key is not present (DupSort).
	NoOverwrite = C.MDB_NOOVERWRITE // Store a new key-value pair only if key is not present.
	Append      = C.MDB_APPEND      // Append an item to the database.
	AppendDup   = C.MDB_APPENDDUP   // Append an item to the database (DupSort).
)

The MDB_MULTIPLE and MDB_RESERVE flags are special and do not fit the calling pattern of other calls to Put. They are not exported because they require special methods, PutMultiple and PutReserve in which the flag is implied and does not need to be passed.

View Source 
const (
	FixedMap    = C.MDB_FIXEDMAP   // Danger zone. Map memory at a fixed address.
	NoSubdir    = C.MDB_NOSUBDIR   // Argument to Open is a file, not a directory.
	Readonly    = C.MDB_RDONLY     // Used in several functions to denote an object as readonly.
	WriteMap    = C.MDB_WRITEMAP   // Use a writable memory map.
	NoMetaSync  = C.MDB_NOMETASYNC // Don't fsync metapage after commit.
	NoSync      = C.MDB_NOSYNC     // Don't fsync after commit.
	MapAsync    = C.MDB_MAPASYNC   // Flush asynchronously when using the WriteMap flag.
	NoTLS       = C.MDB_NOTLS      // Danger zone. When unset reader locktable slots are tied to their thread.
	NoLock      = C.MDB_NOLOCK     // Danger zone. LMDB does not use any locks.
	NoReadahead = C.MDB_NORDAHEAD  // Disable readahead. Requires OS support.
	NoMemInit   = C.MDB_NOMEMINIT  // Disable LMDB memory initialization.
)

These flags are used exclusively for Env types and are set with Env.Open. Some flags may be set/unset later using Env.Set/.Unset methods. Others will produce syscall.EINVAL. Refer to the C documentation for detailed information.

View Source 
const (
	ReverseKey = C.MDB_REVERSEKEY // Use reverse string keys.
	DupSort    = C.MDB_DUPSORT    // Use sorted duplicates.
	DupFixed   = C.MDB_DUPFIXED   // Duplicate items have a fixed size (DupSort).
	ReverseDup = C.MDB_REVERSEDUP // Reverse duplicate values (DupSort).
	Create     = C.MDB_CREATE     // Create DB if not already existing.
)

This flags are used exclusively for Txn.OpenDBI and Txn.OpenRoot. The Create flag must always be supplied when opening a non-root DBI for the first time.

BUG(bmatsuo): MDB_INTEGERKEY and MDB_INTEGERDUP aren't usable. I'm not sure they would be faster with the cgo bridge. They need to be tested and benchmarked.

View Source 
const (
	CopyCompact = C.MDB_CP_COMPACT // Perform compaction while copying
)

These flags are exclusively used in the Env.CopyFlags and Env.CopyFDFlags methods.

Variables

This section is empty.

Functions

func IsErrno 

func IsErrno(err error, errno Errno) bool

IsErrno returns true if err's errno is the given errno.

func IsErrnoFn 

func IsErrnoFn(err error, fn func(error) bool) bool

IsErrnoFn calls fn on the error underlying err and returns the result. If err is an *OpError then err.Errno is passed to fn. Otherwise err is passed directly to fn.

func IsErrnoSys 

func IsErrnoSys(err error, errno syscall.Errno) bool

IsErrnoSys returns true if err's errno is the given errno.

func IsMapFull 

func IsMapFull(err error) bool

IsMapFull returns true if the environment map size has been reached.

func IsMapResized 

func IsMapResized(err error) bool

IsMapResized returns true if the environment has grown too large for the current map after being resized by another process.

func IsNotExist 

func IsNotExist(err error) bool

IsNotExist returns true the path passed to the Env.Open method does not exist.

func IsNotFound 

func IsNotFound(err error) bool

IsNotFound returns true if the key requested in Txn.Get or Cursor.Get does not exist or if the Cursor reached the end of the database without locating a value (EOF).

func Version 

func Version() (major, minor, patch int, s string)

Version return the major, minor, and patch version numbers of the LMDB C library and a string representation of the version.

See mdb_version.

func VersionString 

func VersionString() string

VersionString returns a string representation of the LMDB C library version.

See mdb_version.

Types

type Cursor 

type Cursor struct {
	// contains filtered or unexported fields
}

Cursor operates on data inside a transaction and holds a position in the database.

See MDB_cursor.

func (*Cursor) Close 

func (c *Cursor) Close()

Close the cursor handle and clear the finalizer on c. Cursors belonging to write transactions are closed automatically when the transaction is terminated.

See mdb_cursor_close.

func (*Cursor) Count 

func (c *Cursor) Count() (uint64, error)

Count returns the number of duplicates for the current key.

See mdb_cursor_count.

Example

This example demonstrates how to iterate a database opened with the DupSort flag and get the number of values present for each distinct key. 

package main

import (
	"fmt"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}
		defer cur.Close()

		for {
			k, _, err := cur.Get(nil, nil, lmdb.NextNoDup)
			if lmdb.IsNotFound(err) {
				return nil
			}
			if err != nil {
				return err
			}

			numdup, err := cur.Count()
			if err != nil {
				return err
			}
			fmt.Printf("%d values for key %q", numdup, k)
		}
	})
}

Output:

func (*Cursor) DBI 

func (c *Cursor) DBI() DBI

DBI returns the cursor's database handle. If c has been closed than an invalid DBI is returned.

func (*Cursor) Del 

func (c *Cursor) Del(flags uint) error

Del deletes the item referred to by the cursor from the database.

See mdb_cursor_del.

Example

This example demonstrates how to delete all elements in a database with a key less than a given value (an RFC3339 timestamp in this case). 

package main

import (
	"bytes"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	before := []byte("2014-05-06T03:04:02Z")
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}
		defer cur.Close()

		for {
			k, _, err := cur.Get(nil, nil, lmdb.Next)
			if lmdb.IsNotFound(err) {
				return nil
			}
			if err != nil {
				return err
			}

			if bytes.Compare(k, before) != -1 {
				return nil
			}

			err = cur.Del(0)
			if err != nil {
				return err
			}
		}
	})
}

Output:

func (*Cursor) Get 

func (c *Cursor) Get(setkey, setval []byte, op uint) (key, val []byte, err error)

Get retrieves items from the database. If c.Txn().RawRead is true the slices returned by Get reference readonly sections of memory that must not be accessed after the transaction has terminated.

In a Txn with RawRead set to true the Set op causes the returned key to share its memory with setkey (making it writable memory). In a Txn with RawRead set to false the Set op returns key values with memory distinct from setkey, as is always the case when using RawRead.

Get ignores setval if setkey is empty.

See mdb_cursor_get.

Example

This simple example shows how to iterate a database. The Next flag may be used without an initial call passing the First flag. 

package main

import (
	"fmt"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	err = env.View(func(txn *lmdb.Txn) (err error) {
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}
		defer cur.Close()

		for {
			k, v, err := cur.Get(nil, nil, lmdb.Next)
			if lmdb.IsNotFound(err) {
				return nil
			}
			if err != nil {
				return err
			}

			fmt.Printf("%s %s\n", k, v)
		}
	})
}

Output:
Example (DupFixed)

This simple example shows how to iterate a database opened with the DupSort|DupSort flags. It is not necessary to use the GetMultiple flag before passing the NextMultiple flag. 

package main

import (
	"fmt"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	err = env.View(func(txn *lmdb.Txn) (err error) {
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}
		defer cur.Close()

		for {
			k, first, err := cur.Get(nil, nil, lmdb.NextNoDup)
			if lmdb.IsNotFound(err) {
				return nil
			}
			if err != nil {
				return err
			}

			stride := len(first)

			for {
				_, v, err := cur.Get(nil, nil, lmdb.NextMultiple)
				if lmdb.IsNotFound(err) {
					break
				}
				if err != nil {
					return err
				}

				multi := lmdb.WrapMulti(v, stride)
				for i := 0; i < multi.Len(); i++ {
					fmt.Printf("%s %s\n", k, multi.Val(i))
				}
			}
		}
	})
}

Output:
Example (DupSort)

This example shows how duplicates can be processed using LMDB. It is possible to iterate all key-value pairs (including duplicate key values) by passing Next. But if special handling of duplicates is needed it may be beneficial to use NextNoDup or NextDup. 

package main

import (
	"log"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	err = env.View(func(txn *lmdb.Txn) (err error) {
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}

		for {
			k, v, err := cur.Get(nil, nil, lmdb.NextNoDup)
			if lmdb.IsNotFound(err) {
				// the database was exausted
				return nil
			} else if err != nil {
				return err
			}

			// process duplicates
			var dups [][]byte
			for {
				dups = append(dups, v)

				_, v, err = cur.Get(nil, nil, lmdb.NextDup)
				if lmdb.IsNotFound(err) {
					break
				} else if err != nil {
					return err
				}
			}

			log.Printf("%q %q", k, dups)
		}
	})
}

Output:
Example (Reverse)

This simple example shows how to iterate a database in reverse. As when passing the Next flag, the Prev flag may be used without an initial call using Last. 

package main

import (
	"fmt"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	err = env.View(func(txn *lmdb.Txn) (err error) {
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}
		defer cur.Close()

		for {
			k, v, err := cur.Get(nil, nil, lmdb.Prev)
			if lmdb.IsNotFound(err) {
				return nil
			}
			if err != nil {
				return err
			}

			fmt.Printf("%s %s\n", k, v)
		}
	})
}

Output:

func (*Cursor) Put 

func (c *Cursor) Put(key, val []byte, flags uint) error

Put stores an item in the database.

See mdb_cursor_put.

func (*Cursor) PutMulti 

func (c *Cursor) PutMulti(key []byte, page []byte, stride int, flags uint) error

PutMulti stores a set of contiguous items with stride size under key. PutMulti panics if len(page) is not a multiple of stride. The cursor's database must be DupFixed and DupSort.

See mdb_cursor_put.

Example

This example shows how to write a page of contiguous, fixed-size values to a database opened with DupSort|DupFixed. It doesn't matter if the values are sorted. Values will be stored in sorted order. 

package main

import (
	"bytes"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	key := []byte("k")
	items := [][]byte{
		[]byte("v0"),
		[]byte("v2"),
		[]byte("v1"),
	}
	page := bytes.Join(items, nil)
	stride := 2

	err = env.Update(func(txn *lmdb.Txn) (err error) {
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}
		defer cur.Close()

		return cur.PutMulti(key, page, stride, 0)
	})
}

Output:

func (*Cursor) PutReserve 

func (c *Cursor) PutReserve(key []byte, n int, flags uint) ([]byte, error)

PutReserve returns a []byte of length n that can be written to, potentially avoiding a memcopy. The returned byte slice is only valid in txn's thread, before it has terminated.

func (*Cursor) Renew 

func (c *Cursor) Renew(txn *Txn) error

Renew associates readonly cursor with txn.

See mdb_cursor_renew.

Example

This example shows a trivial case using Renew to service read requests on a database. Close must be called when the cursor will no longer be renewed. Before using Renew benchmark your application to understand its benefits. 

package main

import (
	"log"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error

func main() {
	var cur *lmdb.Cursor
	err = env.View(func(txn *lmdb.Txn) (err error) {
		dbi, err := txn.OpenRoot(0)
		if err != nil {
			return err
		}
		cur, err = txn.OpenCursor(dbi)
		return err
	})
	if err != nil {
		panic(err)
	}

	keys := make(chan []byte)
	go func() {

		// Close must called when the cursor is no longer needed.
		defer cur.Close()

		for key := range keys {
			err := env.View(func(txn *lmdb.Txn) (err error) {
				err = cur.Renew(txn)
				if err != nil {
					return err
				}

				// retrieve the number of items in the database with the given
				// key (DupSort).
				count := uint64(0)
				_, _, err = cur.Get(key, nil, lmdb.SetKey)
				if lmdb.IsNotFound(err) {
					err = nil
				} else if err == nil {
					count, err = cur.Count()
				}
				if err != nil {
					return err
				}

				log.Printf("%d %q", count, key)

				return nil
			})
			if err != nil {
				panic(err)
			}
		}
	}()

	// ...
}

Output:

func (*Cursor) Txn 

func (c *Cursor) Txn() *Txn

Txn returns the cursor's transaction.

type DBI 

type DBI C.MDB_dbi

DBI is a handle for a database in an Env.

See MDB_dbi

type Env 

type Env struct {
	// contains filtered or unexported fields
}

Env is opaque structure for a database environment. A DB environment supports multiple databases, all residing in the same shared-memory map.

See MDB_env.

Example

This example shows the general workflow of LMDB. An environment is created and configured before being opened. After the environment is opened its databases are created and their handles are saved for use in future transactions. 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

func main() {
	// open the LMDB environment and configure common options like its size and
	// maximum number of databases.
	env, err := lmdb.NewEnv()
	if err != nil {
		// ...
	}
	err = env.SetMapSize(100 * 1024 * 1024) // 100MB
	if err != nil {
		// ...
	}
	err = env.SetMaxDBs(1)
	if err != nil {
		// ...
	}

	// open the environment only after the it has been configured.  some
	// settings may only be called before the environment is opened where
	// others may have caveats.
	err = env.Open("mydb/", 0, 0664)
	if err != nil {
		// ...
	}
	defer env.Close()

	var dbi lmdb.DBI
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		// open a database, creating it if necessary.  the database is stored
		// outside the transaction via closure and can be use after the
		// transaction is committed.
		dbi, err = txn.OpenDBI("exampledb", lmdb.Create)
		if err != nil {
			return err
		}

		// commit the transaction, writing an entry for the newly created
		// database if it was just created and allowing the dbi to be used in
		// future transactions.
		return nil
	})
	if err != nil {
		panic(err)
	}
}

Output:

func NewEnv 

func NewEnv() (*Env, error)

NewEnv allocates and initializes a new Env.

See mdb_env_create.

func (*Env) BeginTxn 

func (env *Env) BeginTxn(parent *Txn, flags uint) (*Txn, error)

BeginTxn is an unsafe, low-level method to initialize a new transaction on env. The Txn returned by BeginTxn is unmanaged and must be terminated by calling either its Abort or Commit methods to ensure that its resources are released.

BeginTxn does not call runtime.LockOSThread. Unless the Readonly flag is passed goroutines must call runtime.LockOSThread before calling BeginTxn and the returned Txn must not have its methods called from another goroutine. Failure to meet these restrictions can have undefined results that may include deadlocking your application.

Instead of calling BeginTxn users should prefer calling the View and Update methods, which assist in management of Txn objects and provide OS thread locking required for write transactions.

A finalizer detects unreachable, live transactions and logs thems to standard error. The transactions are aborted, but their presence should be interpreted as an application error which should be patched so transactions are terminated explicitly. Unterminated transactions can adversly effect database performance and cause the database to grow until the map is full.

See mdb_txn_begin.

func (*Env) Close 

func (env *Env) Close() error

Close shuts down the environment, releases the memory map, and clears the finalizer on env.

See mdb_env_close.

func (*Env) CloseDBI 

func (env *Env) CloseDBI(db DBI)

CloseDBI closes the database handle, db. Normally calling CloseDBI explicitly is not necessary.

It is the caller's responsibility to serialize calls to CloseDBI.

See mdb_dbi_close.

func (*Env) Copy 

func (env *Env) Copy(path string) error

Copy copies the data in env to an environment at path.

See mdb_env_copy.

Example

This example uses env.Copy to periodically create atomic backups of an environment. A real implementation would need to solve problems with failures, remote persistence, purging old backups, etc. But the core loop would have the same form. 

package main

import (
	"fmt"
	"os"
	"time"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error
var stop chan struct{}

func main() {
	go func(backup <-chan time.Time) {
		for {
			select {
			case <-backup:
				now := time.Now().UTC()
				backup := fmt.Sprintf("backup-%s", now.Format(time.RFC3339))
				os.Mkdir(backup, 0755)
				err = env.Copy(backup)
				if err != nil {
					// ...
					continue
				}
			case <-stop:
				return
			}
		}
	}(time.Tick(time.Hour))
}

Output:

func (*Env) CopyFD added in v1.1.1 

func (env *Env) CopyFD(fd uintptr) error

CopyFD copies env to the the file descriptor fd.

See mdb_env_copyfd.

func (*Env) CopyFDFlag added in v1.1.1 

func (env *Env) CopyFDFlag(fd uintptr, flags uint) error

CopyFDFlag copies env to the file descriptor fd, with options.

See mdb_env_copyfd2.

func (*Env) CopyFlag 

func (env *Env) CopyFlag(path string, flags uint) error

CopyFlag copies the data in env to an environment at path created with flags.

See mdb_env_copy2.

func (*Env) FD added in v1.2.0 

func (env *Env) FD() (uintptr, error)

FD returns the open file descriptor (or Windows file handle) for the given environment. An error is returned if the environment has not been successfully Opened (where C API just retruns an invalid handle).

See mdb_env_get_fd.

func (*Env) Flags 

func (env *Env) Flags() (uint, error)

Flags returns the flags set in the environment.

See mdb_env_get_flags.

func (*Env) Info 

func (env *Env) Info() (*EnvInfo, error)

Info returns information about the environment.

See mdb_env_info.

func (*Env) MaxKeySize 

func (env *Env) MaxKeySize() int

MaxKeySize returns the maximum allowed length for a key.

See mdb_env_get_maxkeysize.

func (*Env) MaxReaders 

func (env *Env) MaxReaders() (int, error)

MaxReaders returns the maximum number of reader slots for the environment.

See mdb_env_get_maxreaders.

func (*Env) Open 

func (env *Env) Open(path string, flags uint, mode os.FileMode) error

Open an environment handle. If this function fails Close() must be called to discard the Env handle. Open passes flags|NoTLS to mdb_env_open.

See mdb_env_open.

func (*Env) Path 

func (env *Env) Path() (string, error)

Path returns the path argument passed to Open. Path returns a non-nil error if env.Open() was not previously called.

See mdb_env_get_path.

func (*Env) ReaderCheck 

func (env *Env) ReaderCheck() (int, error)

ReaderCheck clears stale entries from the reader lock table and returns the number of entries cleared.

See mdb_reader_check()

func (*Env) ReaderList added in v1.1.1 

func (env *Env) ReaderList(fn func(string) error) error

ReaderList dumps the contents of the reader lock table as text. Readers start on the second line as space-delimited fields described by the first line.

See mdb_reader_list.

func (*Env) RunTxn 

func (env *Env) RunTxn(flags uint, fn TxnOp) error

RunTxn creates a new Txn and calls fn with it as an argument. Run commits the transaction if fn returns nil otherwise the transaction is aborted. Because RunTxn terminates the transaction goroutines should not retain references to it or its data after fn returns.

RunTxn does not call runtime.LockOSThread. Unless the Readonly flag is passed the calling goroutine should ensure it is locked to its thread and any goroutines started by fn must not call methods on the Txn object it is passed.

See mdb_txn_begin.

func (*Env) SetFlags 

func (env *Env) SetFlags(flags uint) error

SetFlags sets flags in the environment.

See mdb_env_set_flags.

func (*Env) SetMapSize 

func (env *Env) SetMapSize(size int64) error

SetMapSize sets the size of the environment memory map.

See mdb_env_set_mapsize.

Example

This example demonstrates how an application typically uses Env.SetMapSize. The call to Env.SetMapSize() is made before calling env.Open(). Any calls after calling Env.Open() must take special care to synchronize with other goroutines. 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

func main() {
	env, err := lmdb.NewEnv()
	if err != nil {
		// ...
	}

	// set the memory map size (maximum database size) to 1GB.
	err = env.SetMapSize(1 << 30)
	if err != nil {
		// ...
	}

	err = env.Open("mydb", 0, 0644)
	if err != nil {
		// ...
	}
	// ...
}

Output:
Example (MapResized)

This example demonstrates how to handle a MapResized error, encountered after another process has called mdb_env_set_mapsize (Env.SetMapSize). Applications which don't expect another process to resize the mmap don't need to check for the MapResized error.

The example is simplified for clarity. Many real applications will need to synchronize calls to Env.SetMapSize using something like a sync.RWMutex to ensure there are no active readonly transactions (those opened successfully before MapResized was encountered). 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

// These values can be used as no-op placeholders in examples.
func doUpdate(txn *lmdb.Txn) error { return nil }

func main() {
retry:
	err := env.Update(doUpdate)
	if lmdb.IsMapResized(err) {
		// If concurrent read transactions are possible then a sync.RWMutex
		// must be used here to ensure they all terminate before calling
		// env.SetMapSize().
		err = env.SetMapSize(0)
		if err != nil {
			panic(err)
		}

		// retry the update. a goto is not necessary but it simplifies error
		// handling with minimal overhead.
		goto retry
	} else if err != nil {
		// ...
	}
	// ...
}

Output:

func (*Env) SetMaxDBs 

func (env *Env) SetMaxDBs(size int) error

SetMaxDBs sets the maximum number of named databases for the environment.

See mdb_env_set_maxdbs.

func (*Env) SetMaxReaders 

func (env *Env) SetMaxReaders(size int) error

SetMaxReaders sets the maximum number of reader slots in the environment.

See mdb_env_set_maxreaders.

func (*Env) Stat 

func (env *Env) Stat() (*Stat, error)

Stat returns statistics about the environment.

See mdb_env_stat.

func (*Env) Sync 

func (env *Env) Sync(force bool) error

Sync flushes buffers to disk. If force is true a synchronous flush occurs and ignores any NoSync or MapAsync flag on the environment.

See mdb_env_sync.

func (*Env) UnsetFlags 

func (env *Env) UnsetFlags(flags uint) error

UnsetFlags clears flags in the environment.

See mdb_env_set_flags.

func (*Env) Update 

func (env *Env) Update(fn TxnOp) error

Update calls fn with a writable transaction. Update commits the transaction if fn returns a nil error otherwise Update aborts the transaction and returns the error.

Update calls runtime.LockOSThread to lock the calling goroutine to its thread and until fn returns and the transaction has been terminated, at which point runtime.UnlockOSThread is called. If the calling goroutine is already known to be locked to a thread, use UpdateLocked instead to avoid premature unlocking of the goroutine.

Neither Update nor UpdateLocked cannot be called safely from a goroutine where it isn't known if runtime.LockOSThread has been called. In such situations writes must either be done in a newly created goroutine which can be safely locked, or through a worker goroutine that accepts updates to apply and delivers transaction results using channels. See the package documentation and examples for more details.

Goroutines created by the operation fn must not use methods on the Txn object that fn is passed. Doing so would have undefined and unpredictable results for your program (likely including data loss, deadlock, etc).

Any call to Commit, Abort, Reset or Renew on a Txn created by Update will panic.

Example

This example shows the basic use of Env.Update, the primary method lmdb-go provides for to store data in an Env. 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	// It is not safe to call runtime.LockOSThread here because Env.Update
	// would later cause premature unlocking of the goroutine.  If an
	// application requires that goroutines be locked to threads before
	// starting an an update on the Env then you must use Env.UpdateLocked
	// instead of Env.Update.

	err = env.Update(func(txn *lmdb.Txn) (err error) {
		// Write several keys to the database within one transaction.  If
		// either write fails and this function returns an error then readers
		// in other transactions will not see either value because Env.Update
		// aborts the transaction if an error is returned.

		err = txn.Put(dbi, []byte("x"), []byte("hello"), 0)
		if err != nil {
			return err
		}
		err = txn.Put(dbi, []byte("y"), []byte("goodbye"), 0)
		if err != nil {
			return err
		}
		return nil
	})
	if err != nil {
		panic(err)
	}
}

Output:

func (*Env) UpdateLocked 

func (env *Env) UpdateLocked(fn TxnOp) error

UpdateLocked behaves like Update but does not lock the calling goroutine to its thread. UpdateLocked should be used if the calling goroutine is already locked to its thread for another purpose.

Neither Update nor UpdateLocked cannot be called safely from a goroutine where it isn't known if runtime.LockOSThread has been called. In such situations writes must either be done in a newly created goroutine which can be safely locked, or through a worker goroutine that accepts updates to apply and delivers transaction results using channels. See the package documentation and examples for more details.

Goroutines created by the operation fn must not use methods on the Txn object that fn is passed. Doing so would have undefined and unpredictable results for your program (likely including data loss, deadlock, etc).

Any call to Commit, Abort, Reset or Renew on a Txn created by UpdateLocked will panic.

Example

In this example, another C library requires the application to lock a goroutine to its thread. When writing to the Env this goroutine must use the method Env.UpdateLocked to prevent premature unlocking of the goroutine.

Note that there is no way for a goroutine to determine if it is locked to a thread failure to call Env.UpdateLocked in a scenario like this can lead to unspecified and hard to debug failure modes for your application. 

package main

import (
	"runtime"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var dbi lmdb.DBI

var err error

func main() {
	runtime.LockOSThread()
	defer runtime.UnlockOSThread()

	// ... Do something that requires the goroutine be locked to its thread.

	// Create a transaction that will not interfere with thread locking and
	// issue some writes with it.
	err = env.UpdateLocked(func(txn *lmdb.Txn) (err error) {
		err = txn.Put(dbi, []byte("x"), []byte("hello"), 0)
		if err != nil {
			return err
		}
		err = txn.Put(dbi, []byte("y"), []byte("goodbye"), 0)
		if err != nil {
			return err
		}
		return nil
	})
	if err != nil {
		panic(err)
	}

	// ... Do something requiring the goroutine still be locked to its thread.
}

Output:

func (*Env) View 

func (env *Env) View(fn TxnOp) error

View creates a readonly transaction with a consistent view of the environment and passes it to fn. View terminates its transaction after fn returns. Any error encountered by View is returned.

Unlike with Update transactions, goroutines created by fn are free to call methods on the Txn passed to fn provided they are synchronized in their accesses (e.g. using a mutex or channel).

Any call to Commit, Abort, Reset or Renew on a Txn created by View will panic.

type EnvInfo 

type EnvInfo struct {
	MapSize    int64 // Size of the data memory map
	LastPNO    int64 // ID of the last used page
	LastTxnID  int64 // ID of the last committed transaction
	MaxReaders uint  // maximum number of threads for the environment
	NumReaders uint  // maximum number of threads used in the environment
}

EnvInfo contains information an environment.

See MDB_envinfo.

type Errno 

type Errno C.int

Errno is an error type that represents the (unique) errno values defined by LMDB. Other errno values (such as EINVAL) are represented with type syscall.Errno. On Windows, LMDB return codes are translated into portable syscall.Errno constants (e.g. syscall.EINVAL, syscall.EACCES, etc.).

Most often helper functions such as IsNotFound may be used instead of dealing with Errno values directly. 

lmdb.IsNotFound(err)
lmdb.IsErrno(err, lmdb.TxnFull)
lmdb.IsErrnoSys(err, syscall.EINVAL)
lmdb.IsErrnoFn(err, os.IsPermission)

const (
	KeyExist        Errno = C.MDB_KEYEXIST
	NotFound        Errno = C.MDB_NOTFOUND
	PageNotFound    Errno = C.MDB_PAGE_NOTFOUND
	Corrupted       Errno = C.MDB_CORRUPTED
	Panic           Errno = C.MDB_PANIC
	VersionMismatch Errno = C.MDB_VERSION_MISMATCH
	Invalid         Errno = C.MDB_INVALID
	MapFull         Errno = C.MDB_MAP_FULL
	DBsFull         Errno = C.MDB_DBS_FULL
	ReadersFull     Errno = C.MDB_READERS_FULL
	TLSFull         Errno = C.MDB_TLS_FULL
	TxnFull         Errno = C.MDB_TXN_FULL
	CursorFull      Errno = C.MDB_CURSOR_FULL
	PageFull        Errno = C.MDB_PAGE_FULL
	MapResized      Errno = C.MDB_MAP_RESIZED
	Incompatible    Errno = C.MDB_INCOMPATIBLE
	BadRSlot        Errno = C.MDB_BAD_RSLOT
	BadTxn          Errno = C.MDB_BAD_TXN
	BadValSize      Errno = C.MDB_BAD_VALSIZE
	BadDBI          Errno = C.MDB_BAD_DBI
)

The most common error codes do not need to be handled explicity. Errors can be checked through helper functions IsNotFound, IsMapFull, etc, Otherwise they should be checked using the IsErrno function instead of direct comparison because they will typically be wrapped with an OpError.

func (Errno) Error 

func (e Errno) Error() string

type Multi 

type Multi struct {
	// contains filtered or unexported fields
}

Multi is a wrapper for a contiguous page of sorted, fixed-length values passed to Cursor.PutMulti or retrieved using Cursor.Get with the GetMultiple/NextMultiple flag.

Multi values are only useful in databases opened with DupSort|DupFixed.

func WrapMulti 

func WrapMulti(page []byte, stride int) *Multi

WrapMulti converts a page of contiguous values with stride size into a Multi. WrapMulti panics if len(page) is not a multiple of stride. 

_, val, _ := cursor.Get(nil, nil, lmdb.FirstDup)
_, page, _ := cursor.Get(nil, nil, lmdb.GetMultiple)
multi := lmdb.WrapMulti(page, len(val))

See mdb_cursor_get and MDB_GET_MULTIPLE.

func (*Multi) Len 

func (m *Multi) Len() int

Len returns the number of values in the Multi.

func (*Multi) Page 

func (m *Multi) Page() []byte

Page returns the Multi page data as a raw slice of bytes with length m.Size().

func (*Multi) Size 

func (m *Multi) Size() int

Size returns the total size of the Multi data and is equal to 

m.Len()*m.Stride()

func (*Multi) Stride 

func (m *Multi) Stride() int

Stride returns the length of an individual value in the m.

func (*Multi) Val 

func (m *Multi) Val(i int) []byte

Val returns the value at index i. Val panics if i is out of range.

func (*Multi) Vals 

func (m *Multi) Vals() [][]byte

Vals returns a slice containing the values in m. The returned slice has length m.Len() and each item has length m.Stride().

type OpError 

type OpError struct {
	Op    string
	Errno error
}

OpError is an error returned by the C API. Not all errors returned by lmdb-go have type OpError but typically they do. The Errno field will either have type Errno or syscall.Errno.

func (*OpError) Error 

func (err *OpError) Error() string

Error implements the error interface.

type Stat 

type Stat struct {
	PSize         uint   // Size of a database page. This is currently the same for all databases.
	Depth         uint   // Depth (height) of the B-tree
	BranchPages   uint64 // Number of internal (non-leaf) pages
	LeafPages     uint64 // Number of leaf pages
	OverflowPages uint64 // Number of overflow pages
	Entries       uint64 // Number of data items
}

Stat contains database status information.

See MDB_stat.

type Txn 

type Txn struct {
	// If RawRead is true []byte values retrieved from Get() calls on the Txn
	// and its cursors will point directly into the memory-mapped structure.
	// Such slices will be readonly and must only be referenced wthin the
	// transaction's lifetime.
	RawRead bool
	// contains filtered or unexported fields
}

Txn is a database transaction in an environment.

WARNING: A writable Txn is not threadsafe and may only be used in the goroutine that created it.

See MDB_txn.

Example

This example shows the basic operations used when creating and working with Txn types. 

package main

import (
	"fmt"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

func main() {
	// open a database.
	var dbi lmdb.DBI
	err := env.Update(func(txn *lmdb.Txn) (err error) {
		dbi, err = txn.OpenDBI("exampledb", lmdb.Create)
		// the transaction will be commited if the database was successfully
		// opened/created.
		return err
	})
	if err != nil {
		// ...
	}

	err = env.Update(func(txn *lmdb.Txn) (err error) {
		return txn.Put(dbi, []byte("k"), []byte("v"), 0)
	})
	if err != nil {
		// ...
	}

	err = env.View(func(txn *lmdb.Txn) (err error) {
		v, err := txn.Get(dbi, []byte("k"))
		if err != nil {
			return err
		}
		fmt.Println(string(v))
		return nil
	})
	if err != nil {
		// ...
	}
}

Output:

func (*Txn) Abort 

func (txn *Txn) Abort()

Abort discards pending writes in the transaction and clears the finalizer on txn. A Txn cannot be used again after Abort is called.

See mdb_txn_abort.

func (*Txn) Commit 

func (txn *Txn) Commit() error

Commit persists all transaction operations to the database and clears the finalizer on txn. A Txn cannot be used again after Commit is called.

See mdb_txn_commit.

func (*Txn) CreateDBI 

func (txn *Txn) CreateDBI(name string) (DBI, error)

CreateDBI is a shorthand for OpenDBI that passed the flag lmdb.Create.

func (*Txn) Del 

func (txn *Txn) Del(dbi DBI, key, val []byte) error

Del deletes an item from database dbi. Del ignores val unless dbi has the DupSort flag.

See mdb_del.

func (*Txn) Drop 

func (txn *Txn) Drop(dbi DBI, del bool) error

Drop empties the database if del is false. Drop deletes and closes the database if del is true.

See mdb_drop.

func (*Txn) Flags 

func (txn *Txn) Flags(dbi DBI) (uint, error)

Flags returns the database flags for handle dbi.

func (*Txn) Get 

func (txn *Txn) Get(dbi DBI, key []byte) ([]byte, error)

Get retrieves items from database dbi. If txn.RawRead is true the slice returned by Get references a readonly section of memory that must not be accessed after txn has terminated.

See mdb_get.

Example

This example shows how to properly handle data retrieved from the database and applies to Txn.Get() as well as Cursor.Get(). It is important to handle data retreival carefully to make sure the application does not retain pointers to memory pages which may be reclaimed by LMDB after the transaction terminates. Typically an application would define helper functions/methods to conveniently handle data safe retrieval. 

package main

import (
	"encoding/json"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values shouldn't actually be assigned to.  The are used as stand-ins
// for tests which do not act as tests.
var EnvEx *lmdb.Env
var DBIEx lmdb.DBI

func main() {
	// variables to hold data extracted from the database
	var point struct{ X, Y int }
	var str string
	var p1, p2 []byte

	// extract data from an example environment/database.  it is critical for application
	// code to handle errors  but that is omitted here to save space.
	EnvEx.View(func(txn *lmdb.Txn) (err error) {
		// OK
		// A []byte to string conversion will always copy the data
		v, _ := txn.Get(DBIEx, []byte("mykey"))
		str = string(v)

		// OK
		// If []byte is the desired data type then an explicit copy is required
		// for safe access after the transaction returns.
		v, _ = txn.Get(DBIEx, []byte("mykey"))
		p1 = make([]byte, len(v))
		copy(p1, v)

		// OK
		// The data does not need be copied because it is parsed while txn is
		// open.
		v, _ = txn.Get(DBIEx, []byte("mykey"))
		_ = json.Unmarshal(v, &point)

		// BAD
		// Assigning the result directly to p2 leaves its pointer volatile
		// after the transaction completes which can result in unpredictable
		// behavior.
		p2, _ = txn.Get(DBIEx, []byte("mykey"))

		return nil
	})
}

Output:

func (*Txn) ID added in v1.6.0 

func (txn *Txn) ID() uintptr

ID returns the identifier for txn. A view transaction identifier corresponds to the Env snapshot being viewed and may be shared with other view transactions.

See mdb_txn_id.

func (*Txn) OpenCursor 

func (txn *Txn) OpenCursor(dbi DBI) (*Cursor, error)

OpenCursor allocates and initializes a Cursor to database dbi.

See mdb_cursor_open.

func (*Txn) OpenDBI 

func (txn *Txn) OpenDBI(name string, flags uint) (DBI, error)

OpenDBI opens a named database in the environment. An error is returned if name is empty. The DBI returned by OpenDBI can be used in other transactions but not before Txn has terminated.

OpenDBI can only be called after env.SetMaxDBs() has been called to set the maximum number of named databases.

The C API uses null terminated strings for database names. A consequence is that names cannot contain null bytes themselves. OpenDBI does not check for null bytes in the name argument.

See mdb_dbi_open.

Example

If the database being opened is known to exist then no flags need to be passed. 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error

func main() {
	// DBI handles can be saved after their opening transaction has committed
	// and may be reused as long as the environment remains open.
	var dbi lmdb.DBI
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		dbi, err = txn.OpenDBI("dbfound", 0)
		return err
	})
	if err != nil {
		panic(err)
	}
}

Output:
Example (Create)

When Create is passed to Txn.OpenDBI() the database will be created if it didn't already exist. An error will be returned if the name is occupied by data written by Txn./Cursor.Put(). 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error

func main() {
	var dbi lmdb.DBI
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		dbi, err = txn.OpenDBI("dbnew", lmdb.Create)
		return err
	})
	if err != nil {
		panic(err)
	}
}

Output:
Example (DBsFull)

When the number of open named databases in an environment reaches the value specified by Env.SetMaxDBs() attempts to open additional databases will return an error with errno DBsFull. If an application needs to handle this case then the function IsError() can test an error for this condition. 

package main

import (
	"log"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error

func main() {
	var dbi lmdb.DBI
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		dbi, err = txn.OpenDBI("dbnotexist", 0)
		return err
	})
	log.Print(err) // mdb_dbi_open: MDB_DBS_FULL: Environment maxdbs limit reached
}

Output:
Example (NotFound)

When a non-existent database is opened without the Create flag the errno is NotFound. If an application needs to handle this case the function IsNotFound() will test an error for this condition. 

package main

import (
	"log"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error

func main() {
	var dbi lmdb.DBI
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		dbi, err = txn.OpenDBI("dbnotfound", 0)
		return err
	})
	log.Print(err) // mdb_dbi_open: MDB_NOTFOUND: No matching key/data pair found
}

Output:

func (*Txn) OpenRoot 

func (txn *Txn) OpenRoot(flags uint) (DBI, error)

OpenRoot opens the root database. OpenRoot behaves similarly to OpenDBI but does not require env.SetMaxDBs() to be called beforehand. And, OpenRoot can be called without flags in a View transaction.

Example

Txn.OpenRoot does not need to be called with the Create flag. And Txn.OpenRoot, unlike Txn.OpenDBI, will never produce the error DBsFull. 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error

func main() {
	var dbi lmdb.DBI
	err = env.Update(func(txn *lmdb.Txn) (err error) {
		dbi, err = txn.OpenRoot(0)
		return err
	})
	if err != nil {
		panic(err)
	}
}

Output:
Example (View)

Txn.OpenRoot may also be called without flags inside View transactions before being openend in an Update transaction. 

package main

import (
	"log"

	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values can only be used is code-only examples (no test output).
var env *lmdb.Env

var err error

func main() {
	err = env.View(func(txn *lmdb.Txn) (err error) {
		dbi, err := txn.OpenRoot(0)
		if err != nil {
			return err
		}
		cur, err := txn.OpenCursor(dbi)
		if err != nil {
			return err
		}
		for {
			k, v, err := cur.Get(nil, nil, lmdb.Next)
			if lmdb.IsNotFound(err) {
				return nil
			}
			if err != nil {
				return err
			}

			log.Printf("%s=%s\n", k, v)
			// ...
		}
	})
	if err != nil {
		// ...
	}
}

Output:

func (*Txn) Put 

func (txn *Txn) Put(dbi DBI, key []byte, val []byte, flags uint) error

Put stores an item in database dbi.

See mdb_put.

func (*Txn) PutReserve 

func (txn *Txn) PutReserve(dbi DBI, key []byte, n int, flags uint) ([]byte, error)

PutReserve returns a []byte of length n that can be written to, potentially avoiding a memcopy. The returned byte slice is only valid in txn's thread, before it has terminated.

Example

This example demonstrates the use of PutReserve to store a string value in the root database. This may be faster than Put alone for large values because a string to []byte conversion is not required. 

package main

import (
	"github.com/bmatsuo/lmdb-go/lmdb"
)

// These values shouldn't actually be assigned to.  The are used as stand-ins
// for tests which do not act as tests.
var EnvEx *lmdb.Env

func main() {
	EnvEx.Update(func(txn *lmdb.Txn) (err error) {
		dbroot, err := txn.OpenRoot(0)
		if err != nil {
			return err
		}

		valstr := "value"
		p, err := txn.PutReserve(dbroot, []byte("key"), len(valstr), 0)
		if err != nil {
			return err
		}
		copy(p, valstr)

		return nil
	})
}

Output:

func (*Txn) Renew 

func (txn *Txn) Renew() error

Renew reuses a transaction that was previously reset by calling txn.Reset(). Renew panics if txn is managed by Update, View, etc.

See mdb_txn_renew.

func (*Txn) Reset 

func (txn *Txn) Reset()

Reset aborts the transaction clears internal state so the transaction may be reused by calling Renew. If txn is not going to be reused txn.Abort() must be called to release its slot in the lock table and free its memory. Reset panics if txn is managed by Update, View, etc.

See mdb_txn_reset.

func (*Txn) Stat 

func (txn *Txn) Stat(dbi DBI) (*Stat, error)

Stat returns a Stat describing the database dbi.

See mdb_stat.

func (*Txn) Sub 

func (txn *Txn) Sub(fn TxnOp) error

Sub executes fn in a subtransaction. Sub commits the subtransaction iff a nil error is returned by fn and otherwise aborts it. Sub returns any error it encounters.

Sub may only be called on an Update Txn (one created without the Readonly flag). Calling Sub on a View transaction will return an error. Sub assumes the calling goroutine is locked to an OS thread and will not call runtime.LockOSThread.

Any call to Abort, Commit, Renew, or Reset on a Txn created by Sub will panic.

type TxnOp 

type TxnOp func(txn *Txn) error

TxnOp is an operation applied to a managed transaction. The Txn passed to a TxnOp is managed and the operation must not call Commit, Abort, Renew, or Reset on it.

IMPORTANT: TxnOps that write to the database (those passed to Env.Update or Txn.Sub) must not use the Txn in another goroutine (passing it directly or otherwise through closure). Doing so has undefined results.

Notes

Bugs

  • MDB_INTEGERKEY and MDB_INTEGERDUP aren't usable. I'm not sure they would be faster with the cgo bridge. They need to be tested and benchmarked.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.