Documentation

Overview

Package lmdb provides bindings to the lmdb C API. The package bindings are fairly low level and are designed to provide a minimal interface that prevents misuse to a reasonable extent. When in doubt refer to the C documentation as a reference.

http://www.lmdb.tech/doc/
http://www.lmdb.tech/doc/starting.html
http://www.lmdb.tech/doc/modules.html

Environment

An LMDB environment holds named databases (key-value stores). An environment is represented as one file on the filesystem (though often a corresponding lock file exists).

LMDB recommends setting an environment's size as large as possible at the time of creation. On filesystems that support sparse files this should not adversely affect disk usage. Resizing an environment is possible but must be handled with care when concurrent access is involved.

Note that the package lmdb forces all Env objects to be opened with the NoTLS (MDB_NOTLS) flag. Without this flag LMDB would not be practically usable in Go (in the author's opinion). However, even for environments opened with this flag there are caveats regarding how transactions are used (see Caveats below).

Databases

A database in an LMDB environment is an ordered key-value store that holds arbitrary binary data. Typically the keys are unique but duplicate keys may be allowed (DupSort), in which case the values for each duplicate key are ordered.

A single LMDB environment can have multiple named databases. But there is also a 'root' (unnamed) database that can be used to store data. Use caution storing data in the root database when named databases are in use. The root database serves as an index for named databases.

A database is referenced by an opaque handle known as its DBI which must be opened inside a transaction with the OpenDBI or OpenRoot methods. DBIs may be closed but it is not required. Typically, applications acquire handles for all their databases immediately after opening an environment and retain them for the lifetime of the process.

Transactions

View (readonly) transactions in LMDB operate on a snapshot of the database at the time the transaction began. The number of simultaneously active view transactions is bounded and configured when the environment is initialized.

Update (read-write) transactions are serialized in LMDB. Attempts to create update transactions block until a lock may be obtained. Update transactions can create subtransactions which may be rolled back independently from their parent.

The lmdb package supplies managed and unmanaged transactions. Managed transactions do not require explicit calling of Abort/Commit and are provided through the Env methods Update, View, and RunTxn. The BeginTxn method on Env creates an unmanaged transaction but its use is not advised in most applications.

Caveats

Write transactions (those created without the Readonly flag) must be created in a goroutine that has been locked to its thread by calling the function runtime.LockOSThread. Futhermore, all methods on such transactions must be called from the goroutine which created them. This is a fundamental limitation of LMDB even when using the NoTLS flag (which the package always uses). The Env.Update method assists the programmer by calling runtime.LockOSThread automatically but it cannot sufficiently abstract write transactions to make them completely safe in Go.

A goroutine must never create a write transaction if the application programmer cannot determine whether the goroutine is locked to an OS thread. This is a consequence of goroutine restrictions on write transactions and limitations in the runtime's thread locking implementation. In such situations updates desired by the goroutine in question must be proxied by a goroutine with a known state (i.e. "locked" or "unlocked"). See the included examples for more details about dealing with such situations.

Example

    This example demonstrates a complete workflow for an lmdb environment. The Env is first created. After being configured the Env is mapped to memory. Once mapped, database handles are opened and normal database operations begin.

    Output:
    
    
    Example (Threads)

      This example demonstrates the simplest (and most naive) way to issue database updates from a goroutine for which it cannot be known ahead of time whether runtime.LockOSThread has been called.

      Output:
      
      
      Example (Worker)

        This example demonstrates a more sophisticated way to issue database updates from a goroutine for which it cannot be known ahead of time whether runtime.LockOSThread has been called.

        Output:
        
        

        Index

        Examples

        Constants

        View Source
        const (
        	First        = C.MDB_FIRST          // The first item.
        	FirstDup     = C.MDB_FIRST_DUP      // The first value of current key (DupSort).
        	GetBoth      = C.MDB_GET_BOTH       // Get the key as well as the value (DupSort).
        	GetBothRange = C.MDB_GET_BOTH_RANGE // Get the key and the nearsest value (DupSort).
        	GetCurrent   = C.MDB_GET_CURRENT    // Get the key and value at the current position.
        	GetMultiple  = C.MDB_GET_MULTIPLE   // Get up to a page dup values for key at current position (DupFixed).
        	Last         = C.MDB_LAST           // Last item.
        	LastDup      = C.MDB_LAST_DUP       // Position at last value of current key (DupSort).
        	Next         = C.MDB_NEXT           // Next value.
        	NextDup      = C.MDB_NEXT_DUP       // Next value of the current key (DupSort).
        	NextMultiple = C.MDB_NEXT_MULTIPLE  // Get key and up to a page of values from the next cursor position (DupFixed).
        	NextNoDup    = C.MDB_NEXT_NODUP     // The first value of the next key (DupSort).
        	Prev         = C.MDB_PREV           // The previous item.
        	PrevDup      = C.MDB_PREV_DUP       // The previous item of the current key (DupSort).
        	PrevNoDup    = C.MDB_PREV_NODUP     // The last data item of the previous key (DupSort).
        	Set          = C.MDB_SET            // The specified key.
        	SetKey       = C.MDB_SET_KEY        // Get key and data at the specified key.
        	SetRange     = C.MDB_SET_RANGE      // The first key no less than the specified key.
        )

          These flags are used exclusively for Cursor.Get.

          View Source
          const (
          	Current     = C.MDB_CURRENT     // Replace the item at the current key position (Cursor only)
          	NoDupData   = C.MDB_NODUPDATA   // Store the key-value pair only if key is not present (DupSort).
          	NoOverwrite = C.MDB_NOOVERWRITE // Store a new key-value pair only if key is not present.
          	Append      = C.MDB_APPEND      // Append an item to the database.
          	AppendDup   = C.MDB_APPENDDUP   // Append an item to the database (DupSort).
          )

            The MDB_MULTIPLE and MDB_RESERVE flags are special and do not fit the calling pattern of other calls to Put. They are not exported because they require special methods, PutMultiple and PutReserve in which the flag is implied and does not need to be passed.

            View Source
            const (
            	FixedMap    = C.MDB_FIXEDMAP   // Danger zone. Map memory at a fixed address.
            	NoSubdir    = C.MDB_NOSUBDIR   // Argument to Open is a file, not a directory.
            	Readonly    = C.MDB_RDONLY     // Used in several functions to denote an object as readonly.
            	WriteMap    = C.MDB_WRITEMAP   // Use a writable memory map.
            	NoMetaSync  = C.MDB_NOMETASYNC // Don't fsync metapage after commit.
            	NoSync      = C.MDB_NOSYNC     // Don't fsync after commit.
            	MapAsync    = C.MDB_MAPASYNC   // Flush asynchronously when using the WriteMap flag.
            	NoTLS       = C.MDB_NOTLS      // Danger zone. When unset reader locktable slots are tied to their thread.
            	NoLock      = C.MDB_NOLOCK     // Danger zone. LMDB does not use any locks.
            	NoReadahead = C.MDB_NORDAHEAD  // Disable readahead. Requires OS support.
            	NoMemInit   = C.MDB_NOMEMINIT  // Disable LMDB memory initialization.
            )

              These flags are used exclusively for Env types and are set with Env.Open. Some flags may be set/unset later using Env.Set/.Unset methods. Others will produce syscall.EINVAL. Refer to the C documentation for detailed information.

              View Source
              const (
              	ReverseKey = C.MDB_REVERSEKEY // Use reverse string keys.
              	DupSort    = C.MDB_DUPSORT    // Use sorted duplicates.
              	DupFixed   = C.MDB_DUPFIXED   // Duplicate items have a fixed size (DupSort).
              	ReverseDup = C.MDB_REVERSEDUP // Reverse duplicate values (DupSort).
              	Create     = C.MDB_CREATE     // Create DB if not already existing.
              )

                This flags are used exclusively for Txn.OpenDBI and Txn.OpenRoot. The Create flag must always be supplied when opening a non-root DBI for the first time.

                BUG(bmatsuo): MDB_INTEGERKEY and MDB_INTEGERDUP aren't usable. I'm not sure they would be faster with the cgo bridge. They need to be tested and benchmarked.

                View Source
                const (
                	CopyCompact = C.MDB_CP_COMPACT // Perform compaction while copying
                )

                  These flags are exclusively used in the Env.CopyFlags and Env.CopyFDFlags methods.

                  Variables

                  This section is empty.

                  Functions

                  func IsErrno

                  func IsErrno(err error, errno Errno) bool

                    IsErrno returns true if err's errno is the given errno.

                    func IsErrnoFn

                    func IsErrnoFn(err error, fn func(error) bool) bool

                      IsErrnoFn calls fn on the error underlying err and returns the result. If err is an *OpError then err.Errno is passed to fn. Otherwise err is passed directly to fn.

                      func IsErrnoSys

                      func IsErrnoSys(err error, errno syscall.Errno) bool

                        IsErrnoSys returns true if err's errno is the given errno.

                        func IsMapFull

                        func IsMapFull(err error) bool

                          IsMapFull returns true if the environment map size has been reached.

                          func IsMapResized

                          func IsMapResized(err error) bool

                            IsMapResized returns true if the environment has grown too large for the current map after being resized by another process.

                            func IsNotExist

                            func IsNotExist(err error) bool

                              IsNotExist returns true the path passed to the Env.Open method does not exist.

                              func IsNotFound

                              func IsNotFound(err error) bool

                                IsNotFound returns true if the key requested in Txn.Get or Cursor.Get does not exist or if the Cursor reached the end of the database without locating a value (EOF).

                                func Version

                                func Version() (major, minor, patch int, s string)

                                  Version return the major, minor, and patch version numbers of the LMDB C library and a string representation of the version.

                                  See mdb_version.

                                  func VersionString

                                  func VersionString() string

                                    VersionString returns a string representation of the LMDB C library version.

                                    See mdb_version.

                                    Types

                                    type Cursor

                                    type Cursor struct {
                                    	// contains filtered or unexported fields
                                    }

                                      Cursor operates on data inside a transaction and holds a position in the database.

                                      See MDB_cursor.

                                      func (*Cursor) Close

                                      func (c *Cursor) Close()

                                        Close the cursor handle and clear the finalizer on c. Cursors belonging to write transactions are closed automatically when the transaction is terminated.

                                        See mdb_cursor_close.

                                        func (*Cursor) Count

                                        func (c *Cursor) Count() (uint64, error)

                                          Count returns the number of duplicates for the current key.

                                          See mdb_cursor_count.

                                          Example

                                            This example demonstrates how to iterate a database opened with the DupSort flag and get the number of values present for each distinct key.

                                            Output:
                                            
                                            

                                            func (*Cursor) DBI

                                            func (c *Cursor) DBI() DBI

                                              DBI returns the cursor's database handle. If c has been closed than an invalid DBI is returned.

                                              func (*Cursor) Del

                                              func (c *Cursor) Del(flags uint) error

                                                Del deletes the item referred to by the cursor from the database.

                                                See mdb_cursor_del.

                                                Example

                                                  This example demonstrates how to delete all elements in a database with a key less than a given value (an RFC3339 timestamp in this case).

                                                  Output:
                                                  
                                                  

                                                  func (*Cursor) Get

                                                  func (c *Cursor) Get(setkey, setval []byte, op uint) (key, val []byte, err error)

                                                    Get retrieves items from the database. If c.Txn().RawRead is true the slices returned by Get reference readonly sections of memory that must not be accessed after the transaction has terminated.

                                                    In a Txn with RawRead set to true the Set op causes the returned key to share its memory with setkey (making it writable memory). In a Txn with RawRead set to false the Set op returns key values with memory distinct from setkey, as is always the case when using RawRead.

                                                    Get ignores setval if setkey is empty.

                                                    See mdb_cursor_get.

                                                    Example

                                                      This simple example shows how to iterate a database. The Next flag may be used without an initial call passing the First flag.

                                                      Output:
                                                      
                                                      
                                                      Example (DupFixed)

                                                        This simple example shows how to iterate a database opened with the DupSort|DupSort flags. It is not necessary to use the GetMultiple flag before passing the NextMultiple flag.

                                                        Output:
                                                        
                                                        
                                                        Example (DupSort)

                                                          This example shows how duplicates can be processed using LMDB. It is possible to iterate all key-value pairs (including duplicate key values) by passing Next. But if special handling of duplicates is needed it may be beneficial to use NextNoDup or NextDup.

                                                          Output:
                                                          
                                                          
                                                          Example (Reverse)

                                                            This simple example shows how to iterate a database in reverse. As when passing the Next flag, the Prev flag may be used without an initial call using Last.

                                                            Output:
                                                            
                                                            

                                                            func (*Cursor) Put

                                                            func (c *Cursor) Put(key, val []byte, flags uint) error

                                                              Put stores an item in the database.

                                                              See mdb_cursor_put.

                                                              func (*Cursor) PutMulti

                                                              func (c *Cursor) PutMulti(key []byte, page []byte, stride int, flags uint) error

                                                                PutMulti stores a set of contiguous items with stride size under key. PutMulti panics if len(page) is not a multiple of stride. The cursor's database must be DupFixed and DupSort.

                                                                See mdb_cursor_put.

                                                                Example

                                                                  This example shows how to write a page of contiguous, fixed-size values to a database opened with DupSort|DupFixed. It doesn't matter if the values are sorted. Values will be stored in sorted order.

                                                                  Output:
                                                                  
                                                                  

                                                                  func (*Cursor) PutReserve

                                                                  func (c *Cursor) PutReserve(key []byte, n int, flags uint) ([]byte, error)

                                                                    PutReserve returns a []byte of length n that can be written to, potentially avoiding a memcopy. The returned byte slice is only valid in txn's thread, before it has terminated.

                                                                    func (*Cursor) Renew

                                                                    func (c *Cursor) Renew(txn *Txn) error

                                                                      Renew associates readonly cursor with txn.

                                                                      See mdb_cursor_renew.

                                                                      Example

                                                                        This example shows a trivial case using Renew to service read requests on a database. Close must be called when the cursor will no longer be renewed. Before using Renew benchmark your application to understand its benefits.

                                                                        Output:
                                                                        
                                                                        

                                                                        func (*Cursor) Txn

                                                                        func (c *Cursor) Txn() *Txn

                                                                          Txn returns the cursor's transaction.

                                                                          type DBI

                                                                          type DBI C.MDB_dbi

                                                                            DBI is a handle for a database in an Env.

                                                                            See MDB_dbi

                                                                            type Env

                                                                            type Env struct {
                                                                            	// contains filtered or unexported fields
                                                                            }

                                                                              Env is opaque structure for a database environment. A DB environment supports multiple databases, all residing in the same shared-memory map.

                                                                              See MDB_env.

                                                                              Example

                                                                                This example shows the general workflow of LMDB. An environment is created and configured before being opened. After the environment is opened its databases are created and their handles are saved for use in future transactions.

                                                                                Output:
                                                                                
                                                                                

                                                                                func NewEnv

                                                                                func NewEnv() (*Env, error)

                                                                                  NewEnv allocates and initializes a new Env.

                                                                                  See mdb_env_create.

                                                                                  func (*Env) BeginTxn

                                                                                  func (env *Env) BeginTxn(parent *Txn, flags uint) (*Txn, error)

                                                                                    BeginTxn is an unsafe, low-level method to initialize a new transaction on env. The Txn returned by BeginTxn is unmanaged and must be terminated by calling either its Abort or Commit methods to ensure that its resources are released.

                                                                                    BeginTxn does not call runtime.LockOSThread. Unless the Readonly flag is passed goroutines must call runtime.LockOSThread before calling BeginTxn and the returned Txn must not have its methods called from another goroutine. Failure to meet these restrictions can have undefined results that may include deadlocking your application.

                                                                                    Instead of calling BeginTxn users should prefer calling the View and Update methods, which assist in management of Txn objects and provide OS thread locking required for write transactions.

                                                                                    A finalizer detects unreachable, live transactions and logs thems to standard error. The transactions are aborted, but their presence should be interpreted as an application error which should be patched so transactions are terminated explicitly. Unterminated transactions can adversly effect database performance and cause the database to grow until the map is full.

                                                                                    See mdb_txn_begin.

                                                                                    func (*Env) Close

                                                                                    func (env *Env) Close() error

                                                                                      Close shuts down the environment, releases the memory map, and clears the finalizer on env.

                                                                                      See mdb_env_close.

                                                                                      func (*Env) CloseDBI

                                                                                      func (env *Env) CloseDBI(db DBI)

                                                                                        CloseDBI closes the database handle, db. Normally calling CloseDBI explicitly is not necessary.

                                                                                        It is the caller's responsibility to serialize calls to CloseDBI.

                                                                                        See mdb_dbi_close.

                                                                                        func (*Env) Copy

                                                                                        func (env *Env) Copy(path string) error

                                                                                          Copy copies the data in env to an environment at path.

                                                                                          See mdb_env_copy.

                                                                                          Example

                                                                                            This example uses env.Copy to periodically create atomic backups of an environment. A real implementation would need to solve problems with failures, remote persistence, purging old backups, etc. But the core loop would have the same form.

                                                                                            Output:
                                                                                            
                                                                                            

                                                                                            func (*Env) CopyFD

                                                                                            func (env *Env) CopyFD(fd uintptr) error

                                                                                              CopyFD copies env to the the file descriptor fd.

                                                                                              See mdb_env_copyfd.

                                                                                              func (*Env) CopyFDFlag

                                                                                              func (env *Env) CopyFDFlag(fd uintptr, flags uint) error

                                                                                                CopyFDFlag copies env to the file descriptor fd, with options.

                                                                                                See mdb_env_copyfd2.

                                                                                                func (*Env) CopyFlag

                                                                                                func (env *Env) CopyFlag(path string, flags uint) error

                                                                                                  CopyFlag copies the data in env to an environment at path created with flags.

                                                                                                  See mdb_env_copy2.

                                                                                                  func (*Env) FD

                                                                                                  func (env *Env) FD() (uintptr, error)

                                                                                                    FD returns the open file descriptor (or Windows file handle) for the given environment. An error is returned if the environment has not been successfully Opened (where C API just retruns an invalid handle).

                                                                                                    See mdb_env_get_fd.

                                                                                                    func (*Env) Flags

                                                                                                    func (env *Env) Flags() (uint, error)

                                                                                                      Flags returns the flags set in the environment.

                                                                                                      See mdb_env_get_flags.

                                                                                                      func (*Env) Info

                                                                                                      func (env *Env) Info() (*EnvInfo, error)

                                                                                                        Info returns information about the environment.

                                                                                                        See mdb_env_info.

                                                                                                        func (*Env) MaxKeySize

                                                                                                        func (env *Env) MaxKeySize() int

                                                                                                          MaxKeySize returns the maximum allowed length for a key.

                                                                                                          See mdb_env_get_maxkeysize.

                                                                                                          func (*Env) MaxReaders

                                                                                                          func (env *Env) MaxReaders() (int, error)

                                                                                                            MaxReaders returns the maximum number of reader slots for the environment.

                                                                                                            See mdb_env_get_maxreaders.

                                                                                                            func (*Env) Open

                                                                                                            func (env *Env) Open(path string, flags uint, mode os.FileMode) error

                                                                                                              Open an environment handle. If this function fails Close() must be called to discard the Env handle. Open passes flags|NoTLS to mdb_env_open.

                                                                                                              See mdb_env_open.

                                                                                                              func (*Env) Path

                                                                                                              func (env *Env) Path() (string, error)

                                                                                                                Path returns the path argument passed to Open. Path returns a non-nil error if env.Open() was not previously called.

                                                                                                                See mdb_env_get_path.

                                                                                                                func (*Env) ReaderCheck

                                                                                                                func (env *Env) ReaderCheck() (int, error)

                                                                                                                  ReaderCheck clears stale entries from the reader lock table and returns the number of entries cleared.

                                                                                                                  See mdb_reader_check()

                                                                                                                  func (*Env) ReaderList

                                                                                                                  func (env *Env) ReaderList(fn func(string) error) error

                                                                                                                    ReaderList dumps the contents of the reader lock table as text. Readers start on the second line as space-delimited fields described by the first line.

                                                                                                                    See mdb_reader_list.

                                                                                                                    func (*Env) RunTxn

                                                                                                                    func (env *Env) RunTxn(flags uint, fn TxnOp) error

                                                                                                                      RunTxn creates a new Txn and calls fn with it as an argument. Run commits the transaction if fn returns nil otherwise the transaction is aborted. Because RunTxn terminates the transaction goroutines should not retain references to it or its data after fn returns.

                                                                                                                      RunTxn does not call runtime.LockOSThread. Unless the Readonly flag is passed the calling goroutine should ensure it is locked to its thread and any goroutines started by fn must not call methods on the Txn object it is passed.

                                                                                                                      See mdb_txn_begin.

                                                                                                                      func (*Env) SetFlags

                                                                                                                      func (env *Env) SetFlags(flags uint) error

                                                                                                                        SetFlags sets flags in the environment.

                                                                                                                        See mdb_env_set_flags.

                                                                                                                        func (*Env) SetMapSize

                                                                                                                        func (env *Env) SetMapSize(size int64) error

                                                                                                                          SetMapSize sets the size of the environment memory map.

                                                                                                                          See mdb_env_set_mapsize.

                                                                                                                          Example

                                                                                                                            This example demonstrates how an application typically uses Env.SetMapSize. The call to Env.SetMapSize() is made before calling env.Open(). Any calls after calling Env.Open() must take special care to synchronize with other goroutines.

                                                                                                                            Output:
                                                                                                                            
                                                                                                                            
                                                                                                                            Example (MapResized)

                                                                                                                              This example demonstrates how to handle a MapResized error, encountered after another process has called mdb_env_set_mapsize (Env.SetMapSize). Applications which don't expect another process to resize the mmap don't need to check for the MapResized error.

                                                                                                                              The example is simplified for clarity. Many real applications will need to synchronize calls to Env.SetMapSize using something like a sync.RWMutex to ensure there are no active readonly transactions (those opened successfully before MapResized was encountered).

                                                                                                                              Output:
                                                                                                                              
                                                                                                                              

                                                                                                                              func (*Env) SetMaxDBs

                                                                                                                              func (env *Env) SetMaxDBs(size int) error

                                                                                                                                SetMaxDBs sets the maximum number of named databases for the environment.

                                                                                                                                See mdb_env_set_maxdbs.

                                                                                                                                func (*Env) SetMaxReaders

                                                                                                                                func (env *Env) SetMaxReaders(size int) error

                                                                                                                                  SetMaxReaders sets the maximum number of reader slots in the environment.

                                                                                                                                  See mdb_env_set_maxreaders.

                                                                                                                                  func (*Env) Stat

                                                                                                                                  func (env *Env) Stat() (*Stat, error)

                                                                                                                                    Stat returns statistics about the environment.

                                                                                                                                    See mdb_env_stat.

                                                                                                                                    func (*Env) Sync

                                                                                                                                    func (env *Env) Sync(force bool) error

                                                                                                                                      Sync flushes buffers to disk. If force is true a synchronous flush occurs and ignores any NoSync or MapAsync flag on the environment.

                                                                                                                                      See mdb_env_sync.

                                                                                                                                      func (*Env) UnsetFlags

                                                                                                                                      func (env *Env) UnsetFlags(flags uint) error

                                                                                                                                        UnsetFlags clears flags in the environment.

                                                                                                                                        See mdb_env_set_flags.

                                                                                                                                        func (*Env) Update

                                                                                                                                        func (env *Env) Update(fn TxnOp) error

                                                                                                                                          Update calls fn with a writable transaction. Update commits the transaction if fn returns a nil error otherwise Update aborts the transaction and returns the error.

                                                                                                                                          Update calls runtime.LockOSThread to lock the calling goroutine to its thread and until fn returns and the transaction has been terminated, at which point runtime.UnlockOSThread is called. If the calling goroutine is already known to be locked to a thread, use UpdateLocked instead to avoid premature unlocking of the goroutine.

                                                                                                                                          Neither Update nor UpdateLocked cannot be called safely from a goroutine where it isn't known if runtime.LockOSThread has been called. In such situations writes must either be done in a newly created goroutine which can be safely locked, or through a worker goroutine that accepts updates to apply and delivers transaction results using channels. See the package documentation and examples for more details.

                                                                                                                                          Goroutines created by the operation fn must not use methods on the Txn object that fn is passed. Doing so would have undefined and unpredictable results for your program (likely including data loss, deadlock, etc).

                                                                                                                                          Any call to Commit, Abort, Reset or Renew on a Txn created by Update will panic.

                                                                                                                                          Example

                                                                                                                                            This example shows the basic use of Env.Update, the primary method lmdb-go provides for to store data in an Env.

                                                                                                                                            Output:
                                                                                                                                            
                                                                                                                                            

                                                                                                                                            func (*Env) UpdateLocked

                                                                                                                                            func (env *Env) UpdateLocked(fn TxnOp) error

                                                                                                                                              UpdateLocked behaves like Update but does not lock the calling goroutine to its thread. UpdateLocked should be used if the calling goroutine is already locked to its thread for another purpose.

                                                                                                                                              Neither Update nor UpdateLocked cannot be called safely from a goroutine where it isn't known if runtime.LockOSThread has been called. In such situations writes must either be done in a newly created goroutine which can be safely locked, or through a worker goroutine that accepts updates to apply and delivers transaction results using channels. See the package documentation and examples for more details.

                                                                                                                                              Goroutines created by the operation fn must not use methods on the Txn object that fn is passed. Doing so would have undefined and unpredictable results for your program (likely including data loss, deadlock, etc).

                                                                                                                                              Any call to Commit, Abort, Reset or Renew on a Txn created by UpdateLocked will panic.

                                                                                                                                              Example

                                                                                                                                                In this example, another C library requires the application to lock a goroutine to its thread. When writing to the Env this goroutine must use the method Env.UpdateLocked to prevent premature unlocking of the goroutine.

                                                                                                                                                Note that there is no way for a goroutine to determine if it is locked to a thread failure to call Env.UpdateLocked in a scenario like this can lead to unspecified and hard to debug failure modes for your application.

                                                                                                                                                Output:
                                                                                                                                                
                                                                                                                                                

                                                                                                                                                func (*Env) View

                                                                                                                                                func (env *Env) View(fn TxnOp) error

                                                                                                                                                  View creates a readonly transaction with a consistent view of the environment and passes it to fn. View terminates its transaction after fn returns. Any error encountered by View is returned.

                                                                                                                                                  Unlike with Update transactions, goroutines created by fn are free to call methods on the Txn passed to fn provided they are synchronized in their accesses (e.g. using a mutex or channel).

                                                                                                                                                  Any call to Commit, Abort, Reset or Renew on a Txn created by View will panic.

                                                                                                                                                  type EnvInfo

                                                                                                                                                  type EnvInfo struct {
                                                                                                                                                  	MapSize    int64 // Size of the data memory map
                                                                                                                                                  	LastPNO    int64 // ID of the last used page
                                                                                                                                                  	LastTxnID  int64 // ID of the last committed transaction
                                                                                                                                                  	MaxReaders uint  // maximum number of threads for the environment
                                                                                                                                                  	NumReaders uint  // maximum number of threads used in the environment
                                                                                                                                                  }

                                                                                                                                                    EnvInfo contains information an environment.

                                                                                                                                                    See MDB_envinfo.

                                                                                                                                                    type Errno

                                                                                                                                                    type Errno C.int

                                                                                                                                                      Errno is an error type that represents the (unique) errno values defined by LMDB. Other errno values (such as EINVAL) are represented with type syscall.Errno. On Windows, LMDB return codes are translated into portable syscall.Errno constants (e.g. syscall.EINVAL, syscall.EACCES, etc.).

                                                                                                                                                      Most often helper functions such as IsNotFound may be used instead of dealing with Errno values directly.

                                                                                                                                                      lmdb.IsNotFound(err)
                                                                                                                                                      lmdb.IsErrno(err, lmdb.TxnFull)
                                                                                                                                                      lmdb.IsErrnoSys(err, syscall.EINVAL)
                                                                                                                                                      lmdb.IsErrnoFn(err, os.IsPermission)
                                                                                                                                                      
                                                                                                                                                      const (
                                                                                                                                                      	KeyExist        Errno = C.MDB_KEYEXIST
                                                                                                                                                      	NotFound        Errno = C.MDB_NOTFOUND
                                                                                                                                                      	PageNotFound    Errno = C.MDB_PAGE_NOTFOUND
                                                                                                                                                      	Corrupted       Errno = C.MDB_CORRUPTED
                                                                                                                                                      	Panic           Errno = C.MDB_PANIC
                                                                                                                                                      	VersionMismatch Errno = C.MDB_VERSION_MISMATCH
                                                                                                                                                      	Invalid         Errno = C.MDB_INVALID
                                                                                                                                                      	MapFull         Errno = C.MDB_MAP_FULL
                                                                                                                                                      	DBsFull         Errno = C.MDB_DBS_FULL
                                                                                                                                                      	ReadersFull     Errno = C.MDB_READERS_FULL
                                                                                                                                                      	TLSFull         Errno = C.MDB_TLS_FULL
                                                                                                                                                      	TxnFull         Errno = C.MDB_TXN_FULL
                                                                                                                                                      	CursorFull      Errno = C.MDB_CURSOR_FULL
                                                                                                                                                      	PageFull        Errno = C.MDB_PAGE_FULL
                                                                                                                                                      	MapResized      Errno = C.MDB_MAP_RESIZED
                                                                                                                                                      	Incompatible    Errno = C.MDB_INCOMPATIBLE
                                                                                                                                                      	BadRSlot        Errno = C.MDB_BAD_RSLOT
                                                                                                                                                      	BadTxn          Errno = C.MDB_BAD_TXN
                                                                                                                                                      	BadValSize      Errno = C.MDB_BAD_VALSIZE
                                                                                                                                                      	BadDBI          Errno = C.MDB_BAD_DBI
                                                                                                                                                      )

                                                                                                                                                        The most common error codes do not need to be handled explicity. Errors can be checked through helper functions IsNotFound, IsMapFull, etc, Otherwise they should be checked using the IsErrno function instead of direct comparison because they will typically be wrapped with an OpError.

                                                                                                                                                        func (Errno) Error

                                                                                                                                                        func (e Errno) Error() string

                                                                                                                                                        type Multi

                                                                                                                                                        type Multi struct {
                                                                                                                                                        	// contains filtered or unexported fields
                                                                                                                                                        }

                                                                                                                                                          Multi is a wrapper for a contiguous page of sorted, fixed-length values passed to Cursor.PutMulti or retrieved using Cursor.Get with the GetMultiple/NextMultiple flag.

                                                                                                                                                          Multi values are only useful in databases opened with DupSort|DupFixed.

                                                                                                                                                          func WrapMulti

                                                                                                                                                          func WrapMulti(page []byte, stride int) *Multi

                                                                                                                                                            WrapMulti converts a page of contiguous values with stride size into a Multi. WrapMulti panics if len(page) is not a multiple of stride.

                                                                                                                                                            _, val, _ := cursor.Get(nil, nil, lmdb.FirstDup)
                                                                                                                                                            _, page, _ := cursor.Get(nil, nil, lmdb.GetMultiple)
                                                                                                                                                            multi := lmdb.WrapMulti(page, len(val))
                                                                                                                                                            

                                                                                                                                                            See mdb_cursor_get and MDB_GET_MULTIPLE.

                                                                                                                                                            func (*Multi) Len

                                                                                                                                                            func (m *Multi) Len() int

                                                                                                                                                              Len returns the number of values in the Multi.

                                                                                                                                                              func (*Multi) Page

                                                                                                                                                              func (m *Multi) Page() []byte

                                                                                                                                                                Page returns the Multi page data as a raw slice of bytes with length m.Size().

                                                                                                                                                                func (*Multi) Size

                                                                                                                                                                func (m *Multi) Size() int

                                                                                                                                                                  Size returns the total size of the Multi data and is equal to

                                                                                                                                                                  m.Len()*m.Stride()
                                                                                                                                                                  

                                                                                                                                                                  func (*Multi) Stride

                                                                                                                                                                  func (m *Multi) Stride() int

                                                                                                                                                                    Stride returns the length of an individual value in the m.

                                                                                                                                                                    func (*Multi) Val

                                                                                                                                                                    func (m *Multi) Val(i int) []byte

                                                                                                                                                                      Val returns the value at index i. Val panics if i is out of range.

                                                                                                                                                                      func (*Multi) Vals

                                                                                                                                                                      func (m *Multi) Vals() [][]byte

                                                                                                                                                                        Vals returns a slice containing the values in m. The returned slice has length m.Len() and each item has length m.Stride().

                                                                                                                                                                        type OpError

                                                                                                                                                                        type OpError struct {
                                                                                                                                                                        	Op    string
                                                                                                                                                                        	Errno error
                                                                                                                                                                        }

                                                                                                                                                                          OpError is an error returned by the C API. Not all errors returned by lmdb-go have type OpError but typically they do. The Errno field will either have type Errno or syscall.Errno.

                                                                                                                                                                          func (*OpError) Error

                                                                                                                                                                          func (err *OpError) Error() string

                                                                                                                                                                            Error implements the error interface.

                                                                                                                                                                            type Stat

                                                                                                                                                                            type Stat struct {
                                                                                                                                                                            	PSize         uint   // Size of a database page. This is currently the same for all databases.
                                                                                                                                                                            	Depth         uint   // Depth (height) of the B-tree
                                                                                                                                                                            	BranchPages   uint64 // Number of internal (non-leaf) pages
                                                                                                                                                                            	LeafPages     uint64 // Number of leaf pages
                                                                                                                                                                            	OverflowPages uint64 // Number of overflow pages
                                                                                                                                                                            	Entries       uint64 // Number of data items
                                                                                                                                                                            }

                                                                                                                                                                              Stat contains database status information.

                                                                                                                                                                              See MDB_stat.

                                                                                                                                                                              type Txn

                                                                                                                                                                              type Txn struct {
                                                                                                                                                                              	// If RawRead is true []byte values retrieved from Get() calls on the Txn
                                                                                                                                                                              	// and its cursors will point directly into the memory-mapped structure.
                                                                                                                                                                              	// Such slices will be readonly and must only be referenced wthin the
                                                                                                                                                                              	// transaction's lifetime.
                                                                                                                                                                              	RawRead bool
                                                                                                                                                                              	// contains filtered or unexported fields
                                                                                                                                                                              }

                                                                                                                                                                                Txn is a database transaction in an environment.

                                                                                                                                                                                WARNING: A writable Txn is not threadsafe and may only be used in the goroutine that created it.

                                                                                                                                                                                See MDB_txn.

                                                                                                                                                                                Example

                                                                                                                                                                                  This example shows the basic operations used when creating and working with Txn types.

                                                                                                                                                                                  Output:
                                                                                                                                                                                  
                                                                                                                                                                                  

                                                                                                                                                                                  func (*Txn) Abort

                                                                                                                                                                                  func (txn *Txn) Abort()

                                                                                                                                                                                    Abort discards pending writes in the transaction and clears the finalizer on txn. A Txn cannot be used again after Abort is called.

                                                                                                                                                                                    See mdb_txn_abort.

                                                                                                                                                                                    func (*Txn) Commit

                                                                                                                                                                                    func (txn *Txn) Commit() error

                                                                                                                                                                                      Commit persists all transaction operations to the database and clears the finalizer on txn. A Txn cannot be used again after Commit is called.

                                                                                                                                                                                      See mdb_txn_commit.

                                                                                                                                                                                      func (*Txn) CreateDBI

                                                                                                                                                                                      func (txn *Txn) CreateDBI(name string) (DBI, error)

                                                                                                                                                                                        CreateDBI is a shorthand for OpenDBI that passed the flag lmdb.Create.

                                                                                                                                                                                        func (*Txn) Del

                                                                                                                                                                                        func (txn *Txn) Del(dbi DBI, key, val []byte) error

                                                                                                                                                                                          Del deletes an item from database dbi. Del ignores val unless dbi has the DupSort flag.

                                                                                                                                                                                          See mdb_del.

                                                                                                                                                                                          func (*Txn) Drop

                                                                                                                                                                                          func (txn *Txn) Drop(dbi DBI, del bool) error

                                                                                                                                                                                            Drop empties the database if del is false. Drop deletes and closes the database if del is true.

                                                                                                                                                                                            See mdb_drop.

                                                                                                                                                                                            func (*Txn) Flags

                                                                                                                                                                                            func (txn *Txn) Flags(dbi DBI) (uint, error)

                                                                                                                                                                                              Flags returns the database flags for handle dbi.

                                                                                                                                                                                              func (*Txn) Get

                                                                                                                                                                                              func (txn *Txn) Get(dbi DBI, key []byte) ([]byte, error)

                                                                                                                                                                                                Get retrieves items from database dbi. If txn.RawRead is true the slice returned by Get references a readonly section of memory that must not be accessed after txn has terminated.

                                                                                                                                                                                                See mdb_get.

                                                                                                                                                                                                Example

                                                                                                                                                                                                  This example shows how to properly handle data retrieved from the database and applies to Txn.Get() as well as Cursor.Get(). It is important to handle data retreival carefully to make sure the application does not retain pointers to memory pages which may be reclaimed by LMDB after the transaction terminates. Typically an application would define helper functions/methods to conveniently handle data safe retrieval.

                                                                                                                                                                                                  Output:
                                                                                                                                                                                                  
                                                                                                                                                                                                  

                                                                                                                                                                                                  func (*Txn) ID

                                                                                                                                                                                                  func (txn *Txn) ID() uintptr

                                                                                                                                                                                                    ID returns the identifier for txn. A view transaction identifier corresponds to the Env snapshot being viewed and may be shared with other view transactions.

                                                                                                                                                                                                    See mdb_txn_id.

                                                                                                                                                                                                    func (*Txn) OpenCursor

                                                                                                                                                                                                    func (txn *Txn) OpenCursor(dbi DBI) (*Cursor, error)

                                                                                                                                                                                                      OpenCursor allocates and initializes a Cursor to database dbi.

                                                                                                                                                                                                      See mdb_cursor_open.

                                                                                                                                                                                                      func (*Txn) OpenDBI

                                                                                                                                                                                                      func (txn *Txn) OpenDBI(name string, flags uint) (DBI, error)

                                                                                                                                                                                                        OpenDBI opens a named database in the environment. An error is returned if name is empty. The DBI returned by OpenDBI can be used in other transactions but not before Txn has terminated.

                                                                                                                                                                                                        OpenDBI can only be called after env.SetMaxDBs() has been called to set the maximum number of named databases.

                                                                                                                                                                                                        The C API uses null terminated strings for database names. A consequence is that names cannot contain null bytes themselves. OpenDBI does not check for null bytes in the name argument.

                                                                                                                                                                                                        See mdb_dbi_open.

                                                                                                                                                                                                        Example

                                                                                                                                                                                                          If the database being opened is known to exist then no flags need to be passed.

                                                                                                                                                                                                          Output:
                                                                                                                                                                                                          
                                                                                                                                                                                                          
                                                                                                                                                                                                          Example (Create)

                                                                                                                                                                                                            When Create is passed to Txn.OpenDBI() the database will be created if it didn't already exist. An error will be returned if the name is occupied by data written by Txn./Cursor.Put().

                                                                                                                                                                                                            Output:
                                                                                                                                                                                                            
                                                                                                                                                                                                            
                                                                                                                                                                                                            Example (DBsFull)

                                                                                                                                                                                                              When the number of open named databases in an environment reaches the value specified by Env.SetMaxDBs() attempts to open additional databases will return an error with errno DBsFull. If an application needs to handle this case then the function IsError() can test an error for this condition.

                                                                                                                                                                                                              Output:
                                                                                                                                                                                                              
                                                                                                                                                                                                              
                                                                                                                                                                                                              Example (NotFound)

                                                                                                                                                                                                                When a non-existent database is opened without the Create flag the errno is NotFound. If an application needs to handle this case the function IsNotFound() will test an error for this condition.

                                                                                                                                                                                                                Output:
                                                                                                                                                                                                                
                                                                                                                                                                                                                

                                                                                                                                                                                                                func (*Txn) OpenRoot

                                                                                                                                                                                                                func (txn *Txn) OpenRoot(flags uint) (DBI, error)

                                                                                                                                                                                                                  OpenRoot opens the root database. OpenRoot behaves similarly to OpenDBI but does not require env.SetMaxDBs() to be called beforehand. And, OpenRoot can be called without flags in a View transaction.

                                                                                                                                                                                                                  Example

                                                                                                                                                                                                                    Txn.OpenRoot does not need to be called with the Create flag. And Txn.OpenRoot, unlike Txn.OpenDBI, will never produce the error DBsFull.

                                                                                                                                                                                                                    Output:
                                                                                                                                                                                                                    
                                                                                                                                                                                                                    
                                                                                                                                                                                                                    Example (View)

                                                                                                                                                                                                                      Txn.OpenRoot may also be called without flags inside View transactions before being openend in an Update transaction.

                                                                                                                                                                                                                      Output:
                                                                                                                                                                                                                      
                                                                                                                                                                                                                      

                                                                                                                                                                                                                      func (*Txn) Put

                                                                                                                                                                                                                      func (txn *Txn) Put(dbi DBI, key []byte, val []byte, flags uint) error

                                                                                                                                                                                                                        Put stores an item in database dbi.

                                                                                                                                                                                                                        See mdb_put.

                                                                                                                                                                                                                        func (*Txn) PutReserve

                                                                                                                                                                                                                        func (txn *Txn) PutReserve(dbi DBI, key []byte, n int, flags uint) ([]byte, error)

                                                                                                                                                                                                                          PutReserve returns a []byte of length n that can be written to, potentially avoiding a memcopy. The returned byte slice is only valid in txn's thread, before it has terminated.

                                                                                                                                                                                                                          Example

                                                                                                                                                                                                                            This example demonstrates the use of PutReserve to store a string value in the root database. This may be faster than Put alone for large values because a string to []byte conversion is not required.

                                                                                                                                                                                                                            Output:
                                                                                                                                                                                                                            
                                                                                                                                                                                                                            

                                                                                                                                                                                                                            func (*Txn) Renew

                                                                                                                                                                                                                            func (txn *Txn) Renew() error

                                                                                                                                                                                                                              Renew reuses a transaction that was previously reset by calling txn.Reset(). Renew panics if txn is managed by Update, View, etc.

                                                                                                                                                                                                                              See mdb_txn_renew.

                                                                                                                                                                                                                              func (*Txn) Reset

                                                                                                                                                                                                                              func (txn *Txn) Reset()

                                                                                                                                                                                                                                Reset aborts the transaction clears internal state so the transaction may be reused by calling Renew. If txn is not going to be reused txn.Abort() must be called to release its slot in the lock table and free its memory. Reset panics if txn is managed by Update, View, etc.

                                                                                                                                                                                                                                See mdb_txn_reset.

                                                                                                                                                                                                                                func (*Txn) Stat

                                                                                                                                                                                                                                func (txn *Txn) Stat(dbi DBI) (*Stat, error)

                                                                                                                                                                                                                                  Stat returns a Stat describing the database dbi.

                                                                                                                                                                                                                                  See mdb_stat.

                                                                                                                                                                                                                                  func (*Txn) Sub

                                                                                                                                                                                                                                  func (txn *Txn) Sub(fn TxnOp) error

                                                                                                                                                                                                                                    Sub executes fn in a subtransaction. Sub commits the subtransaction iff a nil error is returned by fn and otherwise aborts it. Sub returns any error it encounters.

                                                                                                                                                                                                                                    Sub may only be called on an Update Txn (one created without the Readonly flag). Calling Sub on a View transaction will return an error. Sub assumes the calling goroutine is locked to an OS thread and will not call runtime.LockOSThread.

                                                                                                                                                                                                                                    Any call to Abort, Commit, Renew, or Reset on a Txn created by Sub will panic.

                                                                                                                                                                                                                                    type TxnOp

                                                                                                                                                                                                                                    type TxnOp func(txn *Txn) error

                                                                                                                                                                                                                                      TxnOp is an operation applied to a managed transaction. The Txn passed to a TxnOp is managed and the operation must not call Commit, Abort, Renew, or Reset on it.

                                                                                                                                                                                                                                      IMPORTANT: TxnOps that write to the database (those passed to Env.Update or Txn.Sub) must not use the Txn in another goroutine (passing it directly or otherwise through closure). Doing so has undefined results.

                                                                                                                                                                                                                                      Notes

                                                                                                                                                                                                                                      Bugs

                                                                                                                                                                                                                                        • MDB_INTEGERKEY and MDB_INTEGERDUP aren't usable. I'm not sure they would be faster with the cgo bridge. They need to be tested and benchmarked.