Documentation

Overview

    Package lmdbpool provides a TxnPool type that allows lmdb.Readonly transactions to safely be reused by other goroutines when the goroutine that created the transaction no longer has a use for it. The TxnPool type has benefits that would be absent in a naive use of sync.Pool with lmdb.Txn types.

    Naively reusing lmdb.Readonly transactions can cause updates to continually allocate more pages for the database instead of reusing stale pages. The TxnPool type tracks transaction ids to make sure that lmdb.Readonly transactions are not reused when they are known to be holding stale pages which could be reclaimed by LMDB.

    A general downside of pooling lmdb.Readonly transactions using a sync.Pool in applications with a very high rate of transacions is that the number of readers in an environment can be significantly higher than the number of goroutines actively trying to read from that environment. Because of this it is possible that applications may need to increase the maximum number of readers allowed in the environment at initialization time.

    err := env.SetMaxReaders(maxReaders)
    

    In a naive pooling implementation an application compiled with the -race flag may require an extremely large number of open readers. The TxnPool type attempts to keep the value required for Env.SetMaxReaders as low as possible in the presence of -race but there is a limited amount that can be done for a concurrent workload with a rapid enough rate of transactions.

    Index

    Constants

    This section is empty.

    Variables

    This section is empty.

    Functions

    This section is empty.

    Types

    type TxnPool

    type TxnPool struct {
    	// UpdateHandling determines how a TxnPool behaves after updates have been
    	// committed.  It is not safe to modify UpdateHandling if TxnPool is being
    	// used concurrently.
    	UpdateHandling UpdateHandling
    	// contains filtered or unexported fields
    }

      TxnPool is a pool for reusing transactions through their Reset and Renew methods. However, even though TxnPool can only reuse lmdb.Readonly transactions it this way it should be used to create and terminate all Txns if it is used at all. Update transactions can cause LMDB to use excessive numbers of pages when there are long-lived lmdb.Readonly transactions in a TxnPool. Executing all transactions using the TxnPool allows it to track updates and prevent long-lived updates from causing excessive disk utilization.

      func NewTxnPool

      func NewTxnPool(env *lmdb.Env) *TxnPool

        NewTxnPool initializes returns a new TxnPool.

        func (*TxnPool) Abort

        func (p *TxnPool) Abort(txn *lmdb.Txn)

          Abort aborts the txn and allows it to be reused if possible. Abort must only be passed lmdb.Txn objects which it returned from a call to BeginTxn. Aborting a transaction created through other means will have undefined results.

          func (*TxnPool) BeginTxn

          func (p *TxnPool) BeginTxn(flags uint) (*lmdb.Txn, error)

            BeginTxn is analogous to the BeginTxn method on lmdb.Env but may only be used to create lmdb.Readonly transactions. Any call to BeginTxn that does not include lmdb.Readonly will return an error

            func (*TxnPool) Close

            func (p *TxnPool) Close()

              Close flushes the pool of transactions and aborts them to free resources so that the pool Env may be closed.

              func (*TxnPool) CommitID

              func (p *TxnPool) CommitID(id uintptr)

                CommitID sets the TxnPool's last-known transaction id to invalidate previously created lmdb.Readonly transactions and prevent their reuse.

                CommitID should only be called if p is not used to create/commit update transactions.

                func (*TxnPool) Update

                func (p *TxnPool) Update(fn lmdb.TxnOp) error

                  Update is analogous to the Update method on lmdb.Env.

                  func (*TxnPool) View

                  func (p *TxnPool) View(fn lmdb.TxnOp) error

                    View is analogous to the View method on lmdb.Env.

                    type UpdateHandling

                    type UpdateHandling uint

                      UpdateHandling describes how a TxnPool handles existing lmdb.Readonly transactions when an environment update occurs. Applications with a high rate of large updates may need to choose non-default settings to reduce their storage requirements at the cost of read throughput.

                      The zero-value of UpdateHandling causes a TxnPool to ignore all updates and defers to the application and the lmdb.Txn finalizers clear stale readers (pulling an lmdb.Readonly transaction out of the pool is enough to release its stale pages).

                      const (
                      	// HandleOutstanding causes a TxnPool to abort any lmdb.Readonly
                      	// transactions that are being returned to the pool after an update.
                      	HandleOutstanding UpdateHandling = 1 << iota
                      
                      	// HandleIdle causes a TxnPool to actively attempt aborting idle
                      	// transactions in the sync.Pool after an update has been committed.  There
                      	// is no guarantee when using AbortIdle that all idle readers will be
                      	// aborted.
                      	HandleIdle
                      
                      	// HandleRenew modifies how other UpdateHandling flags are interpretted and
                      	// causes a TxnPool to renew transactions and put them back in the pool
                      	// instead of aborting them.
                      	HandleRenew
                      
                      	// HandleAll is a convenient alias for the combination of HandleOutstanding
                      	// and HandleIdle.
                      	HandleAll = HandleOutstanding | HandleIdle
                      )