Documentation

Overview

Package database provides a universal interface for interacting with the database.

A Lazy Database

The database system can handle Go structs as well as serialized data by the dsd package. While data is in transit within the system, it does not know which form it currently has. Only when it reaches its destination, it must ensure that it is either of a certain type or dump it.

Record Interface

The database system uses the Record interface to transparently handle all types of structs that get saved in the database. Structs include the Base struct to fulfill most parts of the Record interface.

Boilerplate Code:

type Example struct {
  record.Base
  sync.Mutex

  Name  string
  Score int
}

var (
  db = database.NewInterface(nil)
)

// GetExample gets an Example from the database.
func GetExample(key string) (*Example, error) {
  r, err := db.Get(key)
  if err != nil {
    return nil, err
  }

  // unwrap
  if r.IsWrapped() {
    // only allocate a new struct, if we need it
    new := &Example{}
    err = record.Unwrap(r, new)
    if err != nil {
      return nil, err
    }
    return new, nil
  }

  // or adjust type
  new, ok := r.(*Example)
  if !ok {
    return nil, fmt.Errorf("record not of type *Example, but %T", r)
  }
  return new, nil
}

func (e *Example) Save() error {
  return db.Put(e)
}

func (e *Example) SaveAs(key string) error {
  e.SetKey(key)
  return db.PutNew(e)
}

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrNotFound         = errors.New("database entry not found")
	ErrPermissionDenied = errors.New("access to database record denied")
	ErrReadOnly         = errors.New("database is read only")
	ErrShuttingDown     = errors.New("database system is shutting down")
	ErrNotImplemented   = errors.New("not implemented by this storage")
)

    Errors

    Functions

    func EnableRegistryPersistence

    func EnableRegistryPersistence()

      EnableRegistryPersistence enables persistence of the database registry.

      func Initialize

      func Initialize(dirStructureRoot *utils.DirStructure) error

        Initialize initializes the database at the specified location using a dir structure.

        func InitializeWithPath

        func InitializeWithPath(dirPath string) error

          InitializeWithPath initializes the database at the specified location using a path.

          func Maintain

          func Maintain(ctx context.Context) (err error)

            Maintain runs the Maintain method on all storages.

            func MaintainRecordStates

            func MaintainRecordStates(ctx context.Context) (err error)

              MaintainRecordStates runs record state lifecycle maintenance on all storages.

              func MaintainThorough

              func MaintainThorough(ctx context.Context) (err error)

                MaintainThorough runs the MaintainThorough method on all storages.

                func Shutdown

                func Shutdown() (err error)

                  Shutdown shuts down the whole database system.

                  Types

                  type Controller

                  type Controller struct {
                  	// contains filtered or unexported fields
                  }

                    A Controller takes care of all the extra database logic.

                    func InjectDatabase

                    func InjectDatabase(name string, storageInt storage.Interface) (*Controller, error)

                      InjectDatabase injects an already running database into the system.

                      func (*Controller) Get

                      func (c *Controller) Get(key string) (record.Record, error)

                        Get return the record with the given key.

                        func (*Controller) Injected

                        func (c *Controller) Injected() bool

                          Injected returns whether the storage is injected.

                          func (*Controller) Maintain

                          func (c *Controller) Maintain(ctx context.Context) error

                            Maintain runs the Maintain method on the storage.

                            func (*Controller) MaintainRecordStates

                            func (c *Controller) MaintainRecordStates(ctx context.Context, purgeDeletedBefore time.Time) error

                              MaintainRecordStates runs the record state lifecycle maintenance on the storage.

                              func (*Controller) MaintainThorough

                              func (c *Controller) MaintainThorough(ctx context.Context) error

                                MaintainThorough runs the MaintainThorough method on the storage.

                                func (*Controller) Purge

                                func (c *Controller) Purge(ctx context.Context, q *query.Query, local, internal bool) (int, error)

                                  Purge deletes all records that match the given query. It returns the number of successful deletes and an error.

                                  func (*Controller) PushUpdate

                                  func (c *Controller) PushUpdate(r record.Record)

                                    PushUpdate pushes a record update to subscribers. The caller must hold the record's lock when calling PushUpdate.

                                    func (*Controller) Put

                                    func (c *Controller) Put(r record.Record) (err error)

                                      Put saves a record in the database, executes any registered pre-put hooks and finally send an update to all subscribers. The record must be locked and secured from concurrent access when calling Put().

                                      func (*Controller) PutMany

                                      func (c *Controller) PutMany() (chan<- record.Record, <-chan error)

                                        PutMany stores many records in the database. It does not process any hooks or update subscriptions. Use with care!

                                        func (*Controller) Query

                                        func (c *Controller) Query(q *query.Query, local, internal bool) (*iterator.Iterator, error)

                                          Query executes the given query on the database.

                                          func (*Controller) ReadOnly

                                          func (c *Controller) ReadOnly() bool

                                            ReadOnly returns whether the storage is read only.

                                            func (*Controller) Shutdown

                                            func (c *Controller) Shutdown() error

                                              Shutdown shuts down the storage.

                                              type Database

                                              type Database struct {
                                              	Name         string
                                              	Description  string
                                              	StorageType  string
                                              	ShadowDelete bool // Whether deleted records should be kept until purged.
                                              	Registered   time.Time
                                              	LastUpdated  time.Time
                                              	LastLoaded   time.Time
                                              }

                                                Database holds information about registered databases

                                                func Register

                                                func Register(new *Database) (*Database, error)

                                                  Register registers a new database. If the database is already registered, only the description and the primary API will be updated and the effective object will be returned.

                                                  func (*Database) Loaded

                                                  func (db *Database) Loaded()

                                                    Loaded updates the LastLoaded timestamp.

                                                    func (*Database) Updated

                                                    func (db *Database) Updated()

                                                      Updated updates the LastUpdated timestamp.

                                                      type Hook

                                                      type Hook interface {
                                                      	// UsesPreGet should return true if the hook's PreGet
                                                      	// should be called prior to loading a database record
                                                      	// from the underlying storage.
                                                      	UsesPreGet() bool
                                                      	// PreGet is called before a database record is loaded from
                                                      	// the underlying storage. A PreGet hookd may be used to
                                                      	// implement more advanced access control on database keys.
                                                      	PreGet(dbKey string) error
                                                      	// UsesPostGet should return true if the hook's PostGet
                                                      	// should be called after loading a database record from
                                                      	// the underlying storage.
                                                      	UsesPostGet() bool
                                                      	// PostGet is called after a record has been loaded form the
                                                      	// underlying storage and may perform additional mutation
                                                      	// or access check based on the records data.
                                                      	// The passed record is already locked by the database system
                                                      	// so users can safely access all data of r.
                                                      	PostGet(r record.Record) (record.Record, error)
                                                      	// UsesPrePut should return true if the hook's PrePut method
                                                      	// should be called prior to saving a record in the database.
                                                      	UsesPrePut() bool
                                                      	// PrePut is called prior to saving (creating or updating) a
                                                      	// record in the database storage. It may be used to perform
                                                      	// extended validation or mutations on the record.
                                                      	// The passed record is already locked by the database system
                                                      	// so users can safely access all data of r.
                                                      	PrePut(r record.Record) (record.Record, error)
                                                      }

                                                        Hook can be registered for a database query and will be executed at certain points during the life cycle of a database record.

                                                        type HookBase

                                                        type HookBase struct {
                                                        }

                                                          HookBase implements the Hook interface and provides dummy functions to reduce boilerplate.

                                                          func (*HookBase) PostGet

                                                          func (b *HookBase) PostGet(r record.Record) (record.Record, error)

                                                            PostGet implements the Hook interface.

                                                            func (*HookBase) PreGet

                                                            func (b *HookBase) PreGet(dbKey string) error

                                                              PreGet implements the Hook interface.

                                                              func (*HookBase) PrePut

                                                              func (b *HookBase) PrePut(r record.Record) (record.Record, error)

                                                                PrePut implements the Hook interface.

                                                                func (*HookBase) UsesPostGet

                                                                func (b *HookBase) UsesPostGet() bool

                                                                  UsesPostGet implements the Hook interface and returns false.

                                                                  func (*HookBase) UsesPreGet

                                                                  func (b *HookBase) UsesPreGet() bool

                                                                    UsesPreGet implements the Hook interface and returns false.

                                                                    func (*HookBase) UsesPrePut

                                                                    func (b *HookBase) UsesPrePut() bool

                                                                      UsesPrePut implements the Hook interface and returns false.

                                                                      type Interface

                                                                      type Interface struct {
                                                                      	// contains filtered or unexported fields
                                                                      }

                                                                        Interface provides a method to access the database with attached options.

                                                                        func NewInterface

                                                                        func NewInterface(opts *Options) *Interface

                                                                          NewInterface returns a new Interface to the database.

                                                                          func (*Interface) ClearCache

                                                                          func (i *Interface) ClearCache()

                                                                            ClearCache clears the read cache.

                                                                            func (*Interface) DelayedCacheWriter

                                                                            func (i *Interface) DelayedCacheWriter(ctx context.Context) error

                                                                              DelayedCacheWriter must be run by the caller of an interface that uses delayed cache writing.

                                                                              func (*Interface) Delete

                                                                              func (i *Interface) Delete(key string) error

                                                                                Delete deletes a record from the database.

                                                                                func (*Interface) Exists

                                                                                func (i *Interface) Exists(key string) (bool, error)

                                                                                  Exists return whether a record with the given key exists.

                                                                                  func (*Interface) FlushCache

                                                                                  func (i *Interface) FlushCache()

                                                                                    FlushCache writes (and thus clears) the write cache.

                                                                                    func (*Interface) Get

                                                                                    func (i *Interface) Get(key string) (record.Record, error)

                                                                                      Get return the record with the given key.

                                                                                      func (*Interface) InsertValue

                                                                                      func (i *Interface) InsertValue(key string, attribute string, value interface{}) error

                                                                                        InsertValue inserts a value into a record.

                                                                                        func (*Interface) MakeCrownJewel

                                                                                        func (i *Interface) MakeCrownJewel(key string) error

                                                                                          MakeCrownJewel marks a record as a crown jewel, meaning it will only be accessible locally.

                                                                                          func (*Interface) MakeSecret

                                                                                          func (i *Interface) MakeSecret(key string) error

                                                                                            MakeSecret marks the record as a secret, meaning interfacing processes, such as an UI, are denied access to the record.

                                                                                            func (*Interface) Purge

                                                                                            func (i *Interface) Purge(ctx context.Context, q *query.Query) (int, error)

                                                                                              Purge deletes all records that match the given query. It returns the number of successful deletes and an error.

                                                                                              func (*Interface) Put

                                                                                              func (i *Interface) Put(r record.Record) (err error)

                                                                                                Put saves a record to the database.

                                                                                                func (*Interface) PutMany

                                                                                                func (i *Interface) PutMany(dbName string) (put func(record.Record) error)

                                                                                                  PutMany stores many records in the database. Warning: This is nearly a direct database access and omits many things: - Record locking - Hooks - Subscriptions - Caching

                                                                                                  func (*Interface) PutNew

                                                                                                  func (i *Interface) PutNew(r record.Record) (err error)

                                                                                                    PutNew saves a record to the database as a new record (ie. with new timestamps).

                                                                                                    func (*Interface) Query

                                                                                                    func (i *Interface) Query(q *query.Query) (*iterator.Iterator, error)

                                                                                                      Query executes the given query on the database.

                                                                                                      func (*Interface) SetAbsoluteExpiry

                                                                                                      func (i *Interface) SetAbsoluteExpiry(key string, time int64) error

                                                                                                        SetAbsoluteExpiry sets an absolute record expiry.

                                                                                                        func (*Interface) SetRelativateExpiry

                                                                                                        func (i *Interface) SetRelativateExpiry(key string, duration int64) error

                                                                                                          SetRelativateExpiry sets a relative (self-updating) record expiry.

                                                                                                          func (*Interface) Subscribe

                                                                                                          func (i *Interface) Subscribe(q *query.Query) (*Subscription, error)

                                                                                                            Subscribe subscribes to updates matching the given query.

                                                                                                            type Options

                                                                                                            type Options struct {
                                                                                                            	// Local specifies if the interface is used by an actor on the local device.
                                                                                                            	// Setting both the Local and Internal flags will bring performance
                                                                                                            	// improvements because less checks are needed.
                                                                                                            	Local bool
                                                                                                            
                                                                                                            	// Internal specifies if the interface is used by an actor within the
                                                                                                            	// software. Setting both the Local and Internal flags will bring performance
                                                                                                            	// improvements because less checks are needed.
                                                                                                            	Internal bool
                                                                                                            
                                                                                                            	// AlwaysMakeSecret will have the interface mark all saved records as secret.
                                                                                                            	// This means that they will be only accessible by an internal interface.
                                                                                                            	AlwaysMakeSecret bool
                                                                                                            
                                                                                                            	// AlwaysMakeCrownjewel will have the interface mark all saved records as
                                                                                                            	// crown jewels. This means that they will be only accessible by a local
                                                                                                            	// interface.
                                                                                                            	AlwaysMakeCrownjewel bool
                                                                                                            
                                                                                                            	// AlwaysSetRelativateExpiry will have the interface set a relative expiry,
                                                                                                            	// based on the current time, on all saved records.
                                                                                                            	AlwaysSetRelativateExpiry int64
                                                                                                            
                                                                                                            	// AlwaysSetAbsoluteExpiry will have the interface set an absolute expiry on
                                                                                                            	// all saved records.
                                                                                                            	AlwaysSetAbsoluteExpiry int64
                                                                                                            
                                                                                                            	// CacheSize defines that a cache should be used for this interface and
                                                                                                            	// defines it's size.
                                                                                                            	// Caching comes with an important caveat: If database records are changed
                                                                                                            	// from another interface, the cache will not be invalidated for these
                                                                                                            	// records. It will therefore serve outdated data until that record is
                                                                                                            	// evicted from the cache.
                                                                                                            	CacheSize int
                                                                                                            
                                                                                                            	// DelayCachedWrites defines a database name for which cache writes should
                                                                                                            	// be cached and batched. The database backend must support the Batcher
                                                                                                            	// interface. This option is only valid if used with a cache.
                                                                                                            	// Additionally, this may only be used for internal and local interfaces.
                                                                                                            	// Please note that this means that other interfaces will not be able to
                                                                                                            	// guarantee to serve the latest record if records are written this way.
                                                                                                            	DelayCachedWrites string
                                                                                                            }

                                                                                                              Options holds options that may be set for an Interface instance.

                                                                                                              func (*Options) Apply

                                                                                                              func (o *Options) Apply(r record.Record)

                                                                                                                Apply applies options to the record metadata.

                                                                                                                func (*Options) HasAllPermissions

                                                                                                                func (o *Options) HasAllPermissions() bool

                                                                                                                  HasAllPermissions returns whether the options specify the highest possible permissions for operations.

                                                                                                                  type RegisteredHook

                                                                                                                  type RegisteredHook struct {
                                                                                                                  	// contains filtered or unexported fields
                                                                                                                  }

                                                                                                                    RegisteredHook is a registered database hook.

                                                                                                                    func RegisterHook

                                                                                                                    func RegisterHook(q *query.Query, hook Hook) (*RegisteredHook, error)

                                                                                                                      RegisterHook registers a hook for records matching the given query in the database.

                                                                                                                      func (*RegisteredHook) Cancel

                                                                                                                      func (h *RegisteredHook) Cancel() error

                                                                                                                        Cancel unregisteres the hook from the database. Once Cancel returned the hook's methods will not be called anymore for updates that matched the registered query.

                                                                                                                        type Subscription

                                                                                                                        type Subscription struct {
                                                                                                                        	Feed chan record.Record
                                                                                                                        	// contains filtered or unexported fields
                                                                                                                        }

                                                                                                                          Subscription is a database subscription for updates.

                                                                                                                          func (*Subscription) Cancel

                                                                                                                          func (s *Subscription) Cancel() error

                                                                                                                            Cancel cancels the subscription.

                                                                                                                            Directories

                                                                                                                            Path Synopsis
                                                                                                                            fstree
                                                                                                                            Package fstree provides a dead simple file-based database storage backend.
                                                                                                                            Package fstree provides a dead simple file-based database storage backend.