datastore

README

Datastore 数据存储组件

将数据依据主键的方式进行存储，并能快速的通过创建函数(creator)和加载函数(loader)从缓存和数据源中进行批量加载。

一般用于作为用户或者其他数据模型的缓存

功能

• 提供store接口，便于根据实际数据类型自定义store实现，快速转换数据
• 使用提供的缓存进行数据的序列化与反序列化
• 预防缓存雪崩

缺点

• 只支持字符串类型的主键

使用方法

Store 数据存储集

储存所有已经从数据源和缓存中取出的数据的接口类型。

需要实现5个方法

Store.Load(key string) (value interface{}, ok bool)

从数据存储集中按给定主键取出对象值。

返回值value为具体的值

返回值ok为是否成功获取。

Store.Store(key string, value interface{})

将给到的数据(value)以指定主键(key)存入数据存储集

Store.Delete(key string)

在数据存储集中按主键删除数据。只有通过这个方法才能删除特定的数据，通过Store方法存入nil并不视为删除数据

Store.Flush()

清除所有数据

Store.LoadInterface(key string) interface{}

将数据强制以 interface{} 的形式取出。与Load方法的区别是并不关注数据是否存在。不存在的数据以nil返回

MapStore与SyncMapStore

MapStore和SyncMapStore的区别是具体实现的驱动

MapStore的实现是map[string]interface{},效率理论较高，非线程安全。创建方式为

 s:=datastore.NewMapStore()

SyncMapStore的实现是sync.Map，线程安全。创建方式为

 s:=datastore.NewSyncMapStore()

Creator/Loader方法

这两个是负责数据初始化和加载的函数

Creator的形式为

func() interface{}

返回一个初始化好的数据

Loader的形式为

func(...string) (map[string]interface{}, error)

通过给定的主键列表加载返回加载成功的数据集，或者任何发生的错误。

###直接通过Store/Creator/Loader 加载数据

err:=Load(store, datacache, loader, creator, key1,key2,key3,key4....)  

将主键列表中的数据加载到store中。同时在给定缓存中对未缓存的数据进行缓存

通过Datasource加载数据

Datasource是一个包含了数据Creator和Loader的结构。创建方式为

datasource:=datastore.NewDataSource()
datasource.Creator=creator
datasource.SourceLoader=loader

通过datasouce加载数据有两种方式

1.传入缓存，直接加载

err:=datasource.Load(store, c cache, key1,key2.key3...) error {

2.传入缓存，生成Loader,通过Loader加载

Loader是一个包含store,cache,datasource的结构

loader:=datasource.NewSyncMapStoreLoader(cache)
//如果需要使用MapStore的话,使用
//loader:=datasource.NewMapStoreLoader(cache)
err:=loader.Load(key1,key2,key3)
store:=loader.Store

对loader可以进行Delete和Flush操作,会同时对对应的Store和缓存进行操作

err:=loader.Delete(key)
//同时删除对应的数据存储集和缓存里的对应主键的值
err:=loader.Flust()
//同时清空对应的数据存储集和缓存

通过实现BatchDataLoader 接口的结构加载

BatchDataLoader 是一个包含数据Creator和Loader的接口。Datasource就是一个BatchDataLoader 的实现。

通过BatchDataLoader加载数据有两种方式

1.传入缓存，直接加载

err:=LoadWithBatchLoader(store, cache, batchdataloader, key1,key2,key3...)

2.传入缓存，生成Loader，通过Loader加载

loader:=NewSyncMapLoader(cache,batchdataloader)
//如果需要使用MapStore的话，使用
//loader:=NewMapLoader(cache,batchdataloader)
err:=loader.Load(key1,key2,key3)
store:=loader.Store

Documentation

Index

• func Load(s Store, c cache.Cacheable, ...) error
• func LoadWithBatchLoader(s Store, c cache.Cacheable, l BatchDataLoader, keys ...string) error
• type BatchDataLoader
• type DataSource
  • func NewDataSource() *DataSource
  • func (s *DataSource) BatchLoadData(keys ...string) (map[string]interface{}, error)
  • func (s *DataSource) Load(m Store, c cache.Cacheable, keys ...string) error
  • func (s *DataSource) NewDataElement() interface{}
  • func (s *DataSource) NewMapStoreLoader(c cache.Cacheable) *Loader
  • func (s *DataSource) NewSyncMapStoreLoader(c cache.Cacheable) *Loader
• type Loader
  • func NewLoader(s Store, c cache.Cacheable, l BatchDataLoader) *Loader
  • func NewMapLoader(c cache.Cacheable, l BatchDataLoader) *Loader
  • func NewSyncMapLoader(c cache.Cacheable, l BatchDataLoader) *Loader
  • func (l *Loader) Delete(key string) error
  • func (l *Loader) Flush() error
  • func (l *Loader) Load(keys ...string) error
• type MapStore
  • func NewMapStore() *MapStore
  • func (m *MapStore) Delete(key string)
  • func (m *MapStore) Flush()
  • func (m *MapStore) Load(key string) (value interface{}, ok bool)
  • func (m *MapStore) LoadInterface(key string) interface{}
  • func (m *MapStore) Store(key string, value interface{})
• type Store
• type SyncMapStore
  • func NewSyncMapStore() *SyncMapStore
  • func (m *SyncMapStore) Delete(key string)
  • func (m *SyncMapStore) Flush()
  • func (m *SyncMapStore) Load(key string) (value interface{}, ok bool)
  • func (m *SyncMapStore) LoadInterface(key string) interface{}
  • func (m *SyncMapStore) Map() *sync.Map
  • func (m *SyncMapStore) SetMap(smap *sync.Map)
  • func (m *SyncMapStore) Store(key string, value interface{})

Functions

func Load

func Load(s Store, c cache.Cacheable, loader func(...string) (map[string]interface{}, error), creator func() interface{}, keys ...string) error

Load load data by given key list into data store. Param s target data store. Param c map cache Param loader func by which data load. Param creator map element creator. Param keys key list to load Return any error if raised.

func LoadWithBatchLoader

func LoadWithBatchLoader(s Store, c cache.Cacheable, l BatchDataLoader, keys ...string) error

LoadWithBatchLoader load data by given key list into data store. Param s target data store. Param c map cache Param l BatchDataLoader Param keys key list to load Return any error if raised.

Types

type BatchDataLoader

type BatchDataLoader interface {
	// NewDataElement create empty data element.
	// Must return pointer of data.
	NewDataElement() interface{}
	//BatchLoadData   load data by given keys to map of data pointers.
	// Return map of data pointers and any error if raised.
	BatchLoadData(...string) (map[string]interface{}, error)
}

BatchDataLoader batch data loader interface

type DataSource

type DataSource struct {
	// Creator empty createor.
	// Must return pointer of data.
	Creator func() interface{}
	//SourceLoader  source loader which load data by given keys to map of data pointers.
	// Return map of data pointers and any error if raised.
	SourceLoader func(...string) (map[string]interface{}, error)
}

DataSource data store datasource

func NewDataSource

func NewDataSource() *DataSource

NewDataSource create new data source.

func (*DataSource) BatchLoadData

func (s *DataSource) BatchLoadData(keys ...string) (map[string]interface{}, error)

BatchLoadData load data by given keys to map of data pointers. Return map of data pointers and any error if raised.

func (*DataSource) Load

func (s *DataSource) Load(m Store, c cache.Cacheable, keys ...string) error

Load load data by given keys to store with given cache Return any error if rasied.

func (*DataSource) NewDataElement

func (s *DataSource) NewDataElement() interface{}

NewDataElement create empty data element. Must return pointer of data.

func (*DataSource) NewMapStoreLoader

func (s *DataSource) NewMapStoreLoader(c cache.Cacheable) *Loader

NewMapStoreLoader create map store loader with given cache

func (*DataSource) NewSyncMapStoreLoader

func (s *DataSource) NewSyncMapStoreLoader(c cache.Cacheable) *Loader

NewSyncMapStoreLoader create sync map store loader with given cache

type Loader

type Loader struct {
	Cache           cache.Cacheable
	Store           Store
	BatchDataLoader BatchDataLoader
}

Loader data store loader

func NewLoader

func NewLoader(s Store, c cache.Cacheable, l BatchDataLoader) *Loader

NewLoader create new loader with given store,cache and batchlaoder

func NewMapLoader

func NewMapLoader(c cache.Cacheable, l BatchDataLoader) *Loader

NewMapLoader create new loader with new MapStore,cache and batchlaoder

func NewSyncMapLoader

func NewSyncMapLoader(c cache.Cacheable, l BatchDataLoader) *Loader

NewSyncMapLoader create new loader with new SyncMapStore,cache and batchlaoder

func (*Loader) Delete

func (l *Loader) Delete(key string) error

Delete delete value from store and cache by given key.

func (*Loader) Flush

func (l *Loader) Flush() error

Flush flush all data from store and cache.

func (*Loader) Load

func (l *Loader) Load(keys ...string) error

Load load data to store by given keys.

type MapStore

type MapStore map[string]interface{}

MapStore store which stores value in map. You should confirm safe for concurrent by yourself.

func NewMapStore

func NewMapStore() *MapStore

NewMapStore create new map store

func (*MapStore) Delete

func (m *MapStore) Delete(key string)

Delete delete value from store with given key

func (*MapStore) Flush

func (m *MapStore) Flush()

Flush flush store

func (*MapStore) Load

func (m *MapStore) Load(key string) (value interface{}, ok bool)

Load load value with given key Return value and whether load successfully

func (*MapStore) LoadInterface

func (m *MapStore) LoadInterface(key string) interface{}

LoadInterface Load load value with given key Return loaded value or nil if load fail

func (*MapStore) Store

func (m *MapStore) Store(key string, value interface{})

Store sotre value with given key

type Store

type Store interface {
	// Load load value with given key
	// Return value and whether load successfully
	Load(key string) (value interface{}, ok bool)
	// Store store value with given key
	Store(key string, value interface{})
	// Delete delete value from store with given key
	Delete(key string)
	// Flush flush store
	Flush()
	// LoadInterface Load load value with given key
	// Return loaded value or nil if load fail
	LoadInterface(key string) interface{}
}

Store interface of data store

type SyncMapStore

type SyncMapStore struct {
	// contains filtered or unexported fields
}

SyncMapStore store which stores value in sync.map.

func NewSyncMapStore

func NewSyncMapStore() *SyncMapStore

NewSyncMapStore create new sync.map store

func (*SyncMapStore) Delete

func (m *SyncMapStore) Delete(key string)

Delete delete value from store with given key

func (*SyncMapStore) Flush

func (m *SyncMapStore) Flush()

Flush flush store

func (*SyncMapStore) Load

func (m *SyncMapStore) Load(key string) (value interface{}, ok bool)

Load load value with given key Return value and whether load successfully

func (*SyncMapStore) LoadInterface

func (m *SyncMapStore) LoadInterface(key string) interface{}

LoadInterface Load load value with given key Return loaded value or nil if load fail

func (*SyncMapStore) Map

func (m *SyncMapStore) Map() *sync.Map

Map return sync map from store

func (*SyncMapStore) SetMap

func (m *SyncMapStore) SetMap(smap *sync.Map)

SetMap set sync map to store

func (*SyncMapStore) Store

func (m *SyncMapStore) Store(key string, value interface{})

Store sotre value with given key