concurrent_map

README

介绍

分片并发map

功能

• 提供基础map操作，Get, Set, Del, Count, Iter(Range), Snapshot, Clear 等功能；
• 通过自定义hash算法生成key进行partition; 比如：google cityhash 为了满足大数据量的需求，减少碰撞

参考

1. concurrent-map
2. allegro/bigcache

Documentation

Index

• type ConcurrentMap
  • func CreateConcurrentMap(numOfPartitions int) *ConcurrentMap
  • func (m *ConcurrentMap) Clear()
  • func (m ConcurrentMap) Count() int
  • func (m *ConcurrentMap) Del(key Partitionable)
  • func (m *ConcurrentMap) Get(key Partitionable) (interface{}, bool)
  • func (m *ConcurrentMap) IterBuffFromSnapshot() <-chan Tuple
  • func (m *ConcurrentMap) Set(key Partitionable, v interface{})
  • func (m *ConcurrentMap) Snapshot() (snapshotChs []chan Tuple)
• type Int64Key
  • func I64Key(key int64) *Int64Key
  • func (i *Int64Key) PartitionKey() int64
  • func (i *Int64Key) Value() interface{}
• type Partitionable
• type StringKey
  • func StrKey(key string) *StringKey
  • func (s *StringKey) PartitionKey() int64
  • func (s *StringKey) Value() interface{}
• type Tuple

Types

type ConcurrentMap

type ConcurrentMap struct {
	// contains filtered or unexported fields
}

ConcurrentMap is a thread safe map collection with better performance. The backend map entries are separated into the different partitions. Threads can access the different partitions safely without lock.

func CreateConcurrentMap

func CreateConcurrentMap(numOfPartitions int) *ConcurrentMap

CreateConcurrentMap is to create a ConcurrentMap with the setting number of the partitions

func (*ConcurrentMap) Clear

func (m *ConcurrentMap) Clear()

clear all shard map

func (ConcurrentMap) Count

func (m ConcurrentMap) Count() int

Count returns the number of elements within the map.

func (*ConcurrentMap) Del

func (m *ConcurrentMap) Del(key Partitionable)

Del is to delete the entries by the key

func (*ConcurrentMap) Get

func (m *ConcurrentMap) Get(key Partitionable) (interface{}, bool)

Get is to get the value by the key

func (*ConcurrentMap) IterBuffFromSnapshot

func (m *ConcurrentMap) IterBuffFromSnapshot() <-chan Tuple

snapshot shard map fan out into channels; then fan in out channels for read;

func (*ConcurrentMap) Set

func (m *ConcurrentMap) Set(key Partitionable, v interface{})

Set is to store the KV entry to the map

func (*ConcurrentMap) Snapshot

func (m *ConcurrentMap) Snapshot() (snapshotChs []chan Tuple)

snapshot shard map fan out into channels

type Int64Key

type Int64Key struct {
	// contains filtered or unexported fields
}

func I64Key

func I64Key(key int64) *Int64Key

StrKey is to convert a string to StringKey

func (*Int64Key) PartitionKey

func (i *Int64Key) PartitionKey() int64

PartitionID is created by string's hash

func (*Int64Key) Value

func (i *Int64Key) Value() interface{}

Value is the raw string

type Partitionable

type Partitionable interface {
	// Value is raw value of the key
	Value() interface{}

	// PartitionKey is used for getting the partition to store the entry with the key.
	// E.g. the key's hash could be used as its PartitionKey
	// The partition for the key is partitions[(PartitionKey % m.numOfBlockets)]
	//
	// 1 Why not provide the default hash function for partition?
	// Ans: As you known, the partition solution would impact the performance significantly.
	// The proper partition solution balances the access to the different partitions and
	// avoid of the hot partition. The access mode highly relates to your business.
	// So, the better partition solution would just be designed according to your business.
	PartitionKey() int64
}

Partitionable is the interface which should be implemented by key type. It is to define how to partition the entries.

type StringKey

type StringKey struct {
	// contains filtered or unexported fields
}

StringKey is for the string type key

func StrKey

func StrKey(key string) *StringKey

StrKey is to convert a string to StringKey

func (*StringKey) PartitionKey

func (s *StringKey) PartitionKey() int64

PartitionID is created by string's hash

func (*StringKey) Value

func (s *StringKey) Value() interface{}

Value is the raw string

type Tuple

type Tuple struct {
	Key interface{}
	Val interface{}
}