cmap

README

concurrent map

As explained here and here, the map type in Go doesn't support concurrent reads and writes. concurrent-map provides a high-performance solution to this by sharding the map with minimal time spent waiting for locks.

usage

Import the package:

import (
	"github.com/orcaman/concurrent-map"
)

go get "github.com/orcaman/concurrent-map"
go test
go test -bench="."

The package is now imported under the "cmap" namespace.

example

	// Create a new map. param is shardCont, for example 128 shards
	map := cmap.New(128)
	
	// Sets item within map, sets "bar" under key "foo"
	map.Set("foo", "bar")

	// Retrieve item from map.
	if tmp, ok := map.Get("foo"); ok {
		bar := tmp.(string)
	}

	// Removes item under key "foo"
	map.Remove("foo")

For more examples have a look at concurrent_map_test.go.

Running tests:

go test "github.com/orcaman/concurrent-map"

templating

To generate your own custom concurrent maps please use concurrent_map_template.txt, the file is a base template for type specific maps. For Example to create a new go source file for a string:int map, in terminal run:

You can change the string and the int in the sed command to whatever you need.

license

MIT (see LICENSE file)

Documentation

Index

• type ConcurrentHashMap
  • func New(shards int) *ConcurrentHashMap
  • func (m *ConcurrentHashMap) AddIfPresent(key string, value interface{}) bool
  • func (m *ConcurrentHashMap) Count() int
  • func (m *ConcurrentHashMap) Get(key string) (interface{}, bool)
  • func (m *ConcurrentHashMap) GetShard(key string) *ConcurrentMapShared
  • func (m *ConcurrentHashMap) Has(key string) bool
  • func (m *ConcurrentHashMap) IsEmpty() bool
  • func (m *ConcurrentHashMap) Items() map[string]interface{}
  • func (m *ConcurrentHashMap) ItemsLike(like string) map[string]interface{}
  • func (m *ConcurrentHashMap) Iter() <-chan Tuple
  • func (m *ConcurrentHashMap) IterBuffered() <-chan Tuple
  • func (m *ConcurrentHashMap) IterBufferedLike(k string) <-chan Tuple
  • func (m *ConcurrentHashMap) IterCb(fn IterCb)
  • func (m *ConcurrentHashMap) Keys() []string
  • func (m *ConcurrentHashMap) MSet(data map[string]interface{})
  • func (m *ConcurrentHashMap) MarshalJSON() ([]byte, error)
  • func (m *ConcurrentHashMap) Pop(key string) (v interface{}, exists bool)
  • func (m *ConcurrentHashMap) Remove(key string)
  • func (m *ConcurrentHashMap) Set(key string, value interface{})
  • func (m *ConcurrentHashMap) SetIfAbsent(key string, value interface{}) bool
  • func (m *ConcurrentHashMap) SetIfPresent(key string, newValue, oldValue interface{}) bool
  • func (m *ConcurrentHashMap) Upsert(key string, value interface{}, cb UpsertCb) (res interface{})
• type ConcurrentMap
• type ConcurrentMapShared
• type IterCb
• type Tuple
• type UpsertCb

Types

type ConcurrentHashMap

type ConcurrentHashMap struct {
	Shards  int
	HashMap ConcurrentMap
}

func New

func New(shards int) *ConcurrentHashMap

Creates a new concurrent map.

func (*ConcurrentHashMap) AddIfPresent

func (m *ConcurrentHashMap) AddIfPresent(key string, value interface{}) bool

Sets the given value under the specified key if oldValue was associated with it.

func (*ConcurrentHashMap) Count

func (m *ConcurrentHashMap) Count() int

Returns the number of elements within the map.

func (*ConcurrentHashMap) Get

func (m *ConcurrentHashMap) Get(key string) (interface{}, bool)

Retrieves an element from map under given key.

func (*ConcurrentHashMap) GetShard

func (m *ConcurrentHashMap) GetShard(key string) *ConcurrentMapShared

Returns shard under given key

func (*ConcurrentHashMap) Has

func (m *ConcurrentHashMap) Has(key string) bool

Looks up an item under specified key

func (*ConcurrentHashMap) IsEmpty

func (m *ConcurrentHashMap) IsEmpty() bool

Checks if map is empty.

func (*ConcurrentHashMap) Items

func (m *ConcurrentHashMap) Items() map[string]interface{}

Returns all items as map[string]interface{}

func (*ConcurrentHashMap) ItemsLike

func (m *ConcurrentHashMap) ItemsLike(like string) map[string]interface{}

Returns all items as map[string]interface{}

func (*ConcurrentHashMap) Iter

func (m *ConcurrentHashMap) Iter() <-chan Tuple

Returns an iterator which could be used in a for range loop.

Deprecated: using IterBuffered() will get a better performence

func (*ConcurrentHashMap) IterBuffered

func (m *ConcurrentHashMap) IterBuffered() <-chan Tuple

Returns a buffered iterator which could be used in a for range loop.

func (*ConcurrentHashMap) IterBufferedLike

func (m *ConcurrentHashMap) IterBufferedLike(k string) <-chan Tuple

Returns a buffered iterator which could be used in a for range loop.

func (*ConcurrentHashMap) IterCb

func (m *ConcurrentHashMap) IterCb(fn IterCb)

Callback based iterator, cheapest way to read all elements in a map.

func (*ConcurrentHashMap) Keys

func (m *ConcurrentHashMap) Keys() []string

Return all keys as []string

func (*ConcurrentHashMap) MSet

func (m *ConcurrentHashMap) MSet(data map[string]interface{})

Sets the given map

func (*ConcurrentHashMap) MarshalJSON

func (m *ConcurrentHashMap) MarshalJSON() ([]byte, error)

Reviles ConcurrentHashMap "private" variables to json marshal.

func (*ConcurrentHashMap) Pop

func (m *ConcurrentHashMap) Pop(key string) (v interface{}, exists bool)

Removes an element from the map and returns it

func (*ConcurrentHashMap) Remove

func (m *ConcurrentHashMap) Remove(key string)

Removes an element from the map.

func (*ConcurrentHashMap) Set

func (m *ConcurrentHashMap) Set(key string, value interface{})

Sets the given value under the specified key.

func (*ConcurrentHashMap) SetIfAbsent

func (m *ConcurrentHashMap) SetIfAbsent(key string, value interface{}) bool

Sets the given value under the specified key if no value was associated with it.

func (*ConcurrentHashMap) SetIfPresent

func (m *ConcurrentHashMap) SetIfPresent(key string, newValue, oldValue interface{}) bool

Sets the given value under the specified key if oldValue was associated with it.

func (*ConcurrentHashMap) Upsert

func (m *ConcurrentHashMap) Upsert(key string, value interface{}, cb UpsertCb) (res interface{})

Insert or Update - updates existing element or inserts a new one using UpsertCb

type ConcurrentMap

type ConcurrentMap []*ConcurrentMapShared

A "thread" safe map of type string:Anything. To avoid lock bottlenecks this map is dived to several (Shards) map shards.

type ConcurrentMapShared

type ConcurrentMapShared struct {
	sync.RWMutex // Read Write mutex, guards access to internal map.
	// contains filtered or unexported fields
}

A "thread" safe string to anything map.

type IterCb

type IterCb func(key string, v interface{})

Iterator callback,called for every key,value found in maps. RLock is held for all calls for a given shard therefore callback sess consistent view of a shard, but not across the shards

type Tuple

type Tuple struct {
	Key string
	Val interface{}
}

Used by the Iter & IterBuffered functions to wrap two variables together over a channel,

type UpsertCb

type UpsertCb func(exist bool, valueInMap interface{}, newValue interface{}) interface{}

Callback to return new element to be inserted into the map It is called while lock is held, therefore it MUST NOT try to access other keys in same map, as it can lead to deadlock since Go sync.RWLock is not reentrant