nitecache

README

nitecache

golang cache library

Table of Contents

• Getting started
• Roadmap
• Contributing
• License

About The Project

disclaimer : This project is still experimental. Expect breaking changes.

nitecache is an embedded and distributed cache library for golang that supports:

• sharding
• explicit cache eviction
• ttl
• lfu/lru eviction policies
• rpc for value updates
• type safety using generics

Getting started

Installation

• requires go version >= 1.21

go get github.com/MysteriousPotato/nitecache@v0.4.4

Usage

Creating a cache instance:

// Both ID and Addr must be unique across peers
self := {ID: "1", Addr: "node1:8100"},
members := []nitecache.Member{
    self, // You can omit self from the members list
    {ID: "2", Addr: "node2:8100"},
}

cache, err := nitecache.NewCache(self, members)
...
go func() {
    err := cache.ListenAndServe()
    ...
}

Creating a table:

// Specify the name of the table
table := nitecache.NewTable[string]("sessions").
    // If WithEvictionPolicy is omitted, nitecache won't apply any eviction policy
    WithStorage(nitecache.LRU(1024)).
    // Option to specify the cache-aside getter
    // If WithGetter is omitted, nitecache will return an error on cache miss. 
    WithGetter(func(ctx context.Context, key string) (Session, time.Duration, error) {
        sess, err := getSessionFromSomewhere()
        if err != nil {
            return Session{}, 0, err
        }
        //Return the value and a ttl (optional)
        return sess, time.Hour, nil
    }).
    Build(c) // Pass cache instance to Build method

Retrieving a value by key:

// If no corresponding value is found and no cache-aside getter was provided, returns ErrKeyNotFound.
value, err := table.Get(ctx, "key")
if err != nil {
}

Create a new entry:

if err := table.Put(ctx, "key", session, time.Hour); err != nil {
}

Evicting a value by key:

if err := table.Evict(ctx, "key"); err != nil {
}

Registering a RPC for value updates:

// WithProcedure is used to register the RPC.
table := nitecache.NewTable[RateLimitEntry]("rate-limiter").
    WithProcedure("incrementRL", func(r RateLimitEntry, params []byte) (RateLimitEntry, time.Duration, error) {
        r.Count++
        return r, 0, nil
    }).
    Build(c)
    
// You can pass parameters as bytes to the function call for more flexibility.
rlEntry, err = table.Call(ctx, "key", "incrementRL", []byte("Hello there!)])
if err != nil {
}

Roadmap

See the open issues for a full list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".

License

Distributed under the MIT License. See LICENSE for more information.

Documentation

Index

• Variables
• func GRPCServerOpts(opts ...grpc.ServerOption) func(c *Cache)
• func GRPCTransportCredentials(opts credentials.TransportCredentials) func(c *Cache)
• func HashFuncOpt(hashFunc hashring.HashFunc) func(c *Cache)
• func LFU(threshold int) inmem.Storage[string, []byte]
• func LRU(threshold int) inmem.Storage[string, []byte]
• func TimeoutOpt(timeout time.Duration) func(c *Cache)
• func VirtualNodeOpt(nodes int) func(c *Cache)
• type BatchEvictionErrs
  • func (b BatchEvictionErrs) AffectedKeys() []string
  • func (b BatchEvictionErrs) Error() string
• type Cache
  • func NewCache(self Member, peers []Member, opts ...CacheOpt) (*Cache, error)
  • func (c *Cache) GetMetrics() (Metrics, error)
  • func (c *Cache) HealthCheckPeers(ctx context.Context) error
  • func (c *Cache) ListenAndServe() error
  • func (c *Cache) SetPeers(peers []Member) error
  • func (c *Cache) TearDown() error
• type CacheOpt
• type Codec
• type GobCodec
  • func (c GobCodec[T]) Decode(b []byte, v *T) error
  • func (c GobCodec[T]) Encode(v T) ([]byte, error)
• type JsonCodec
  • func (c JsonCodec[T]) Decode(b []byte, v *T) error
  • func (c JsonCodec[T]) Encode(v T) ([]byte, error)
• type Member
• type Metrics
• type Procedure
• type StringCodec
  • func (c StringCodec[T]) Decode(b []byte, v *T) error
  • func (c StringCodec[T]) Encode(v T) ([]byte, error)
• type Table
  • func (t *Table[T]) Call(ctx context.Context, key, function string, args []byte) (T, error)
  • func (t *Table[T]) Evict(ctx context.Context, key string) error
  • func (t *Table[T]) EvictAll(ctx context.Context, keys []string) error
  • func (t *Table[T]) Get(ctx context.Context, key string) (T, error)
  • func (t *Table[T]) GetHot(key string) (T, error)
  • func (t *Table[T]) GetMetrics() (Metrics, error)
  • func (t *Table[T]) Put(ctx context.Context, key string, value T, ttl time.Duration) error
• type TableBuilder
  • func NewTable[T any](name string) *TableBuilder[T]
  • func (tb *TableBuilder[T]) Build(c *Cache) *Table[T]
  • func (tb *TableBuilder[T]) WithCodec(codec Codec[T]) *TableBuilder[T]
  • func (tb *TableBuilder[T]) WithGetter(fn inmem.Getter[string, T]) *TableBuilder[T]
  • func (tb *TableBuilder[T]) WithHotCache(storage inmem.Storage[string, []byte]) *TableBuilder[T]
  • func (tb *TableBuilder[T]) WithProcedure(name string, function Procedure[T]) *TableBuilder[T]
  • func (tb *TableBuilder[T]) WithStorage(storage inmem.Storage[string, []byte]) *TableBuilder[T]

Variables

var (
	ErrDuplicatePeer  = errors.New("duplicate peer detected")
	ErrTableNotFound  = errors.New("table not found")
	ErrMissingMembers = errors.New("peers must contain at least one member")
	ErrCacheDestroyed = errors.New("can't use cache after tear down")
)

var (
	ErrRPCNotFound = errors.New("RPC not found")
	ErrKeyNotFound = errors.New("key not found")
)

Functions

func GRPCServerOpts

func GRPCServerOpts(opts ...grpc.ServerOption) func(c *Cache)

GRPCServerOpts sets the options when creating the gRPC service.

func GRPCTransportCredentials

func GRPCTransportCredentials(opts credentials.TransportCredentials) func(c *Cache)

GRPCTransportCredentials sets the credentials for the gRPC server

func HashFuncOpt

func HashFuncOpt(hashFunc hashring.HashFunc) func(c *Cache)

HashFuncOpt sets the hash function used to determine hashring keys Defaults to FNV-1 algorithm

func LFU

func LFU(threshold int) inmem.Storage[string, []byte]

func LRU

func LRU(threshold int) inmem.Storage[string, []byte]

func TimeoutOpt

func TimeoutOpt(timeout time.Duration) func(c *Cache)

TimeoutOpt sets the timeout for grpc clients Defaults to 3 seconds

func VirtualNodeOpt

func VirtualNodeOpt(nodes int) func(c *Cache)

VirtualNodeOpt sets the number of points/node on the hashring Defaults to 32

Types

type BatchEvictionErrs

type BatchEvictionErrs []batchEvictionErr

func (BatchEvictionErrs) AffectedKeys

func (b BatchEvictionErrs) AffectedKeys() []string

AffectedKeys returns a list of keys owned by clients who returned an error.

As a result, the list may contain keys that were successfully evicted.

func (BatchEvictionErrs) Error

func (b BatchEvictionErrs) Error() string

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache hold the nitecache instance. The zero value is not read for use.

Refer to NewCache for creating an instance.

func NewCache

func NewCache(self Member, peers []Member, opts ...CacheOpt) (*Cache, error)

NewCache Creates a new Cache instance

This should only be called once for a same set of peers, so that connections can be reused.

Create a new Table using NewTable if you need to store different values.

Members must have a unique ID and Addr.

Ex.:

func() {
		self := nitecache.Member{ID: "1", Addr: "localhost:8000"}
		c, err := nitecache.NewCache(self, nitecache.Member{self})
		...
}

func (*Cache) GetMetrics

func (c *Cache) GetMetrics() (Metrics, error)

GetMetrics Returns a copy of the current cache Metrics. For Metrics specific to a Table, refer to Table.GetMetrics.

func (*Cache) HealthCheckPeers

func (c *Cache) HealthCheckPeers(ctx context.Context) error

HealthCheckPeers checks the status of the cache's grpc clients

func (*Cache) ListenAndServe

func (c *Cache) ListenAndServe() error

ListenAndServe starts the cache grpc server

func (*Cache) SetPeers

func (c *Cache) SetPeers(peers []Member) error

SetPeers will update the cache members to the new value.

func (*Cache) TearDown

func (c *Cache) TearDown() error

TearDown properly tears down all Table from Cache, closes all client connections and stops the grpc server.

Once called, using it or any of its table references cause ErrCacheDestroyed to be returned.

type CacheOpt

type CacheOpt func(c *Cache)

type Codec

type Codec[T any] interface {
	Encode(value T) ([]byte, error)
	Decode(bytes []byte, value *T) error
}

Codec defines the interface to implement custom marshalling/unmarshalling

If a generic type of string or []byte is supplied to the table, nitecache will automatically use StringCodec Otherwise nitecache will default to JsonCodec.

Example:

type (
	Coord struct {
		x, y float64
	}
	CoordCodec struct{}
)

func (c CoordCodec) Decode(b []byte, v *Coord) error {
	buf := bytes.NewBuffer(b)
	_, err := fmt.Fscanln(buf, &v.x, &v.y)
	return err
}

func (c CoordCodec) Encode(v Coord) ([]byte, error) {
	var buf bytes.Buffer
	if _, err := fmt.Fprintln(&buf, v.x, v.y); err != nil {
		return nil, err
	}
	return buf.Bytes(), nil
}

type GobCodec

type GobCodec[T any] struct{}

GobCodec implements Codec using encoding/gob

func (GobCodec[T]) Decode

func (c GobCodec[T]) Decode(b []byte, v *T) error

func (GobCodec[T]) Encode

func (c GobCodec[T]) Encode(v T) ([]byte, error)

type JsonCodec

type JsonCodec[T any] struct{}

JsonCodec implements Codec using encoding/json

This is the default codec for all type except []byte and string.

func (JsonCodec[T]) Decode

func (c JsonCodec[T]) Decode(b []byte, v *T) error

func (JsonCodec[T]) Encode

func (c JsonCodec[T]) Encode(v T) ([]byte, error)

type Member

type Member struct {
	ID   string
	Addr string
}

type Metrics

type Metrics struct {
	Miss  int64
	Get   int64
	Put   int64
	Evict int64
	Call  map[string]int64
}

Metrics are limited to the scope of the owner nodes.

For example, if node-1 queries node-2, metrics will be registered on node-2 only.

type Procedure

type Procedure[T any] func(ctx context.Context, v T, args []byte) (T, time.Duration, error)

Procedure defines the type used for registering RPCs through TableBuilder.WithProcedure.

type StringCodec

type StringCodec[T ~string | ~[]byte] struct{}

StringCodec implements Codec by preventing unnecessary marshalling/unmarshalling for string-like tables.

This codec will automatically be used for string/[]byte tables. However, nitecache won't use it for string/[]byte alias, so you'll have to supply the codec yourself.

func (StringCodec[T]) Decode

func (c StringCodec[T]) Decode(b []byte, v *T) error

func (StringCodec[T]) Encode

func (c StringCodec[T]) Encode(v T) ([]byte, error)

type Table

type Table[T any] struct {
	// contains filtered or unexported fields
}

func (*Table[T]) Call

func (t *Table[T]) Call(ctx context.Context, key, function string, args []byte) (T, error)

Call calls an RPC previously registered through TableBuilder.WithProcedure on the owner node to update the value for the given key.

Call acquires a lock exclusive to the given key until the RPC has finished executing.

func (*Table[T]) Evict

func (t *Table[T]) Evict(ctx context.Context, key string) error

func (*Table[T]) EvictAll

func (t *Table[T]) EvictAll(ctx context.Context, keys []string) error

EvictAll attempts to remove all entries from the Table for the given keys.

Keys owned by the same client are batched together for efficiency.

After the operation, a BatchEvictionErrs detailing which keys (if any) failed to be evicted can be retrieved when checking the returned error. Example:

if errs, ok := err.(nitecache.BatchEvictionErrs); ok {
	// Note that keys that AffectedKeys may return keys that were actually evicted successfully.
	keysThatFailed := errs.AffectedKeys()
}

func (*Table[T]) Get

func (t *Table[T]) Get(ctx context.Context, key string) (T, error)

func (*Table[T]) GetHot

func (t *Table[T]) GetHot(key string) (T, error)

GetHot looks up local cache if the current node is the owner, otherwise looks up hot cache.

GetHot does not call the getter to autofill cache, does not increment metrics and does not affect the main cache's LFU/LRU (if used).

func (*Table[T]) GetMetrics

func (t *Table[T]) GetMetrics() (Metrics, error)

GetMetrics returns a copy of the current table Metrics. For global cache Metrics, refer to Cache.GetMetrics

func (*Table[T]) Put

func (t *Table[T]) Put(ctx context.Context, key string, value T, ttl time.Duration) error

type TableBuilder

type TableBuilder[T any] struct {
	// contains filtered or unexported fields
}

func NewTable

func NewTable[T any](name string) *TableBuilder[T]

func (*TableBuilder[T]) Build

func (tb *TableBuilder[T]) Build(c *Cache) *Table[T]

func (*TableBuilder[T]) WithCodec

func (tb *TableBuilder[T]) WithCodec(codec Codec[T]) *TableBuilder[T]

WithCodec overrides the default encoding/decoding behavior.

Defaults to [BytesCodec] for []byte tables and JsonCodec for any other types. See Codec for custom implementation.

func (*TableBuilder[T]) WithGetter

func (tb *TableBuilder[T]) WithGetter(fn inmem.Getter[string, T]) *TableBuilder[T]

WithGetter sets the auto cache filling function.

func (*TableBuilder[T]) WithHotCache

func (tb *TableBuilder[T]) WithHotCache(storage inmem.Storage[string, []byte]) *TableBuilder[T]

WithHotCache enables hot cache.

If hot cache is enabled, a new cache will be populated with values gotten from other peers that can be accessed only through Table.GetHot.

The owner of the hot cache is responsible for keeping it up to date (i.e. calls to Table.Put and Table.Evict won't update hot cache of other peers).

func (*TableBuilder[T]) WithProcedure

func (tb *TableBuilder[T]) WithProcedure(name string, function Procedure[T]) *TableBuilder[T]

WithProcedure Registers an RPC that can be called using Table.Call.

func (*TableBuilder[T]) WithStorage

func (tb *TableBuilder[T]) WithStorage(storage inmem.Storage[string, []byte]) *TableBuilder[T]

WithStorage specifies how to store values.

Must be one of LFU, LRU or nil.

if nil, the table will always grow unless keys are explicitly evicted.