pool

README

pool

Package pool connects to multiple physical Redis instances, and emulates a single logical Redis instance. Clients are expected (but not required) to use their Redis keys as hash keys to select a Redis instance. The package maintains a connection pool to each instance.

Usage

Simple usage with a single key.

p := pool.New(...)
defer p.Close()

key, value := "foo", "bar"
if err := p.With(key, func(c redis.Conn) error {
	_, err := c.Do("SET", key, value)
	return err
}); err != nil {
	log.Printf("Failure: %s", err)
}

Keys may be pre-hashed with the Index method, and connections used for pipelining.

m := map[int][]string{} // index: keys to INCR
for _, key := range keys {
	index := p.Index(key)
	m[index] = append(m[index], key)
}

wg := sync.WaitGroup{}
wg.Add(len(m))
for index, keys := range m {
	// Pool is safe for concurrent access.
	go p.WithIndex(index, func(c redis.Conn) error) {
		defer wg.Done()
		for _, key := range keys {
			if err := c.Send("INCR", key); err != nil {
				return err
			}
		}
		if err := c.Flush(); err != nil {
			return err
		}
		for _ = range keys {
			if _, err := c.Receive(); err != nil {
				return err
			}
		}
		return nil
	})
}
wg.Wait()

Documentation

Overview

Package pool performs key-based sharding over multiple Redis instances.

Index

• func FNV(s string) uint32
• func FNVa(s string) uint32
• func Murmur3(s string) uint32
• type Pool
  • func New(addresses []string, connectTimeout, readTimeout, writeTimeout time.Duration, ...) *Pool
  • func (p *Pool) Close() error
  • func (p *Pool) ID(index int) string
  • func (p *Pool) Index(key string) int
  • func (p *Pool) Size() int
  • func (p *Pool) With(key string, do func(redis.Conn) error) error
  • func (p *Pool) WithIndex(index int, do func(redis.Conn) error) error

Functions

func FNV

func FNV(s string) uint32

FNV implements the FNV-1 string hashing function. It can be passed to NewCluster.

func FNVa

func FNVa(s string) uint32

FNVa implements the FNV-1a string hashing function. It can be passed to NewCluster.

func Murmur3

func Murmur3(s string) uint32

Murmur3 implements the Murmur3 string hashing function. It can be passed to NewCluster.

https://github.com/reusee/mmh3

Types

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool maintains a connection pool for multiple Redis instances.

func New

func New(
	addresses []string,
	connectTimeout, readTimeout, writeTimeout time.Duration,
	maxConnectionsPerInstance int,
	hash func(string) uint32,
) *Pool

New creates and returns a new Pool object.

Addresses are host:port strings for each underlying Redis instance. The number and order of the addresses determines the hash slots, so be careful to make that deterministic.

Connect timeout is the timeout for establishing a connection to any underlying Redis instance. Read timeout is the timeout for reading a reply to a command via an established connection. Write timeout is the timeout for writing a command to an established connection.

Max connections per instance is the size of the connection pool for each Redis instance. Hash defines the hash function used by the With methods. Any function that takes a string and returns a uint32 may be used. Package pool ships with several options, including Murmur3, FNV, and FNVa.

func (*Pool) Close

func (p *Pool) Close() error

Close closes all available (idle) connections in the cluster. Close does not affect oustanding (in-use) connections.

func (*Pool) ID

func (p *Pool) ID(index int) string

ID returns a unique identifier for the Redis instance represented by index, or an error if the index is invalid.

func (*Pool) Index

func (p *Pool) Index(key string) int

Index returns a reference to the connection pool that will be used to satisfy any request for the given key. Pass that value to WithIndex.

func (*Pool) Size

func (p *Pool) Size() int

Size returns how many instances the pool sits over. Useful for ranging over with WithIndex.

func (*Pool) With

func (p *Pool) With(key string, do func(redis.Conn) error) error

With is a convenience function that combines Index and WithIndex, for simple/single Redis requests on a single key.

func (*Pool) WithIndex

func (p *Pool) WithIndex(index int, do func(redis.Conn) error) error

WithIndex selects a single Redis instance from the referenced connection pool, and then calls the given function with that connection. If the function returns a nil error, WithIndex returns the connection to the pool after it's used. Otherwise, WithIndex discards the connection.

WithIndex will return an error if it wasn't able to successfully retrieve a connection from the referenced connection pool, and will forward any error returned by the `do` function.