Documentation
¶
Overview ¶
Package gore is a full feature Redis client for Go:
- Convenient command building and reply parsing
- Pipeline, multi-exec, LUA scripting
- Pubsub
- Connection pool
- Redis sentinel
- Client implementation of sharding
Connections ¶
Gore only supports TCP connection for Redis. The connection is thread-safe and can be auto-repaired with or without sentinel.
conn, err := gore.Dial("localhost:6379") //Connect to redis server at localhost:6379
if err != nil {
return
}
defer conn.Close()
Command ¶
Redis command is built with NewCommand
gore.NewCommand("SET", "kirisame", "marisa") // SET kirisame marisa
gore.NewCommand("ZADD", "magician", 1337, "alice") // ZADD magician 1337 alice
gore.NewCommand("HSET", "sdm", "sakuya", 99) // HSET smd sakuya 99
In the last command, the value stored by redis will be the string "99", not the integer 99.
Integer and float values are converted to string using strconv Boolean values are convert to "1" and "0" Nil values are stored as zero length string Other types are converted to string using standard fmt.Sprint
To efficiently store integer, you can use gore.FixInt or gore.VarInt ¶
Compact integer ¶
Gore supports compacting integer to reduce memory used by redis. There are 2 ways of compacting integer:
gore.FixInt stores an integer as a fixed 8 bytes []byte.
gore.VarInt encodes an integer with variable length []byte.
gore.NewCommand("SET", "fixint", gore.FixInt(1337)) // Set fixint as an 8 bytes []byte
gore.NewCommand("SET", "varint", gore.VarInt(1337)) // varint only takes 3 bytes
Reply ¶
A redis reply is return when the command is run on a connection
rep, err := gore.NewCommand("GET", "kirisame").Run(conn)
Parsing the reply is straightforward:
s, _ := rep.String() // Return string value if reply is simple string (status) or bulk string b, _ := rep.Bytes() // Return a byte array x, _ := rep.Integer() // Return integer value if reply type is integer (INCR, DEL) e, _ := rep.Error() // Return error message if reply type is error a, _ := rep.Array() // Return reply list if reply type is array (MGET, ZRANGE)
Reply converting ¶
Reply support convenient methods to convert to other types
x, _ := rep.Int() // Convert string value to int64. This method is different from rep.Integer() f, _ := rep.Float() // Convert string value to float64 t, _ := rep.Bool() // Convert string value to boolean, where "1" is true and "0" is false x, _ := rep.FixInt() // Convert string value to FixInt x, _ := rep.VarInt() // Convert string value to VarInt
To convert an array reply to a slice, you can use Slice method:
s := []int err := rep.Slice(&s) // Convert an array reply to a slice of integer
The following slice element types are supported:
- integer (int, int64)
- float (float64)
- string and []byte
- FixInt and VarInt
- *gore.Pair for converting map data from HGETALL or ZRANGE WITHSCORES
Reply returns from HGETALL or SENTINEL master can be converted into a map using Map:
m, err:= rep.Map()
Pipeline ¶
Gore supports pipelining using gore.Pipeline:
p := gore.NewPipeline()
p.Add(gore.NewCommand("SET", "kirisame", "marisa"))
p.Add(gore.NewCommand("SET", "alice", "margatroid"))
replies, _ := p.Run(conn)
for _, r := range replies {
// Deal with individual reply here
}
Script ¶
Script can be set from a string or read from a file, and can be executed over a connection. Gore makes sure to use EVALSHA before using EVAL to save bandwidth.
s := gore.NewScript()
s.SetBody("return redis.call('SET', KEYS[1], ARGV[1])")
rep, err := s.Execute(conn, 1, "kirisame", "marisa")
Script can be loaded from a file:
s := gore.NewScript()
s.ReadFromFile("scripts/set.lua")
rep, err := s.Execute(conn, 1, "kirisame", "marisa")
Script map ¶
If your application use a lot of script files, you can manage them through ScriptMap
gore.LoadScripts("scripts", ".*\\.lua") // Load all .lua file from scripts folder
s := gore.GetScripts("set.lua") // Get script from set.lua file
rep, err := s.Execute(conn, 1, "kirisame", "marisa") // And execute
Pubsub ¶
Publish message to a channel is easy, you can use gore.Command to issue a PUBLISH over a connection, or use gore.Publish method:
gore.Publish(conn, "touhou", "Hello!")
To handle subscriptions, you should allocate a dedicated connection and assign it to gore.Subscriptions:
subs := gore.NewSubscriptions(conn)
subs.Subscribe("test")
subs.PSubscribe("tou*")
To receive messages, the subcriber should spawn a new goroutine and use Subscriptions Message channel:
go func() {
for message := range subs.Message() {
if message == nil {
break
}
fmt.Println("Got message from %s, originate from %s: %s", message.Channel, message.OriginalChannel, message.Message)
}
}()
Connection pool ¶
To use connection pool, a Pool should be created when application startup. The Dial() method of the pool should be called to make initial connection to the redis server. If Dial() fail, it is up to the application to decide to fail fast, or wait and connect again later.
pool := &gore.Pool{
InitialConn: 5, // Initial number of connections to open
MaximumConn: 10, // Maximum number of connections to open
}
err := pool.Dial("localhost:6379")
if err != nil {
log.Error(err)
return
}
...
In each goroutine, a connection from the pool can be get by Acquire() method. Release() method should always be called later to return the connection to the pool, even in error situation.
// Inside a goroutine
conn, err := pool.Acquire()
if err != nil {
// Error can happens when goroutine try to acquire a conn
// from the pool. Application should fail fast here.
return
}
defer pool.Release(conn)
if conn == nil {
// This happens when the pool was closed. Application should
// fail here.
return
}
// Do every thing with the conn, exclusively.
...
To gracefully close the pool, call Close() method anywhere in your program.
Transaction ¶
Transaction is implemented using MULTI, EXEC and WATCH. Using transaction directly with a Conn is not goroutine-safe, so transaction should be used with connection pool only.
tr := gore.NewTransaction(conn)
tr.Watch("a key") // Watch a key
tr.Watch("another key")
rep, _ := NewCommand("GET", "a key").Run(conn)
value, _ := rep.Int()
tr.Add(NewCommand("SET", "a key", value + 1)) // Add a command to the transaction
_, err := tr.Commit() // Commit the transaction
if err == nil {
// Transaction OK!!!
} else if err == gore.ErrKeyChanged {
// Watched key has been changed, transaction should be started over.
} else {
// Other errors, transaction should be aborted
}
Authentication ¶
Gore supports Redis authentication in single connection, pool, sentinel.
Authentication with a single connection can be done by sending "AUTH" command to redis server as normal, but this can be trouble some when the server is down, and is reconnected after that. To deal with this problem, gore provides conn.Auth() method:
conn.Auth("secret password")
This method should be called when the connection is initialized. By calling Auth(), when gore tries to reconnect, is will also attempt to send AUTH command to redis server right after the connection is made.
To configure Auth password with gore.Pool, you can set pool.Password before calling pool.Dial(). Like gore.Conn, gore.Pool also automatically send AUTH command when reconnected.
If you are using sentinel to retrieve Pool or Cluster, instead of using GetPool or GetCluster method, you can use GetPoolWithPassword or GetClusterWithPassword to connect with a password-protected pool/cluster
Sentinel ¶
Redis Sentinel is a system that monitors other Redis instance, notify application when something is wrong with monitored Redis instance, and do automatic failover. Please note that Redis Sentinel is still in beta stage, and only supported fully in Redis version 2.8 and above. For more information about setting up Redis Sentinel, please refer to the official document at http://redis.io/topics/sentinel
Using Redis Sentinel with gore is simple:
// First, you need to create a Sentinel object:
s := gore.NewSentinel()
// Add some Sentinel servers to this object.
// In production environment, you should have at least 3 Sentinel Servers
s.AddServer("127.0.0.1:26379", "127.0.0.1:26380", "127.0.0.1:26381")
// Initialize the Sentinel
err := s.Dial()
if err != nil {
return
}
// Now, the Sentinel is ready, you can get a monitored pool of connection from the
// sentinel by using one function:
pool, err := s.GetPool("mymaster")
The name of the pool ("mymaster") must be an already monitored instance name, otherwise, the function will return ErrNil. The application also should not call GetPool function repeatedly because internal locking may cause dropping in performance. It should assign and reuse the pool variable instead. Because the GetPool function is normally used when the application starts up, it will fail immediately if the redis instance is still down. Application can use a for loop and sleep to retry to connect.
Sharding ¶
Gore supports simple sharding strategy: a fixed number of Redis instances are grouped into "cluster", each instance holds a portion of the cluster keyset.
When a single command needed to be execute on the cluster, gore will redirect the command to approriate instance based on the key. Gore makes sure that each key will be redirected to only one instance consistently. Because of the nature of the fixed-sharding, the number of Redis instances in the cluster should never change, and pipeline or transaction is not supported.
Gore provides two ways to connect to a cluster.
The first way is using Sentinel. All Redis instances in the same cluster should have the same prefix, and the suffix should be a number. For example: "mycluster1", "mycluster2", ..., "mycluster20". Using Sentinel, you can get a cluster relatively easy:
s := NewSentinel()
s.AddServer("127.0.0.1:26379", "127.0.0.1:26380", "127.0.0.1:26381")
err := s.Dial()
if err != nil {
return
}
c, err := s.GetCluster("mycluster")
The second way is to add shard to a cluster manually:
c := NewCluster()
c.AddShard("127.0.0.1:6379", "127.0.0.1:6380")
err := c.Dial()
if err != nil {
return
}
Using cluster ¶
A single command can be ran on the cluster with Execute:
rep, err := c.Execute(NewCommand("SET", "kirisame", "marisa"))
if err != nil || !rep.IsOk() {
return
}
rep, err := c.Execute(NewCommand("GET", "kirisame"))
if err != nil {
return
}
value, _ := rep.String() // value should be "marisa"
Index ¶
- Constants
- Variables
- func AddScript(scriptName string, script *Script)
- func DefaultShardStrategy(key string, size int) int
- func DeleteScript(scriptName string)
- func LoadScripts(folder, pattern string) error
- func Publish(conn *Conn, channel string, message interface{}) error
- func ToFixInt(b []byte) (int64, error)
- func ToVarInt(b []byte) (int64, error)
- type Cluster
- type Command
- type Conn
- type FixInt
- type Message
- type Pair
- type Pipeline
- type Pool
- type Reply
- func (r *Reply) Array() ([]*Reply, error)
- func (r *Reply) Bool() (bool, error)
- func (r *Reply) Bytes() ([]byte, error)
- func (r *Reply) Error() (string, error)
- func (r *Reply) FixInt() (int64, error)
- func (r *Reply) Float() (float64, error)
- func (r *Reply) Int() (int64, error)
- func (r *Reply) Integer() (int64, error)
- func (r *Reply) IsArray() bool
- func (r *Reply) IsError() bool
- func (r *Reply) IsInteger() bool
- func (r *Reply) IsNil() bool
- func (r *Reply) IsOk() bool
- func (r *Reply) IsStatus() bool
- func (r *Reply) IsString() bool
- func (r *Reply) Map() (map[string]string, error)
- func (r *Reply) Slice(s interface{}) error
- func (r *Reply) String() (string, error)
- func (r *Reply) Type() int
- func (r *Reply) VarInt() (int64, error)
- type Script
- type ScriptMap
- type Sentinel
- func (s *Sentinel) AddServer(addresses ...string)
- func (s *Sentinel) Close()
- func (s *Sentinel) Dial() (err error)
- func (s *Sentinel) GetCluster(name string) (c *Cluster, err error)
- func (s *Sentinel) GetClusterWithPassword(name string, password string) (c *Cluster, err error)
- func (s *Sentinel) GetPool(name string) (*Pool, error)
- func (s *Sentinel) GetPoolWithPassword(name string, password string) (*Pool, error)
- type Subscriptions
- func (s *Subscriptions) Close()
- func (s *Subscriptions) Message() chan *Message
- func (s *Subscriptions) PSubscribe(channel ...string) error
- func (s *Subscriptions) PUnsubscribe(channel ...string) error
- func (s *Subscriptions) Subscribe(channel ...string) error
- func (s *Subscriptions) Unsubscribe(channel ...string) error
- type Transaction
- type VarInt
Constants ¶
const ( ReplyString = 1 ReplyArray = 2 ReplyInteger = 3 ReplyNil = 4 ReplyStatus = 5 ReplyError = 6 )
Reply type, similar to Hiredis
Variables ¶
var ( // ErrNotConnected is returned when attempt to send command when connection is down ErrNotConnected = errors.New("not connected") // ErrEmptyScript is returned when try to execute an empty script ErrEmptyScript = errors.New("empty script") // ErrType is returned when convert between different reply types ErrType = errors.New("type error") // ErrConvert is returned when convert between data types ErrConvert = errors.New("convert error") // ErrKeyChanged is returned when transaction fails because watched keys have been changed ErrKeyChanged = errors.New("key changed") // ErrTransactionAborted is returned when tracsaction fails because of other reasons ErrTransactionAborted = errors.New("transaction aborted") // ErrNil is for nil reply ErrNil = errors.New("nil value") // ErrAuth is returned when redis AUTH fail ErrAuth = errors.New("authentication fail") //ErrNoShard is returned when trying to connect with a cluster with no shard ErrNoShard = errors.New("no shard") // ErrNoKey is returned when sending command with no key to the cluster ErrNoKey = errors.New("no key") // ErrWrite is returned when connection cannot be written ErrWrite = errors.New("write error") // ErrRead is returned when connection cannot be read ErrRead = errors.New("read error") )
var Config = &struct { ConnectTimeout int RequestTimeout int ReconnectTime int PoolInitialSize int PoolMaximumSize int }{ ConnectTimeout: 5, RequestTimeout: 10, ReconnectTime: 2, PoolInitialSize: 5, PoolMaximumSize: 10, }
Config keeps some default configurations. Time is measured in second
var ( // ErrNumberFormat is returned when formatting number fails. ErrNumberFormat = errors.New("number format error") )
Functions ¶
func DefaultShardStrategy ¶
DefaultShardStrategy converts a string key into number and takes modulo with the size of cluster
func DeleteScript ¶
func DeleteScript(scriptName string)
DeleteScript a script from the default script map
func LoadScripts ¶
LoadScripts loads all script files from a folder with a regular expression pattern to the default script map. Loaded script will be keyed by its file name. This method can be called many times to reload script files.
Types ¶
type Cluster ¶
type Cluster struct {
ShardStrategy func(string, int) int
// contains filtered or unexported fields
}
Cluster consists of fix number of shards, with each shard holds a portion of the keyset. Cluster can be created by adding shards, or using sentinel.
func NewCluster ¶
func NewCluster() *Cluster
NewCluster creates new cluster. You must add shards to this cluster manually
func (*Cluster) AddShardWithPassword ¶
AddShardWithPassword add a password-protected shard
type Command ¶
type Command struct {
// contains filtered or unexported fields
}
Command sent to redis
func NewCommand ¶
NewCommand returns a new Command
type Conn ¶
Conn holds a persistent connection to a redis server
func DialTimeout ¶
DialTimeout opens a TCP connection with a redis server with a connection timeout
func (*Conn) IsConnected ¶
IsConnected returns true if connection is okay
type Message ¶
type Message struct {
// message or pmessage
Type string
// The channel/pchannel the client subscribed to. For example: "test", "te*"
Channel string
// The channel that publisher published to. For example "test", "text"
OriginalChannel string
// The payload
Message []byte
}
Message is a nofitication from a subscribed channel or pchannel
type Pipeline ¶
type Pipeline struct {
// contains filtered or unexported fields
}
Pipeline keeps a list of command for sending to redis once, saving network roundtrip
type Pool ¶
type Pool struct {
// Request timeout for each connection
RequestTimeout time.Duration
// Initial number of connection to open
InitialConn int
// Maximum number of connection to open
MaximumConn int
// Password to send after connection is opened
Password string
// contains filtered or unexported fields
}
Pool is a pool of connection. The application acquires connection from pool using Acquire() method, and when done, returns it to the pool with Release().
func (*Pool) Acquire ¶
Acquire returns a usable, exclusive connection for the goroutine. If this function return a nil connection, application can check the error to know whether there is really an error or it is because the pool was closed. If the pool was closed, the returned error will also be nil.
func (*Pool) Dial ¶
Dial initializes connection from the pool to redis server. If the redis server cannot be connected, this function returns an error, and the application should fail accordingly.
func (*Pool) IsConnected ¶
IsConnected returns pool connection status. This function only works when sentinel is enabled. When sentinel is disabled, false positive may occur.
type Reply ¶
type Reply struct {
// contains filtered or unexported fields
}
Reply holds redis reply
func (*Reply) Bool ¶
Bool parses string value to boolean. String "0" returns false while string "1" returns true. Other values will cause ErrConvert
func (*Reply) Int ¶
Int parses string value to int64. To get the integer value from command like INCR, use Integer()
func (*Reply) Map ¶
Map converts the reply into a map[string]string. It will return error unless the reply is an array reply from HGETALL, or SENTINEL master
func (*Reply) Slice ¶
Slice parses the reply to a slice. The element of the destination slice must be integer, float, boolean, string, []byte, FixInt, VarInt, or a Pair
type Script ¶
type Script struct {
// contains filtered or unexported fields
}
Script represents a Lua script.
func GetScript ¶
GetScript a script by its name from defaultScriptMap . Nil value will be returned if the name is not found
func (*Script) ReadFromFile ¶
ReadFromFile reads the script from a file
type ScriptMap ¶
type ScriptMap struct {
// contains filtered or unexported fields
}
ScriptMap is a thread-safe map from script name to its content
type Sentinel ¶
type Sentinel struct {
// contains filtered or unexported fields
}
Sentinel is a special Redis process that monitors other Redis instances, does fail-over, notifies client status of all monitored instances.
func (*Sentinel) AddServer ¶
AddServer adds new sentinel servers. Only one sentinel server is active at any time. If this server fails, gore will connect to other sentinel servers immediately.
AddServer can be called at anytime, to add new server on the fly. In production environment, you should always have at least 3 sentinel servers up and running.
func (*Sentinel) Close ¶
func (s *Sentinel) Close()
Close gracefully closes the sentinel and all monitored connections
func (*Sentinel) Dial ¶
Dial connects to one sentinel server in the list. If it fails to connect, it moves to the next on the list. If all servers cannot be connected, Init return error.
func (*Sentinel) GetCluster ¶
GetCluster returns a cluster monitored by the sentinel. The name of the cluster will determine name of Redis instances. For example, if the cluster name is "mycluster", the instances' name maybe "mycluster1", "mycluster2", ...
func (*Sentinel) GetClusterWithPassword ¶
GetClusterWithPassword returns a password-protected cluster monitored by the sentinel.
func (*Sentinel) GetPool ¶
GetPool returns a pool of connection from a pool name. If the pool has not been retrieved before, gore will attempt to fetch the address from the sentinel server, and initialize connections with this address. The application should never call this function repeatedly to get the same pool, because internal locking can cause performance to drop. An error can be returned if the pool name is not monitored by the sentinel, or the redis server is currently dead, or the redis server cannot be connected (for example: firewall issues).
type Subscriptions ¶
type Subscriptions struct {
// contains filtered or unexported fields
}
Subscriptions keeps all SUBSCRIBE and PSUBSCRIBE channels, and handles all errors and re-subcription process when the connection is down then reconnected. A dedicated connection should be used for Subscriptions
func NewSubscriptions ¶
func NewSubscriptions(conn *Conn) *Subscriptions
NewSubscriptions returns new Subscriptions
func (*Subscriptions) Close ¶
func (s *Subscriptions) Close()
Close terminates the subscriptions. The connection is NOT closed. You should close it if you do not want to use anymore.
func (*Subscriptions) Message ¶
func (s *Subscriptions) Message() chan *Message
Message returns a channel for receiving message event. A nil message indicates the channel is closed. The channel should be used from a separated goroutine. For example:
for message := range subs.Message() {
if message == nil {
break
}
...
}
func (*Subscriptions) PSubscribe ¶
func (s *Subscriptions) PSubscribe(channel ...string) error
PSubscribe subscribes to a list of channels with given pattern
func (*Subscriptions) PUnsubscribe ¶
func (s *Subscriptions) PUnsubscribe(channel ...string) error
PUnsubscribe unsubscribes to a list of channels with given pattern
func (*Subscriptions) Subscribe ¶
func (s *Subscriptions) Subscribe(channel ...string) error
Subscribe subscribes to a list of channels
func (*Subscriptions) Unsubscribe ¶
func (s *Subscriptions) Unsubscribe(channel ...string) error
Unsubscribe unsubscribes to a list of channels
type Transaction ¶
type Transaction struct {
// contains filtered or unexported fields
}
Transaction implements MULTI/EXEC/WATCH protocol of redis. Transaction must be used with connection pool, or undefined behavior. may orcur when used from multiple goroutine.
func NewTransaction ¶
func NewTransaction(conn *Conn) *Transaction
NewTransaction returns new transaction
func (*Transaction) Add ¶
func (t *Transaction) Add(cmd ...*Command)
Add appends commands to the transaction
func (*Transaction) Commit ¶
func (t *Transaction) Commit() ([]*Reply, error)
Commit commits the whole transaction. If transaction fail, ErrTransactionAborted is returned. If watched key has been modified, ErrKeyChanged is returned.
func (*Transaction) Discard ¶
func (t *Transaction) Discard() error
Discard discards the transaction
func (*Transaction) Watch ¶
func (t *Transaction) Watch(key ...string) error
Watch watches some keys. If the key has been changed before Exec, the transaction will be aborted
