redis

README

单元测试可以使用 https://github.com/alicebob/miniredis, 可以开启一个本地的模拟redis

• 在单元测试中模拟Redis

案例

• Redis分布式锁没用明白，搞出了大故障…
• 看完这篇Redis缓存三大问题

Redis 优化方向

参数优化

maxIdle设置高点，可以保证突发流量情况下，能够有足够的连接去获取redis，不用在高流量情况下建立连接

go-redis参数优化

  min_idle_conn: 30               
  dial_timeout: "1s"
  read_timeout: "500ms"
  write_timeout: "500ms"
  pool_size: 500
  pool_timeout: "60s"

redisgo参数优化

maxIdle = 30
maxActive = 500
dialTimeout = "1s"
readTimeout = "500ms"
writeTimeout = "500ms"
idleTimeout = "60s"

使用优化

• 增加redis从库
• 对批量数据，根据redis从库数量，并发goroutine拉取数据
• 对批量数据大量使用pipeline指令
• 精简key字段
• redis的value存储编解码改为msgpack

Pipeline

• https://redis.io/topics/pipelining
• 兼容go redis cluster的pipeline批量
• https://www.tizi365.com/archives/309.html

Documentation

Overview

trace redis link: https://github.com/smacker/opentracing-go-redis

Index

• Constants
• Variables
• func Init(c *Config) *redis.Client
• func InitTestRedis()
• func WrapRedisClient(ctx context.Context, client *redis.Client) *redis.Client
• type CheckRepeat
  • func NewCheckRepeat(client *redis.Client) CheckRepeat
• type Config
• type IDAlloc
  • func NewIDAlloc(conn *redis.Client) *IDAlloc
  • func (ia *IDAlloc) GetCurrentID(key string) (int64, error)
  • func (ia *IDAlloc) GetKey(key string) string
  • func (ia *IDAlloc) GetNewID(key string, step int64) (int64, error)
• type Lock
  • func NewLock(conn *redis.Client, key string, opts ...Option) *Lock
  • func (l *Lock) GetKey() string
  • func (l *Lock) Lock(ctx context.Context) (bool, error)
  • func (l *Lock) Unlock(ctx context.Context) error
• type Option
  • func WithTimeout(expiration time.Duration) Option

Constants

const (
	// PrefixCheckRepeat check repeat key
	PrefixCheckRepeat = "CHECK_REPEAT"
	// RepeatDefaultTimeout define default timeout
	RepeatDefaultTimeout = 60
)

const (
	// LockKey redis lock key
	LockKey = "snake:redis:lock:%s"
	// DefaultTimeout default expire time
	DefaultTimeout = 2 * time.Second
)

const ErrRedisNotFound = redis.Nil

ErrRedisNotFound not exist in redis

Variables

var RedisClient *redis.Client

RedisClient redis 客户端

Functions

func Init

func Init(c *Config) *redis.Client

Init 实例化一个redis client

func InitTestRedis

func InitTestRedis()

InitTestRedis 实例化一个可以用于单元测试的redis

func WrapRedisClient

func WrapRedisClient(ctx context.Context, client *redis.Client) *redis.Client

WrapRedisClient adds opentracing measurements for commands and returns cloned client

Types

type CheckRepeat

type CheckRepeat interface {
	Set(key string, value interface{}, expiration time.Duration) error
	Get(key string) (string, error)
	SetNX(key string, value interface{}, expiration time.Duration) (bool, error)
	Del(keys string) int64
}

CheckRepeat define interface

func NewCheckRepeat

func NewCheckRepeat(client *redis.Client) CheckRepeat

NewCheckRepeat create a check repeat

type Config

type Config struct {
	Addr         string
	Password     string
	DB           int
	MinIdleConn  int
	DialTimeout  time.Duration
	ReadTimeout  time.Duration
	WriteTimeout time.Duration
	PoolSize     int
	PoolTimeout  time.Duration
}

Config redis config

type IDAlloc

type IDAlloc struct {
	// contains filtered or unexported fields
}

IDAlloc id生成器 key 为业务key, 由业务前缀+功能前缀+具体场景id组成 比如生成用户id, 可以传入user_id， 完整示例: snake:idalloc:user_id

func NewIDAlloc

func NewIDAlloc(conn *redis.Client) *IDAlloc

NewIDAlloc create a id alloc instance

func (*IDAlloc) GetCurrentID

func (ia *IDAlloc) GetCurrentID(key string) (int64, error)

GetCurrentID 获取当前id

func (*IDAlloc) GetKey

func (ia *IDAlloc) GetKey(key string) string

GetKey 获取key

func (*IDAlloc) GetNewID

func (ia *IDAlloc) GetNewID(key string, step int64) (int64, error)

GetNewID 生成id

type Lock

type Lock struct {
	// contains filtered or unexported fields
}

Lock 定义lock结构体

func NewLock

func NewLock(conn *redis.Client, key string, opts ...Option) *Lock

NewLock new a lock instance

func (*Lock) GetKey

func (l *Lock) GetKey() string

GetKey 获取key

func (*Lock) Lock

func (l *Lock) Lock(ctx context.Context) (bool, error)

Lock 加锁

func (*Lock) Unlock

func (l *Lock) Unlock(ctx context.Context) error

Unlock 解锁 token 一致才会执行删除，避免误删，这里用了lua脚本进行事务处理

type Option

type Option func(*Lock)

func WithTimeout

func WithTimeout(expiration time.Duration) Option