redfi

README

RedFI acts as a proxy between the client and Redis with the capability of injecting faults on the fly, based on the rules given by you.

Features

• Simple to use. It is just a binary that you execute.
• Transparent to the client.
• Flexibility to inject failures on Redis Commands you define.
• Limit failure injection to a percentage of the commands.
• Limit failure injection to certain clients only.
• Inject latency, drop connections, return empty responses.

Why?

We believe that to gain trust in your system, you must test it against various scenarios of failures, before deploying it to production.
These tests should also be part of your CI, to catch code changes that might introduce regressions.
This would help you understand the limits of your system, and how resilient it is against Redis' failures.

How it Works

RedFI is a proxy that sets between the client and the actual Redis server. On every incoming command from the client, it checks the list of failure rules provided by you, and then it applies the first rule that matches the request.

Usage

First, download the latest binary.

After that, execute the binary we downloaded earlier

$ ./redfi -addr 127.0.0.1:8083 -redis 127.0.0.1:6379

RedFI is listening on 127.0.0.1:8083
Don't forget to point your client to that address.

RedFI Controller is listening on :6380

• addr: is the address on which the proxy listens on for new connections.
• redis: address of the actual Redis server to proxy commands/connections to.
• plan: path to the json file that contains the rules/scenarios for fault injection.

An example of controlling the proxy rules

Config Commands

Point redis-cli to RedFI Controller port

$ redi-cli -p 6380

RULEADD rule_name option=value [option=value]

Adds a rule to the engine of RedFI

Faults Options

• delay: adds a delay, the value unit is milliseconds. Example: delay=200.
• return_empty: returns an empty response. Example: return_empty=true.
• return_err: returns an error response. Example: return_err=ERR bad command.
• drop: drop connections, the value is a boolean. Example: drop=true.

Blast limiters

They limit the radius of the blast

• command: only apply failure on certain commands. For example command=HGET.
• percentage: limits how many times it applies the rule. For example percentage=25 applies only to 25% of matching commands.
• client_addr: scopes the radius to clients coming from a certain subnet,ip,ip:port. For example client_addr=192.0.0.1

Project Status

RedFI is still in its early stages, so expect rapid changes.
Also, for now, we advise you to use the proxy in development and staging area only.
If you want to use it in production, you should limit the percentage of connections you send to the proxy.

Documentation

Index

• Constants
• Variables
• type API
  • func NewAPI(p *Plan) *API
• type Controller
  • func (c *Controller) Start() error
• type Plan
  • func NewPlan() *Plan
  • func Parse(planPath string) (*Plan, error)
  • func (p *Plan) AddRule(r Rule) error
  • func (p *Plan) DeleteRule(name string) error
  • func (p *Plan) GetRule(name string) (Rule, error)
  • func (p *Plan) ListRules() []Rule
  • func (p *Plan) MarshalCommands()
  • func (p *Plan) SelectRule(clientAddr string, buf []byte) *Rule
• type Proxy
  • func New(planPath, server, addr string) (*Proxy, error)
  • func (p *Proxy) Start() error
• type Response
• type Rule
  • func (r Rule) String() string

Constants

const (
	RULEADD      = "ruleadd"
	RULEDEL      = "ruledel"
	RULELIST     = "rulelist"
	RULEGET      = "ruleget"
	RULECOUNTERS = "rulecounters"
)

const (
	COMMAND    = "command"
	DELAY      = "delay"
	DROP       = "drop"
	RETEMPTY   = "return_empty"
	RETERR     = "return_err"
	CLIENTADDR = "client_addr"
	PERCENTAGE = "percentage"
)

Variables

var ErrMsg = `{"ok": false, "msg": %s}`

var (
	// ErrNotFound is returned iff SelectRule can't find a Rule that applies
	ErrNotFound = errors.New("no matching rule found")
)

Types

type API

type API struct {
	// contains filtered or unexported fields
}

func NewAPI

func NewAPI(p *Plan) *API

type Controller

type Controller struct {
	// contains filtered or unexported fields
}

func (*Controller) Start

func (c *Controller) Start() error

type Plan

type Plan struct {
	Rules []*Rule `json:"rules,omitempty"`
	// contains filtered or unexported fields
}

Plan defines a set of rules to be applied by the proxy

func NewPlan

func NewPlan() *Plan

func Parse

func Parse(planPath string) (*Plan, error)

Parse the plan.json file

func (*Plan) AddRule

func (p *Plan) AddRule(r Rule) error

AddRule adds a rule to the current working plan

func (*Plan) DeleteRule

func (p *Plan) DeleteRule(name string) error

DeleteRule deletes the given ruleName if found otherwise it returns ErrNotFound

func (*Plan) GetRule

func (p *Plan) GetRule(name string) (Rule, error)

GetRule returns the rule that matches the given name

func (*Plan) ListRules

func (p *Plan) ListRules() []Rule

ListRules returns a slice of all the existing rules the slice will be empty if Plan has no rules

func (*Plan) MarshalCommands

func (p *Plan) MarshalCommands()

func (*Plan) SelectRule

func (p *Plan) SelectRule(clientAddr string, buf []byte) *Rule

SelectRule finds the first rule that applies to the given variables

type Proxy

type Proxy struct {
	// contains filtered or unexported fields
}

func New

func New(planPath, server, addr string) (*Proxy, error)

func (*Proxy) Start

func (p *Proxy) Start() error

type Response

type Response struct {
	OK      bool   `json:"ok"`
	Message string `json:"msg,omitempty"`
	Rules   []Rule `json:"rules,omitempty"`
}

type Rule

type Rule struct {
	Name        string `json:"name,omiempty"`
	Delay       int    `json:"delay,omitempty"`
	Drop        bool   `json:"drop,omitempty"`
	ReturnEmpty bool   `json:"return_empty,omitempty"`
	ReturnErr   string `json:"return_err,omitempty"`
	Percentage  int    `json:"percentage,omitempty"`
	// SelectRule does prefix matching on this value
	ClientAddr string `json:"client_addr,omitempty"`
	Command    string `json:"command,omitempty"`
	// contains filtered or unexported fields
}

Rule is what get's applied on every client message iff it matches it

func (Rule) String

func (r Rule) String() string