redcon

package module
Version: v1.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 25, 2021 License: MIT Imports: 14 Imported by: 101

README

REDCON
GoDoc

Redis compatible server framework for Go

Features

  • Create a Fast custom Redis compatible server in Go
  • Simple interface. One function ListenAndServe and two types Conn & Command
  • Support for pipelining and telnet commands
  • Works with Redis clients such as redigo, redis-py, node_redis, and jedis
  • TLS Support
  • Compatible pub/sub support
  • Multithreaded

Installing

go get -u github.com/tidwall/redcon

Example

Here's a full example of a Redis clone that accepts:

  • SET key value
  • GET key
  • DEL key
  • PING
  • QUIT
  • PUBLISH channel message
  • SUBSCRIBE channel

You can run this example from a terminal:

go run example/clone.go
package main

import (
	"log"
	"strings"
	"sync"

	"github.com/tidwall/redcon"
)

var addr = ":6380"

func main() {
	var mu sync.RWMutex
	var items = make(map[string][]byte)
	var ps redcon.PubSub
	go log.Printf("started server at %s", addr)
	err := redcon.ListenAndServe(addr,
		func(conn redcon.Conn, cmd redcon.Command) {
			switch strings.ToLower(string(cmd.Args[0])) {
			default:
				conn.WriteError("ERR unknown command '" + string(cmd.Args[0]) + "'")
			case "ping":
				conn.WriteString("PONG")
			case "quit":
				conn.WriteString("OK")
				conn.Close()
			case "set":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				items[string(cmd.Args[1])] = cmd.Args[2]
				mu.Unlock()
				conn.WriteString("OK")
			case "get":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.RLock()
				val, ok := items[string(cmd.Args[1])]
				mu.RUnlock()
				if !ok {
					conn.WriteNull()
				} else {
					conn.WriteBulk(val)
				}
			case "del":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				_, ok := items[string(cmd.Args[1])]
				delete(items, string(cmd.Args[1]))
				mu.Unlock()
				if !ok {
					conn.WriteInt(0)
				} else {
					conn.WriteInt(1)
				}
			case "publish":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				conn.WriteInt(ps.Publish(string(cmd.Args[1]), string(cmd.Args[2])))
			case "subscribe", "psubscribe":
				if len(cmd.Args) < 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				command := strings.ToLower(string(cmd.Args[0]))
				for i := 1; i < len(cmd.Args); i++ {
					if command == "psubscribe" {
						ps.Psubscribe(conn, string(cmd.Args[i]))
					} else {
						ps.Subscribe(conn, string(cmd.Args[i]))
					}
				}
			}
		},
		func(conn redcon.Conn) bool {
			// Use this function to accept or deny the connection.
			// log.Printf("accept: %s", conn.RemoteAddr())
			return true
		},
		func(conn redcon.Conn, err error) {
			// This is called when the connection has been closed
			// log.Printf("closed: %s, err: %v", conn.RemoteAddr(), err)
		},
	)
	if err != nil {
		log.Fatal(err)
	}
}

TLS Example

Redcon has full TLS support through the ListenAndServeTLS function.

The same example is also provided for serving Redcon over TLS.

go run example/tls/clone.go

Benchmarks

Redis: Single-threaded, no disk persistence.

$ redis-server --port 6379 --appendonly no
redis-benchmark -p 6379 -t set,get -n 10000000 -q -P 512 -c 512
SET: 941265.12 requests per second
GET: 1189909.50 requests per second

Redcon: Single-threaded, no disk persistence.

$ GOMAXPROCS=1 go run example/clone.go
redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 2018570.88 requests per second
GET: 2403846.25 requests per second

Redcon: Multi-threaded, no disk persistence.

$ GOMAXPROCS=0 go run example/clone.go
$ redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 1944390.38 requests per second
GET: 3993610.25 requests per second

Running on a MacBook Pro 15" 2.8 GHz Intel Core i7 using Go 1.7

Contact

Josh Baker @tidwall

License

Redcon source code is available under the MIT License.

Documentation

Overview

Package redcon implements a Redis compatible server framework

Index

Constants

View Source
const (
	Integer = ':'
	String  = '+'
	Bulk    = '$'
	Array   = '*'
	Error   = '-'
)

Various RESP kinds

Variables

This section is empty.

Functions

func AppendAny added in v1.3.0

func AppendAny(b []byte, v interface{}) []byte

AppendAny appends any type to valid Redis type.

nil             -> null
error           -> error (adds "ERR " when first word is not uppercase)
string          -> bulk-string
numbers         -> bulk-string
[]byte          -> bulk-string
bool            -> bulk-string ("0" or "1")
slice           -> array
map             -> array with key/value pairs
SimpleString    -> string
SimpleInt       -> integer
Marshaler       -> raw bytes
everything-else -> bulk-string representation using fmt.Sprint()

func AppendArray

func AppendArray(b []byte, n int) []byte

AppendArray appends a Redis protocol array to the input bytes.

func AppendBulk

func AppendBulk(b []byte, bulk []byte) []byte

AppendBulk appends a Redis protocol bulk byte slice to the input bytes.

func AppendBulkFloat added in v1.0.2

func AppendBulkFloat(dst []byte, f float64) []byte

AppendBulkFloat appends a float64, as bulk bytes.

func AppendBulkInt added in v1.0.2

func AppendBulkInt(dst []byte, x int64) []byte

AppendBulkInt appends an int64, as bulk bytes.

func AppendBulkString

func AppendBulkString(b []byte, bulk string) []byte

AppendBulkString appends a Redis protocol bulk string to the input bytes.

func AppendBulkUint added in v1.0.2

func AppendBulkUint(dst []byte, x uint64) []byte

AppendBulkUint appends an uint64, as bulk bytes.

func AppendError

func AppendError(b []byte, s string) []byte

AppendError appends a Redis protocol error to the input bytes.

func AppendInt

func AppendInt(b []byte, n int64) []byte

AppendInt appends a Redis protocol int64 to the input bytes.

func AppendNull

func AppendNull(b []byte) []byte

AppendNull appends a Redis protocol null to the input bytes.

func AppendOK

func AppendOK(b []byte) []byte

AppendOK appends a Redis protocol OK to the input bytes.

func AppendString

func AppendString(b []byte, s string) []byte

AppendString appends a Redis protocol string to the input bytes.

func AppendTile38

func AppendTile38(b []byte, data []byte) []byte

AppendTile38 appends a Tile38 message to the input bytes.

func AppendUint

func AppendUint(b []byte, n uint64) []byte

AppendUint appends a Redis protocol uint64 to the input bytes.

func ListenAndServe

func ListenAndServe(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) error

ListenAndServe creates a new server and binds to addr configured on "tcp" network net.

func ListenAndServeNetwork

func ListenAndServeNetwork(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) error

ListenAndServeNetwork creates a new server and binds to addr. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func ListenAndServeNetworkTLS

func ListenAndServeNetworkTLS(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) error

ListenAndServeNetworkTLS creates a new TLS server and binds to addr. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func ListenAndServeTLS

func ListenAndServeTLS(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) error

ListenAndServeTLS creates a new TLS server and binds to addr configured on "tcp" network net.

func Serve added in v1.0.0

func Serve(ln net.Listener,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) error

Serve creates a new server and serves with the given net.Listener.

Types

type Command

type Command struct {
	// Raw is a encoded RESP message.
	Raw []byte
	// Args is a series of arguments that make up the command.
	Args [][]byte
}

Command represent a command

func Parse

func Parse(raw []byte) (Command, error)

Parse parses a raw RESP message and returns a command.

type Conn

type Conn interface {
	// RemoteAddr returns the remote address of the client connection.
	RemoteAddr() string
	// Close closes the connection.
	Close() error
	// WriteError writes an error to the client.
	WriteError(msg string)
	// WriteString writes a string to the client.
	WriteString(str string)
	// WriteBulk writes bulk bytes to the client.
	WriteBulk(bulk []byte)
	// WriteBulkString writes a bulk string to the client.
	WriteBulkString(bulk string)
	// WriteInt writes an integer to the client.
	WriteInt(num int)
	// WriteInt64 writes a 64-bit signed integer to the client.
	WriteInt64(num int64)
	// WriteUint64 writes a 64-bit unsigned integer to the client.
	WriteUint64(num uint64)
	// WriteArray writes an array header. You must then write additional
	// sub-responses to the client to complete the response.
	// For example to write two strings:
	//
	//   c.WriteArray(2)
	//   c.WriteBulk("item 1")
	//   c.WriteBulk("item 2")
	WriteArray(count int)
	// WriteNull writes a null to the client
	WriteNull()
	// WriteRaw writes raw data to the client.
	WriteRaw(data []byte)
	// WriteAny writes any type to the client.
	//   nil             -> null
	//   error           -> error (adds "ERR " when first word is not uppercase)
	//   string          -> bulk-string
	//   numbers         -> bulk-string
	//   []byte          -> bulk-string
	//   bool            -> bulk-string ("0" or "1")
	//   slice           -> array
	//   map             -> array with key/value pairs
	//   SimpleString    -> string
	//   SimpleInt       -> integer
	//   everything-else -> bulk-string representation using fmt.Sprint()
	WriteAny(any interface{})
	// Context returns a user-defined context
	Context() interface{}
	// SetContext sets a user-defined context
	SetContext(v interface{})
	// SetReadBuffer updates the buffer read size for the connection
	SetReadBuffer(bytes int)
	// Detach return a connection that is detached from the server.
	// Useful for operations like PubSub.
	//
	//   dconn := conn.Detach()
	//   go func(){
	//       defer dconn.Close()
	//       cmd, err := dconn.ReadCommand()
	//       if err != nil{
	//           fmt.Printf("read failed: %v\n", err)
	//	         return
	//       }
	//       fmt.Printf("received command: %v", cmd)
	//       hconn.WriteString("OK")
	//       if err := dconn.Flush(); err != nil{
	//           fmt.Printf("write failed: %v\n", err)
	//	         return
	//       }
	//   }()
	Detach() DetachedConn
	// ReadPipeline returns all commands in current pipeline, if any
	// The commands are removed from the pipeline.
	ReadPipeline() []Command
	// PeekPipeline returns all commands in current pipeline, if any.
	// The commands remain in the pipeline.
	PeekPipeline() []Command
	// NetConn returns the base net.Conn connection
	NetConn() net.Conn
}

Conn represents a client connection

type DetachedConn

type DetachedConn interface {
	// Conn is the original connection
	Conn
	// ReadCommand reads the next client command.
	ReadCommand() (Command, error)
	// Flush flushes any writes to the network.
	Flush() error
}

DetachedConn represents a connection that is detached from the server

type Handler added in v1.0.1

type Handler interface {
	ServeRESP(conn Conn, cmd Command)
}

A Handler responds to an RESP request.

type HandlerFunc added in v1.0.1

type HandlerFunc func(conn Conn, cmd Command)

The HandlerFunc type is an adapter to allow the use of ordinary functions as RESP handlers. If f is a function with the appropriate signature, HandlerFunc(f) is a Handler that calls f.

func (HandlerFunc) ServeRESP added in v1.0.1

func (f HandlerFunc) ServeRESP(conn Conn, cmd Command)

ServeRESP calls f(w, r)

type Kind

type Kind int

Kind is the kind of command

const (
	// Redis is returned for Redis protocol commands
	Redis Kind = iota
	// Tile38 is returnd for Tile38 native protocol commands
	Tile38
	// Telnet is returnd for plain telnet commands
	Telnet
)

func ReadNextCommand

func ReadNextCommand(packet []byte, argsbuf [][]byte) (
	complete bool, args [][]byte, kind Kind, leftover []byte, err error,
)

ReadNextCommand reads the next command from the provided packet. It's possible that the packet contains multiple commands, or zero commands when the packet is incomplete. 'argsbuf' is an optional reusable buffer and it can be nil. 'complete' indicates that a command was read. false means no more commands. 'args' are the output arguments for the command. 'kind' is the type of command that was read. 'leftover' is any remaining unused bytes which belong to the next command. 'err' is returned when a protocol error was encountered.

type Marshaler added in v1.3.3

type Marshaler interface {
	MarshalRESP() []byte
}

Marshaler is the interface implemented by types that can marshal themselves into a Redis response type from an *Any call. The return value is not check for validity.

type PubSub added in v1.4.0

type PubSub struct {
	// contains filtered or unexported fields
}

PubSub is a Redis compatible pub/sub server

func (*PubSub) Psubscribe added in v1.4.0

func (ps *PubSub) Psubscribe(conn Conn, channel string)

Psubscribe a connection to PubSub

func (*PubSub) Publish added in v1.4.0

func (ps *PubSub) Publish(channel, message string) int

Publish a message to subscribers

func (*PubSub) Subscribe added in v1.4.0

func (ps *PubSub) Subscribe(conn Conn, channel string)

Subscribe a connection to PubSub

type RESP added in v1.2.0

type RESP struct {
	Type  Type
	Raw   []byte
	Data  []byte
	Count int
}

RESP ...

func ReadNextRESP added in v1.2.0

func ReadNextRESP(b []byte) (n int, resp RESP)

ReadNextRESP returns the next resp in b and returns the number of bytes the took up the result.

func (*RESP) ForEach added in v1.2.0

func (r *RESP) ForEach(iter func(resp RESP) bool)

ForEach iterates over each Array element

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader represent a reader for RESP or telnet commands.

func NewReader

func NewReader(rd io.Reader) *Reader

NewReader returns a command reader which will read RESP or telnet commands.

func (*Reader) ReadCommand

func (rd *Reader) ReadCommand() (Command, error)

ReadCommand reads the next command.

func (*Reader) ReadCommands added in v1.4.1

func (rd *Reader) ReadCommands() ([]Command, error)

ReadCommands reads the next pipeline commands.

type ServeMux added in v1.0.1

type ServeMux struct {
	// contains filtered or unexported fields
}

ServeMux is an RESP command multiplexer.

func NewServeMux added in v1.0.1

func NewServeMux() *ServeMux

NewServeMux allocates and returns a new ServeMux.

func (*ServeMux) Handle added in v1.0.1

func (m *ServeMux) Handle(command string, handler Handler)

Handle registers the handler for the given command. If a handler already exists for command, Handle panics.

func (*ServeMux) HandleFunc added in v1.0.1

func (m *ServeMux) HandleFunc(command string, handler func(conn Conn, cmd Command))

HandleFunc registers the handler function for the given command.

func (*ServeMux) ServeRESP added in v1.0.1

func (m *ServeMux) ServeRESP(conn Conn, cmd Command)

ServeRESP dispatches the command to the handler.

type Server

type Server struct {

	// AcceptError is an optional function used to handle Accept errors.
	AcceptError func(err error)
	// contains filtered or unexported fields
}

Server defines a server for clients for managing client connections.

func NewServer

func NewServer(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) *Server

NewServer returns a new Redcon server configured on "tcp" network net.

func NewServerNetwork

func NewServerNetwork(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
) *Server

NewServerNetwork returns a new Redcon server. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func (*Server) Addr

func (s *Server) Addr() net.Addr

Addr returns server's listen address

func (*Server) Close

func (s *Server) Close() error

Close stops listening on the TCP address. Already Accepted connections will be closed.

func (*Server) ListenAndServe

func (s *Server) ListenAndServe() error

ListenAndServe serves incoming connections.

func (*Server) ListenServeAndSignal

func (s *Server) ListenServeAndSignal(signal chan error) error

ListenServeAndSignal serves incoming connections and passes nil or error when listening. signal can be nil.

func (*Server) Serve added in v1.0.0

func (s *Server) Serve(ln net.Listener) error

Serve serves incoming connections with the given net.Listener.

func (*Server) SetIdleClose added in v1.4.0

func (s *Server) SetIdleClose(dur time.Duration)

SetIdleClose will automatically close idle connections after the specified duration. Use zero to disable this feature.

type SimpleInt added in v1.3.2

type SimpleInt int

SimpleInt is for representing a non-bulk representation of a int from an *Any call.

type SimpleString added in v1.3.2

type SimpleString string

SimpleString is for representing a non-bulk representation of a string from an *Any call.

type TLSServer

type TLSServer struct {
	*Server
	// contains filtered or unexported fields
}

TLSServer defines a server for clients for managing client connections.

func NewServerNetworkTLS

func NewServerNetworkTLS(
	net, laddr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) *TLSServer

NewServerNetworkTLS returns a new TLS Redcon server. The network net must be a stream-oriented network: "tcp", "tcp4", "tcp6", "unix" or "unixpacket"

func NewServerTLS

func NewServerTLS(addr string,
	handler func(conn Conn, cmd Command),
	accept func(conn Conn) bool,
	closed func(conn Conn, err error),
	config *tls.Config,
) *TLSServer

NewServerTLS returns a new Redcon TLS server configured on "tcp" network net.

func (*TLSServer) Close

func (s *TLSServer) Close() error

Close stops listening on the TCP address. Already Accepted connections will be closed.

func (*TLSServer) ListenAndServe

func (s *TLSServer) ListenAndServe() error

ListenAndServe serves incoming connections.

func (*TLSServer) ListenServeAndSignal

func (s *TLSServer) ListenServeAndSignal(signal chan error) error

ListenServeAndSignal serves incoming connections and passes nil or error when listening. signal can be nil.

type Type added in v1.2.0

type Type byte

Type of RESP

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer allows for writing RESP messages.

func BaseWriter

func BaseWriter(c Conn) *Writer

BaseWriter returns the underlying connection writer, if any

func NewWriter

func NewWriter(wr io.Writer) *Writer

NewWriter creates a new RESP writer.

func (*Writer) Buffer

func (w *Writer) Buffer() []byte

Buffer returns the unflushed buffer. This is a copy so changes to the resulting []byte will not affect the writer.

func (*Writer) Flush

func (w *Writer) Flush() error

Flush writes all unflushed Write* calls to the underlying writer.

func (*Writer) SetBuffer

func (w *Writer) SetBuffer(raw []byte)

SetBuffer replaces the unflushed buffer with new bytes.

func (*Writer) WriteAny added in v1.3.0

func (w *Writer) WriteAny(v interface{})

WriteAny writes any type to client.

nil             -> null
error           -> error (adds "ERR " when first word is not uppercase)
string          -> bulk-string
numbers         -> bulk-string
[]byte          -> bulk-string
bool            -> bulk-string ("0" or "1")
slice           -> array
map             -> array with key/value pairs
SimpleString    -> string
SimpleInt       -> integer
everything-else -> bulk-string representation using fmt.Sprint()

func (*Writer) WriteArray

func (w *Writer) WriteArray(count int)

WriteArray writes an array header. You must then write additional sub-responses to the client to complete the response. For example to write two strings:

c.WriteArray(2)
c.WriteBulk("item 1")
c.WriteBulk("item 2")

func (*Writer) WriteBulk

func (w *Writer) WriteBulk(bulk []byte)

WriteBulk writes bulk bytes to the client.

func (*Writer) WriteBulkString

func (w *Writer) WriteBulkString(bulk string)

WriteBulkString writes a bulk string to the client.

func (*Writer) WriteError

func (w *Writer) WriteError(msg string)

WriteError writes an error to the client.

func (*Writer) WriteInt

func (w *Writer) WriteInt(num int)

WriteInt writes an integer to the client.

func (*Writer) WriteInt64

func (w *Writer) WriteInt64(num int64)

WriteInt64 writes a 64-bit signed integer to the client.

func (*Writer) WriteNull

func (w *Writer) WriteNull()

WriteNull writes a null to the client

func (*Writer) WriteRaw

func (w *Writer) WriteRaw(data []byte)

WriteRaw writes raw data to the client.

func (*Writer) WriteString

func (w *Writer) WriteString(msg string)

WriteString writes a string to the client.

func (*Writer) WriteUint64 added in v1.0.1

func (w *Writer) WriteUint64(num uint64)

WriteUint64 writes a 64-bit unsigned integer to the client.

Source Files

Directories

Path Synopsis
mux
tls

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL