uhaha

package module
v0.11.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 7, 2022 License: MIT Imports: 32 Imported by: 5

README

uhaha

GoDoc

High Availabilty Framework for Happy Data

Uhaha is a framework for building highly available Raft-based data applications in Go. This is basically an upgrade to the Finn project, but has an updated API, better security features (TLS and auth passwords), customizable services, deterministic time, recalculable random numbers, simpler snapshots, a smaller network footprint, and more. Under the hood it utilizes hashicorp/raft, tidwall/redcon, and syndtr/goleveldb.

Features

  • Simple API for quickly creating a custom Raft-based application.
  • Deterministic monotonic time that does not drift and stays in sync with the internet.
  • APIs for building custom services such as HTTP and gRPC. Supports the Redis protocol by default, so most Redis client library will work with Uhaha.
  • TLS and Auth password support.
  • Multiple examples to help jumpstart integration, including a Key-value DB, a Timeseries DB, and a Ticket Service.

Example

Below a simple example of a service for monotonically increasing tickets.

package main

import "github.com/tidwall/uhaha"

type data struct {
	Ticket int64
}

func main() {
	// Set up a uhaha configuration
	var conf uhaha.Config
	
	// Give the application a name. All servers in the cluster should use the
	// same name.
	conf.Name = "ticket"
	
	// Set the initial data. This is state of the data when first server in the 
	// cluster starts for the first time ever.
	conf.InitialData = new(data)

	// Since we are not holding onto much data we can used the built-in JSON
	// snapshot system. You just need to make sure all the important fields in
	// the data are exportable (capitalized) to JSON. In this case there is
	// only the one field "Ticket".
	conf.UseJSONSnapshots = true
	
	// Add a command that will change the value of a Ticket. 
	conf.AddWriteCommand("ticket", cmdTICKET)

	// Finally, hand off all processing to uhaha.
	uhaha.Main(conf)
}

// TICKET
// help: returns a new ticket that has a value that is at least one greater
// than the previous TICKET call.
func cmdTICKET(m uhaha.Machine, args []string) (interface{}, error) {
	// The the current data from the machine
	data := m.Data().(*data)

	// Increment the ticket
	data.Ticket++

	// Return the new ticket to caller
	return data.Ticket, nil
}

Building

Using the source file from the examples directory, we'll build an application named "ticket"

go build -o ticket examples/ticket/main.go

Running

It's ideal to have three, five, or seven nodes in your cluster.

Let's create the first node.

./ticket -n 1 -a :11001

This will create a node named 1 and bind the address to :11001

Now let's create two more nodes and add them to the cluster.

./ticket -n 2 -a :11002 -j :11001
./ticket -n 3 -a :11003 -j :11001

Now we have a fault-tolerant three node cluster up and running.

Using

You can use any Redis compatible client, such as the redis-cli, telnet, or netcat.

I'll use the redis-cli in the example below.

Connect to the leader. This will probably be the first node you created.

redis-cli -p 11001

Send the server a TICKET command and receive the first ticket.

> TICKET
"1"

From here on every TICKET command will guarentee to generate a value larger than the previous TICKET command.

> TICKET
"2"
> TICKET
"3"
> TICKET
"4"
> TICKET
"5"

Built-in Commands

There are a number built-in commands for managing and monitor the cluster.

VERSION                                 # show the application version
MACHINE                                 # show information about the state machine
RAFT LEADER                             # show the address of the current raft leader
RAFT INFO [pattern]                     # show information about the raft server and cluster
RAFT SERVER LIST                        # show all servers in cluster
RAFT SERVER ADD id address              # add a server to cluster
RAFT SERVER REMOVE id                   # remove a server from the cluster
RAFT SNAPSHOT NOW                       # make a snapshot of the data
RAFT SNAPSHOT LIST                      # show a list of all snapshots on server
RAFT SNAPSHOT FILE id                   # show the file path of a snapshot on server
RAFT SNAPSHOT READ id [RANGE start end] # download all or part of a snapshot

And also some client commands.

QUIT                                    # close the client connection
PING                                    # ping the server
ECHO [message]                          # echo a message to the server
AUTH password                           # authenticate with a password

Network and security considerations (TLS and Auth password)

By default a single Uhaha instance is bound to the local 127.0.0.1 IP address. Thus nothing outside that machine, including other servers in the cluster or machines on the same local network will be able communicate with this instance.

Network security

To open up the service you will need to provide an IP address that can be reached from the outside. For example, let's say you want to set up three servers on a local 10.0.0.0 network.

On server 1:

./ticket -n 1 -a 10.0.0.1:11001

On server 2:

./ticket -n 2 -a 10.0.0.2:11001 -j 10.0.0.1:11001

On server 3:

./ticket -n 3 -a 10.0.0.3:11001 -j 10.0.0.1:11001

Now you have a Raft cluster running on three distinct servers in the same local network. This may be enough for applications that only require a network security policy. Basically any server on the local network can access the cluster.

Auth password

If you want to lock down the cluster further you can provide a secret auth, which is more or less a password that the cluster and client will need to communicate with each other.

./ticket -n 1 -a 10.0.0.1:11001 --auth my-secret

All the servers will need to be started with the same auth.

./ticket -n 2 -a 10.0.0.2:11001 --auth my-secret -j 10.0.0.1:11001

./ticket -n 2 -a 10.0.0.3:11001 --auth my-secret -j 10.0.0.1:11001

The client will also need the same auth to talk with cluster. All redis clients support an auth password, such as:

redis-cli -h 10.0.0.1 -p 11001 -a my-secret

This may be enough if you keep all your machines on the same private network, but you don't want all machines or applications to have unfettered access to the cluster.

TLS

Finally you can use TLS, which I recommend along with an auth password.

In this example a custom cert and key are created using the mkcert tool.

mkcert uhaha-example
# produces uhaha-example.pem, uhaha-example-key.pem, and a rootCA.pem

Then create a cluster using the cert & key files. Along with an auth.

./ticket -n 1 -a 10.0.0.1:11001 --tls-cert uhaha-example.pem --tls-key uhaha-example-key.pem --auth my-secret

./ticket -n 2 -a 10.0.0.2:11001 --tls-cert uhaha-example.pem --tls-key uhaha-example-key.pem --auth my-secret -j 10.0.0.1:11001

./ticket -n 2 -a 10.0.0.3:11001 --tls-cert uhaha-example.pem --tls-key uhaha-example-key.pem --auth my-secret -j 10.0.0.1:11001

Now you can connect to the server from a client that has the rootCA.pem. You can find the location of your rootCA.pem file in the running ls "$(mkcert -CAROOT)/rootCA.pem".

redis-cli -h 10.0.0.1 -p 11001 --tls --cacert rootCA.pem -a my-secret

Command-line options

Below are all of the command line options.

Usage: my-uhaha-app [-n id] [-a addr] [options]

Basic options:
  -v               : display version
  -h               : display help, this screen
  -a addr          : bind to address  (default: 127.0.0.1:11001)
  -n id            : node ID  (default: 1)
  -d dir           : data directory  (default: data)
  -j addr          : leader address of a cluster to join
  -l level         : log level  (default: info) [debug,verb,info,warn,silent]

Security options:
  --tls-cert path  : path to TLS certificate
  --tls-key path   : path to TLS private key
  --auth auth      : cluster authorization, shared by all servers and clients

Networking options:
  --advertise addr : advertise address  (default: network bound address)

Advanced options:
  --nosync         : turn off syncing data to disk after every write. This leads
                     to faster write operations but opens up the chance for data
                     loss due to catastrophic events such as power failure.
  --openreads      : allow followers to process read commands, but with the
                     possibility of returning stale data.
  --localtime      : have the raft machine time synchronized with the local
                     server rather than the public internet. This will run the
                     risk of time shifts when the local server time is
                     drastically changed during live operation.
  --restore path   : restore a raft machine from a snapshot file. This will
                     start a brand new single-node cluster using the snapshot as
                     initial data. The other nodes must be re-joined. This
                     operation is ignored when a data directory already exists.
                     Cannot be used with -j flag.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrCorrupt = errors.New("corrupt")

ErrCorrupt is returned when a data is invalid or corrupt

View Source 
var ErrInvalid = errors.New("invalid")

ErrInvalid is returned when an operation has invalid arguments or options

View Source 
var ErrNotLeader = raft.ErrNotLeader

ErrNotLeader is returned when the raft leader is unknown

View Source 
var ErrSyntax = errors.New("syntax error")

ErrSyntax is returned where there was a syntax error

View Source 
var ErrUnauthorized = errors.New("unauthorized")

ErrUnauthorized is returned when a client connection has not been authorized

View Source 
var ErrUnknownCommand = errors.New("unknown command")

ErrUnknownCommand is returned when a command is not known

View Source 
var ErrWrongNumArgs = errors.New("wrong number of arguments")

ErrWrongNumArgs is returned when the arg count is wrong

Functions

func Main 

func Main(conf Config)

Main entrypoint for the cluster node. This must be called once and only once, and as the last call in the Go main() function. There are no return values as all application operations, logging, and I/O will be forever transferred.

func ReadRawMachineInfo added in v0.1.5 

func ReadRawMachineInfo(m Machine, info *RawMachineInfo)

ReadRawMachineInfo reads the raw machine components.

func RedisDial added in v0.6.0 

func RedisDial(addr, auth string, tlscfg *tls.Config) (redis.Conn, error)

RedisDial is a helper function that dials out to another Uhaha server with redis protocol and using the provded TLS config and Auth token. The TLS/Auth must be correct in order to establish a connection.

func WriteRawMachineInfo added in v0.1.5 

func WriteRawMachineInfo(m Machine, info *RawMachineInfo)

WriteRawMachineInfo writes raw components to the machine. Use with care as this operation may destroy the consistency of your cluster.

Types

type Backend 

type Backend int

The Backend database format used for storing Raft logs and meta data. 

const (
	// LevelDB is an on-disk LSM (LSM log-structured merge-tree) database. This
	// format is optimized for fast sequential writes, which is ideal for most
	// Raft implementations. This is the default format used by Uhaha.
	LevelDB Backend = iota
	// Bolt is an on-disk single-file b+tree database. This format has been a
	// popular choice for Go-based Raft implementations for years.
	Bolt
)

type Config 

type Config struct {

	// Name gives the server application a name. Default "uhaha-app"
	Name string

	// Version of the application. Default "0.0.0"
	Version string

	// GitSHA of the application.
	GitSHA string

	// Flag is used to manage the application startup flags.
	Flag struct {
		// Custom tells Main to not automatically parse the application startup
		// flags. When set it is up to the user to parse the os.Args manually
		// or with a different library.
		Custom bool
		// Usage is an optional function that allows for altering the usage
		// message.
		Usage func(usage string) string
		// PreParse is an optional function that allows for adding command line
		// flags before the user flags are parsed.
		PreParse func()
		// PostParse is an optional function that fires after user flags are
		// parsed.
		PostParse func()
	}

	// Snapshot fires when a snapshot
	Snapshot func(data interface{}) (Snapshot, error)

	// Restore returns a data object that is fully restored from the previous
	// snapshot using the input Reader. A restore operation on happens once,
	// if needed, at the start of the application.
	Restore func(rd io.Reader) (data interface{}, err error)

	// UseJSONSnapshots is a convienence field that tells the machine to use
	// JSON as the format for all snapshots and restores. This may be good for
	// small simple data models which have types that can be fully marshalled
	// into JSON, ie. all imporant data fields need to exportable (Capitalized).
	// For more complicated or specialized data, it's proabably best to assign
	// custom functions to the Config.Snapshot and Config.Restore fields.
	// It's invalid to set this field while also setting Snapshot and/or
	// Restore. Default false
	UseJSONSnapshots bool

	// Tick fires at regular intervals as specified by TickDelay. This function
	// can be used to make updates to the database.
	Tick func(m Machine)

	// DataDirReady is an optional callback function that fires containing the
	// path to the directory where all the logs and snapshots are stored.
	DataDirReady func(dir string)

	// LogReady is an optional callback function that fires when the logger has
	// been initialized. The logger is can be safely used concurrently.
	LogReady func(log Logger)

	// ServerReady is an optional callback function that fires when the server
	// socket is listening and is ready to accept incoming connections. The
	// network address, auth, and tls-config are provided to allow for
	// background connections to be made to self, if desired.
	ServerReady func(addr, auth string, tlscfg *tls.Config)

	// ConnOpened is an optional callback function that fires when a new
	// network connection was opened on this machine. You can accept or deny
	// the connection, and optionally provide a client-specific context that
	// stick around until the connection is closed with ConnClosed.
	ConnOpened func(addr string) (context interface{}, accept bool)

	// ConnClosed is an optional callback function that fires when a network
	// connection has been closed on this machine.
	ConnClosed func(context interface{}, addr string)

	// ResponseFilter is and options function used to filter every response
	// prior to send into a client connection.
	ResponseFilter ResponseFilter

	// StateChange is an optional callback function that fires when the raft
	// state has changed.
	StateChange func(state State)

	// LocalConnector is an optional callback function that returns a new
	// connector that allows for establishing "local" connections through
	// the Redis protocol. A local connection bypasses the network and
	// communicates directly with this server, though the same process.
	LocalConnector func(lconn LocalConnector)

	LocalTime   bool          // default false
	TickDelay   time.Duration // default 200ms
	BackupPath  string        // default ""
	InitialData interface{}   // default nil
	NodeID      string        // default "1"
	Addr        string        // default ":11001"
	DataDir     string        // default "data"
	LogOutput   io.Writer     // default os.Stderr
	LogLevel    string        // default "notice"
	JoinAddr    string        // default ""
	Backend     Backend       // default LevelDB
	NoSync      bool          // default false
	OpenReads   bool          // default false
	MaxPool     int           // default 8
	TLSCertPath string        // default ""
	TLSKeyPath  string        // default ""
	Auth        string        // default ""
	Advertise   string        // default ""
	TryErrors   bool          // default false (return TRY instead of MOVED)
	InitRunQuit bool          // default false
	// contains filtered or unexported fields
}

Config is the configuration for managing the behavior of the application. This must be fill out prior and then passed to the uhaha.Main() function.

func (*Config) AddCatchallCommand added in v0.2.1 

func (conf *Config) AddCatchallCommand(
	fn func(m Machine, args []string) (interface{}, error),
)

AddCatchallCommand adds a intermediate command that will execute for any input that was not previously defined with AddIntermediateCommand, AddWriteCommand, or AddReadCommand.

func (*Config) AddIntermediateCommand added in v0.5.0 

func (conf *Config) AddIntermediateCommand(name string,
	fn func(m Machine, args []string) (interface{}, error),
)

AddIntermediateCommand adds a command that is for peforming client and system specific operations. It *is not* intended for working with the machine data, and doing so will risk data corruption.

func (*Config) AddReadCommand 

func (conf *Config) AddReadCommand(name string,
	fn func(m Machine, args []string) (interface{}, error),
)

AddReadCommand adds a command for reading machine data.

func (*Config) AddService 

func (conf *Config) AddService(
	name string,
	sniff func(rd io.Reader) bool,
	acceptor func(s Service, ln net.Listener),
)

AddService adds a custom client network service, such as HTTP or gRPC. By default, a Redis compatible service is already included.

func (*Config) AddWriteCommand 

func (conf *Config) AddWriteCommand(name string,
	fn func(m Machine, args []string) (interface{}, error),
)

AddWriteCommand adds a command for reading or altering machine data.

type FilterArgs added in v0.1.1 

type FilterArgs []string

FilterArgs ...

type Hijack 

type Hijack func(s Service, conn HijackedConn)

Hijack is a function type that can be used to "hijack" a service client connection and allowing to perform I/O operations outside the standard network loop. An example of it's usage can be found in the examples/kvdb project.

type HijackedConn 

type HijackedConn interface {
	// RemoteAddr is the connection remote tcp address.
	RemoteAddr() string
	// ReadCommands is an iterator function that reads pipelined commands.
	// Returns a error when the connection encountared and error.
	ReadCommands(func(args []string) bool) error
	// ReadCommand reads one command at a time.
	ReadCommand() (args []string, err error)
	// WriteAny writes any type to the write buffer using the format rules that
	// are defined by the original Service.
	WriteAny(v interface{})
	// WriteRaw writes raw data to the write buffer.
	WriteRaw(data []byte)
	// Flush the write write buffer and send data to the connection.
	Flush() error
	// Close the connection
	Close() error
}

HijackedConn is a connection that has been detached from the main service network loop. It's entirely up to the hijacker to performs all I/O operations. The Write* functions buffer write data and the Flush must be called to do the actual sending of the data to the connection. Close the connection to when done.

type LocalConn added in v0.11.0 

type LocalConn interface {
	Do(args ...string) redcon.RESP
	Close()
}

type LocalConnector added in v0.11.0 

type LocalConnector interface {
	Open() (LocalConn, error)
}

type Logger 

type Logger interface {
	Debugf(format string, args ...interface{})
	Debug(args ...interface{})
	Debugln(args ...interface{})
	Verbf(format string, args ...interface{})
	Verb(args ...interface{})
	Verbln(args ...interface{})
	Noticef(format string, args ...interface{})
	Notice(args ...interface{})
	Noticeln(args ...interface{})
	Printf(format string, args ...interface{})
	Print(args ...interface{})
	Println(args ...interface{})
	Warningf(format string, args ...interface{})
	Warning(args ...interface{})
	Warningln(args ...interface{})
	Fatalf(format string, args ...interface{})
	Fatal(args ...interface{})
	Fatalln(args ...interface{})
	Panicf(format string, args ...interface{})
	Panic(args ...interface{})
	Panicln(args ...interface{})
	Errorf(format string, args ...interface{})
	Error(args ...interface{})
	Errorln(args ...interface{})
}

Logger is the logger used by Uhaha for printing all console messages.

type Machine 

type Machine interface {
	// Data is the original user data interface that was assigned at startup.
	// It's safe to alter the data in this interface while inside a Write
	// command, but it's only safe to read the data from this interface for
	// Read commands.
	// For Intermediate commands, it's not safe to read or write data from
	// this interface and you should use at your own risk.
	Data() interface{}
	// Now generates a stable timestamp that is synced with internet time
	// and for Write commands is always monotonical increasing. It's made to
	// be a trusted source of time for performing operations on the user data.
	// Always use this function instead of the builtin time.Now().
	// Returns nil for Intermediate Commands.
	Now() time.Time
	// Rand is a random number generator that must be used instead of the
	// standard Go packages `crypto/rand` and `math/rand`. For Write commands
	// the values returned from this generator are crypto seeded, guaranteed
	// to be reproduced in exact order when the server restarts, and identical
	// across all machines in the cluster. The underlying implementation is
	// PCG. Check out http://www.pcg-random.org/ for more information.
	// Returns nil for Intermediate Commands.
	Rand() Rand
	// Utility logger for printing information to the local server log.
	Log() Logger
	// Context returns the connection context that was defined in from the
	// Config.ConnOpened callback.
	// Only available for Intermediate and Read commands.
	// Returns nil for Write Commands.
	Context() interface{}
}

The Machine interface is passed to every command. It includes the user data and various utilities that should be used from Write, Read, and Intermediate commands.

It's important to note that the Data(), Now(), and Rand() functions can be used safely for Write and Read commands, but are not available for Intermediate commands. The Context() is ONLY available for Intermediate commands.

A call to Rand() and Now() from inside of a Read command will always return back the same last known value of it's respective type. While, from a Write command, you'll get freshly generated values. This is to ensure that the every single command ALWAYS generates the same series of data on every server.

type Message 

type Message struct {
	// Args are the original command arguments.
	Args []string
	// Resp is the command reponse, if not an error.
	Resp interface{}
	// Err is the command error, if not successful.
	Err error
	// Elapsed is the amount of time that the command took to process.
	Elapsed time.Duration
	// Addr is the remote TCP address of the connection that generated
	// this message.
	Addr string
}

A Message represents a command and is in a format that is consumed by an Observer.

type Monitor 

type Monitor interface {
	// Send a message to observers
	Send(msg Message)
	// NewObjser returns a new Observer containing a channel that will send the
	// messages for every command processed by the service.
	// Stop the observer to release associated resources.
	NewObserver() Observer
}

Monitor represents an interface for sending and consuming command messages that are processed by a Service.

type Observer 

type Observer interface {
	Stop()
	C() <-chan Message
}

An Observer holds a channel that delivers the messages for all commands processed by a Service.

type Rand 

type Rand interface {
	Int() int
	Uint64() uint64
	Uint32() uint32
	Float64() float64
	Read([]byte) (n int, err error)
	UUID() string
}

Rand is a random number interface used by Machine

type RawMachineInfo added in v0.1.5 

type RawMachineInfo struct {
	TS   int64
	Seed int64
}

RawMachineInfo represents the raw components of the machine

type Receiver 

type Receiver interface {
	Args() []string
	Recv() (interface{}, time.Duration, error)
}

Receiver ...

func Response 

func Response(args []string, v interface{}, elapsed time.Duration, err error,
) Receiver

Response ...

type ResponseFilter added in v0.10.0 

type ResponseFilter func(
	serviceName string, context interface{},
	args []string, v interface{},
) interface{}

type SendOptions 

type SendOptions struct {
	Context        interface{}
	From           interface{}
	AllowOpenReads bool
	DenyOpenReads  bool
}

SendOptions ...

type Service 

type Service interface {
	// The name of the service
	Name() string
	// Send a command with args from a client
	Send(args []string, opts *SendOptions) Receiver
	// Auth authorizes a client
	Auth(auth string) error
	// Log is shared logger
	Log() Logger
	// Monitor returns a service monitor for observing client commands.
	Monitor() Monitor
	// Opened
	Opened(addr string) (context interface{}, accept bool)
	// Closed
	Closed(context interface{}, addr string)
	// ResponseFilter
	ResponseFilter() ResponseFilter
}

Service is a client facing service.

type Snapshot 

type Snapshot interface {
	Persist(io.Writer) error
	Done(path string)
}

A Snapshot is an interface that allows for Raft snapshots to be taken.

type State added in v0.11.0 

type State byte

State captures the state of a Raft node: Follower, Candidate, Leader, or Shutdown. 

const (
	// Follower is the initial state of a Raft node.
	Follower State = iota

	// Candidate is one of the valid states of a Raft node.
	Candidate

	// Leader is one of the valid states of a Raft node.
	Leader

	// Shutdown is the terminal state of a Raft node.
	Shutdown
)

func (State) String added in v0.11.0 

func (state State) String() string

Source Files

View all Source files

Directories

Path Synopsis
examples
kvdb
ticket
timeseries

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.