gomemcached

package
v0.0.0-...-bea4ea4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 2, 2016 License: MIT, AGPL-3.0-or-later Imports: 6 Imported by: 0

README

gomemcached

This is a memcached binary protocol toolkit in go.

It provides client and server functionality as well as a little sample server showing how I might make a server if I valued purity over performance.

Server Design

overview

The basic design can be seen in gocache. A storage server is run as a goroutine that receives a MCRequest on a channel, and then issues an MCResponse to a channel contained within the request.

Each connection is a separate goroutine, of course, and is responsible for all IO for that connection until the connection drops or the dataServer decides it's stupid and sends a fatal response back over the channel.

There is currently no work at all in making the thing perform (there are specific areas I know need work). This is just my attempt to learn the language somewhat.

Documentation

Overview

Memcached binary protocol packet formats and constants.

Index

Constants

View Source 
const (
	REQ_MAGIC = 0x80
	RES_MAGIC = 0x81
)
View Source 
const (
	GET        = CommandCode(0x00)
	SET        = CommandCode(0x01)
	ADD        = CommandCode(0x02)
	REPLACE    = CommandCode(0x03)
	DELETE     = CommandCode(0x04)
	INCREMENT  = CommandCode(0x05)
	DECREMENT  = CommandCode(0x06)
	QUIT       = CommandCode(0x07)
	FLUSH      = CommandCode(0x08)
	GETQ       = CommandCode(0x09)
	NOOP       = CommandCode(0x0a)
	VERSION    = CommandCode(0x0b)
	GETK       = CommandCode(0x0c)
	GETKQ      = CommandCode(0x0d)
	APPEND     = CommandCode(0x0e)
	PREPEND    = CommandCode(0x0f)
	STAT       = CommandCode(0x10)
	SETQ       = CommandCode(0x11)
	ADDQ       = CommandCode(0x12)
	REPLACEQ   = CommandCode(0x13)
	DELETEQ    = CommandCode(0x14)
	INCREMENTQ = CommandCode(0x15)
	DECREMENTQ = CommandCode(0x16)
	QUITQ      = CommandCode(0x17)
	FLUSHQ     = CommandCode(0x18)
	APPENDQ    = CommandCode(0x19)
	PREPENDQ   = CommandCode(0x1a)
	RGET       = CommandCode(0x30)
	RSET       = CommandCode(0x31)
	RSETQ      = CommandCode(0x32)
	RAPPEND    = CommandCode(0x33)
	RAPPENDQ   = CommandCode(0x34)
	RPREPEND   = CommandCode(0x35)
	RPREPENDQ  = CommandCode(0x36)
	RDELETE    = CommandCode(0x37)
	RDELETEQ   = CommandCode(0x38)
	RINCR      = CommandCode(0x39)
	RINCRQ     = CommandCode(0x3a)
	RDECR      = CommandCode(0x3b)
	RDECRQ     = CommandCode(0x3c)

	SASL_LIST_MECHS = CommandCode(0x20)
	SASL_AUTH       = CommandCode(0x21)
	SASL_STEP       = CommandCode(0x22)

	TAP_CONNECT          = CommandCode(0x40) // Client-sent request to initiate Tap feed
	TAP_MUTATION         = CommandCode(0x41) // Notification of a SET/ADD/REPLACE/etc. on the server
	TAP_DELETE           = CommandCode(0x42) // Notification of a DELETE on the server
	TAP_FLUSH            = CommandCode(0x43) // Replicates a flush_all command
	TAP_OPAQUE           = CommandCode(0x44) // Opaque control data from the engine
	TAP_VBUCKET_SET      = CommandCode(0x45) // Sets state of vbucket in receiver (used in takeover)
	TAP_CHECKPOINT_START = CommandCode(0x46) // Notifies start of new checkpoint
	TAP_CHECKPOINT_END   = CommandCode(0x47) // Notifies end of checkpoint

	OBSERVE = CommandCode(0x92)
)
View Source 
const (
	SUCCESS         = Status(0x00)
	KEY_ENOENT      = Status(0x01)
	KEY_EEXISTS     = Status(0x02)
	E2BIG           = Status(0x03)
	EINVAL          = Status(0x04)
	NOT_STORED      = Status(0x05)
	DELTA_BADVAL    = Status(0x06)
	NOT_MY_VBUCKET  = Status(0x07)
	UNKNOWN_COMMAND = Status(0x81)
	ENOMEM          = Status(0x82)
	TMPFAIL         = Status(0x86)
)
View Source 
const (
	BACKFILL           = TapConnectFlag(0x01)
	DUMP               = TapConnectFlag(0x02)
	LIST_VBUCKETS      = TapConnectFlag(0x04)
	TAKEOVER_VBUCKETS  = TapConnectFlag(0x08)
	SUPPORT_ACK        = TapConnectFlag(0x10)
	REQUEST_KEYS_ONLY  = TapConnectFlag(0x20)
	CHECKPOINT         = TapConnectFlag(0x40)
	REGISTERED_CLIENT  = TapConnectFlag(0x80)
	FIX_FLAG_BYTEORDER = TapConnectFlag(0x100)
)

Tap connect option flags

View Source 
const (
	TAP_OPAQUE_ENABLE_AUTO_NACK       = 0
	TAP_OPAQUE_INITIAL_VBUCKET_STREAM = 1
	TAP_OPAQUE_ENABLE_CHECKPOINT_SYNC = 2
	TAP_OPAQUE_CLOSE_TAP_STREAM       = 7
	TAP_OPAQUE_CLOSE_BACKFILL         = 8
)

Tap opaque event subtypes

View Source 
const (
	TAP_ACK                     = 1
	TAP_NO_VALUE                = 2
	TAP_FLAG_NETWORK_BYTE_ORDER = 4
)

Tap item flags

View Source 
const HDR_LEN = 24

Number of bytes in a binary protocol header.

Variables

View Source 
var CommandNames map[CommandCode]string

Mapping of CommandCode -> name of command (not exhaustive)

View Source 
var MaxBodyLen = int(1e6)

The maximum reasonable body length to expect. Anything larger than this will result in an error.

View Source 
var StatusNames map[Status]string
View Source 
var TapConnectFlagNames = map[TapConnectFlag]string{
	BACKFILL:           "BACKFILL",
	DUMP:               "DUMP",
	LIST_VBUCKETS:      "LIST_VBUCKETS",
	TAKEOVER_VBUCKETS:  "TAKEOVER_VBUCKETS",
	SUPPORT_ACK:        "SUPPORT_ACK",
	REQUEST_KEYS_ONLY:  "REQUEST_KEYS_ONLY",
	CHECKPOINT:         "CHECKPOINT",
	REGISTERED_CLIENT:  "REGISTERED_CLIENT",
	FIX_FLAG_BYTEORDER: "FIX_FLAG_BYTEORDER",
}
View Source 
var TapFlagParsers = map[TapConnectFlag]TapItemParser{
	BACKFILL:      TapParseUint64,
	LIST_VBUCKETS: TapParseVBList,
}

Functions

func IsFatal 

func IsFatal(e error) bool

False if this error isn't believed to be fatal to a connection.

func IsNotFound 

func IsNotFound(e error) bool

True if this error represents a "not found" response.

func TapParseBool 

func TapParseBool(r io.Reader) (interface{}, error)

func TapParseUint16 

func TapParseUint16(r io.Reader) (interface{}, error)

func TapParseUint64 

func TapParseUint64(r io.Reader) (interface{}, error)

func TapParseVBList 

func TapParseVBList(r io.Reader) (interface{}, error)

Types

type CommandCode 

type CommandCode uint8

func (CommandCode) IsQuiet 

func (o CommandCode) IsQuiet() bool

Return true if a command is a "quiet" command.

func (CommandCode) String 

func (o CommandCode) String() (rv string)

String an op code.

type MCItem 

type MCItem struct {
	Cas               uint64
	Flags, Expiration uint32
	Data              []byte
}

An internal representation of an item.

type MCRequest 

type MCRequest struct {
	// The command being issued
	Opcode CommandCode
	// The CAS (if applicable, or 0)
	Cas uint64
	// An opaque value to be returned with this request
	Opaque uint32
	// The vbucket to which this command belongs
	VBucket uint16
	// Command extras, key, and body
	Extras, Key, Body []byte
}

A Memcached Request

func (*MCRequest) Bytes 

func (req *MCRequest) Bytes() []byte

The wire representation of this request.

func (*MCRequest) HeaderBytes 

func (req *MCRequest) HeaderBytes() []byte

The wire representation of the header (with the extras and key)

func (*MCRequest) ParseTapCommands 

func (req *MCRequest) ParseTapCommands() (TapConnect, error)

Parse the tap request into the interesting bits we may need to do something with.

func (*MCRequest) Receive 

func (req *MCRequest) Receive(r io.Reader, hdrBytes []byte) (int, error)

Fill this MCRequest with the data from this reader.

func (*MCRequest) Size 

func (req *MCRequest) Size() int

The number of bytes this request requires.

func (MCRequest) String 

func (req MCRequest) String() string

A debugging string representation of this request

func (*MCRequest) Transmit 

func (req *MCRequest) Transmit(w io.Writer) (n int, err error)

Send this request message across a writer.

type MCResponse 

type MCResponse struct {
	// The command opcode of the command that sent the request
	Opcode CommandCode
	// The status of the response
	Status Status
	// The opaque sent in the request
	Opaque uint32
	// The CAS identifier (if applicable)
	Cas uint64
	// Extras, key, and body for this response
	Extras, Key, Body []byte
	// If true, this represents a fatal condition and we should hang up
	Fatal bool
}

A memcached response

func (*MCResponse) Bytes 

func (res *MCResponse) Bytes() []byte

The actual bytes transmitted for this response.

func (*MCResponse) Error 

func (res *MCResponse) Error() string

Response as an error.

func (*MCResponse) HeaderBytes 

func (res *MCResponse) HeaderBytes() []byte

Get just the header bytes for this response.

func (*MCResponse) Receive 

func (req *MCResponse) Receive(r io.Reader, hdrBytes []byte) (int, error)

Fill this MCResponse with the data from this reader.

func (*MCResponse) Size 

func (res *MCResponse) Size() int

Number of bytes this response consumes on the wire.

func (MCResponse) String 

func (res MCResponse) String() string

A debugging string representation of this response

func (*MCResponse) Transmit 

func (res *MCResponse) Transmit(w io.Writer) (n int, err error)

Send this response message across a writer.

type Status 

type Status uint16

func (Status) String 

func (s Status) String() (rv string)

String an op code.

type TapConnect 

type TapConnect struct {
	Flags         map[TapConnectFlag]interface{}
	RemainingBody []byte
	Name          string
}

type TapConnectFlag 

type TapConnectFlag uint32

func (TapConnectFlag) SplitFlags 

func (f TapConnectFlag) SplitFlags() []TapConnectFlag

Split the ORed flags into the individual bit flags.

func (TapConnectFlag) String 

func (f TapConnectFlag) String() string

type TapItemParser 

type TapItemParser func(io.Reader) (interface{}, error)

A function to parse a single tap extra

Source Files

View all Source files

Directories

Path Synopsis
Package memcached provides a memcached binary protocol client.
 Package memcached provides a memcached binary protocol client.
example
Package mcdebug provides memcached client op statistics via expvar.
 Package mcdebug provides memcached client op statistics via expvar.
Useful functions for building your own memcached server.
 Useful functions for building your own memcached server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.