gomemcached

package module
Version: v0.0.0-...-a592032 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 16, 2015 License: MIT Imports: 6 Imported by: 0

README

gomemcached

This is a memcached binary protocol toolkit in go.

It provides client and server functionality as well as a little sample server showing how I might make a server if I valued purity over performance.

Server Design

overview

The basic design can be seen in gocache. A storage server is run as a goroutine that receives a MCRequest on a channel, and then issues an MCResponse to a channel contained within the request.

Each connection is a separate goroutine, of course, and is responsible for all IO for that connection until the connection drops or the dataServer decides it's stupid and sends a fatal response back over the channel.

There is currently no work at all in making the thing perform (there are specific areas I know need work). This is just my attempt to learn the language somewhat.

Documentation

Overview

Package gomemcached is binary protocol packet formats and constants.

Index

Constants

View Source
const (
	REQ_MAGIC = 0x80
	RES_MAGIC = 0x81
)
View Source
const (
	GET        = CommandCode(0x00)
	SET        = CommandCode(0x01)
	ADD        = CommandCode(0x02)
	REPLACE    = CommandCode(0x03)
	DELETE     = CommandCode(0x04)
	INCREMENT  = CommandCode(0x05)
	DECREMENT  = CommandCode(0x06)
	QUIT       = CommandCode(0x07)
	FLUSH      = CommandCode(0x08)
	GETQ       = CommandCode(0x09)
	NOOP       = CommandCode(0x0a)
	VERSION    = CommandCode(0x0b)
	GETK       = CommandCode(0x0c)
	GETKQ      = CommandCode(0x0d)
	APPEND     = CommandCode(0x0e)
	PREPEND    = CommandCode(0x0f)
	STAT       = CommandCode(0x10)
	SETQ       = CommandCode(0x11)
	ADDQ       = CommandCode(0x12)
	REPLACEQ   = CommandCode(0x13)
	DELETEQ    = CommandCode(0x14)
	INCREMENTQ = CommandCode(0x15)
	DECREMENTQ = CommandCode(0x16)
	QUITQ      = CommandCode(0x17)
	FLUSHQ     = CommandCode(0x18)
	APPENDQ    = CommandCode(0x19)
	PREPENDQ   = CommandCode(0x1a)
	RGET       = CommandCode(0x30)
	RSET       = CommandCode(0x31)
	RSETQ      = CommandCode(0x32)
	RAPPEND    = CommandCode(0x33)
	RAPPENDQ   = CommandCode(0x34)
	RPREPEND   = CommandCode(0x35)
	RPREPENDQ  = CommandCode(0x36)
	RDELETE    = CommandCode(0x37)
	RDELETEQ   = CommandCode(0x38)
	RINCR      = CommandCode(0x39)
	RINCRQ     = CommandCode(0x3a)
	RDECR      = CommandCode(0x3b)
	RDECRQ     = CommandCode(0x3c)

	SASL_LIST_MECHS = CommandCode(0x20)
	SASL_AUTH       = CommandCode(0x21)
	SASL_STEP       = CommandCode(0x22)

	TAP_CONNECT          = CommandCode(0x40) // Client-sent request to initiate Tap feed
	TAP_MUTATION         = CommandCode(0x41) // Notification of a SET/ADD/REPLACE/etc. on the server
	TAP_DELETE           = CommandCode(0x42) // Notification of a DELETE on the server
	TAP_FLUSH            = CommandCode(0x43) // Replicates a flush_all command
	TAP_OPAQUE           = CommandCode(0x44) // Opaque control data from the engine
	TAP_VBUCKET_SET      = CommandCode(0x45) // Sets state of vbucket in receiver (used in takeover)
	TAP_CHECKPOINT_START = CommandCode(0x46) // Notifies start of new checkpoint
	TAP_CHECKPOINT_END   = CommandCode(0x47) // Notifies end of checkpoint

	UPR_OPEN        = CommandCode(0x50) // Open a UPR connection with a name
	UPR_ADDSTREAM   = CommandCode(0x51) // Sent by ebucketMigrator to UPR Consumer
	UPR_CLOSESTREAM = CommandCode(0x52) // Sent by eBucketMigrator to UPR Consumer
	UPR_FAILOVERLOG = CommandCode(0x54) // Request failover logs
	UPR_STREAMREQ   = CommandCode(0x53) // Stream request from consumer to producer
	UPR_STREAMEND   = CommandCode(0x55) // Sent by producer when it has no more messages to stream
	UPR_SNAPSHOT    = CommandCode(0x56) // Start of a new snapshot
	UPR_MUTATION    = CommandCode(0x57) // Key mutation
	UPR_DELETION    = CommandCode(0x58) // Key deletion
	UPR_EXPIRATION  = CommandCode(0x59) // Key expiration
	UPR_FLUSH       = CommandCode(0x5a) // Delete all the data for a vbucket
	UPR_NOOP        = CommandCode(0x5c) // UPR NOOP
	UPR_BUFFERACK   = CommandCode(0x5d) // UPR Buffer Acknowledgement
	UPR_CONTROL     = CommandCode(0x5e) // Set flow control params

	SELECT_BUCKET = CommandCode(0x89) // Select bucket

	OBSERVE = CommandCode(0x92)
)
View Source
const (
	SUCCESS         = Status(0x00)
	KEY_ENOENT      = Status(0x01)
	KEY_EEXISTS     = Status(0x02)
	E2BIG           = Status(0x03)
	EINVAL          = Status(0x04)
	NOT_STORED      = Status(0x05)
	DELTA_BADVAL    = Status(0x06)
	NOT_MY_VBUCKET  = Status(0x07)
	ERANGE          = Status(0x22)
	ROLLBACK        = Status(0x23)
	UNKNOWN_COMMAND = Status(0x81)
	ENOMEM          = Status(0x82)
	TMPFAIL         = Status(0x86)
)
View Source
const (
	BACKFILL           = TapConnectFlag(0x01)
	DUMP               = TapConnectFlag(0x02)
	LIST_VBUCKETS      = TapConnectFlag(0x04)
	TAKEOVER_VBUCKETS  = TapConnectFlag(0x08)
	SUPPORT_ACK        = TapConnectFlag(0x10)
	REQUEST_KEYS_ONLY  = TapConnectFlag(0x20)
	CHECKPOINT         = TapConnectFlag(0x40)
	REGISTERED_CLIENT  = TapConnectFlag(0x80)
	FIX_FLAG_BYTEORDER = TapConnectFlag(0x100)
)

Tap connect option flags

View Source
const (
	TAP_OPAQUE_ENABLE_AUTO_NACK       = 0
	TAP_OPAQUE_INITIAL_VBUCKET_STREAM = 1
	TAP_OPAQUE_ENABLE_CHECKPOINT_SYNC = 2
	TAP_OPAQUE_CLOSE_TAP_STREAM       = 7
	TAP_OPAQUE_CLOSE_BACKFILL         = 8
)

Tap opaque event subtypes

View Source
const (
	TAP_ACK                     = 1
	TAP_NO_VALUE                = 2
	TAP_FLAG_NETWORK_BYTE_ORDER = 4
)

Tap item flags

View Source
const HDR_LEN = 24

Number of bytes in a binary protocol header.

Variables

View Source
var CommandNames map[CommandCode]string

Mapping of CommandCode -> name of command (not exhaustive)

View Source
var MaxBodyLen = int(1e6)

The maximum reasonable body length to expect. Anything larger than this will result in an error.

View Source
var StatusNames map[Status]string

StatusNames human readable names for memcached response.

View Source
var TapConnectFlagNames = map[TapConnectFlag]string{
	BACKFILL:           "BACKFILL",
	DUMP:               "DUMP",
	LIST_VBUCKETS:      "LIST_VBUCKETS",
	TAKEOVER_VBUCKETS:  "TAKEOVER_VBUCKETS",
	SUPPORT_ACK:        "SUPPORT_ACK",
	REQUEST_KEYS_ONLY:  "REQUEST_KEYS_ONLY",
	CHECKPOINT:         "CHECKPOINT",
	REGISTERED_CLIENT:  "REGISTERED_CLIENT",
	FIX_FLAG_BYTEORDER: "FIX_FLAG_BYTEORDER",
}

TapConnectFlagNames for TapConnectFlag

TapFlagParsers parser functions for TAP fields.

Functions

func IsFatal

func IsFatal(e error) bool

IsFatal is false if this error isn't believed to be fatal to a connection.

func IsNotFound

func IsNotFound(e error) bool

IsNotFound is true if this error represents a "not found" response.

func TapParseBool

func TapParseBool(r io.Reader) (interface{}, error)

TapParseBool is a function to parse a single tap boolean.

func TapParseUint16

func TapParseUint16(r io.Reader) (interface{}, error)

TapParseUint16 is a function to parse a single tap uint16.

func TapParseUint64

func TapParseUint64(r io.Reader) (interface{}, error)

TapParseUint64 is a function to parse a single tap uint64.

func TapParseVBList

func TapParseVBList(r io.Reader) (interface{}, error)

TapParseVBList parses a list of vBucket numbers as []uint16.

Types

type CommandCode

type CommandCode uint8

CommandCode for memcached packets.

func (CommandCode) IsQuiet

func (o CommandCode) IsQuiet() bool

IsQuiet will return true if a command is a "quiet" command.

func (CommandCode) String

func (o CommandCode) String() (rv string)

String an op code.

type MCItem

type MCItem struct {
	Cas               uint64
	Flags, Expiration uint32
	Data              []byte
}

MCItem is an internal representation of an item.

type MCRequest

type MCRequest struct {
	// The command being issued
	Opcode CommandCode
	// The CAS (if applicable, or 0)
	Cas uint64
	// An opaque value to be returned with this request
	Opaque uint32
	// The vbucket to which this command belongs
	VBucket uint16
	// Command extras, key, and body
	Extras, Key, Body []byte
}

MCRequest is memcached Request

func (*MCRequest) Bytes

func (req *MCRequest) Bytes() []byte

Bytes will return the wire representation of this request.

func (*MCRequest) HeaderBytes

func (req *MCRequest) HeaderBytes() []byte

HeaderBytes will return the wire representation of the request header (with the extras and key).

func (*MCRequest) ParseTapCommands

func (req *MCRequest) ParseTapCommands() (TapConnect, error)

ParseTapCommands parse the tap request into the interesting bits we may need to do something with.

func (*MCRequest) Receive

func (req *MCRequest) Receive(r io.Reader, hdrBytes []byte) (int, error)

Receive will fill this MCRequest with the data from a reader.

func (*MCRequest) Size

func (req *MCRequest) Size() int

Size gives the number of bytes this request requires.

func (MCRequest) String

func (req MCRequest) String() string

A debugging string representation of this request

func (*MCRequest) Transmit

func (req *MCRequest) Transmit(w io.Writer) (n int, err error)

Transmit will send this request message across a writer.

type MCResponse

type MCResponse struct {
	// The command opcode of the command that sent the request
	Opcode CommandCode
	// The status of the response
	Status Status
	// The opaque sent in the request
	Opaque uint32
	// The CAS identifier (if applicable)
	Cas uint64
	// Extras, key, and body for this response
	Extras, Key, Body []byte
	// If true, this represents a fatal condition and we should hang up
	Fatal bool
}

MCResponse is memcached response

func (*MCResponse) Bytes

func (res *MCResponse) Bytes() []byte

Bytes will return the actual bytes transmitted for this response.

func (*MCResponse) Error

func (res *MCResponse) Error() string

Response as an error.

func (*MCResponse) HeaderBytes

func (res *MCResponse) HeaderBytes() []byte

HeaderBytes will get just the header bytes for this response.

func (*MCResponse) Receive

func (res *MCResponse) Receive(r io.Reader, hdrBytes []byte) (int, error)

Receive will fill this MCResponse with the data from this reader.

func (*MCResponse) Size

func (res *MCResponse) Size() int

Size is number of bytes this response consumes on the wire.

func (MCResponse) String

func (res MCResponse) String() string

A debugging string representation of this response

func (*MCResponse) Transmit

func (res *MCResponse) Transmit(w io.Writer) (n int, err error)

Transmit will send this response message across a writer.

type Status

type Status uint16

Status field for memcached response.

func (Status) String

func (s Status) String() (rv string)

String an op code.

type TapConnect

type TapConnect struct {
	Flags         map[TapConnectFlag]interface{}
	RemainingBody []byte
	Name          string
}

type TapConnectFlag

type TapConnectFlag uint32

func (TapConnectFlag) SplitFlags

func (f TapConnectFlag) SplitFlags() []TapConnectFlag

SplitFlags will split the ORed flags into the individual bit flags.

func (TapConnectFlag) String

func (f TapConnectFlag) String() string

type TapItemParser

type TapItemParser func(io.Reader) (interface{}, error)

TapItemParser is a function to parse a single tap extra.

Directories

Path Synopsis
Package memcached provides a memcached binary protocol client.
Package memcached provides a memcached binary protocol client.
Package mcdebug provides memcached client op statistics via expvar.
Package mcdebug provides memcached client op statistics via expvar.
Package memcached provides useful functions for building your own memcached server.
Package memcached provides useful functions for building your own memcached server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL