gomemcached

package module
v0.0.0-...-a592032 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 16, 2015 License: MIT Imports: 6 Imported by: 0

README

gomemcached

This is a memcached binary protocol toolkit in go.

It provides client and server functionality as well as a little sample server showing how I might make a server if I valued purity over performance.

Server Design

overview

The basic design can be seen in gocache. A storage server is run as a goroutine that receives a MCRequest on a channel, and then issues an MCResponse to a channel contained within the request.

Each connection is a separate goroutine, of course, and is responsible for all IO for that connection until the connection drops or the dataServer decides it's stupid and sends a fatal response back over the channel.

There is currently no work at all in making the thing perform (there are specific areas I know need work). This is just my attempt to learn the language somewhat.

Documentation

Overview

Package gomemcached is binary protocol packet formats and constants.

Index

Constants

View Source 
const (
	REQ_MAGIC = 0x80
	RES_MAGIC = 0x81
)
View Source 
const (
	GET        = CommandCode(0x00)
	SET        = CommandCode(0x01)
	ADD        = CommandCode(0x02)
	REPLACE    = CommandCode(0x03)
	DELETE     = CommandCode(0x04)
	INCREMENT  = CommandCode(0x05)
	DECREMENT  = CommandCode(0x06)
	QUIT       = CommandCode(0x07)
	FLUSH      = CommandCode(0x08)
	GETQ       = CommandCode(0x09)
	NOOP       = CommandCode(0x0a)
	VERSION    = CommandCode(0x0b)
	GETK       = CommandCode(0x0c)
	GETKQ      = CommandCode(0x0d)
	APPEND     = CommandCode(0x0e)
	PREPEND    = CommandCode(0x0f)
	STAT       = CommandCode(0x10)
	SETQ       = CommandCode(0x11)
	ADDQ       = CommandCode(0x12)
	REPLACEQ   = CommandCode(0x13)
	DELETEQ    = CommandCode(0x14)
	INCREMENTQ = CommandCode(0x15)
	DECREMENTQ = CommandCode(0x16)
	QUITQ      = CommandCode(0x17)
	FLUSHQ     = CommandCode(0x18)
	APPENDQ    = CommandCode(0x19)
	PREPENDQ   = CommandCode(0x1a)
	RGET       = CommandCode(0x30)
	RSET       = CommandCode(0x31)
	RSETQ      = CommandCode(0x32)
	RAPPEND    = CommandCode(0x33)
	RAPPENDQ   = CommandCode(0x34)
	RPREPEND   = CommandCode(0x35)
	RPREPENDQ  = CommandCode(0x36)
	RDELETE    = CommandCode(0x37)
	RDELETEQ   = CommandCode(0x38)
	RINCR      = CommandCode(0x39)
	RINCRQ     = CommandCode(0x3a)
	RDECR      = CommandCode(0x3b)
	RDECRQ     = CommandCode(0x3c)

	SASL_LIST_MECHS = CommandCode(0x20)
	SASL_AUTH       = CommandCode(0x21)
	SASL_STEP       = CommandCode(0x22)

	TAP_CONNECT          = CommandCode(0x40) // Client-sent request to initiate Tap feed
	TAP_MUTATION         = CommandCode(0x41) // Notification of a SET/ADD/REPLACE/etc. on the server
	TAP_DELETE           = CommandCode(0x42) // Notification of a DELETE on the server
	TAP_FLUSH            = CommandCode(0x43) // Replicates a flush_all command
	TAP_OPAQUE           = CommandCode(0x44) // Opaque control data from the engine
	TAP_VBUCKET_SET      = CommandCode(0x45) // Sets state of vbucket in receiver (used in takeover)
	TAP_CHECKPOINT_START = CommandCode(0x46) // Notifies start of new checkpoint
	TAP_CHECKPOINT_END   = CommandCode(0x47) // Notifies end of checkpoint

	UPR_OPEN        = CommandCode(0x50) // Open a UPR connection with a name
	UPR_ADDSTREAM   = CommandCode(0x51) // Sent by ebucketMigrator to UPR Consumer
	UPR_CLOSESTREAM = CommandCode(0x52) // Sent by eBucketMigrator to UPR Consumer
	UPR_FAILOVERLOG = CommandCode(0x54) // Request failover logs
	UPR_STREAMREQ   = CommandCode(0x53) // Stream request from consumer to producer
	UPR_STREAMEND   = CommandCode(0x55) // Sent by producer when it has no more messages to stream
	UPR_SNAPSHOT    = CommandCode(0x56) // Start of a new snapshot
	UPR_MUTATION    = CommandCode(0x57) // Key mutation
	UPR_DELETION    = CommandCode(0x58) // Key deletion
	UPR_EXPIRATION  = CommandCode(0x59) // Key expiration
	UPR_FLUSH       = CommandCode(0x5a) // Delete all the data for a vbucket
	UPR_NOOP        = CommandCode(0x5c) // UPR NOOP
	UPR_BUFFERACK   = CommandCode(0x5d) // UPR Buffer Acknowledgement
	UPR_CONTROL     = CommandCode(0x5e) // Set flow control params

	SELECT_BUCKET = CommandCode(0x89) // Select bucket

	OBSERVE = CommandCode(0x92)
)
View Source 
const (
	SUCCESS         = Status(0x00)
	KEY_ENOENT      = Status(0x01)
	KEY_EEXISTS     = Status(0x02)
	E2BIG           = Status(0x03)
	EINVAL          = Status(0x04)
	NOT_STORED      = Status(0x05)
	DELTA_BADVAL    = Status(0x06)
	NOT_MY_VBUCKET  = Status(0x07)
	ERANGE          = Status(0x22)
	ROLLBACK        = Status(0x23)
	UNKNOWN_COMMAND = Status(0x81)
	ENOMEM          = Status(0x82)
	TMPFAIL         = Status(0x86)
)
View Source 
const (
	BACKFILL           = TapConnectFlag(0x01)
	DUMP               = TapConnectFlag(0x02)
	LIST_VBUCKETS      = TapConnectFlag(0x04)
	TAKEOVER_VBUCKETS  = TapConnectFlag(0x08)
	SUPPORT_ACK        = TapConnectFlag(0x10)
	REQUEST_KEYS_ONLY  = TapConnectFlag(0x20)
	CHECKPOINT         = TapConnectFlag(0x40)
	REGISTERED_CLIENT  = TapConnectFlag(0x80)
	FIX_FLAG_BYTEORDER = TapConnectFlag(0x100)
)

Tap connect option flags

View Source 
const (
	TAP_OPAQUE_ENABLE_AUTO_NACK       = 0
	TAP_OPAQUE_INITIAL_VBUCKET_STREAM = 1
	TAP_OPAQUE_ENABLE_CHECKPOINT_SYNC = 2
	TAP_OPAQUE_CLOSE_TAP_STREAM       = 7
	TAP_OPAQUE_CLOSE_BACKFILL         = 8
)

Tap opaque event subtypes

View Source 
const (
	TAP_ACK                     = 1
	TAP_NO_VALUE                = 2
	TAP_FLAG_NETWORK_BYTE_ORDER = 4
)

Tap item flags

View Source 
const HDR_LEN = 24

Number of bytes in a binary protocol header.

Variables

View Source 
var CommandNames map[CommandCode]string

Mapping of CommandCode -> name of command (not exhaustive)

View Source 
var MaxBodyLen = int(1e6)

The maximum reasonable body length to expect. Anything larger than this will result in an error.

View Source 
var StatusNames map[Status]string

StatusNames human readable names for memcached response.

View Source 
var TapConnectFlagNames = map[TapConnectFlag]string{
	BACKFILL:           "BACKFILL",
	DUMP:               "DUMP",
	LIST_VBUCKETS:      "LIST_VBUCKETS",
	TAKEOVER_VBUCKETS:  "TAKEOVER_VBUCKETS",
	SUPPORT_ACK:        "SUPPORT_ACK",
	REQUEST_KEYS_ONLY:  "REQUEST_KEYS_ONLY",
	CHECKPOINT:         "CHECKPOINT",
	REGISTERED_CLIENT:  "REGISTERED_CLIENT",
	FIX_FLAG_BYTEORDER: "FIX_FLAG_BYTEORDER",
}

TapConnectFlagNames for TapConnectFlag

View Source 
var TapFlagParsers = map[TapConnectFlag]TapItemParser{
	BACKFILL:      TapParseUint64,
	LIST_VBUCKETS: TapParseVBList,
}

TapFlagParsers parser functions for TAP fields.

Functions

func IsFatal 

func IsFatal(e error) bool

IsFatal is false if this error isn't believed to be fatal to a connection.

func IsNotFound 

func IsNotFound(e error) bool

IsNotFound is true if this error represents a "not found" response.

func TapParseBool 

func TapParseBool(r io.Reader) (interface{}, error)

TapParseBool is a function to parse a single tap boolean.

func TapParseUint16 

func TapParseUint16(r io.Reader) (interface{}, error)

TapParseUint16 is a function to parse a single tap uint16.

func TapParseUint64 

func TapParseUint64(r io.Reader) (interface{}, error)

TapParseUint64 is a function to parse a single tap uint64.

func TapParseVBList 

func TapParseVBList(r io.Reader) (interface{}, error)

TapParseVBList parses a list of vBucket numbers as []uint16.

Types

type CommandCode 

type CommandCode uint8

CommandCode for memcached packets.

func (CommandCode) IsQuiet 

func (o CommandCode) IsQuiet() bool

IsQuiet will return true if a command is a "quiet" command.

func (CommandCode) String 

func (o CommandCode) String() (rv string)

String an op code.

type MCItem 

type MCItem struct {
	Cas               uint64
	Flags, Expiration uint32
	Data              []byte
}

MCItem is an internal representation of an item.

type MCRequest 

type MCRequest struct {
	// The command being issued
	Opcode CommandCode
	// The CAS (if applicable, or 0)
	Cas uint64
	// An opaque value to be returned with this request
	Opaque uint32
	// The vbucket to which this command belongs
	VBucket uint16
	// Command extras, key, and body
	Extras, Key, Body []byte
}

MCRequest is memcached Request

func (*MCRequest) Bytes 

func (req *MCRequest) Bytes() []byte

Bytes will return the wire representation of this request.

func (*MCRequest) HeaderBytes 

func (req *MCRequest) HeaderBytes() []byte

HeaderBytes will return the wire representation of the request header (with the extras and key).

func (*MCRequest) ParseTapCommands 

func (req *MCRequest) ParseTapCommands() (TapConnect, error)

ParseTapCommands parse the tap request into the interesting bits we may need to do something with.

func (*MCRequest) Receive 

func (req *MCRequest) Receive(r io.Reader, hdrBytes []byte) (int, error)

Receive will fill this MCRequest with the data from a reader.

func (*MCRequest) Size 

func (req *MCRequest) Size() int

Size gives the number of bytes this request requires.

func (MCRequest) String 

func (req MCRequest) String() string

A debugging string representation of this request

func (*MCRequest) Transmit 

func (req *MCRequest) Transmit(w io.Writer) (n int, err error)

Transmit will send this request message across a writer.

type MCResponse 

type MCResponse struct {
	// The command opcode of the command that sent the request
	Opcode CommandCode
	// The status of the response
	Status Status
	// The opaque sent in the request
	Opaque uint32
	// The CAS identifier (if applicable)
	Cas uint64
	// Extras, key, and body for this response
	Extras, Key, Body []byte
	// If true, this represents a fatal condition and we should hang up
	Fatal bool
}

MCResponse is memcached response

func (*MCResponse) Bytes 

func (res *MCResponse) Bytes() []byte

Bytes will return the actual bytes transmitted for this response.

func (*MCResponse) Error 

func (res *MCResponse) Error() string

Response as an error.

func (*MCResponse) HeaderBytes 

func (res *MCResponse) HeaderBytes() []byte

HeaderBytes will get just the header bytes for this response.

func (*MCResponse) Receive 

func (res *MCResponse) Receive(r io.Reader, hdrBytes []byte) (int, error)

Receive will fill this MCResponse with the data from this reader.

func (*MCResponse) Size 

func (res *MCResponse) Size() int

Size is number of bytes this response consumes on the wire.

func (MCResponse) String 

func (res MCResponse) String() string

A debugging string representation of this response

func (*MCResponse) Transmit 

func (res *MCResponse) Transmit(w io.Writer) (n int, err error)

Transmit will send this response message across a writer.

type Status 

type Status uint16

Status field for memcached response.

func (Status) String 

func (s Status) String() (rv string)

String an op code.

type TapConnect 

type TapConnect struct {
	Flags         map[TapConnectFlag]interface{}
	RemainingBody []byte
	Name          string
}

type TapConnectFlag 

type TapConnectFlag uint32

func (TapConnectFlag) SplitFlags 

func (f TapConnectFlag) SplitFlags() []TapConnectFlag

SplitFlags will split the ORed flags into the individual bit flags.

func (TapConnectFlag) String 

func (f TapConnectFlag) String() string

type TapItemParser 

type TapItemParser func(io.Reader) (interface{}, error)

TapItemParser is a function to parse a single tap extra.

Source Files

View all Source files

Directories

Path Synopsis
Package memcached provides a memcached binary protocol client.
 Package memcached provides a memcached binary protocol client.
example
upr_example
Package mcdebug provides memcached client op statistics via expvar.
 Package mcdebug provides memcached client op statistics via expvar.
Package memcached provides useful functions for building your own memcached server.
 Package memcached provides useful functions for building your own memcached server.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.