README

groupcache

Summary

groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many cases.

For API docs and examples, see http://godoc.org/github.com/golang/groupcache

Comparison to memcached

Like memcached, groupcache:
  • shards by key to select which peer is responsible for that key
Unlike memcached, groupcache:
  • does not require running a separate set of servers, thus massively reducing deployment/configuration pain. groupcache is a client library as well as a server. It connects to its own peers, forming a distributed cache.

  • comes with a cache filling mechanism. Whereas memcached just says "Sorry, cache miss", often resulting in a thundering herd of database (or whatever) loads from an unbounded number of clients (which has resulted in several fun outages), groupcache coordinates cache fills such that only one load in one process of an entire replicated set of processes populates the cache, then multiplexes the loaded value to all callers.

  • does not support versioned values. If key "foo" is value "bar", key "foo" must always be "bar". There are neither cache expiration times, nor explicit cache evictions. Thus there is also no CAS, nor Increment/Decrement. This also means that groupcache....

  • ... supports automatic mirroring of super-hot items to multiple processes. This prevents memcached hot spotting where a machine's CPU and/or NIC are overloaded by very popular keys/values.

  • is currently only available for Go. It's very unlikely that I (bradfitz@) will port the code to any other language.

Loading process

In a nutshell, a groupcache lookup of Get("foo") looks like:

(On machine #5 of a set of N machines running the same code)

  1. Is the value of "foo" in local memory because it's super hot? If so, use it.

  2. Is the value of "foo" in local memory because peer #5 (the current peer) is the owner of it? If so, use it.

  3. Amongst all the peers in my set of N, am I the owner of the key "foo"? (e.g. does it consistent hash to 5?) If so, load it. If other callers come in, via the same process or via RPC requests from peers, they block waiting for the load to finish and get the same answer. If not, RPC to the peer that's the owner and get the answer. If the RPC fails, just load it locally (still with local dup suppression).

Users

groupcache is in production use by dl.google.com (its original user), parts of Blogger, parts of Google Code, parts of Google Fiber, parts of Google production monitoring systems, etc.

Presentations

See http://talks.golang.org/2013/oscon-dl.slide

Help

Use the golang-nuts mailing list for any discussion or questions.

Expand ▾ Collapse ▴

Documentation

Overview

    Package groupcache provides a data loading mechanism with caching and de-duplication that works across a set of peer processes.

    Each data Get first consults its local cache, otherwise delegates to the requested key's canonical owner, which then checks its cache or finally gets the data. In the common case, many concurrent cache misses across a set of peers for the same key result in just one cache fill.

    Index

    Constants

    This section is empty.

    Variables

    This section is empty.

    Functions

    func RegisterNewGroupHook

    func RegisterNewGroupHook(fn func(*Group))

      RegisterNewGroupHook registers a hook that is run each time a group is created.

      func RegisterPeerPicker

      func RegisterPeerPicker(fn func() PeerPicker)

        RegisterPeerPicker registers the peer initialization function. It is called once, when the first group is created. Either RegisterPeerPicker or RegisterPerGroupPeerPicker should be called exactly once, but not both.

        func RegisterPerGroupPeerPicker

        func RegisterPerGroupPeerPicker(fn func(groupName string) PeerPicker)

          RegisterPerGroupPeerPicker registers the peer initialization function, which takes the groupName, to be used in choosing a PeerPicker. It is called once, when the first group is created. Either RegisterPeerPicker or RegisterPerGroupPeerPicker should be called exactly once, but not both.

          func RegisterServerStart

          func RegisterServerStart(fn func())

            RegisterServerStart registers a hook that is run when the first group is created.

            Types

            type AtomicInt

            type AtomicInt int64

              An AtomicInt is an int64 to be accessed atomically.

              func (*AtomicInt) Add

              func (i *AtomicInt) Add(n int64)

                Add atomically adds n to i.

                func (*AtomicInt) Get

                func (i *AtomicInt) Get() int64

                  Get atomically gets the value of i.

                  func (*AtomicInt) String

                  func (i *AtomicInt) String() string

                  type ByteView

                  type ByteView struct {
                  	// contains filtered or unexported fields
                  }

                    A ByteView holds an immutable view of bytes. Internally it wraps either a []byte or a string, but that detail is invisible to callers.

                    A ByteView is meant to be used as a value type, not a pointer (like a time.Time).

                    func (ByteView) At

                    func (v ByteView) At(i int) byte

                      At returns the byte at index i.

                      func (ByteView) ByteSlice

                      func (v ByteView) ByteSlice() []byte

                        ByteSlice returns a copy of the data as a byte slice.

                        func (ByteView) Copy

                        func (v ByteView) Copy(dest []byte) int

                          Copy copies b into dest and returns the number of bytes copied.

                          func (ByteView) Equal

                          func (v ByteView) Equal(b2 ByteView) bool

                            Equal returns whether the bytes in b are the same as the bytes in b2.

                            func (ByteView) EqualBytes

                            func (v ByteView) EqualBytes(b2 []byte) bool

                              EqualBytes returns whether the bytes in b are the same as the bytes in b2.

                              func (ByteView) EqualString

                              func (v ByteView) EqualString(s string) bool

                                EqualString returns whether the bytes in b are the same as the bytes in s.

                                func (ByteView) Len

                                func (v ByteView) Len() int

                                  Len returns the view's length.

                                  func (ByteView) ReadAt

                                  func (v ByteView) ReadAt(p []byte, off int64) (n int, err error)

                                    ReadAt implements io.ReaderAt on the bytes in v.

                                    func (ByteView) Reader

                                    func (v ByteView) Reader() io.ReadSeeker

                                      Reader returns an io.ReadSeeker for the bytes in v.

                                      func (ByteView) Slice

                                      func (v ByteView) Slice(from, to int) ByteView

                                        Slice slices the view between the provided from and to indices.

                                        func (ByteView) SliceFrom

                                        func (v ByteView) SliceFrom(from int) ByteView

                                          SliceFrom slices the view from the provided index until the end.

                                          func (ByteView) String

                                          func (v ByteView) String() string

                                            String returns the data as a string, making a copy if necessary.

                                            func (ByteView) WriteTo

                                            func (v ByteView) WriteTo(w io.Writer) (n int64, err error)

                                              WriteTo implements io.WriterTo on the bytes in v.

                                              type CacheStats

                                              type CacheStats struct {
                                              	Bytes     int64
                                              	Items     int64
                                              	Gets      int64
                                              	Hits      int64
                                              	Evictions int64
                                              }

                                                CacheStats are returned by stats accessors on Group.

                                                type CacheType

                                                type CacheType int

                                                  CacheType represents a type of cache.

                                                  const (
                                                  	// The MainCache is the cache for items that this peer is the
                                                  	// owner for.
                                                  	MainCache CacheType = iota + 1
                                                  
                                                  	// The HotCache is the cache for items that seem popular
                                                  	// enough to replicate to this node, even though it's not the
                                                  	// owner.
                                                  	HotCache
                                                  )

                                                  type Context

                                                  type Context = context.Context

                                                    Context is an alias to context.Context for backwards compatibility purposes.

                                                    type Getter

                                                    type Getter interface {
                                                    	// Get returns the value identified by key, populating dest.
                                                    	//
                                                    	// The returned data must be unversioned. That is, key must
                                                    	// uniquely describe the loaded data, without an implicit
                                                    	// current time, and without relying on cache expiration
                                                    	// mechanisms.
                                                    	Get(ctx context.Context, key string, dest Sink) error
                                                    }

                                                      A Getter loads data for a key.

                                                      type GetterFunc

                                                      type GetterFunc func(ctx context.Context, key string, dest Sink) error

                                                        A GetterFunc implements Getter with a function.

                                                        func (GetterFunc) Get

                                                        func (f GetterFunc) Get(ctx context.Context, key string, dest Sink) error

                                                        type Group

                                                        type Group struct {
                                                        
                                                        	// Stats are statistics on the group.
                                                        	Stats Stats
                                                        	// contains filtered or unexported fields
                                                        }

                                                          A Group is a cache namespace and associated data loaded spread over a group of 1 or more machines.

                                                          func GetGroup

                                                          func GetGroup(name string) *Group

                                                            GetGroup returns the named group previously created with NewGroup, or nil if there's no such group.

                                                            func NewGroup

                                                            func NewGroup(name string, cacheBytes int64, getter Getter) *Group

                                                              NewGroup creates a coordinated group-aware Getter from a Getter.

                                                              The returned Getter tries (but does not guarantee) to run only one Get call at once for a given key across an entire set of peer processes. Concurrent callers both in the local process and in other processes receive copies of the answer once the original Get completes.

                                                              The group name must be unique for each getter.

                                                              func (*Group) CacheStats

                                                              func (g *Group) CacheStats(which CacheType) CacheStats

                                                                CacheStats returns stats about the provided cache within the group.

                                                                func (*Group) Get

                                                                func (g *Group) Get(ctx context.Context, key string, dest Sink) error

                                                                func (*Group) Name

                                                                func (g *Group) Name() string

                                                                  Name returns the name of the group.

                                                                  type HTTPPool

                                                                  type HTTPPool struct {
                                                                  	// Context optionally specifies a context for the server to use when it
                                                                  	// receives a request.
                                                                  	// If nil, the server uses the request's context
                                                                  	Context func(*http.Request) context.Context
                                                                  
                                                                  	// Transport optionally specifies an http.RoundTripper for the client
                                                                  	// to use when it makes a request.
                                                                  	// If nil, the client uses http.DefaultTransport.
                                                                  	Transport func(context.Context) http.RoundTripper
                                                                  	// contains filtered or unexported fields
                                                                  }

                                                                    HTTPPool implements PeerPicker for a pool of HTTP peers.

                                                                    func NewHTTPPool

                                                                    func NewHTTPPool(self string) *HTTPPool

                                                                      NewHTTPPool initializes an HTTP pool of peers, and registers itself as a PeerPicker. For convenience, it also registers itself as an http.Handler with http.DefaultServeMux. The self argument should be a valid base URL that points to the current server, for example "http://example.net:8000".

                                                                      func NewHTTPPoolOpts

                                                                      func NewHTTPPoolOpts(self string, o *HTTPPoolOptions) *HTTPPool

                                                                        NewHTTPPoolOpts initializes an HTTP pool of peers with the given options. Unlike NewHTTPPool, this function does not register the created pool as an HTTP handler. The returned *HTTPPool implements http.Handler and must be registered using http.Handle.

                                                                        func (*HTTPPool) PickPeer

                                                                        func (p *HTTPPool) PickPeer(key string) (ProtoGetter, bool)

                                                                        func (*HTTPPool) ServeHTTP

                                                                        func (p *HTTPPool) ServeHTTP(w http.ResponseWriter, r *http.Request)

                                                                        func (*HTTPPool) Set

                                                                        func (p *HTTPPool) Set(peers ...string)

                                                                          Set updates the pool's list of peers. Each peer value should be a valid base URL, for example "http://example.net:8000".

                                                                          type HTTPPoolOptions

                                                                          type HTTPPoolOptions struct {
                                                                          	// BasePath specifies the HTTP path that will serve groupcache requests.
                                                                          	// If blank, it defaults to "/_groupcache/".
                                                                          	BasePath string
                                                                          
                                                                          	// Replicas specifies the number of key replicas on the consistent hash.
                                                                          	// If blank, it defaults to 50.
                                                                          	Replicas int
                                                                          
                                                                          	// HashFn specifies the hash function of the consistent hash.
                                                                          	// If blank, it defaults to crc32.ChecksumIEEE.
                                                                          	HashFn consistenthash.Hash
                                                                          }

                                                                            HTTPPoolOptions are the configurations of a HTTPPool.

                                                                            type NoPeers

                                                                            type NoPeers struct{}

                                                                              NoPeers is an implementation of PeerPicker that never finds a peer.

                                                                              func (NoPeers) PickPeer

                                                                              func (NoPeers) PickPeer(key string) (peer ProtoGetter, ok bool)

                                                                              type PeerPicker

                                                                              type PeerPicker interface {
                                                                              	// PickPeer returns the peer that owns the specific key
                                                                              	// and true to indicate that a remote peer was nominated.
                                                                              	// It returns nil, false if the key owner is the current peer.
                                                                              	PickPeer(key string) (peer ProtoGetter, ok bool)
                                                                              }

                                                                                PeerPicker is the interface that must be implemented to locate the peer that owns a specific key.

                                                                                type ProtoGetter

                                                                                type ProtoGetter interface {
                                                                                	Get(ctx context.Context, in *pb.GetRequest, out *pb.GetResponse) error
                                                                                }

                                                                                  ProtoGetter is the interface that must be implemented by a peer.

                                                                                  type Sink

                                                                                  type Sink interface {
                                                                                  	// SetString sets the value to s.
                                                                                  	SetString(s string) error
                                                                                  
                                                                                  	// SetBytes sets the value to the contents of v.
                                                                                  	// The caller retains ownership of v.
                                                                                  	SetBytes(v []byte) error
                                                                                  
                                                                                  	// SetProto sets the value to the encoded version of m.
                                                                                  	// The caller retains ownership of m.
                                                                                  	SetProto(m proto.Message) error
                                                                                  	// contains filtered or unexported methods
                                                                                  }

                                                                                    A Sink receives data from a Get call.

                                                                                    Implementation of Getter must call exactly one of the Set methods on success.

                                                                                    func AllocatingByteSliceSink

                                                                                    func AllocatingByteSliceSink(dst *[]byte) Sink

                                                                                      AllocatingByteSliceSink returns a Sink that allocates a byte slice to hold the received value and assigns it to *dst. The memory is not retained by groupcache.

                                                                                      func ByteViewSink

                                                                                      func ByteViewSink(dst *ByteView) Sink

                                                                                        ByteViewSink returns a Sink that populates a ByteView.

                                                                                        func ProtoSink

                                                                                        func ProtoSink(m proto.Message) Sink

                                                                                          ProtoSink returns a sink that unmarshals binary proto values into m.

                                                                                          func StringSink

                                                                                          func StringSink(sp *string) Sink

                                                                                            StringSink returns a Sink that populates the provided string pointer.

                                                                                            func TruncatingByteSliceSink

                                                                                            func TruncatingByteSliceSink(dst *[]byte) Sink

                                                                                              TruncatingByteSliceSink returns a Sink that writes up to len(*dst) bytes to *dst. If more bytes are available, they're silently truncated. If fewer bytes are available than len(*dst), *dst is shrunk to fit the number of bytes available.

                                                                                              type Stats

                                                                                              type Stats struct {
                                                                                              	Gets           AtomicInt // any Get request, including from peers
                                                                                              	CacheHits      AtomicInt // either cache was good
                                                                                              	PeerLoads      AtomicInt // either remote load or remote cache hit (not an error)
                                                                                              	PeerErrors     AtomicInt
                                                                                              	Loads          AtomicInt // (gets - cacheHits)
                                                                                              	LoadsDeduped   AtomicInt // after singleflight
                                                                                              	LocalLoads     AtomicInt // total good local loads
                                                                                              	LocalLoadErrs  AtomicInt // total bad local loads
                                                                                              	ServerRequests AtomicInt // gets that came over the network from peers
                                                                                              }

                                                                                                Stats are per-group statistics.

                                                                                                Directories

                                                                                                Path Synopsis
                                                                                                Package consistenthash provides an implementation of a ring hash.
                                                                                                Package consistenthash provides an implementation of a ring hash.
                                                                                                Package lru implements an LRU cache.
                                                                                                Package lru implements an LRU cache.
                                                                                                Package singleflight provides a duplicate function call suppression mechanism.
                                                                                                Package singleflight provides a duplicate function call suppression mechanism.