README
¶
go-memoize
Add simple memcache-backed function memoization to your gRPC clients and servers, or anywhere else that you pass protocol buffers around.
Documentation
¶
Overview ¶
Package memoize implements simple function memoization for protobuf messages using a memcache-like cache.
This package favors simplicity and generality over maximal deduplication of work — in many common cases, there will be more calls to the underlying function than strictly necessary. For instance, this package does nothing about concurrency: multiple parallel calls with the same inputs will all go to the underlying function until one of them returns. As well, in the vein of generality, a default key function is provided for all proto messages, even though proto serialization is not canonical so the same messages will sometimes have different keys.
For best results, memoizing should be done “close to” requests — so e.g. on the client rather than the server side, where it is more likely that all of the relevant fields to the request will be accounted for. This also has the advantage that clients will talk directly to your memcached instance rather than having to go through a gRPC server.
The main API exported by this package is Wrap. This function is well-suited to gRPC servers, and is in fact shaped the same way as a gRPC RPC modulo the target. Wrap uses a package-default memoizer constructed via New; if a custom Memoizer is desired, then WrapWithMemoizer may be used instead.
A client-side grpc.UnaryInterceptor is also exported as Intercept, which corresponds to Wrap; or InterceptWithMemoizer, which corresponds to WrapWithMemoizer.
This library generally assumes that the wrapped function’s inputs (but not necessarily its outputs) are trusted, e.g. not user-generated. The full inputs are not stored in the cache entry, so if two different inputs hash to the same key, this can poison the cache. (To emphasize this point, the default key function uses crypto.SHA1 as its hash, and SHA-1 has known collisions on untrusted inputs.)
Index ¶
- Variables
- func Intercept(cache Cache, opts ...Option) grpc.UnaryClientInterceptor
- func InterceptWithMemoizer(m Memoizer) grpc.UnaryClientInterceptor
- func New(c Cache, opts ...Option) *memoizer
- type Cache
- type ErrFunc
- type Errer
- type Expirer
- type Flagger
- type Func
- type HasErrer
- type Item
- type LocalCache
- type Memoizer
- type Option
- func WithCustomKeyFunc(f func(context.Context, proto.Message) (string, error)) Option
- func WithCustomKeyer(k keyer.Keyer) Option
- func WithErrorFunc(f func(error)) Option
- func WithErrorHandler(errer Errer) Option
- func WithFlags(flags uint32) Option
- func WithHashKeyerOpts(opts ...keyer.HashKeyerOption) Option
- func WithTTL(ttl time.Duration) Option
Constants ¶
This section is empty.
Variables ¶
var ( ErrCacheMiss = memcache.ErrCacheMiss ErrNotStored = memcache.ErrNotStored )
These two reexported errors from memcache are treated specially by this library, in that ErrCacheMiss from a Cache Get — and ErrNotStored from an Add — are ignored by the error handling logic in this package, as neither of them indicates an erroneous condition.
memcache.ErrCacheMiss is returned by Get if there was no item at the requested key.
memcache.ErrNotStored is returned in different cases in memcache.Client, but for our purposes only its return from Add is relevant: this happens when there is already a value stored at the requested key.
var ErrNotProto = errors.New("not a proto")
ErrNotProto is sent to the Errer if the functions intercepted by Intercept or InterceptWithMemoizer are called with something that is not a proto.Message.
var NilCache = (*nilCache)(nil)
NilCache is a Cache that never stores or retrieves anything.
Gets fail with ErrCacheMiss, and Adds fail with ErrNotStored. As both of these errors are ignored by the error handling logic in this library, this means a function wrapped with NilCache will report no errors.
This is sort of lying, in that ErrNotStored from memcache.Client.Add is supposed to only be returned if there already was a value for that key in the cache. But it is not hard to imagine a cache that winds up behaving like this in practice on some pathological workload, say getting an Add for the same key from elsewhere before every Add, but then having that key evicted before every Get.
Functions ¶
func Intercept ¶
func Intercept(cache Cache, opts ...Option) grpc.UnaryClientInterceptor
Intercept is a grpc.UnaryClientInterceptor that memoizes gRPC clients. It uses the passed Cache with a package-default memoizer (see New.)
Note that the keyer.HashKeyer will be constructed the option keyer.WithTypePrefix set to true by default, as otherwise it is quite common for protobuf messages of different types to share the same serialization and therefore cache key.
func InterceptWithMemoizer ¶
func InterceptWithMemoizer(m Memoizer) grpc.UnaryClientInterceptor
InterceptWithMemoizer is a grpc.UnaryClientInterceptor that memoizes gRPC clients with the passed Memoizer.
It is strongly encouraged for the Memoizer to take care to produce different keys for different input types.
Types ¶
type Cache ¶
Cache implements a minimal subset of memcache.Client for use elsewhere in this library.
type ErrFunc ¶
type ErrFunc func(error)
ErrFunc is an interface wrapper for functions allowing them to be used as an Errer.
type Errer ¶
type Errer interface {
Error(error)
}
Errer receives non-fatal errors that can occur in the memoization logic but that do not affect function outputs. It does not receive ErrCacheMiss or ErrNotStored, but otherwise if anything goes wrong with cache retreival or storage, or with proto serialization or deserialization, it will be reported for either recovery or logging.
var ErrorHandler Errer
ErrorHandler is the default Errer for the memoize package. It is nil by default. If it is set to non-nil, and there is not a custom Errer configured for a given Memoizer, then it will receive all non-fatal errors. E.g. it may make sense to wire this up to a custom log.Errorf function, or even simply:
memoize.ErrorHandler = func(err error) {
log.Printf("ERROR: %v", err)
}
type Expirer ¶
Expirer returns the expiration time for the cache entry corresponding to the passed context, and request/result messages, in the format expected by Item. That is: “the cache expiration time, in seconds: either a relative time from now (up to 1 month), or an absolute Unix epoch time. Zero means […] no expiration time.”
type Flagger ¶
Flagger sets custom flags on cache entries, corresponding to the Flags field on Item.
type Func ¶
Func is the basic unit that is memoized by this package. It is shaped like a gRPC RPC call but without the target.
func Wrap ¶
func Wrap[T any, Req proto.Message, Res protoMessage[T]]( c Cache, f Func[Req, Res], opts ...Option, ) Func[Req, Res]
Wrap returns a memoized version of the Func f using the Cache c. The type should normally be inferred; for further discussion, see WrapWithMemoizer.
It should be relatively simple to use this function on a gRPC server like so:
type Server struct {
pb.UnimplementedFooServer
memoSayHello memoize.Func[*pb.HelloRequest, *pb.HelloReply]
}
func New(cache memoize.Cache) *Server {
s := &Server{}
s.memoSayHello = Wrap(cache, func(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
return s.SayHello_Raw(ctx, in)
})
}
func (s *Server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
return s.memoSayHello(ctx, in)
}
func (s *Server) SayHello_Raw(_ context.Context, in *pb.HelloReqest) (*pb.HelloReply, error) {
// do the actual work
}
func WrapWithMemoizer ¶
func WrapWithMemoizer[T any, Req proto.Message, Res protoMessage[T]]( m Memoizer, f Func[Req, Res], ) Func[Req, Res]
WrapWithMemoizer returns a memoized version of f using the passed Memoizer. The type should normally be inferred; for example, given:
var myFunc func(context.Context, *pb.RpcRequest) (*pb.RpcReply, error)
MyFunc can be memoized by simply saying:
memo := WrapWithMemoizer(myMemoizer, myFunc)
The returned memo function will have the Func type:
Func[*pb.RpcRequest, *pb.RpcReply]
which corresponds to the raw type:
func(context.Context, *pb.RpcRequest) (*pb.RpcReply, error)
This is the same type as that of a gRPC server function with request type RpcRequest and reply type RpcReply.
The protoMessage[T] in the type of WrapWithMemoizer is an implementation detail; it should be read “Res is a proto.Message and also a *T for some T.” It exists to allow a zero value of type T to be created without having to use reflection.
type HasErrer ¶
type HasErrer interface {
Errer
// IsErrerEnabled returns true if this Errer is live and receiving errors;
// this allows e.g. a type to implement Errer but have it be conditional
// whether any individual instance should receive errors.
IsErrerEnabled() bool
}
HasErrer is an Errer that is optionally enabled or not.
type LocalCache ¶
LocalCache is a test-only in-memory Cache. Stores always succeed unless Full is set to true. Gets always succeed if there is an unexpired Item at that key unless Down is set to true.
This type should be constructed via NewLocalCache as it contains an unexported map field.
func NewLocalCache ¶
func NewLocalCache() *LocalCache
func (*LocalCache) Add ¶
func (c *LocalCache) Add(item *Item) error
func (*LocalCache) AdvanceTime ¶
func (c *LocalCache) AdvanceTime(d time.Duration)
AdvanceTime advances this cache’s clock by the passed duration.
func (*LocalCache) Now ¶
func (c *LocalCache) Now() time.Time
Now implements time.Now with the skew applied by LocalCache.AdvanceTime.
type Memoizer ¶
Memoizer is the interface required to provide memoization for functions in this API. It consists of:
- A Cache to store and retreive values.
- A keyer.Keyer to generate keys from inputs.
A Memoizer may additionally implement the following optional interfaces to opt in to additional functionality:
type Option ¶
type Option func(*builder)
Option customizes a memoizer’s behavior.
func WithCustomKeyFunc ¶
WithCustomKeyFunc allows supplying a raw function to produce cache keys.
func WithCustomKeyer ¶
WithCustomKeyer allows supplying a completely custom keyer.Keyer for cache keys.
func WithErrorFunc ¶
WithErrorFunc allows passing a raw function to receive memoization errors.
func WithErrorHandler ¶
WithErrorHandler allows setting a custom Errer for this memoizer.
func WithFlags ¶
WithFlags allows setting constant flags to be set on all cache entries from this memoizer.
func WithHashKeyerOpts ¶
func WithHashKeyerOpts(opts ...keyer.HashKeyerOption) Option
WithHashKeyerOpts appends to the options passed to the default keyer.HashKeyer that is constructed if no other keyer.Keyer or keyer.KeyFunc is passed.
Source Files
Directories