goon

package module
v2.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 26, 2021 License: ISC Imports: 19 Imported by: 0

README

goon

An autocaching interface to the app engine datastore for Go. Designed to be similar to the python NDB package.

The different flavors of App Engine

You must choose the goon major version based on which App Engine library you are using.

App Engine Go library Include in your project Correct goon version
google.golang.org/appengine/v2 github.com/mjibson/goon/v2 goon v2.0.0
google.golang.org/appengine github.com/mjibson/goon goon v1.1.0
appengine github.com/mjibson/goon goon v0.9.0
cloud.google.com/go N/A Not supported (issue #74)

Documentation

https://pkg.go.dev/github.com/mjibson/goon

Documentation

Overview

Package goon provides an autocaching interface to the app engine datastore similar to the python NDB package.

Goon differs from the datastore package in various ways: it remembers the appengine Context, which need only be specified once at creation time; kinds need not be specified as they are computed, by default, from a type's name; keys are inferred from specially-tagged fields on types, removing the need to pass key objects around.

In general, the difference is that Goon's API is identical to the datastore API, it's just shorter.

Keys in Goon are stored in the structs themselves. Below is an example struct with a field to specify the id (see the Key Specifications section below for full documentation).

type User struct {
	Id    string `datastore:"-" goon:"id"`
	Name  string
}

Thus, to get a User with id 2:

userid := 2
g := goon.NewGoon(r)
u := &User{Id: userid}
g.Get(u)

Key Specifications

For both the Key and KeyError functions, src must be a S or *S for some struct type S. The key is extracted based on various fields of S. If a field of type int64 or string has a struct tag named goon with value "id", it is used as the key's id. If a field of type *datastore.Key has a struct tag named goon with value "parent", it is used as the key's parent. If a field of type string has a struct tag named goon with value "kind", it is used as the key's kind. The "kind" field supports an optional second parameter which is the default kind name. If no kind field exists, the struct's name is used. These fields should all have their datastore field marked as "-".

Example, with kind User:

type User struct {
	Id    string `datastore:"-" goon:"id"`
	Read  time.Time
}

Example, with kind U if _kind is the empty string:

type User struct {
	_kind string `goon:"kind,U"`
	Id    string `datastore:"-" goon:"id"`
	Read  time.Time
}

To override kind of a single entity to UserKind:

u := User{_kind: "UserKind"}

An example with both parent and kind:

type UserData struct {
	Id     string         `datastore:"-" goon:"id"`
	_kind  string         `goon:"kind,UD"`
	Parent *datastore.Key `datastore:"-" goon:"parent"`
	Data   []byte
}

Features

Datastore interaction with: Get, GetMulti, Put, PutMulti, Delete, DeleteMulti, Queries.

All key-based operations backed by memory and memcache.

Per-request, in-memory cache: fetch the same key twice, the second request is served from local memory.

Intelligent multi support: running GetMulti correctly fetches from memory, then memcache, then the datastore; each tier only sends keys off to the next one if they were missing.

Memcache control variance: long memcache requests are cancelled.

Transactions use a separate context, but locally cache any results on success.

Automatic kind naming: struct names are inferred by reflection, removing the need to manually specify key kinds.

Simpler API than appengine/datastore.

API comparison between goon and datastore

put with incomplete key

datastore:

type Group struct {
	Name string
}
c := appengine.NewContext(r)
g := &Group{Name: "name"}
k := datastore.NewIncompleteKey(c, "Group", nil)
err := datastore.Put(c, k, g)

goon:

type Group struct {
	Id   int64 `datastore:"-" goon:"id"`
	Name string
}
n := goon.NewGoon(r)
g := &Group{Name: "name"}
err := n.Put(g)

get with known key

datastore:

type Group struct {
	Name string
}
c := appengine.NewContext(r)
g := &Group{}
k := datastore.NewKey(c, "Group", "", 1, nil)
err := datastore.Get(c, k, g)

goon:

type Group struct {
	Id   int64 `datastore:"-" goon:"id"`
	Name string
}
n := goon.NewGoon(r)
g := &Group{Id: 1}
err := n.Get(g)

Memcache Control Variance

Memcache is generally fast. When it is slow, goon will timeout the memcache requests and proceed to use the datastore directly. The memcache put and get timeout variables determine how long to wait for various kinds of requests. The default settings were determined experimentally and should provide reasonable defaults for most applications.

See: http://talks.golang.org/2013/highperf.slide#23

PropertyLoadSaver support

Structs that implement the PropertyLoadSaver interface are guaranteed to call the Save() method once and only once per Put/PutMulti call and never elsewhere. Similarly the Load() method is guaranteed to be called once and only once per Get/GetMulti/GetAll/Next call and never elsewhere.

Index

Constants

This section is empty.

Variables

View Source
var (
	// LogErrors issues appengine.Context.Errorf on any error.
	LogErrors = true
	// LogTimeoutErrors issues appengine.Context.Warningf on memcache timeout errors.
	LogTimeoutErrors = false

	// MemcachePutTimeoutThreshold is the number of bytes after which the large
	// timeout setting is added to the small timeout setting. This repeats.
	// Which means that with a 20 KiB threshold and a 100 KiB memcache payload,
	// the final timeout is MemcachePutTimeoutSmall + 5*MemcachePutTimeoutLarge
	MemcachePutTimeoutThreshold = 20 * 1024 // 20 KiB
	// MemcachePutTimeoutSmall is the minimum time to wait during memcache
	// Put operations before aborting them and using the datastore.
	MemcachePutTimeoutSmall = 5 * time.Millisecond
	// MemcachePutTimeoutLarge is the amount of extra time to wait for larger
	// memcache Put requests. See also MemcachePutTimeoutThreshold.
	MemcachePutTimeoutLarge = 1 * time.Millisecond
	// MemcacheGetTimeout is the amount of time to wait for all memcache Get
	// requests, per key fetched. Because we can't really know how big entities
	// we are requesting, this setting should be for the maximum size entity.
	// The final timeout is limited to the number of maximum sized entities
	// an RPC result can contain, so the timeout won't grow insanely large
	// if you're fetching a ton of small entities.
	MemcacheGetTimeout = 31250 * time.Microsecond // 31.25 milliseconds

	// IgnoreFieldMismatch decides whether *datastore.ErrFieldMismatch errors
	// should be silently ignored. This allows you to easily remove fields from structs.
	IgnoreFieldMismatch = true
)
View Source
var MemcacheKey = func(k *datastore.Key) string {
	return k.Encode()
}

MemcacheKey returns the string form of the provided datastore key.

Functions

func DefaultKindName

func DefaultKindName(src interface{}) string

DefaultKindName is the default implementation to determine the Kind an Entity has. Returns the basic Type of the src (no package name included).

func NotFound

func NotFound(err error, idx int) bool

NotFound returns true if err is an appengine.MultiError and err[idx] is a datastore.ErrNoSuchEntity.

Types

type Goon

type Goon struct {
	Context context.Context

	// KindNameResolver is used to determine what Kind to give an Entity.
	// Defaults to DefaultKindName
	KindNameResolver KindNameResolver
	// contains filtered or unexported fields
}

Goon holds the app engine context and the request memory cache.

func FromContext

func FromContext(c context.Context) *Goon

FromContext creates a new Goon object from the given appengine Context. Useful with profiling packages like appstats.

func NewGoon

func NewGoon(r *http.Request) *Goon

NewGoon creates a new Goon object from the given request.

func (*Goon) ClearCache

func (g *Goon) ClearCache(src interface{}, mem, local bool) error

ClearCache removes the provided entity from cache. Takes either *S or *datastore.Key. The 'mem' and 'local' booleans dictate the type of caches to clear.

func (*Goon) ClearCacheMulti

func (g *Goon) ClearCacheMulti(src interface{}, mem, local bool) error

ClearCacheMulti removes the provided entities from cache. Takes either []*S or []*datastore.Key. The 'mem' and 'local' booleans dictate the type of caches to clear.

func (*Goon) Count

func (g *Goon) Count(q *datastore.Query) (int, error)

Count returns the number of results for the query.

func (*Goon) Delete

func (g *Goon) Delete(src interface{}) error

Delete deletes the provided entity. Takes either *S or *datastore.Key.

func (*Goon) DeleteMulti

func (g *Goon) DeleteMulti(src interface{}) error

DeleteMulti is a batch version of Delete. Takes either []*S or []*datastore.Key.

func (*Goon) FlushLocalCache

func (g *Goon) FlushLocalCache()

FlushLocalCache clears the local memory cache.

func (*Goon) Get

func (g *Goon) Get(dst interface{}) error

Get loads the entity based on dst's key into dst If there is no such entity for the key, Get returns datastore.ErrNoSuchEntity.

func (*Goon) GetAll

func (g *Goon) GetAll(q *datastore.Query, dst interface{}) ([]*datastore.Key, error)

GetAll runs the query and returns all the keys that match the query, as well as appending the values to dst, setting the goon key fields of dst, and caching the returned data in local memory.

For "keys-only" queries dst can be nil, however if it is not, then GetAll appends zero value structs to dst, only setting the goon key fields.

No data is cached with projection or "keys-only" queries.

See: https://developers.google.com/appengine/docs/go/datastore/reference#Query.GetAll

func (*Goon) GetMulti

func (g *Goon) GetMulti(dst interface{}) error

GetMulti is a batch version of Get.

dst must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.

func (*Goon) Key

func (g *Goon) Key(src interface{}) *datastore.Key

Key is the same as KeyError, except nil is returned on error or if the key is incomplete.

func (*Goon) KeyError

func (g *Goon) KeyError(src interface{}) (*datastore.Key, error)

KeyError returns the key of src based on its properties.

func (*Goon) Kind

func (g *Goon) Kind(src interface{}) string

Kind returns src's datastore Kind or "" on error.

func (*Goon) Put

func (g *Goon) Put(src interface{}) (*datastore.Key, error)

Put saves the entity src into the datastore based on src's key k. If k is an incomplete key, the returned key will be a unique key generated by the datastore.

func (*Goon) PutMulti

func (g *Goon) PutMulti(src interface{}) ([]*datastore.Key, error)

PutMulti is a batch version of Put.

src must be a *[]S, *[]*S, *[]I, []S, []*S, or []I, for some struct type S, or some interface type I. If *[]I or []I, each element must be a struct pointer.

func (*Goon) Run

func (g *Goon) Run(q *datastore.Query) *Iterator

Run runs the query.

func (*Goon) RunInTransaction

func (g *Goon) RunInTransaction(f func(tg *Goon) error, opts *datastore.TransactionOptions) error

RunInTransaction runs f in a transaction. It calls f with a transaction context tg that f should use for all App Engine operations. Neither cache nor memcache are used or set during a transaction.

Otherwise similar to appengine/datastore.RunInTransaction: https://developers.google.com/appengine/docs/go/datastore/reference#RunInTransaction

type Iterator

type Iterator struct {
	// contains filtered or unexported fields
}

Iterator is the result of running a query.

func (*Iterator) Cursor

func (t *Iterator) Cursor() (datastore.Cursor, error)

Cursor returns a cursor for the iterator's current location.

func (*Iterator) Next

func (t *Iterator) Next(dst interface{}) (*datastore.Key, error)

Next returns the entity of the next result. When there are no more results, datastore.Done is returned as the error.

If the query is not keys only and dst is non-nil, it also loads the entity stored for that key into the struct pointer dst, with the same semantics and possible errors as for the Get function.

This result is cached in memory, unless it's a projection query.

If the query is keys only and dst is non-nil, dst will be given the right id.

Refer to appengine/datastore.Iterator.Next: https://developers.google.com/appengine/docs/go/datastore/reference#Iterator.Next

type KindNameResolver

type KindNameResolver func(src interface{}) string

KindNameResolver takes an Entity and returns what the Kind should be for Datastore.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL