README
¶
dataloader
Inspired by https://www.npmjs.com/package/dataloader, this is a generic utility to be used as part of your application's data fetching layer to allow naive calls to databases and other services without sacrificing performance via caching and batching.
QueryBatcher usage
func getUsers(userIds []string) (map[string]User, map[string]error) {
...
}
const maxConcurrentBatches = 3
const maxBatchSize = 9999
batcher := NewQueryBatcher(getUsers, maxConcurrentBatches, maxBatchSize)
user, err := batcher.Load("user-id-0001")
QueryBatcher's name says it all: it represents a pool that limits the number and size of simultaneous requests a service can make to a resource like a database. When more requests come in at once than are allowed by maxConcurrentBatches, these excess requests will be added to a batch (with a size capped at maxBatchSize) which will all be queried at once as soon as a current request finishes.
DataLoader usage
DataLoader is functionally the same as QueryBatcher, but with an added cache to prevent repeating calls after they've already been made.
gorm
For convenience, there are also the GormGetter and GormListGetter functions, which simplify lookups in databases managed by gorm.io/gorm
type User struct {
ID string `gorm:"primaryKey"`
Name string // not unique
...
}
userLoader := dataloader.NewDataLoader(
GormGetter(db, "id", func (usr User) string { return usr.ID }),
maxConcurrentBatches,
maxBatchSize,
)
user, err := batcher.Load("user-id-0001")
usersByNameLoader := dataloader.NewDataLoader(
GormListGetter(db, "name", func (tst User) string { return usr.Name }),
maxConcurrentBatches,
maxBatchSize,
)
users, err := batcher.Load("bob")
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrMissingResponse = errors.New("no data or explicit error was returned for the given key")
Functions ¶
func ErrForAll ¶
func ErrForAll[KEY_TYPE comparable](keys []KEY_TYPE, err error) map[KEY_TYPE]error
reduces boilerplate for getters that want to respond to all keys with a shared error (such as database failure)
func FillEmpty ¶
func FillEmpty[KEY_TYPE comparable, VALUE_TYPE any](keys []KEY_TYPE, current map[KEY_TYPE][]VALUE_TYPE) map[KEY_TYPE][]VALUE_TYPE
reduces boilerplate for getters that return slices, where an empty slice is an expected possibility
Types ¶
type DataLoader ¶
type DataLoader[KEY_TYPE comparable, VALUE_TYPE any] struct { // contains filtered or unexported fields }
func NewDataLoader ¶
func NewDataLoader[KEY_TYPE comparable, VALUE_TYPE any](getter Getter[KEY_TYPE, VALUE_TYPE], maxConcurrentBatches, maxBatchSize int) *DataLoader[KEY_TYPE, VALUE_TYPE]
func (*DataLoader[KEY_TYPE, VALUE_TYPE]) Close ¶
func (dataLoader *DataLoader[KEY_TYPE, VALUE_TYPE]) Close()
func (*DataLoader[KEY_TYPE, VALUE_TYPE]) Load ¶
func (dataLoader *DataLoader[KEY_TYPE, VALUE_TYPE]) Load(key KEY_TYPE) (VALUE_TYPE, error)
func (*DataLoader[KEY_TYPE, VALUE_TYPE]) LoadPromise ¶
func (dataLoader *DataLoader[KEY_TYPE, VALUE_TYPE]) LoadPromise(key KEY_TYPE) *promises.Promise[VALUE_TYPE]
type Getter ¶
type Getter[KEY_TYPE comparable, VALUE_TYPE any] func([]KEY_TYPE) (map[KEY_TYPE]VALUE_TYPE, map[KEY_TYPE]error)
A getter function accepts a list of de-duplicated keys, and returns a pair of maps from keys to values (for successful lookups) and keys to errors (for unsuccessful lookups)
type GetterPanicError ¶
type GetterPanicError struct {
// contains filtered or unexported fields
}
func (GetterPanicError) Error ¶
func (gpe GetterPanicError) Error() string
type QueryBatcher ¶
type QueryBatcher[KEY_TYPE comparable, VALUE_TYPE any] struct { // contains filtered or unexported fields }
func NewQueryBatcher ¶
func NewQueryBatcher[KEY_TYPE comparable, VALUE_TYPE any](getter Getter[KEY_TYPE, VALUE_TYPE], maxConcurrentBatches, maxBatchSize int) *QueryBatcher[KEY_TYPE, VALUE_TYPE]
func (*QueryBatcher[KEY_TYPE, VALUE_TYPE]) Close ¶
func (batcher *QueryBatcher[KEY_TYPE, VALUE_TYPE]) Close()
func (*QueryBatcher[KEY_TYPE, VALUE_TYPE]) Load ¶
func (batcher *QueryBatcher[KEY_TYPE, VALUE_TYPE]) Load(key KEY_TYPE) (VALUE_TYPE, error)
func (*QueryBatcher[KEY_TYPE, VALUE_TYPE]) LoadPromise ¶
func (batcher *QueryBatcher[KEY_TYPE, VALUE_TYPE]) LoadPromise(key KEY_TYPE) *promises.Promise[VALUE_TYPE]
Source Files
Directories