dataloadgen

README

dataloadgen

This is a fork of https://github.com/vektah/dataloaden that uses generics instead of code genation. See the docs there for the motivation and usage details.

With this package you can skip the generation step! That's why it's dataloadgen (gen stands for generics).

To add this package as a dependency:

go get github.com/vikstrous/dataloadgen

See the example in the documentation: https://pkg.go.dev/github.com/vikstrous/dataloadgen

This package has evolved to include some ideas from dataloader https://github.com/graph-gophers/dataloader

Benchmarks show that this package is faster than both of the above and offers the best of both worlds.

BenchmarkDataloader/caches-8                     3963708               301.9 ns/op           162 B/op          4 allocs/op
BenchmarkDataloader/random_spread-8               642184              1961 ns/op             736 B/op         14 allocs/op
BenchmarkDataloader/concurently-8                  17191             85223 ns/op           45449 B/op        200 allocs/op

BenchmarkDataloaden/caches-8                    10976748               108.6 ns/op            26 B/op          1 allocs/op
BenchmarkDataloaden/random_spread-8              1000000              1107 ns/op             331 B/op          4 allocs/op
BenchmarkDataloaden/concurently-8                  23605             53127 ns/op            2984 B/op         68 allocs/op

BenchmarkDataloadgen/caches-8                   12434064                97.04 ns/op           10 B/op          0 allocs/op
BenchmarkDataloadgen/random_spread-8             1000000              1088 ns/op             307 B/op          3 allocs/op
BenchmarkDataloadgen/concurently-8                 39688             30788 ns/op            2729 B/op         60 allocs/op

Documentation

Index

• type Loader
  • func NewLoader[KeyT comparable, ValueT any](fetch func(keys []KeyT) ([]ValueT, []error), options ...Option[KeyT, ValueT]) *Loader[KeyT, ValueT]
  • func (l *Loader[KeyT, ValueT]) Clear(key KeyT)
  • func (l *Loader[KeyT, ValueT]) Load(key KeyT) (ValueT, error)
  • func (l *Loader[KeyT, ValueT]) LoadAll(keys []KeyT) ([]ValueT, []error)
  • func (l *Loader[KeyT, ValueT]) LoadAllThunk(keys []KeyT) func() ([]ValueT, []error)
  • func (l *Loader[KeyT, ValueT]) LoadThunk(key KeyT) func() (ValueT, error)
  • func (l *Loader[KeyT, ValueT]) Prime(key KeyT, value ValueT) bool
• type Option
  • func WithBatchCapacity[KeyT comparable, ValueT any](c int) Option[KeyT, ValueT]
  • func WithWait[KeyT comparable, ValueT any](d time.Duration) Option[KeyT, ValueT]

Examples

• Loader

Types

type Loader

type Loader[KeyT comparable, ValueT any] struct {
	// contains filtered or unexported fields
}

Loader batches and caches requests

Example

package main

import (
	"fmt"
	"strconv"
	"time"

	"github.com/vikstrous/dataloadgen"
)

func main() {
	loader := dataloadgen.NewLoader(func(keys []string) (ret []int, errs []error) {
		for _, key := range keys {
			num, err := strconv.ParseInt(key, 10, 32)
			ret = append(ret, int(num))
			errs = append(errs, err)
		}
		return
	},
		dataloadgen.WithBatchCapacity[string, int](1),
		dataloadgen.WithWait[string, int](16*time.Millisecond),
	)
	one, err := loader.Load("1")
	if err != nil {
		panic(err)
	}
	fmt.Println(one)
}

Output:

1

func NewLoader

func NewLoader[KeyT comparable, ValueT any](fetch func(keys []KeyT) ([]ValueT, []error), options ...Option[KeyT, ValueT]) *Loader[KeyT, ValueT]

NewLoader creates a new GenreicLoader given a fetch, wait, and maxBatch

func (*Loader[KeyT, ValueT]) Clear

func (l *Loader[KeyT, ValueT]) Clear(key KeyT)

Clear the value at key from the cache, if it exists

func (*Loader[KeyT, ValueT]) Load

func (l *Loader[KeyT, ValueT]) Load(key KeyT) (ValueT, error)

Load a ValueT by key, batching and caching will be applied automatically

func (*Loader[KeyT, ValueT]) LoadAll

func (l *Loader[KeyT, ValueT]) LoadAll(keys []KeyT) ([]ValueT, []error)

LoadAll fetches many keys at once. It will be broken into appropriate sized sub batches depending on how the loader is configured

func (*Loader[KeyT, ValueT]) LoadAllThunk

func (l *Loader[KeyT, ValueT]) LoadAllThunk(keys []KeyT) func() ([]ValueT, []error)

LoadAllThunk returns a function that when called will block waiting for a ValueT. This method should be used if you want one goroutine to make requests to many different data loaders without blocking until the thunk is called.

func (*Loader[KeyT, ValueT]) LoadThunk

func (l *Loader[KeyT, ValueT]) LoadThunk(key KeyT) func() (ValueT, error)

LoadThunk returns a function that when called will block waiting for a ValueT. This method should be used if you want one goroutine to make requests to many different data loaders without blocking until the thunk is called.

func (*Loader[KeyT, ValueT]) Prime

func (l *Loader[KeyT, ValueT]) Prime(key KeyT, value ValueT) bool

Prime the cache with the provided key and value. If the key already exists, no change is made and false is returned. (To forcefully prime the cache, clear the key first with loader.clear(key).prime(key, value).)

type Option

type Option[KeyT comparable, ValueT any] func(*Loader[KeyT, ValueT])

Option allows for configuration of loader fields.

func WithBatchCapacity

func WithBatchCapacity[KeyT comparable, ValueT any](c int) Option[KeyT, ValueT]

WithBatchCapacity sets the batch capacity. Default is 0 (unbounded)

func WithWait

func WithWait[KeyT comparable, ValueT any](d time.Duration) Option[KeyT, ValueT]

WithWait sets the amount of time to wait before triggering a batch. Default duration is 16 milliseconds.