dataloader

package module
Version: v5.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2018 License: MIT Imports: 7 Imported by: 8

README

DataLoader

GoDoc Build Status

This is an implementation of Facebook's DataLoader in Golang.

Install

go get -u github.com/graph-gophers/dataloader

Usage

// setup batch function
batchFn := func(ctx context.Context, keys dataloader.Keys) []*dataloader.Result {
  var results []*dataloader.Result
  // do some aync work to get data for specified keys
  // append to this list resolved values
  return results
}

// create Loader with an in-memory cache
loader := dataloader.NewBatchedLoader(batchFn)

/**
 * Use loader
 *
 * A thunk is a function returned from a function that is a
 * closure over a value (in this case an interface value and error).
 * When called, it will block until the value is resolved.
 */
thunk := loader.Load(ctx.TODO(), dataloader.StringKey("key1")) // StringKey is a convenience method that make wraps string to implement `Key` interface
result, err := thunk()
if err != nil {
  // handle data error
}

log.Printf("value: %#v", result)
Don't need/want to use context?

You're welcome to install the v1 version of this library.

Cache

This implementation contains a very basic cache that is intended only to be used for short lived DataLoaders (i.e. DataLoaders that ony exsist for the life of an http request). You may use your own implementation if you want.

it also has a NoCache type that implements the cache interface but all methods are noop. If you do not wish to cache anyting.

Examples

There are a few basic examples in the example folder.

Documentation

Overview

Package dataloader is an implimentation of facebook's dataloader in go. See https://github.com/facebook/dataloader for more information

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BatchFunc

type BatchFunc func(context.Context, Keys) []*Result

BatchFunc is a function, which when given a slice of keys (string), returns an slice of `results`. It's important that the length of the input keys matches the length of the output results.

The keys passed to this function are guaranteed to be unique

type Cache

type Cache interface {
	Get(context.Context, Key) (Thunk, bool)
	Set(context.Context, Key, Thunk)
	Delete(context.Context, Key) bool
	Clear()
}

The Cache interface. If a custom cache is provided, it must implement this interface.

type InMemoryCache

type InMemoryCache struct {
	// contains filtered or unexported fields
}

InMemoryCache is an in memory implementation of Cache interface. this simple implementation is well suited for a "per-request" dataloader (i.e. one that only lives for the life of an http request) but it not well suited for long lived cached items.

func NewCache

func NewCache() *InMemoryCache

NewCache constructs a new InMemoryCache

func (*InMemoryCache) Clear

func (c *InMemoryCache) Clear()

Clear clears the entire cache

func (*InMemoryCache) Delete

func (c *InMemoryCache) Delete(_ context.Context, key Key) bool

Delete deletes item at `key` from cache

func (*InMemoryCache) Get

func (c *InMemoryCache) Get(_ context.Context, key Key) (Thunk, bool)

Get gets the value at `key` if it exsits, returns value (or nil) and bool indicating of value was found

func (*InMemoryCache) Set

func (c *InMemoryCache) Set(_ context.Context, key Key, value Thunk)

Set sets the `value` at `key` in the cache

type Interface

type Interface interface {
	Load(context.Context, Key) Thunk
	LoadMany(context.Context, Keys) ThunkMany
	Clear(context.Context, Key) Interface
	ClearAll() Interface
	Prime(ctx context.Context, key Key, value interface{}) Interface
}

Interface is a `DataLoader` Interface which defines a public API for loading data from a particular data back-end with unique keys such as the `id` column of a SQL table or document name in a MongoDB database, given a batch loading function.

Each `DataLoader` instance should contain a unique memoized cache. Use caution when used in long-lived applications or those which serve many users with different access permissions and consider creating a new instance per web request.

type Key

type Key interface {
	// String returns a guaranteed unique string that can be used to identify an object
	String() string
	// Raw returns the raw, underlaying value of the key
	Raw() interface{}
}

Key is the interface that all keys need to implement

type Keys

type Keys []Key

Keys wraps a slice of Key types to provide some convenience methods.

func NewKeysFromStrings

func NewKeysFromStrings(strings []string) Keys

NewKeysFromStrings converts a `[]strings` to a `Keys` ([]Key)

func (Keys) Keys

func (l Keys) Keys() []string

Keys returns the list of strings. One for each "Key" in the list

type Loader

type Loader struct {
	// contains filtered or unexported fields
}

Loader implements the dataloader.Interface.

func NewBatchedLoader

func NewBatchedLoader(batchFn BatchFunc, opts ...Option) *Loader

NewBatchedLoader constructs a new Loader with given options.

func (*Loader) Clear

func (l *Loader) Clear(ctx context.Context, key Key) Interface

Clear clears the value at `key` from the cache, it it exsits. Returs self for method chaining

func (*Loader) ClearAll

func (l *Loader) ClearAll() Interface

ClearAll clears the entire cache. To be used when some event results in unknown invalidations. Returns self for method chaining.

func (*Loader) Load

func (l *Loader) Load(originalContext context.Context, key Key) Thunk

Load load/resolves the given key, returning a channel that will contain the value and error

func (*Loader) LoadMany

func (l *Loader) LoadMany(originalContext context.Context, keys Keys) ThunkMany

LoadMany loads mulitiple keys, returning a thunk (type: ThunkMany) that will resolve the keys passed in.

func (*Loader) Prime

func (l *Loader) Prime(ctx context.Context, key Key, value interface{}) Interface

Prime adds the provided key and value to the cache. If the key already exists, no change is made. Returns self for method chaining

type NoCache

type NoCache struct{}

NoCache implements Cache interface where all methods are noops. This is useful for when you don't want to cache items but still want to use a data loader

func (*NoCache) Clear

func (c *NoCache) Clear()

Clear is a NOOP

func (*NoCache) Delete

func (c *NoCache) Delete(context.Context, Key) bool

Delete is a NOOP

func (*NoCache) Get

func (c *NoCache) Get(context.Context, Key) (Thunk, bool)

Get is a NOOP

func (*NoCache) Set

func (c *NoCache) Set(context.Context, Key, Thunk)

Set is a NOOP

type NoopTracer

type NoopTracer struct{}

NoopTracer is the default (noop) tracer

func (NoopTracer) TraceBatch

TraceBatch is a noop function

func (NoopTracer) TraceLoad

TraceLoad is a noop function

func (NoopTracer) TraceLoadMany

TraceLoadMany is a noop function

type OpenTracingTracer

type OpenTracingTracer struct{}

OpenTracing Tracer implements a tracer that can be used with the Open Tracing standard.

func (OpenTracingTracer) TraceBatch

TraceBatch will trace a call to dataloader.LoadMany with Open Tracing

func (OpenTracingTracer) TraceLoad

TraceLoad will trace a call to dataloader.LoadMany with Open Tracing

func (OpenTracingTracer) TraceLoadMany

TraceLoadMany will trace a call to dataloader.LoadMany with Open Tracing

type Option

type Option func(*Loader)

Option allows for configuration of Loader fields.

func WithBatchCapacity

func WithBatchCapacity(c int) Option

WithBatchCapacity sets the batch capacity. Default is 0 (unbounded).

func WithCache

func WithCache(c Cache) Option

WithCache sets the BatchedLoader cache. Defaults to InMemoryCache if a Cache is not set.

func WithClearCacheOnBatch

func WithClearCacheOnBatch() Option

WithClearCacheOnBatch allows batching of items but no long term caching. It accomplishes this by clearing the cache after each batch operation.

func WithInputCapacity

func WithInputCapacity(c int) Option

WithInputCapacity sets the input capacity. Default is 1000.

func WithOpenTracingTracer

func WithOpenTracingTracer() Option

WithOpenTracingTracer allows tracing of calls to Load and LoadMany

func WithTracer

func WithTracer(tracer Tracer) Option

WithTracer allows tracing of calls to Load and LoadMany

func WithWait

func WithWait(d time.Duration) Option

WithWait sets the amount of time to wait before triggering a batch. Default duration is 16 milliseconds.

type Result

type Result struct {
	Data  interface{}
	Error error
}

Result is the data structure that a BatchFunc returns. It contains the resolved data, and any errors that may have occurred while fetching the data.

type ResultMany

type ResultMany struct {
	Data  []interface{}
	Error []error
}

ResultMany is used by the LoadMany method. It contains a list of resolved data and a list of errors. The lengths of the data list and error list will match, and elements at each index correspond to each other.

type StringKey

type StringKey string

StringKey implements the Key interface for a string

func (StringKey) Raw

func (k StringKey) Raw() interface{}

String is an identity method. Used to implement Key Raw

func (StringKey) String

func (k StringKey) String() string

String is an identity method. Used to implement String interface

type Thunk

type Thunk func() (interface{}, error)

Thunk is a function that will block until the value (*Result) it contains is resolved. After the value it contains is resolved, this function will return the result. This function can be called many times, much like a Promise is other languages. The value will only need to be resolved once so subsequent calls will return immediately.

type ThunkMany

type ThunkMany func() ([]interface{}, []error)

ThunkMany is much like the Thunk func type but it contains a list of results.

type TraceBatchFinishFunc

type TraceBatchFinishFunc func([]*Result)

type TraceLoadFinishFunc

type TraceLoadFinishFunc func(Thunk)

type TraceLoadManyFinishFunc

type TraceLoadManyFinishFunc func(ThunkMany)

type Tracer

type Tracer interface {
	// TraceLoad will trace the calls to Load
	TraceLoad(ctx context.Context, key Key) (context.Context, TraceLoadFinishFunc)
	// TraceLoadMany will trace the calls to LoadMany
	TraceLoadMany(ctx context.Context, keys Keys) (context.Context, TraceLoadManyFinishFunc)
	// TraceBatch will trace data loader batches
	TraceBatch(ctx context.Context, keys Keys) (context.Context, TraceBatchFinishFunc)
}

Tracer is an interface that may be used to implement tracing.

Directories

Path Synopsis
example
lru-cache
This is an exmaple of using go-cache as a long term cache solution for dataloader.
This is an exmaple of using go-cache as a long term cache solution for dataloader.
ttl-cache
This is an exmaple of using go-cache as a long term cache solution for dataloader.
This is an exmaple of using go-cache as a long term cache solution for dataloader.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL