moredis

Documentation

Index

• func ApplyTemplate(templateString string, payload bson.M) (string, error)
• func BuildCache(cacheConfig Config, params Params, redisURL string, mongoURL string) error
• func ParseTemplatedJSON(query string, params Params) (map[string]interface{}, error)
• func ParseTemplates(collection *CollectionConfig) error
• func ProcessQuery(writer RedisWriter, iter MongoIter, maps []MapConfig) error
• func SetRedisHashKeys(conn redis.Conn, collection *CollectionConfig) error
• func SetupDbs(mongoURL, redisURL string) (*mgo.Database, redis.Conn, error)
• func UpdateRedisMapReference(conn redis.Conn, params Params, mapConfig MapConfig) error
• type CollectionConfig
• type Config
  • func LoadConfig(path string) (Config, error)
• type MapConfig
• type MongoIter
• type Params
  • func (p *Params) Bson() bson.M
  • func (p *Params) Set(value string) error
  • func (p *Params) String() string
• type RedisWriter
  • func NewRedisWriter(conn redis.Conn) RedisWriter

Functions

func ApplyTemplate

func ApplyTemplate(templateString string, payload bson.M) (string, error)

ApplyTemplate takes a template string and a map of values to use for evaluating the template. Returns the evaluated template as a string or an error.

func BuildCache

func BuildCache(cacheConfig Config, params Params, redisURL string, mongoURL string) error

BuildCache builds a redis cache according to the passed in config.

func ParseTemplatedJSON

func ParseTemplatedJSON(query string, params Params) (map[string]interface{}, error)

ParseTemplatedJSON takes a template string for a json object and a map of values for substitution and returns a map that can be used in mgo queries. The template string must evaluate to a valid JSON object. This means that all keys must be strings. For mongo operators, you should encase them in quotes, for example "$or". For ObjectIds, you should use the hex string in place of the ObjectId.

func ParseTemplates

func ParseTemplates(collection *CollectionConfig) error

ParseTemplates takes a collection config and parses the templates for all of the contained maps.

func ProcessQuery

func ProcessQuery(writer RedisWriter, iter MongoIter, maps []MapConfig) error

ProcessQuery iterates through all of the documents contained within iter, and maps keys to values in a redis hash according to your mapping config.

func SetRedisHashKeys

func SetRedisHashKeys(conn redis.Conn, collection *CollectionConfig) error

SetRedisHashKeys determines the correct keys to use for the redis hashes that will be created to store the mapped values. These keys are generated in an atomic fashion and will not interfere with any other running instances of moredis

func SetupDbs

func SetupDbs(mongoURL, redisURL string) (*mgo.Database, redis.Conn, error)

SetupDbs takes connection parameters for redis and mongo and returns active sessions. The caller is responsible for closing the returned connections.

func UpdateRedisMapReference

func UpdateRedisMapReference(conn redis.Conn, params Params, mapConfig MapConfig) error

UpdateRedisMapReference updates the map specified in redis to point to the newly populated hashes, then deletes the previously referenced hash. The hash reference is updated atomically.

Types

type CollectionConfig

type CollectionConfig struct {
	Collection string      `yaml:"collection"`
	Query      string      `yaml:"query"`
	Projection string      `yaml:"projection"`
	Maps       []MapConfig `yaml:"maps"`
}

CollectionConfig is the config for a specific collection

type Config

type Config struct {
	Name        string             `yaml:"name"`
	Collections []CollectionConfig `yaml:"collections"`
}

Config is the config for the cache

func LoadConfig

func LoadConfig(path string) (Config, error)

LoadConfig takes a path to a config yaml file and loads it into the appropriate structs.

type MapConfig

type MapConfig struct {
	Name          string `yaml:"name"`
	Key           string `yaml:"key"`
	Value         string `yaml:"val"`
	HashKey       string
	KeyTemplate   *template.Template
	ValueTemplate *template.Template
}

MapConfig is the config for a specific map.

type MongoIter

type MongoIter interface {
	Next(result interface{}) bool
	Err() error
	Close() error
}

MongoIter defines an interface that must be met by types we use as mongo iterators. The main purpose of breaking this out into an interface is for ease of mocking in tests.

type Params

type Params map[string]string

Params holds the params the user passes in for substitution into config templates.

func (*Params) Bson

func (p *Params) Bson() bson.M

Bson makes a copy of the Params object in a bson map for interaction with mgo.

func (*Params) Set

func (p *Params) Set(value string) error

Set parses a Param object from a string into the method receiver.

func (*Params) String

func (p *Params) String() string

String gives a string representation of a Params object

type RedisWriter

type RedisWriter interface {
	Send(cmd string, args ...interface{}) error
	Flush() error
}

RedisWriter is an interface for types that can write to redis using send/flush (pipelined operations) The main purpose of breaking this out into an interface is for ease of mocking in tests.

func NewRedisWriter

func NewRedisWriter(conn redis.Conn) RedisWriter

NewRedisWriter creates a new RedisWriter. We wrap redis.Conn here so that we can specify how many documents we want to allow buffered before flushing automatically.