riak

package module
v0.0.0-...-8179165 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 6, 2013 License: Apache-2.0 Imports: 12 Imported by: 0

README

riak (goriakpbc) Build Status

Package riak is a riak-client, inspired by the Ruby riak-client gem and the riakpbc go package from mrb (github.com/mrb/riakpbc). It implements a connection to Riak using Protocol Buffers.

A simple program using goriakpbc:

package main

import (
	"fmt"
	"github.com/tpjg/goriakpbc"
)

func main() {
	err := riak.ConnectClient("127.0.0.1:8087")
	if err != nil {
		fmt.Println("Cannot connect, is Riak running?")
		return
	}

	bucket, _ := riak.NewBucket("tstriak")
	obj := bucket.NewObject("tstobj")
	obj.ContentType = "application/json"
	obj.Data = []byte("{'field':'value'}")
	obj.Store()

	fmt.Printf("Stored an object in Riak, vclock = %v\n", obj.Vclock)

	riak.Close()
}

Parts of the library are specifically designed to facilitate projects that use both Ruby and Go. See the "Document Models" below. To install run go get github.com/tpjg/goriakpbc and use import as in the example above. If the Document Models (ORM) features are not needed simply run rm $GOPATH/src/github.com/tpjg/goriakpbc/model*.go after go get.

Documentation

More documentation is available in the Wiki (https://github.com/tpjg/goriakpbc/wiki), below are some examples of the features implemented in this library. Full API documentation (automatically generated including protobuf definitions) is available at http://go.pkgdoc.org/github.com/tpjg/goriakpbc or through go doc.

Secondary indexes (2i)

Secondary indexes are supported and can be queried for equality using IndexQuery and for a range using IndexQueryRange. Indexes must be added as strings, even when adding a "_int" index. See the example below, taken from riak_test.go:


obj, _ := bucket.NewObject("some_key")
obj.ContentType = "text/plain"
obj.Data = []byte("testing indexes")
obj.Indexes["test_int"] = strconv.Itoa(123)
err := obj.Store()
...
keys, err := bucket.IndexQuery("test_int", strconv.Itoa(123))
...
keys, err = bucket.IndexQueryRange("test_int", strconv.Itoa(120), strconv.Itoa(130))

Using Riak 1.4 and greater, you can pagination through keys in secondary indexes:

keys, continuation, err := bucket.IndexQueryPage("test_int", strconv.Itoa(123), 10, "")
...
keys, continuation, err = bucket.IndexQueryPage("test_int", strconv.Itoa(123), 10, continuation)
...
keys, continuation, err = bucket.IndexQueryRangePage("test_int", strconv.Itoa(120), strconv.Itoa(130), 10, "")
...
keys, continuation, err = bucket.IndexQueryRangePage("test_int", strconv.Itoa(120), strconv.Itoa(130), 10, continuation)

Map Reduce

There is a function to run a MapReduce directly:

func (c *Client) RunMapReduce(query string) (resp [][]byte, err error)

And MapReduce queries can be build similar to how the MapReduce class from the Ruby riak-client works:

mr := riak.NewMapReduce()
mr.Add("bucket", "key")
mr.LinkBucket("otherbucket", false)
mr.Map("function(v) {return [JSON.parse(v.values[0].data)];}", true)
res, err := mr.Run()

Map functions using Erlang instead of Javascript must be added using "MapErlang" instead of "Map" and there is a predefined function "MapObjectValue" that uses the riak_kv_mapreduce module's map_object_value function.

If the backend supports secondary indexes a whole bucket can be added as input to a MapReduce query. Alternatively range queries and single key queries on 2i are also supported:

mr := riak.NewMapReduce()
mr.AddBucket("bucket")
// mr.AddBucketRange("bucket", "a", "k")
// mr.AddIndexRange("bucket", "key", "a", "k")
// mr.AddIndex("bucket", "key", "somekey1234")
mr.MapObjectValue(true)
res, err := mr.Run()
Riak Document Models

Document Models, commonly referred to as ORM (Object-Relational Mapping) in other database drivers, maps Go structs to an object in Riak and supports links between objects. This is done by parsing the JSON data from an object in Riak and mapping it to a struct's fields.

The library allows for easy integration of a Go application into a project that also uses Ruby (on Rails) with the "ripple" gem (https://github.com/basho/ripple). To enable easy integration with Ruby/ripple projects the struct "tag" feature of Go is used to get around the naming convention differences between Go and Ruby (Uppercase starting letter required for export versus Uppercase being constants and typically CamelCase versus snake_case). Also it stores the model/struct name as _type in Riak just like ripple does.

For example the following Ruby/Ripple class:

    class Device
      include Ripple::Document
      property :ip, String
      property :description, String
      property :download_enabled, Boolean
    end

can be mapped to the following Go struct:

    type Device struct {
        DownloadEnabled  bool    `riak:"download_enabled"`
        Ip               string  `riak:"ip"`
        Description      string  `riak:"description"`
        riak.Model       `riak:"devices"`
    }

Note that it is required to have an (anonymous) riak.Model field. If the riak.Model field is an anonymous field this has the benefit that the functions like "Save" or "SaveAs" can be called directly as in the example below.

To get an instantiated struct from Riak would then require only a call to the riak.Client "Load" function, and to store it call "Save" or "SaveAs":

err := riak.ConnectClient("127.0.0.1:8087")
var dev Device 
err = riak.LoadModel("abcdefghijklm", &dev)
dev.Description = "something else"
err = dev.SaveAs("newkey")
Licensing

goriakpbc is distributed under the Apache license, see LICENSE.txt file or http://www.apache.org/licenses/LICENSE-2.0 for details. The model_json_*.go files are a copy from the original Go distribution with minor changes and are governed by a BSD-style license, see LICENSE.go.txt.

Documentation

Overview

Package riak is a riak-client, inspired by the Ruby riak-client gem and the riakpbc go package from mrb. It implements a connection to Riak using protobuf.

Index

Constants

This section is empty.

Variables

View Source
var (
	R1  = map[string]uint32{"r": 1}
	PR1 = map[string]uint32{"pr": 1}
	W1  = map[string]uint32{"w": 1}
	DW1 = map[string]uint32{"dw": 1}
	PW1 = map[string]uint32{"pw": 1}
)

Options for storing and retrieving data, only a few are defined, different values can be supplied by creating a map in the application, for example:

bucket.Get("key", map[string]int{"r":2})
View Source
var (
	BadNumberOfConnections = errors.New("Connection count <= 0")
	BadResponseLength      = errors.New("Response length too short")
	NoBucketName           = errors.New("No bucket name")
	BadMapReduceInputs     = errors.New("MapReduce inputs should be either a (single) index or bucket,key pairs - not both at")
	ChanWaitTimeout        = errors.New("Waiting for an available connection timed out")
)

Error definitions

View Source
var (
	ResolveNotImplemented     = errors.New("Resolve not implemented")
	DestinationError          = errors.New("Destination is not a pointer (to a struct)")
	DestinationIsNotModel     = errors.New("Destination has no riak.Model field")
	DestinationIsNotSlice     = errors.New("Must supply a slice to GetSiblings")
	DestinationLengthError    = errors.New("Length of slice does not match number of siblings")
	DestinationNotInitialized = errors.New("Destination struct is not initialized (correctly) using riak.New or riak.Load")
	ModelDoesNotMatch         = errors.New("Warning: struct name does not match _type in Riak")
	ModelNotNew               = errors.New("Destination struct already has an instantiated riak.Model (this struct is probably not new)")
	NoSiblingData             = errors.New("No non-empty sibling data")
)

Error definitions

View Source
var (
	NoDefaultClientConnection = errors.New("No (default) client connection")
)
View Source
var (
	NotFound = errors.New("Object not found")
)

Error definitions

Functions

func Close

func Close()

Closes the connection of the default client.

func ConnectClient

func ConnectClient(addr string) (err error)

Create the default client with a single connection to Riak.

func ConnectClientPool

func ConnectClientPool(addr string, count int) (err error)

Create the default client, using a pool of connections. This is the recommended method to connect to Riak in an application. A single client instance can be used by multiple threads/goroutines and will not block operations if there are enough connections in the pool. NOTE: If an application needs connections to different Riak clusters it can use riak.NewClientPool or riak.NewClient.

func DeleteFrom

func DeleteFrom(bucketname string, key string, options ...map[string]uint32) (err error)

Delete directly from a bucket, without creating a bucket object first

func ExistsIn

func ExistsIn(bucketname string, key string, options ...map[string]uint32) (exists bool, err error)

Test if an object exists in a bucket directly, without creating a bucket object first

func Id

func Id() (id string, err error)

Get the client Id

func IsWarning

func IsWarning(err error) bool

Return is an error is really a warning, e.g. a common json error, or ModelDoesNotMatch.

func LoadModel

func LoadModel(key string, dest Resolver, options ...map[string]uint32) (err error)

Load data into the model using the default bucket (from the Model's struct definition)

func LoadModelFrom

func LoadModelFrom(bucketname string, key string, dest Resolver, options ...map[string]uint32) (err error)

The LoadModelFrom function retrieves the data from Riak using the default client and stores it in the struct that is passed as destination.

func NewModel

func NewModel(key string, dest Resolver, options ...map[string]uint32) (err error)

Instantiate a new model (using the default client), setting the necessary fields, like the client. If key is not empty that key will be used, otherwise Riak will choose a key.

func NewModelIn

func NewModelIn(bucketname string, key string, dest Resolver, options ...map[string]uint32) (err error)

Create a new Document Model (using the default client), passing in the bucketname and key.

func Ping

func Ping() (err error)

Ping the server

func RunMapReduce

func RunMapReduce(query string) (resp [][]byte, err error)

Run a MapReduce query directly

func ServerVersion

func ServerVersion() (node string, version string, err error)

Get the server version for the default client

func SetId

func SetId(id string) (err error)

Set the client Id

Types

type Bucket

type Bucket struct {
	// contains filtered or unexported fields
}

Implements access to a bucket and its properties

func NewBucket

func NewBucket(name string) (*Bucket, error)

Return a new bucket using the client connection

func (*Bucket) AllowMult

func (b *Bucket) AllowMult() bool

Return the allowMult property of a bucket

func (*Bucket) Delete

func (b *Bucket) Delete(key string, options ...map[string]uint32) (err error)

Delete a key/value from the bucket

func (*Bucket) Exists

func (b *Bucket) Exists(key string, options ...map[string]uint32) (exists bool, err error)

Test if an object exists

func (*Bucket) Get

func (b *Bucket) Get(key string, options ...map[string]uint32) (obj *RObject, err error)

Get an object

func (*Bucket) IndexQuery

func (b *Bucket) IndexQuery(index string, key string) (keys []string, err error)

Return a list of keys using the index for a single key

func (*Bucket) IndexQueryPage

func (b *Bucket) IndexQueryPage(index string, key string, results uint32, continuation string) (keys []string, next string, err error)

Return a page of keys using the index for a single key

func (*Bucket) IndexQueryRange

func (b *Bucket) IndexQueryRange(index string, min string, max string) (keys []string, err error)

Return a list of keys using the index range query

func (*Bucket) IndexQueryRangePage

func (b *Bucket) IndexQueryRangePage(index string, min string, max string, results uint32, continuation string) (keys []string, next string, err error)

Return a page of keys using the index range query

func (*Bucket) ListKeys

func (b *Bucket) ListKeys() (response [][]byte, err error)

List all keys from bucket

func (*Bucket) NVal

func (b *Bucket) NVal() uint32

Return the nval property of a bucket

func (*Bucket) Name

func (b *Bucket) Name() string

Return the bucket name

func (*Bucket) New

func (b *Bucket) New(key string, options ...map[string]uint32) *RObject

Create a new RObject. DEPRECATED, use NewObject instead

func (*Bucket) NewObject

func (b *Bucket) NewObject(key string, options ...map[string]uint32) *RObject

Create a new RObject

func (*Bucket) SetAllowMult

func (b *Bucket) SetAllowMult(allowMult bool) (err error)

Set the allowMult property of a bucket

func (*Bucket) SetNVal

func (b *Bucket) SetNVal(nval uint32) (err error)

Set the nval property of a bucket

type Client

type Client struct {
	// contains filtered or unexported fields
}

riak.Client the client interface

func New

func New(addr string) *Client

Returns a new Client connection. DEPRECATED, use NewClient instead

func NewClient

func NewClient(addr string) *Client

Returns a new Client connection

func NewClientPool

func NewClientPool(addr string, count int) *Client

Returns a new Client with multiple connections to Riak

func NewPool

func NewPool(addr string, count int) *Client

Returns a new Client with multiple connections to Riak. DEPRECATED, use NewClientPool instead

func (*Client) Bucket

func (c *Client) Bucket(name string) (*Bucket, error)

Return a new bucket object. DEPRECATED, use NewBucket instead.

func (*Client) Close

func (c *Client) Close()

Close the connection

func (*Client) Connect

func (c *Client) Connect() (err error)

Connects to a Riak server.

func (*Client) DeleteFrom

func (c *Client) DeleteFrom(bucketname string, key string, options ...map[string]uint32) (err error)

Delete directly from a bucket, without creating a bucket object first

func (*Client) ExistsIn

func (c *Client) ExistsIn(bucketname string, key string, options ...map[string]uint32) (exists bool, err error)

Test if an object exists in a bucket directly, without creating a bucket object first

func (*Client) GetFrom

func (c *Client) GetFrom(bucketname string, key string, options ...map[string]uint32) (obj *RObject, err error)

Get directly from a bucket, without creating a bucket first

func (*Client) Id

func (c *Client) Id() (id string, err error)

Get the client Id

func (*Client) Key

func (c *Client) Key(dest interface{}) (key string, err error)

Get a models Key, e.g. needed when Riak has picked it

func (*Client) Load

func (c *Client) Load(bucketname string, key string, dest Resolver, options ...map[string]uint32) (err error)

Load data into model. DEPRECATED, use LoadModelFrom instead.

func (*Client) LoadModel

func (c *Client) LoadModel(key string, dest Resolver, options ...map[string]uint32) (err error)

Load data into the model using the default bucket (from the Model's struct definition)

func (*Client) LoadModelFrom

func (c *Client) LoadModelFrom(bucketname string, key string, dest Resolver, options ...map[string]uint32) (err error)

The LoadModelFrom function retrieves the data from Riak and stores it in the struct that is passed as destination. It stores some necessary information in the riak.Model field so it can be used later in other (Save) operations.

If the bucketname is empty ("") it'll be the default bucket, based on the riak.Model tag.

Using the "Device" struct as an example:

dev := &Device{} err := client.Load("devices", "12345", dev)

func (*Client) MapReduce

func (c *Client) MapReduce() *MapReduce

func (*Client) New

func (c *Client) New(bucketname string, key string, dest Resolver, options ...map[string]uint32) (err error)

Create a new model. DEPRECATED, use NewModelIn instead.

func (*Client) NewBucket

func (c *Client) NewBucket(name string) (*Bucket, error)

Return a new bucket object

func (*Client) NewModel

func (c *Client) NewModel(key string, dest Resolver, options ...map[string]uint32) (err error)

Instantiate a new model, setting the necessary fields, like the client. If key is not empty that key will be used, otherwise Riak will choose a key.

func (*Client) NewModelIn

func (c *Client) NewModelIn(bucketname string, key string, dest Resolver, options ...map[string]uint32) (err error)

Create a new Document Model, passing in the bucketname and key. The key can be empty in which case Riak will pick a key. The destination must be a pointer to a struct that has the riak.Model field. If the bucketname is empty the default bucketname, based on the riak.Model tag will be used.

func (*Client) NewObjectIn

func (c *Client) NewObjectIn(bucketname string, key string, options ...map[string]uint32) (*RObject, error)

Create a new RObject in a bucket directly, without creating a bucket object first

func (*Client) Ping

func (c *Client) Ping() (err error)

Ping the server

func (*Client) RunMapReduce

func (c *Client) RunMapReduce(query string) (resp [][]byte, err error)

Run a MapReduce query

func (*Client) Save

func (c *Client) Save(dest Resolver) (err error)

Save a Document Model to Riak

func (*Client) SaveAs

func (c *Client) SaveAs(newKey string, dest Resolver) (err error)

Save a Document Model to Riak under a new key, if empty a Key will be choosen by Riak

func (*Client) ServerVersion

func (c *Client) ServerVersion() (node string, version string, err error)

Get the server version

func (*Client) SetChanWaitTimeout

func (c *Client) SetChanWaitTimeout(waitTimeout int)

Set the maximum time (in milliseconds) to wait for a connection to be available in the pool. By default getConn() will wait forever.

func (*Client) SetId

func (c *Client) SetId(id string) (err error)

Set the client Id

func (*Client) SetKey

func (c *Client) SetKey(newKey string, dest interface{}) (err error)

Set the Key value, note that this does not save the model, it only changes the data structure

type Link struct {
	Bucket string
	Key    string
	Tag    string
}

A Riak link

type Many

type Many []One

Link to many other models

func (*Many) Add

func (m *Many) Add(dest Resolver) (err error)

Add a Link to the given Model (dest)

func (*Many) Len

func (m *Many) Len() int

Return the number of Links

type MapReduce

type MapReduce struct {
	// contains filtered or unexported fields
}

An object to build a MapReduce job similar to how the Ruby client can build it by adding different stages.

func NewMapReduce

func NewMapReduce() *MapReduce

Create a MapReduce object that can be used to build a MR query

func (*MapReduce) Add

func (mr *MapReduce) Add(bucket string, key string) (err error)

func (*MapReduce) AddBucket

func (mr *MapReduce) AddBucket(bucket string) (err error)

Add a whole bucket as input. Note that this ONLY works on buckets that have secondary indexes (2i) enabled since listing keys on a bucket without using indexes is dangerous on production clusters.

func (*MapReduce) AddBucketRange

func (mr *MapReduce) AddBucketRange(bucket string, start string, end string) (err error)

Add a range of keys from one bucket using secondary indexes.

func (*MapReduce) AddIndex

func (mr *MapReduce) AddIndex(bucket string, index string, key string) (err error)

Add a keys using a secondary index.

func (*MapReduce) AddIndexRange

func (mr *MapReduce) AddIndexRange(bucket string, index string, start string, end string) (err error)

Add a range of keys using a secondary index.

func (*MapReduce) LinkBucket

func (mr *MapReduce) LinkBucket(name string, keep bool)

func (*MapReduce) Map

func (mr *MapReduce) Map(fun string, keep bool)

func (*MapReduce) MapErlang

func (mr *MapReduce) MapErlang(module string, fun string, keep bool)

func (*MapReduce) MapObjectValue

func (mr *MapReduce) MapObjectValue(keep bool)

func (*MapReduce) Query

func (mr *MapReduce) Query() (query []byte, err error)

Generate the Query string

func (*MapReduce) Reduce

func (mr *MapReduce) Reduce(fun string, keep bool)

func (*MapReduce) Run

func (mr *MapReduce) Run() (resp [][]byte, err error)

type Model

type Model struct {
	// contains filtered or unexported fields
}

Make structs work like a Document Model, similar to how the Ruby based "ripple" gem works. This is done by parsing the JSON data and mapping it to the struct's fields. To enable easy integration with Ruby/ripple projects the struct "tag" feature of Go is used to possibly get around the naming convention differences between Go and Ruby (Uppercase starting letter required for export and typically CamelCase versus underscores). Also it stores the model/struct name as _type in Riak.

For example the following Ruby/Ripple class:

class Device
  include Ripple::Document
  property :ip, String
  property :description, String
  property :download_enabled, Boolean
end

can be mapped to the following Go class:

type Device struct {
	riak.Model       `riak:devices`
	Download_enabled bool    `riak:"download_enabled"`
	Ip               string  `riak:"ip"`
	Description      string  `riak:"description"`
}

Note that it is required to have a riak.Model field. Also if the field name in Ripple is equal the extra tag is not needed, (e.g. if the Ripple class above would have a "property :Ip, String").

func (*Model) Delete

func (m *Model) Delete() (err error)

Delete a Document Model

func (*Model) GetSiblings

func (m *Model) GetSiblings(dest interface{}) (err error)

func (*Model) Indexes

func (m *Model) Indexes() map[string]string

Return the object's indexes. This allows an application to set custom secondary indexes on the object for later querying.

func (Model) Key

func (m Model) Key() (key string)

Get a models Key, e.g. needed when Riak has picked it

func (*Model) Reload

func (m *Model) Reload() (err error)

Reload a Document Model

func (*Model) Resolve

func (*Model) Resolve(count int) (err error)

func (*Model) Save

func (m *Model) Save() (err error)

Save a Document Model to Riak

func (*Model) SaveAs

func (m *Model) SaveAs(newKey string) (err error)

Save a Document Model to Riak under a new key, if empty a Key will be choosen by Riak

func (Model) SetKey

func (m Model) SetKey(newKey string) (err error)

Set the Key value, note that this does not save the model, it only changes the data structure

func (*Model) Siblings

func (m *Model) Siblings(dest Resolver) (result interface{}, err error)

Allocates a slice of models for the result and populates it with the data from the siblings held in Riak.

func (*Model) Vclock

func (m *Model) Vclock() (vclock []byte)

Return the object Vclock - this allows an application to detect whether Reload() loaded a newer version of the object

type One

type One struct {
	// contains filtered or unexported fields
}

Link to one other model

func (*One) Get

func (o *One) Get(dest Resolver) (err error)
func (o One) Link() (link Link)

func (*One) Set

func (o *One) Set(dest Resolver) (err error)

Set the link to a given Model (dest)

type RObject

type RObject struct {
	Bucket      *Bucket
	Vclock      []byte
	Key         string
	ContentType string
	Data        []byte
	Links       []Link
	Meta        map[string]string
	Indexes     map[string]string

	Siblings []Sibling
	Options  []map[string]uint32
	// contains filtered or unexported fields
}

An RObject is an object or document that is or can be stored in Riak

func GetFrom

func GetFrom(bucketname string, key string, options ...map[string]uint32) (obj *RObject, err error)

Get directly from a bucket, without creating a bucket first

func NewObjectIn

func NewObjectIn(bucketname string, key string, options ...map[string]uint32) (*RObject, error)

Create a new RObject in a bucket directly, without creating a bucket object first

func (obj *RObject) AddLink(link Link) bool

Add a link if it is not already in the Links slics, returns false if already present

func (*RObject) Conflict

func (obj *RObject) Conflict() bool

Returns true if the object was fetched with multiple siblings (AllowMult=true on the bucket)

func (*RObject) Destroy

func (obj *RObject) Destroy() (err error)

Delete the object from Riak

func (*RObject) LinkTo

func (obj *RObject) LinkTo(target *RObject, tag string)

Add a link to another object (does not store the object, must explicitly call "Store()")

func (*RObject) Reload

func (obj *RObject) Reload() (err error)

Reload an object if it has changed (new Vclock)

func (*RObject) Store

func (obj *RObject) Store() (err error)

Store an RObject

type Resolver

type Resolver interface {
	Resolve(int) error
}

type Sibling

type Sibling struct {
	ContentType string
	Data        []byte
	Links       []Link
	Meta        map[string]string
	Indexes     map[string]string
}

An object van have siblings that can each have their own content

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL