collection

Documentation

Index

• Constants
• Variables
• func IsErrExists(e error) bool
• func IsErrMalformedValue(e error) bool
• func IsErrNotFound(e error) bool
• func NewDryrunSTM(ctx context.Context, c *v3.Client, apply func(STM) error) error
• func NewSTM(ctx context.Context, c *v3.Client, apply func(STM) error) (*v3.TxnResponse, error)
• type Collection
  • func NewCollection(etcdClient *etcd.Client, prefix string, indexes []*Index, ...) Collection
• type ErrExists
  • func (e ErrExists) Error() string
• type ErrMalformedValue
  • func (e ErrMalformedValue) Error() string
• type ErrNotFound
  • func (e ErrNotFound) Error() string
• type Index
• type Options
• type ReadWriteCollection
• type ReadWriteIntCollection
• type ReadonlyCollection
• type STM

Constants

const (
	DefaultPrefix string = "pachyderm/1.7.0"
)

defaultLimit was experimentally determined to be the highest value that could work (It gets scaled down for specific collections if they trip the max-message size.)

Variables

var DefaultOptions = &Options{etcd.SortByCreateRevision, etcd.SortDescend, false}

DefaultOptions are the default sort options when iterating through etcd key/values.

var (
	// ErrNotClaimed is an error used to indicate that a different requester beat
	// the current requester to a key claim.
	ErrNotClaimed = fmt.Errorf("NOT_CLAIMED")
)

Functions

func IsErrExists

func IsErrExists(e error) bool

IsErrExists determines if an error is an ErrExists error

func IsErrMalformedValue

func IsErrMalformedValue(e error) bool

IsErrMalformedValue determines if an error is an ErrMalformedValue error

func IsErrNotFound

func IsErrNotFound(e error) bool

IsErrNotFound determines if an error is an ErrNotFound error

func NewDryrunSTM

func NewDryrunSTM(ctx context.Context, c *v3.Client, apply func(STM) error) error

NewDryrunSTM intiates a new STM operation, but the final commit is skipped. It uses a serializable model.

func NewSTM

func NewSTM(ctx context.Context, c *v3.Client, apply func(STM) error) (*v3.TxnResponse, error)

NewSTM intiates a new STM operation. It uses a serializable model.

Types

type Collection

type Collection interface {
	// Path returns the full etcd path of the given key in the collection
	Path(string) string
	// ReadWrite enables reads and writes on a collection in a
	// transactional manner.  Specifically, all writes are applied
	// atomically, and writes are only applied if reads have not been
	// invalidated at the end of the transaction.  Basically, it's
	// software transactional memory.  See this blog post for details:
	// https://coreos.com/blog/transactional-memory-with-etcd3.html
	ReadWrite(stm STM) ReadWriteCollection
	// ReadWriteInt is the same as ReadWrite except that it operates on
	// integral items, as opposed to protobuf items
	ReadWriteInt(stm STM) ReadWriteIntCollection
	// For read-only operatons, use the ReadOnly for better performance
	ReadOnly(ctx context.Context) ReadonlyCollection
	// Claim attempts to claim a key and run the passed in callback with
	// the context for the claim.
	Claim(ctx context.Context, key string, val proto.Message, f func(context.Context) error) error
}

Collection implements helper functions that makes common operations on top of etcd more pleasant to work with. It's called collection because most of our data is modelled as collections, such as repos, commits, branches, etc.

func NewCollection

func NewCollection(etcdClient *etcd.Client, prefix string, indexes []*Index, template proto.Message, keyCheck func(string) error, valCheck func(proto.Message) error) Collection

NewCollection creates a new collection.

type ErrExists

type ErrExists struct {
	Type string
	Key  string
}

ErrExists indicates that a key was found to exist when it was expected not to.

func (ErrExists) Error

func (e ErrExists) Error() string

type ErrMalformedValue

type ErrMalformedValue struct {
	Type string
	Key  string
	Val  string
}

ErrMalformedValue indicates that a value was malformed, such as when it was supposed to be parseable as an int but wasn't.

func (ErrMalformedValue) Error

func (e ErrMalformedValue) Error() string

type ErrNotFound

type ErrNotFound struct {
	Type string
	Key  string
}

ErrNotFound indicates that a key was not found when it was expected to exist.

func (ErrNotFound) Error

func (e ErrNotFound) Error() string

type Index

type Index struct {
	Field string
	Multi bool
	// contains filtered or unexported fields
}

Index specifies a secondary index on a collection.

Indexes are created in a transactional manner thanks to etcd's transactional support.

A secondary index for collection "foo" on field "bar" will reside under the path `/foo__index_bar`. Each item under the path is in turn a directory whose name is the value of the field `bar`. For instance, if you have a object in collection `foo` whose `bar` field is `test`, then you will see a directory at path `/foo__index_bar/test`.

Under that directory, you have keys that point to items in the collection. For instance, if the aforementioned object has the key "buzz", then you will see an item at `/foo__index_bar/test/buzz`. The value of this item is empty. Thus, to get all items in collection `foo` whose values of field `bar` is `test`, we issue a query for all items under `foo__index_bar/test`.

Multi specifies whether this is a multi-index. A multi-index is an index on a field that's a slice. The item is then indexed on each element of the slice.

type Options

type Options struct {
	Target   etcd.SortTarget
	Order    etcd.SortOrder
	SelfSort bool
}

Options are the sort options when iterating through etcd key/values. Currently implemented sort targets are CreateRevision and ModRevision. The sorting can be done in the calling process by setting SelfSort to true.

type ReadWriteCollection

type ReadWriteCollection interface {
	Get(key string, val proto.Message) error
	Put(key string, val proto.Message) error
	// TTL returns the amount of time that 'key' will continue to exist in the
	// collection, or '0' if 'key' will remain in the collection indefinitely
	TTL(key string) (int64, error)
	// PutTTL is the same as Put except that the object is removed after
	// TTL seconds.
	// WARNING: using PutTTL with a collection that has secondary indices
	// can result in inconsistency, as the indices are removed at roughly
	// but not exactly the same time as the documents.
	PutTTL(key string, val proto.Message, ttl int64) error
	// Update reads the current value associated with 'key', calls 'f' to update
	// the value, and writes the new value back to the collection. 'key' must be
	// present in the collection, or a 'Not Found' error is returned
	Update(key string, val proto.Message, f func() error) error
	// Upsert is like Update but 'key' is not required to be present
	Upsert(key string, val proto.Message, f func() error) error
	Create(key string, val proto.Message) error
	Delete(key string) error
	DeleteAll()
	DeleteAllPrefix(prefix string)
}

ReadWriteCollection is a collection interface that supports read,write and delete operations.

type ReadWriteIntCollection

type ReadWriteIntCollection interface {
	Create(key string, val int) error
	Get(key string) (int, error)
	Increment(key string) error
	IncrementBy(key string, n int) error
	Decrement(key string) error
	DecrementBy(key string, n int) error
	Delete(key string) error
}

ReadWriteIntCollection is a ReadonlyCollection interface specifically for ints.

type ReadonlyCollection

type ReadonlyCollection interface {
	Get(key string, val proto.Message) error
	GetByIndex(index *Index, indexVal interface{}, val proto.Message, opts *Options, f func(key string) error) error
	// GetBlock is like Get but waits for the key to exist if it doesn't already.
	GetBlock(key string, val proto.Message) error
	// TTL returns the number of seconds that 'key' will continue to exist in the
	// collection, or '0' if 'key' will remain in the collection indefinitely
	TTL(key string) (int64, error)
	List(val proto.Message, opts *Options, f func(key string) error) error
	ListPrefix(prefix string, val proto.Message, opts *Options, f func(string) error) error
	Count() (int64, error)
	Watch(opts ...watch.OpOption) (watch.Watcher, error)
	WatchOne(key string) (watch.Watcher, error)
	WatchOneF(key string, f func(*watch.Event) error) error
	WatchByIndex(index *Index, val interface{}) (watch.Watcher, error)
}

ReadonlyCollection is a collection interface that only supports read ops.

type STM

type STM interface {
	// Get returns the value for a key and inserts the key in the txn's read set.
	// If Get fails, it aborts the transaction with an error, never returning.
	Get(key string) (string, error)
	// Put adds a value for a key to the write set.
	Put(key, val string, ttl int64, ptr uintptr) error
	// Rev returns the revision of a key in the read set.
	Rev(key string) int64
	// Del deletes a key.
	Del(key string)
	// TTL returns the remaining time to live for 'key', or 0 if 'key' has no TTL
	TTL(key string) (int64, error)
	// DelAll deletes all keys with the given prefix
	// Note that the current implementation of DelAll is incomplete.
	// To use DelAll safely, do not issue any Get/Put operations after
	// DelAll is called.
	DelAll(key string)
	Context() context.Context
	// SetSafePutCheck sets the bit pattern to check if a put is safe.
	SetSafePutCheck(key string, ptr uintptr)
	// IsSafePut checks against the bit pattern for a key to see if it is safe to put.
	IsSafePut(key string, ptr uintptr) bool
	// contains filtered or unexported methods
}

STM is an interface for software transactional memory.