manipulate

package module
v1.54.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 17, 2019 License: Apache-2.0 Imports: 7 Imported by: 42

README

Manipulate

codecov

README IS A WORK IN PROGRESS AS WE ARE WRITTING MORE DOCUMENTATION ABOUT THIS PACKAGE.

Package manipulate provides everything needed to perform CRUD operations on an elemental based data model.

The main interface is Manipulator. This interface provides various methods for creation, modification, retrieval and so on.

A Manipulator works with elemental.Identifiable.

The storage engine used by a Manipulator is abstracted. By default manipulate provides implementations for Mongo, ReST HTTP, Websocket and a Memory backed datastore. You can of course implement Your own storage implementation.

Each method of a Manipulator is taking a manipulate.Context as argument. The context is used to pass additional informations like a Filter, or some Parameters.

Example for creating an object

// Create a User from a generated Elemental model.
user := models.NewUser() // always use the initializer to get various default value correctly set.
user.FullName := "Antoine Mercadal"
user.Login := "primalmotion"

// Create Mongo Manipulator.
m := manipmongo.New("127.0.0.1", "test")

// Then create the User.
m.Create(nil, user)

Example for retreving an object

// Create a Context with a filter.
ctx := manipulate.NewContextWithFilter(manipulate.NewFilterComposer().
    WithKey("login").Equals("primalmotion").
    Done(),
)

// Retrieve the users matching the filter.
var users models.UserLists
m.RetrieveMany(ctx, &users)

Documentation

Overview

Package manipulate provides everything needed to perform CRUD operations on an https://go.aporeto.io/elemental based data model.

The main interface is Manipulator. This interface provides various methods for creation, modification, retrieval and so on. TransactionalManipulator, which is an extension of the Manipulator add methods to manage transactions, like Commit and Abort.

A Manipulator works with some elemental.Identifiables.

The storage engine used by a Manipulator is abstracted. By default manipulate provides implementations for Rest API over HTTP or websocket, Mongo DB, Memory and a mock Manipulator for unit testing. You can of course create your own implementation.

Each method of a Manipulator is taking a manipulate.Context as argument. The context is used to pass additional informations like a Filter or some Parameters.

Example for creating an object:

// Create a User from a generated Elemental model.
user := models.NewUser()
user.FullName, user.Login := "Antoine Mercadal", "primalmotion"

// Create Mongo Manipulator.
m := manipmongo.NewMongoManipulator([]{"127.0.0.1"}, "test", "db-username", "db-password", "db-authsource", 512)

// Then create the User.
m.Create(nil, user)

Example for retreving an object:

// Create a Context with a filter.
ctx := manipulate.NewContextWithFilter(
    manipulate.NewFilterComposer().WithKey("login").Equals("primalmotion").
    Done())

// Retrieve the users matching the filter.
var users models.UserLists
m.RetrieveMany(ctx, models.UserIdentity, &users)

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsCannotBuildQueryError

func IsCannotBuildQueryError(err error) bool

IsCannotBuildQueryError returns true if the given error is am ErrCannotBuildQuery.

func IsCannotCommitError

func IsCannotCommitError(err error) bool

IsCannotCommitError returns true if the given error is am ErrCannotCommit.

func IsCannotCommunicateError

func IsCannotCommunicateError(err error) bool

IsCannotCommunicateError returns true if the given error is am ErrCannotCommunicate.

func IsCannotExecuteQueryError

func IsCannotExecuteQueryError(err error) bool

IsCannotExecuteQueryError returns true if the given error is am ErrCannotExecuteQuery.

func IsCannotMarshalError

func IsCannotMarshalError(err error) bool

IsCannotMarshalError returns true if the given error is am ErrCannotMarshal.

func IsCannotUnmarshalError

func IsCannotUnmarshalError(err error) bool

IsCannotUnmarshalError returns true if the given error is am ErrCannotUnmarshal.

func IsConstraintViolationError

func IsConstraintViolationError(err error) bool

IsConstraintViolationError returns true if the given error is am ErrConstraintViolation.

func IsDisconnectedError

func IsDisconnectedError(err error) bool

IsDisconnectedError returns true if the given error is am ErrDisconnected.

func IsLockedError

func IsLockedError(err error) bool

IsLockedError returns true if the given error is am ErrLocked.

func IsMultipleObjectsFoundError

func IsMultipleObjectsFoundError(err error) bool

IsMultipleObjectsFoundError returns true if the given error is am ErrMultipleObjectsFound.

func IsNotImplementedError

func IsNotImplementedError(err error) bool

IsNotImplementedError returns true if the given error is am ErrNotImplemented.

func IsObjectNotFoundError

func IsObjectNotFoundError(err error) bool

IsObjectNotFoundError returns true if the given error is am ErrObjectNotFound.

func IsTLSError

func IsTLSError(err error) bool

IsTLSError returns true if the given error is am ErrTLS.

func IsTooManyRequestsError

func IsTooManyRequestsError(err error) bool

IsTooManyRequestsError returns true if the given error is am ErrTooManyRequests.

func IsTransactionNotFoundError

func IsTransactionNotFoundError(err error) bool

IsTransactionNotFoundError returns true if the given error is am ErrTransactionNotFound.

func Iter

func Iter(
	ctx context.Context,
	m Manipulator,
	mctx Context,
	identifiablesTemplate elemental.Identifiables,
	blockSize int,
) (elemental.Identifiables, error)

Iter is a helper function for IterFunc.

It will simply iterates on the object with identity of the given elemental.Identifiables. Not that this function cannot populate the data in the identifiable parameter. Instead It will return the destination.

Always pass an empty elemental.Identifiables to this function

For more information, please check IterFunc documentation.

Example:

dest, err := Iter(context.Background(), m, mctx, model.ThingsList{}, 100)

func IterFunc

func IterFunc(
	ctx context.Context,
	manipulator Manipulator,
	identifiablesTemplate elemental.Identifiables,
	mctx Context,
	iteratorFunc func(block elemental.Identifiables) error,
	blockSize int,
) error

IterFunc calls RetrieveMany on the given Manipulator, and will retrieve the data by block of the given blockSize.

IterFunc will naturally ends and return when there is no more data to pull.

For each retrieved block, the given func will be called with the current data block. If the function returns an error, the error is returned to the caller of IterFunc and the iteration stops.

The given context will be used if the underlying manipulator honors it.

The given manipulate.Context will be used to retry any failed batch recovery.

The identifiablesTemplate parameter is must be an empty elemental.Identifiables that will be used to hold the data block. It is reset at every iteration. Do not rely on it to be filled once IterFunc is complete.

Finally, if the given blockSize is <= 0, then it will use the default that is 10000.

func Retry

func Retry(ctx context.Context, manipulateFunc func() error, onRetryFunc func(int, error) error) error

Retry will retry the given function that performs a manipulate operation if it fails and the error is a manipulate.ErrCannotCommunicate.

It will retry with exponential backoff (up to 8s) until the provided context is canceled.

If the onRetryFunc is not nil and returns an error, the retrying process will be interrupted and manipulate.Retry will return the provided error. The retry function gets the retry number and the error produced by manipulateFunc.

Example:

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

if err := manipulate.Retry(
    ctx,
    func() error {
        return m.Create(nil, obj)
    },
    func(t int, e error) error {
        if _, ok := e.(manipulate.ErrLocked); ok {
            return errors.New("nah, I don't wanna retry")
        }
        return nil
    },
); err != nil {
    // do interesting stuff.
}

Types

type BufferedManipulator

type BufferedManipulator interface {
	FlushableManipulator
	Manipulator
}

A BufferedManipulator is a Manipulator with a local cache

type Context

type Context interface {
	Count() int
	SetCount(count int)
	Filter() *elemental.Filter
	Finalizer() FinalizerFunc
	Version() int
	TransactionID() TransactionID
	Page() int
	PageSize() int
	Override() bool
	Recursive() bool
	Namespace() string
	Credentials() (string, string)
	Parameters() url.Values
	Parent() elemental.Identifiable
	ExternalTrackingID() string
	ExternalTrackingType() string
	Order() []string
	Context() context.Context
	Derive(...ContextOption) Context
	Fields() []string
	ReadConsistency() ReadConsistency
	WriteConsistency() WriteConsistency
	Messages() []string
	SetMessages([]string)
	ClientIP() string

	fmt.Stringer
}

A Context holds all information regarding a particular manipulate operation.

func NewContext

func NewContext(ctx context.Context, options ...ContextOption) Context

NewContext creates a context with the given ContextOption.

type ContextOption

type ContextOption func(*mcontext)

ContextOption represents an option can can be passed to NewContext.

func ContextOptionClientIP added in v1.54.1

func ContextOptionClientIP(clientIP string) ContextOption

ContextOptionClientIP sets the optional headers for the request.

func ContextOptionCredentials added in v1.54.1

func ContextOptionCredentials(username, password string) ContextOption

ContextOptionCredentials sets user name and password for this context.

func ContextOptionFields

func ContextOptionFields(fields []string) ContextOption

ContextOptionFields sets the list of fields to include in the reply.

func ContextOptionFilter

func ContextOptionFilter(f *elemental.Filter) ContextOption

ContextOptionFilter sets the filter.

func ContextOptionFinalizer

func ContextOptionFinalizer(f FinalizerFunc) ContextOption

ContextOptionFinalizer sets the create finalizer option of the context.

func ContextOptionNamespace

func ContextOptionNamespace(n string) ContextOption

ContextOptionNamespace sets the namespace.

func ContextOptionOrder

func ContextOptionOrder(orders ...string) ContextOption

ContextOptionOrder sets the ordering option of the context.

func ContextOptionOverride

func ContextOptionOverride(o bool) ContextOption

ContextOptionOverride sets the override option of the context.

func ContextOptionPage

func ContextOptionPage(n, size int) ContextOption

ContextOptionPage sets the pagination option of the context.

func ContextOptionParameters

func ContextOptionParameters(p url.Values) ContextOption

ContextOptionParameters sets the parameters option of the context.

func ContextOptionParent

func ContextOptionParent(i elemental.Identifiable) ContextOption

ContextOptionParent sets the parent option of the context.

func ContextOptionReadConsistency

func ContextOptionReadConsistency(consistency ReadConsistency) ContextOption

ContextOptionReadConsistency sets the desired read consistency of the request.

func ContextOptionRecursive

func ContextOptionRecursive(r bool) ContextOption

ContextOptionRecursive sets the recursive option of the context.

func ContextOptionToken added in v1.54.1

func ContextOptionToken(token string) ContextOption

ContextOptionToken sets the token for this request.

func ContextOptionTracking

func ContextOptionTracking(identifier, typ string) ContextOption

ContextOptionTracking sets the opentracing tracking option of the context.

func ContextOptionTransationID

func ContextOptionTransationID(tid TransactionID) ContextOption

ContextOptionTransationID sets the parameters option of the context.

func ContextOptionVersion

func ContextOptionVersion(v int) ContextOption

ContextOptionVersion sets the version option of the context.

func ContextOptionWriteConsistency

func ContextOptionWriteConsistency(consistency WriteConsistency) ContextOption

ContextOptionWriteConsistency sets the desired write consistency of the request.

type ErrCannotBuildQuery

type ErrCannotBuildQuery struct {
	// contains filtered or unexported fields
}

ErrCannotBuildQuery represents query building error.

func NewErrCannotBuildQuery

func NewErrCannotBuildQuery(message string) ErrCannotBuildQuery

NewErrCannotBuildQuery returns a new ErrCannotBuildQuery.

func (ErrCannotBuildQuery) Error

func (e ErrCannotBuildQuery) Error() string

type ErrCannotCommit

type ErrCannotCommit struct {
	// contains filtered or unexported fields
}

ErrCannotCommit represents commit execution error.

func NewErrCannotCommit

func NewErrCannotCommit(message string) ErrCannotCommit

NewErrCannotCommit returns a new ErrCannotCommit.

func (ErrCannotCommit) Error

func (e ErrCannotCommit) Error() string

type ErrCannotCommunicate

type ErrCannotCommunicate struct {
	// contains filtered or unexported fields
}

ErrCannotCommunicate represents a failure in backend communication.

func NewErrCannotCommunicate

func NewErrCannotCommunicate(message string) ErrCannotCommunicate

NewErrCannotCommunicate returns a new ErrCannotCommunicate.

func (ErrCannotCommunicate) Error

func (e ErrCannotCommunicate) Error() string

type ErrCannotExecuteQuery

type ErrCannotExecuteQuery struct {
	// contains filtered or unexported fields
}

ErrCannotExecuteQuery represents query execution error.

func NewErrCannotExecuteQuery

func NewErrCannotExecuteQuery(message string) ErrCannotExecuteQuery

NewErrCannotExecuteQuery returns a new ErrCannotExecuteQuery.

func (ErrCannotExecuteQuery) Error

func (e ErrCannotExecuteQuery) Error() string

type ErrCannotMarshal

type ErrCannotMarshal struct {
	// contains filtered or unexported fields
}

ErrCannotMarshal represents marshaling error.

func NewErrCannotMarshal

func NewErrCannotMarshal(message string) ErrCannotMarshal

NewErrCannotMarshal returns a new ErrCannotMarshal.

func (ErrCannotMarshal) Error

func (e ErrCannotMarshal) Error() string

type ErrCannotUnmarshal

type ErrCannotUnmarshal struct {
	// contains filtered or unexported fields
}

ErrCannotUnmarshal represents unmarshaling error.

func NewErrCannotUnmarshal

func NewErrCannotUnmarshal(message string) ErrCannotUnmarshal

NewErrCannotUnmarshal returns a new ErrCannotUnmarshal.

func (ErrCannotUnmarshal) Error

func (e ErrCannotUnmarshal) Error() string

type ErrConstraintViolation

type ErrConstraintViolation struct {
	// contains filtered or unexported fields
}

ErrConstraintViolation represents a failure to find a transaction.

func NewErrConstraintViolation

func NewErrConstraintViolation(message string) ErrConstraintViolation

NewErrConstraintViolation returns a new ErrConstraintViolation.

func (ErrConstraintViolation) Error

func (e ErrConstraintViolation) Error() string

type ErrDisconnected

type ErrDisconnected struct {
	// contains filtered or unexported fields
}

ErrDisconnected represents an error due user disconnection.

func NewErrDisconnected

func NewErrDisconnected(message string) ErrDisconnected

NewErrDisconnected returns a new ErrDisconnected.

func (ErrDisconnected) Error

func (e ErrDisconnected) Error() string

type ErrLocked

type ErrLocked struct {
	// contains filtered or unexported fields
}

ErrLocked represents the error returned when the server api is locked..

func NewErrLocked

func NewErrLocked(message string) ErrLocked

NewErrLocked returns a new ErrCannotCommunicate.

func (ErrLocked) Error

func (e ErrLocked) Error() string

type ErrMultipleObjectsFound

type ErrMultipleObjectsFound struct {
	// contains filtered or unexported fields
}

ErrMultipleObjectsFound represents too many object found error.

func NewErrMultipleObjectsFound

func NewErrMultipleObjectsFound(message string) ErrMultipleObjectsFound

NewErrMultipleObjectsFound returns a new ErrMultipleObjectsFound.

func (ErrMultipleObjectsFound) Error

func (e ErrMultipleObjectsFound) Error() string

type ErrNotImplemented

type ErrNotImplemented struct {
	// contains filtered or unexported fields
}

ErrNotImplemented represents a non implemented function.

func NewErrNotImplemented

func NewErrNotImplemented(message string) ErrNotImplemented

NewErrNotImplemented returns a new ErrNotImplemented.

func (ErrNotImplemented) Error

func (e ErrNotImplemented) Error() string

type ErrObjectNotFound

type ErrObjectNotFound struct {
	// contains filtered or unexported fields
}

ErrObjectNotFound represents object not found error.

func NewErrObjectNotFound

func NewErrObjectNotFound(message string) ErrObjectNotFound

NewErrObjectNotFound returns a new ErrObjectNotFound.

func (ErrObjectNotFound) Error

func (e ErrObjectNotFound) Error() string

type ErrTLS

type ErrTLS struct {
	// contains filtered or unexported fields
}

ErrTLS represents the error returned when there is a TLS error.

func NewErrTLS

func NewErrTLS(message string) ErrTLS

NewErrTLS returns a new ErrTLS.

func (ErrTLS) Error

func (e ErrTLS) Error() string

type ErrTooManyRequests

type ErrTooManyRequests struct {
	// contains filtered or unexported fields
}

ErrTooManyRequests represents the error returned when the server api is locked.

func NewErrTooManyRequests

func NewErrTooManyRequests(message string) ErrTooManyRequests

NewErrTooManyRequests returns a new ErrTooManyRequests.

func (ErrTooManyRequests) Error

func (e ErrTooManyRequests) Error() string

type ErrTransactionNotFound

type ErrTransactionNotFound struct {
	// contains filtered or unexported fields
}

ErrTransactionNotFound represents a failure to find a transaction.

func NewErrTransactionNotFound

func NewErrTransactionNotFound(message string) ErrTransactionNotFound

NewErrTransactionNotFound returns a new ErrTransactionNotFound.

func (ErrTransactionNotFound) Error

func (e ErrTransactionNotFound) Error() string

type Filter

type Filter = elemental.Filter

Filter is an alias of elemental.Filter

func NewFilter

func NewFilter() *Filter

NewFilter returns a new Filter using the aliased type.

func NewFilterFromString

func NewFilterFromString(filter string) (*Filter, error)

NewFilterFromString returns a new NewFilterFromString using the aliased type.

type FilterKeyComposer

type FilterKeyComposer = elemental.FilterKeyComposer

FilterKeyComposer is an alias of elemental.FilterKeyComposer

func NewFilterComposer

func NewFilterComposer() FilterKeyComposer

NewFilterComposer returns a new FilterKeyComposer using the aliased type.

type FilterParser

type FilterParser = elemental.FilterParser

FilterParser is an alias of elemental.FilterParser

func NewFilterParser

func NewFilterParser(input string) *FilterParser

NewFilterParser returns a new NewFilterParser using the aliased type.

type FinalizerFunc

type FinalizerFunc func(o elemental.Identifiable) error

A FinalizerFunc is the type of a function that can be used as a creation finalizer. This is only supported by manipulators that generate an ID to let a chance to the user. to now the intended ID before actually creating the object.

type FlushableManipulator

type FlushableManipulator interface {

	// Flush flushes and empties the cache.
	Flush(ctx context.Context) error
}

A FlushableManipulator is a manipulator that can flush its content to somewhere, like a file.

type Manipulator

type Manipulator interface {

	// RetrieveMany retrieves the a list of objects with the given elemental.Identity and put them in the given dest.
	RetrieveMany(mctx Context, dest elemental.Identifiables) error

	// Retrieve retrieves one or multiple elemental.Identifiables.
	// In order to be retrievable, the elemental.Identifiable needs to have their Identifier correctly set.
	Retrieve(mctx Context, object elemental.Identifiable) error

	// Create creates a the given elemental.Identifiables.
	Create(mctx Context, object elemental.Identifiable) error

	// Update updates one or multiple elemental.Identifiables.
	// In order to be updatable, the elemental.Identifiable needs to have their Identifier correctly set.
	Update(mctx Context, object elemental.Identifiable) error

	// Delete deletes one or multiple elemental.Identifiables.
	// In order to be deletable, the elemental.Identifiable needs to have their Identifier correctly set.
	Delete(mctx Context, object elemental.Identifiable) error

	// DeleteMany deletes all objects of with the given identity or
	// all the ones matching the filter in the given context.
	DeleteMany(mctx Context, identity elemental.Identity) error

	// Count returns the number of objects with the given identity.
	Count(mctx Context, identity elemental.Identity) (int, error)
}

Manipulator is the interface of a storage backend.

type ReadConsistency

type ReadConsistency string

ReadConsistency represents the desired consistency of the request. Not all driver may implement this.

const (
	ReadConsistencyDefault   ReadConsistency = "default"
	ReadConsistencyNearest   ReadConsistency = "nearest"
	ReadConsistencyEventual  ReadConsistency = "eventual"
	ReadConsistencyMonotonic ReadConsistency = "monotonic"
	ReadConsistencyStrong    ReadConsistency = "strong"
)

Various values for Consistency

type Subscriber

type Subscriber interface {

	// Start connects to the websocket and starts collecting events
	// until the given context is canceled or any non communication error is
	// received. The eventual error will be received in the Errors() channel.
	// If not nil, the given filter will be applied right away.
	Start(context.Context, *elemental.PushFilter)

	// UpdateFilter updates the current filter.
	UpdateFilter(*elemental.PushFilter)

	// Events returns the events channel.
	Events() chan *elemental.Event

	// Errors returns the errors channel.
	Errors() chan error

	// Status returns the status channel.
	Status() chan SubscriberStatus
}

A Subscriber is the interface to control a push event subscription.

type SubscriberStatus

type SubscriberStatus int

SubscriberStatus is the type of a subscriber status.

const (
	SubscriberStatusInitialConnection SubscriberStatus = iota + 1
	SubscriberStatusInitialConnectionFailure
	SubscriberStatusReconnection
	SubscriberStatusReconnectionFailure
	SubscriberStatusDisconnection
	SubscriberStatusFinalDisconnection
	SubscriberStatusTokenRenewal
)

Various values of SubscriberEvent.

type TokenManager

type TokenManager interface {

	// Issues isses a new token.
	Issue(context.Context) (string, error)

	// Run runs the token renewal job and published the new token in the
	// given channel.
	Run(ctx context.Context, tokenCh chan string)
}

A TokenManager issues an renew tokens periodically.

type TransactionID

type TransactionID string

TransactionID is the type used to define a transcation ID of a store

func NewTransactionID

func NewTransactionID() TransactionID

NewTransactionID returns a new transaction ID.

type TransactionalManipulator

type TransactionalManipulator interface {

	// Commit commits the given TransactionID.
	Commit(id TransactionID) error

	// Abort aborts the give TransactionID. It returns true if
	// a transaction has been effectively aborted, otherwise it returns false.
	Abort(id TransactionID) bool

	Manipulator
}

A TransactionalManipulator is a Manipulator that handles transactions.

type WriteConsistency

type WriteConsistency string

WriteConsistency represents the desired consistency of the request. Not all driver may implement this.

const (
	WriteConsistencyDefault   WriteConsistency = "default"
	WriteConsistencyNone      WriteConsistency = "none"
	WriteConsistencyStrong    WriteConsistency = "strong"
	WriteConsistencyStrongest WriteConsistency = "strongest"
)

Various values for Consistency

Directories

Path Synopsis
internal
Package maniphttp provides a ReST backed Manipulator.
Package maniphttp provides a ReST backed Manipulator.
Package manipmemory provides a go-memdb backed Manipulator.
Package manipmemory provides a go-memdb backed Manipulator.
Package manipmongo provides a MongoDB backed TransactionalManipulator.
Package manipmongo provides a MongoDB backed TransactionalManipulator.
Package maniptest contains a Mockable TransactionalManipulator.
Package maniptest contains a Mockable TransactionalManipulator.
Package manipvortex contains a Manipulator that can be used as cache in front of another Manipulator.
Package manipvortex contains a Manipulator that can be used as cache in front of another Manipulator.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL