v0.99.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Apr 22, 2024 License: MIT Imports: 14 Imported by: 2



Package keyspace implements an efficient mechanism to mirror a decoded Etcd key/value space into a local KeySpace, which may be kept updated via a long- lived Watch operation. Each key & value of a KeySpace is decoded with a user provided decoder, and validated. Clients of a KeySpace are thus ensured that only validated keys & values are captured, while the KeySpace maintains consistency of the key/value set despite the potential for validation errors.

KeySpace instances may be "observed", which allows additional states to be derived from and updated by the KeySpace while being protected by the KeySpace Mutex. For example, if a value `foo` is mutated by a function of KeySpace.Observers, then any reader which properly synchronizes over KeySpace.Mu is guaranteed to see values of `foo` which reflect the current KeySpace state. Formally, readers are assured atomicity of a combined update to the KeySpace and the derived value.

KeySpace scales efficiently to Watches over 100's of thousands of keys by amortizing updates with a short Nagle-like delay, while providing fast range and point queries powered by its packed, sorted ordering.



This section is empty.


This section is empty.


This section is empty.


type KeySpace

type KeySpace struct {
	// Key prefix which roots this KeySpace.
	Root string
	// Header is the last Etcd operation revision which actually updated this KeySpace:
	// in other words, `Header.Revision` is the greatest `ModRevision` of any key under
	// the `Root`. It is not incremented by other Etcd operations which don't actually
	// change the KeySpace, such as progress notifications. This property means, for
	// all watched KeySpaces having a common Root, their revisions are always directly
	// comparable and will be equal if (and only if) they reflect the same keys and
	// values.
	Header etcdserverpb.ResponseHeader
	// KeyValues is a complete and decoded mirror of the (prefixed) Etcd key/value space.
	// Observers called upon each mutation of the KeySpace. Observer calls occur
	// in-order, after the KeySpace itself has been updated, and while a write-
	// lock over the KeySpace is still held (which Observers must not release).
	// Observers is intended to simplify atomicity of stateful representations
	// deriving themselves from the KeySpace: any mutations performed by Observers
	// will appear synchronously with changes to the KeySpace itself, from the
	// perspective of a client appropriately utilizing a read-lock.
	Observers []func()
	// WatchApplyDelay is the duration for which KeySpace should allow Etcd
	// WatchResponses to queue before applying all responses to the KeySpace.
	// This Nagle-like mechanism amortizes the cost of applying many
	// WatchResponses arriving in close succession. Default is 30ms.
	WatchApplyDelay time.Duration
	// Mu guards Header, KeyValues, and Observers. It must be locked before any are accessed.
	Mu sync.RWMutex
	// contains filtered or unexported fields

A KeySpace is a local mirror of a decoded portion of the Etcd key/value space, which is kept in sync with Etcd via long-lived Watch operations. KeySpace must be read-locked before access, to guard against concurrent updates.

func NewKeySpace

func NewKeySpace(prefix string, decoder KeyValueDecoder) *KeySpace

NewKeySpace returns a KeySpace with the configured key |prefix| and |decoder|. |prefix| must be a "Clean" path, as defined by path.Clean, or NewKeySpace panics. This check limits the space of possible prefixes somewhat, but guards against many common and unintentional errors (eg, mixed use of trailing slashes).

func (*KeySpace) Apply

func (ks *KeySpace) Apply(responses ...clientv3.WatchResponse) error

Apply one or more Etcd WatchResponses to the KeySpace. Apply returns only unrecoverable Application errors; inconsistencies in the updates themselves are instead logged. Apply is exported principally in support of testing fixtures; most clients should instead use Watch. Clients must ensure concurrent calls to Apply are not made.

func (*KeySpace) Load

func (ks *KeySpace) Load(ctx context.Context, client *clientv3.Client, rev int64) error

Load loads a snapshot of the prefixed KeySpace at revision |rev|, or if |rev| is zero, at the current revision.

func (*KeySpace) Update added in v0.89.0

func (ks *KeySpace) Update() <-chan struct{}

Update returns a channel which will signal on the next KeySpace update. It's not required to hold a read lock of the KeySpace Mutex, and a write lock must not be held or Update will deadlock.

func (*KeySpace) WaitForRevision

func (ks *KeySpace) WaitForRevision(ctx context.Context, revision int64) error

WaitForRevision blocks until the KeySpace Revision is at least |revision|, or until the context is done. A read lock of the KeySpace Mutex must be held at invocation, and will be re-acquired before WaitForRevision returns.

func (*KeySpace) Watch

func (ks *KeySpace) Watch(ctx context.Context, client clientv3.Watcher) error

Watch a loaded KeySpace and apply updates as they are received.

type KeyValue

type KeyValue struct {
	Raw     mvccpb.KeyValue
	Decoded interface{}

KeyValue composes a "raw" Etcd KeyValue with its user-defined, decoded representation.

type KeyValueDecoder

type KeyValueDecoder func(raw *mvccpb.KeyValue) (interface{}, error)

A KeyValueDecoder decodes raw KeyValue instances into a user-defined representation. KeyValueDecoder returns an error if the KeyValue cannot be decoded. KeySpace will log decoding errors and not incorporate them into the current KeyValues, but will also treat them as recoverable and in all cases seek to bring the KeyValues representation to consistency with Etcd. In practice, this means bad values written to Etcd which fail to decode may be later corrected with valid representations: KeySpace will ignore the bad update and then reflect the corrected one once available.

type KeyValues

type KeyValues []KeyValue

KeyValues is a collection of KeyValue naturally ordered on keys.

func (KeyValues) Copy

func (kv KeyValues) Copy() KeyValues

Copy returns a deep-copy of the KeyValues.

func (KeyValues) EqualKeyRevisions

func (kv KeyValues) EqualKeyRevisions(other KeyValues) bool

EqualKeyRevisions returns true if keys and revisions of |other| matches this KeyValues.

func (KeyValues) Prefixed

func (kv KeyValues) Prefixed(prefix string) KeyValues

Prefixed returns the sub-slice of KeyValues prefixed by |prefix|.

func (KeyValues) Range

func (kv KeyValues) Range(from, to string) KeyValues

Range returns the sub-slice of KeyValues spanning range [from, to).

func (KeyValues) Search

func (kv KeyValues) Search(key string) (ind int, found bool)

Search returns the index at which |key| is found to be present, or should be inserted to maintain ordering.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL