Documentation ¶
Overview ¶
Package cache is a client-side caching mechanism. It is useful for reducing the number of server calls you'd otherwise need to make. Reflector watches a server and updates a Store. Two stores are provided; one that simply caches objects (for example, to allow a scheduler to list currently available minions), and one that additionally acts as a FIFO queue (for example, to allow a scheduler to process incoming pods).
Index ¶
- Variables
- func MetaNamespaceIndexFunc(obj interface{}) (string, error)
- func MetaNamespaceKeyFunc(obj interface{}) (string, error)
- func NewNamespaceKeyedIndexerAndReflector(lw ListerWatcher, expectedType interface{}, resyncPeriod time.Duration) (indexer Indexer, reflector *Reflector)
- func SplitMetaNamespaceKey(key string) (namespace, name string, err error)
- type DeletedFinalStateUnknown
- type Delta
- type DeltaCompressor
- type DeltaCompressorFunc
- type DeltaFIFO
- func (f *DeltaFIFO) Add(obj interface{}) error
- func (f *DeltaFIFO) AddIfNotPresent(obj interface{}) error
- func (f *DeltaFIFO) Delete(obj interface{}) error
- func (f *DeltaFIFO) Get(obj interface{}) (item interface{}, exists bool, err error)
- func (f *DeltaFIFO) GetByKey(key string) (item interface{}, exists bool, err error)
- func (f *DeltaFIFO) KeyOf(obj interface{}) (string, error)
- func (f *DeltaFIFO) List() []interface{}
- func (f *DeltaFIFO) ListKeys() []string
- func (f *DeltaFIFO) Pop() interface{}
- func (f *DeltaFIFO) Replace(list []interface{}) error
- func (f *DeltaFIFO) Update(obj interface{}) error
- type DeltaType
- type Deltas
- type Enumerator
- type ExpirationCache
- func (c *ExpirationCache) Add(obj interface{}) error
- func (c *ExpirationCache) Delete(obj interface{}) error
- func (c *ExpirationCache) Get(obj interface{}) (interface{}, bool, error)
- func (c *ExpirationCache) GetByKey(key string) (interface{}, bool, error)
- func (c *ExpirationCache) List() []interface{}
- func (c *ExpirationCache) ListKeys() []string
- func (c *ExpirationCache) Replace(list []interface{}) error
- func (c *ExpirationCache) Update(obj interface{}) error
- type ExpirationPolicy
- type ExplicitKey
- type FIFO
- func (f *FIFO) Add(obj interface{}) error
- func (f *FIFO) AddIfNotPresent(obj interface{}) error
- func (f *FIFO) Delete(obj interface{}) error
- func (f *FIFO) Get(obj interface{}) (item interface{}, exists bool, err error)
- func (f *FIFO) GetByKey(key string) (item interface{}, exists bool, err error)
- func (f *FIFO) List() []interface{}
- func (f *FIFO) ListKeys() []string
- func (f *FIFO) Pop() interface{}
- func (f *FIFO) Replace(list []interface{}) error
- func (f *FIFO) Update(obj interface{}) error
- type FakeExpirationPolicy
- type GetFunc
- type Index
- type IndexFunc
- type Indexer
- type Indexers
- type Indices
- type KeyError
- type KeyFunc
- type KeyLister
- type KeyListerFunc
- type ListFunc
- type ListWatch
- type ListerWatcher
- type Poller
- type Queue
- type Reflector
- type Store
- type StoreToNodeLister
- type StoreToPodLister
- type StoreToServiceLister
- type TTLPolicy
- type ThreadSafeStore
- type UndeltaStore
- type WatchFunc
Constants ¶
This section is empty.
Variables ¶
var ( // ErrZeroLengthDeltasObject is returned in a KeyError if a Deltas // object with zero length is encountered (should be impossible, // even if such an object is accidentally produced by a DeltaCompressor-- // but included for completeness). ErrZeroLengthDeltasObject = errors.New("0 length Deltas object; can't get key") )
Functions ¶
func MetaNamespaceIndexFunc ¶
MetaNamespaceIndexFunc is a default index function that indexes based on an object's namespace
func MetaNamespaceKeyFunc ¶
MetaNamespaceKeyFunc is a convenient default KeyFunc which knows how to make keys for API objects which implement meta.Interface. The key uses the format <namespace>/<name> unless <namespace> is empty, then it's just <name>.
TODO: replace key-as-string with a key-as-struct so that this packing/unpacking won't be necessary.
func NewNamespaceKeyedIndexerAndReflector ¶
func NewNamespaceKeyedIndexerAndReflector(lw ListerWatcher, expectedType interface{}, resyncPeriod time.Duration) (indexer Indexer, reflector *Reflector)
NewNamespaceKeyedIndexerAndReflector creates an Indexer and a Reflector The indexer is configured to key on namespace
func SplitMetaNamespaceKey ¶
SplitMetaNamespaceKey returns the namespace and name that MetaNamespaceKeyFunc encoded into key.
TODO: replace key-as-string with a key-as-struct so that this packing/unpacking won't be necessary.
Types ¶
type DeletedFinalStateUnknown ¶
type DeletedFinalStateUnknown struct {
Key string
}
DeletedFinalStateUnknown is placed into a DeltaFIFO in the case where an object was deleted but the watch deletion event was was missed. In this case we don't know the final "resting" state of the object.
type Delta ¶
type Delta struct { Type DeltaType Object interface{} }
Delta is the type stored by a DeltaFIFO. It tells you what change happened, and the object's state after* that change.
[*] Unless the change is a deletion, and then you'll get the final
state of the object before it was deleted.
type DeltaCompressor ¶
DeltaCompressor is an algorithm that removes redundant changes.
type DeltaCompressorFunc ¶
DeltaCompressorFunc should remove redundant changes; but changes that are redundant depend on one's desired semantics, so this is an injectable function.
DeltaCompressorFunc adapts a raw function to be a DeltaCompressor.
func (DeltaCompressorFunc) Compress ¶
func (dc DeltaCompressorFunc) Compress(d Deltas) Deltas
Compress just calls dc.
type DeltaFIFO ¶
type DeltaFIFO struct {
// contains filtered or unexported fields
}
DeltaFIFO is like FIFO, but allows you to process deletes.
DeltaFIFO is a producer-consumer queue, where a Reflector is intended to be the producer, and the consumer is whatever calls the Pop() method.
DeltaFIFO solves this use case:
- You want to process every object change (delta) at most once.
- When you process an object, you want to see everything that's happened to it since you last processed it.
- You want to process the deletion of objects.
- You might want to periodically reprocess objects.
DeltaFIFO's Pop(), Get(), and GetByKey() methods return interface{} to satisfy the Store/Queue interfaces, but it will always return an object of type Deltas.
A note on threading: If you call Pop() in parallel from multiple threads, you could end up with multiple threads processing slightly different versions of the same object.
You may provide a function to compress deltas (e.g., represent a series of Updates as a single Update).
func NewDeltaFIFO ¶
func NewDeltaFIFO(keyFunc KeyFunc, compressor DeltaCompressor, knownObjectKeys KeyLister) *DeltaFIFO
NewDeltaFIFO returns a Store which can be used process changes to items.
keyFunc is used to figure out what key an object should have. (It's exposed in the returned DeltaFIFO's KeyOf() method, with bonus features.)
'compressor' may compress as many or as few items as it wants (including returning an empty slice), but it should do what it does quickly since it is called while the queue is locked. 'compressor' may be nil if you don't want any delta compression.
'keyLister' is expected to return a list of keys that the consumer of this queue "knows about". It is used to decide which items are missing when Replace() is called; 'Deleted' deltas are produced for these items. It may be nil if you don't need to detect all deletions. TODO: consider merging keyLister with this object, tracking a list of
"known" keys when Pop() is called. Have to think about how that affects error retrying.
Also see the comment on DeltaFIFO.
func (*DeltaFIFO) Add ¶
Add inserts an item, and puts it in the queue. The item is only enqueued if it doesn't already exist in the set.
func (*DeltaFIFO) AddIfNotPresent ¶
AddIfNotPresent inserts an item, and puts it in the queue. If the item is already present in the set, it is neither enqueued nor added to the set.
This is useful in a single producer/consumer scenario so that the consumer can safely retry items without contending with the producer and potentially enqueueing stale items.
Important: obj must be a Deltas (the output of the Pop() function). Yes, this is different from the Add/Update/Delete functions.
func (*DeltaFIFO) Get ¶
Get returns the complete list of deltas for the requested item, or sets exists=false. You should treat the items returned inside the deltas as immutable.
func (*DeltaFIFO) GetByKey ¶
GetByKey returns the complete list of deltas for the requested item, setting exists=false if that list is empty. You should treat the items returned inside the deltas as immutable.
func (*DeltaFIFO) KeyOf ¶
KeyOf exposes f's keyFunc, but also detects the key of a Deltas object or DeletedFinalStateUnknown objects.
func (*DeltaFIFO) List ¶
func (f *DeltaFIFO) List() []interface{}
List returns a list of all the items; it returns the object from the most recent Delta. You should treat the items returned inside the deltas as immutable.
func (*DeltaFIFO) ListKeys ¶
ListKeys returns a list of all the keys of the objects currently in the FIFO.
func (*DeltaFIFO) Pop ¶
func (f *DeltaFIFO) Pop() interface{}
Pop blocks until an item is added to the queue, and then returns it. If multiple items are ready, they are returned in the order in which they were added/updated. The item is removed from the queue (and the store) before it is returned, so if you don't succesfully process it, you need to add it back with AddIfNotPresent().
Pop returns a 'Deltas', which has a complete list of all the things that happened to the object (deltas) while it was sitting in the queue.
type DeltaType ¶
type DeltaType string
DeltaType is the type of a change (addition, deletion, etc)
const ( Added DeltaType = "Added" Updated DeltaType = "Updated" Deleted DeltaType = "Deleted" // The other types are obvious. You'll get Sync deltas when: // * A watch expires/errors out and a new list/watch cycle is started. // * You've turned on periodic syncs. // (Anything that trigger's DeltaFIFO's Replace() method.) Sync DeltaType = "Sync" )
type Deltas ¶
type Deltas []Delta
Deltas is a list of one or more 'Delta's to an individual object. The oldest delta is at index 0, the newest delta is the last one.
type Enumerator ¶
Enumerator should be able to return the list of objects to be synced with one object at a time.
type ExpirationCache ¶
type ExpirationCache struct {
// contains filtered or unexported fields
}
ExpirationCache implements the store interface
- All entries are automatically time stamped on insert a. The key is computed based off the original item/keyFunc b. The value inserted under that key is the timestamped item
- Expiration happens lazily on read based on the expiration policy
- Time-stamps are stripped off unexpired entries before return
func (*ExpirationCache) Add ¶
func (c *ExpirationCache) Add(obj interface{}) error
Add timestamps an item and inserts it into the cache, overwriting entries that might exist under the same key.
func (*ExpirationCache) Delete ¶
func (c *ExpirationCache) Delete(obj interface{}) error
Delete removes an item from the cache.
func (*ExpirationCache) Get ¶
func (c *ExpirationCache) Get(obj interface{}) (interface{}, bool, error)
Get returns unexpired items. It purges the cache of expired items in the process.
func (*ExpirationCache) GetByKey ¶
func (c *ExpirationCache) GetByKey(key string) (interface{}, bool, error)
GetByKey returns the item stored under the key, or sets exists=false.
func (*ExpirationCache) List ¶
func (c *ExpirationCache) List() []interface{}
List retrieves a list of unexpired items. It purges the cache of expired items in the process.
func (*ExpirationCache) ListKeys ¶
func (c *ExpirationCache) ListKeys() []string
ListKeys returns a list of all keys in the expiration cache.
func (*ExpirationCache) Replace ¶
func (c *ExpirationCache) Replace(list []interface{}) error
Replace will convert all items in the given list to TimestampedEntries before attempting the replace operation. The replace operation will delete the contents of the ExpirationCache `c`.
func (*ExpirationCache) Update ¶
func (c *ExpirationCache) Update(obj interface{}) error
Update has not been implemented yet for lack of a use case, so this method simply calls `Add`. This effectively refreshes the timestamp.
type ExpirationPolicy ¶
type ExpirationPolicy interface {
IsExpired(obj *timestampedEntry) bool
}
ExpirationPolicy dictates when an object expires. Currently only abstracted out so unittests don't rely on the system clock.
type ExplicitKey ¶
type ExplicitKey string
ExplicitKey can be passed to MetaNamespaceKeyFunc if you have the key for the object but not the object itself.
type FIFO ¶
type FIFO struct {
// contains filtered or unexported fields
}
FIFO receives adds and updates from a Reflector, and puts them in a queue for FIFO order processing. If multiple adds/updates of a single item happen while an item is in the queue before it has been processed, it will only be processed once, and when it is processed, the most recent version will be processed. This can't be done with a channel.
FIFO solves this use case:
- You want to process every object (exactly) once.
- You want to process the most recent version of the object when you process it.
- You do not want to process deleted objects, they should be removed from the queue.
- You do not want to periodically reprocess objects.
Compare with DeltaFIFO for other use cases.
func (*FIFO) Add ¶
Add inserts an item, and puts it in the queue. The item is only enqueued if it doesn't already exist in the set.
func (*FIFO) AddIfNotPresent ¶
AddIfNotPresent inserts an item, and puts it in the queue. If the item is already present in the set, it is neither enqueued nor added to the set.
This is useful in a single producer/consumer scenario so that the consumer can safely retry items without contending with the producer and potentially enqueueing stale items.
func (*FIFO) Delete ¶
Delete removes an item. It doesn't add it to the queue, because this implementation assumes the consumer only cares about the objects, not the order in which they were created/added.
func (*FIFO) ListKeys ¶
ListKeys returns a list of all the keys of the objects currently in the FIFO.
func (*FIFO) Pop ¶
func (f *FIFO) Pop() interface{}
Pop waits until an item is ready and returns it. If multiple items are ready, they are returned in the order in which they were added/updated. The item is removed from the queue (and the store) before it is returned, so if you don't succesfully process it, you need to add it back with AddIfNotPresent().
type FakeExpirationPolicy ¶
func (*FakeExpirationPolicy) IsExpired ¶
func (p *FakeExpirationPolicy) IsExpired(obj *timestampedEntry) bool
type GetFunc ¶
type GetFunc func() (Enumerator, error)
GetFunc should return an enumerator that you wish the Poller to proccess.
type Indexer ¶
type Indexer interface { Store // Retrieve list of objects that match on the named indexing function Index(indexName string, obj interface{}) ([]interface{}, error) }
Indexer is a storage interface that lets you list objects using multiple indexing functions
func NewIndexer ¶
NewIndexer returns an Indexer implemented simply with a map and a lock.
type KeyError ¶
type KeyError struct { Obj interface{} Err error }
KeyError will be returned any time a KeyFunc gives an error; it includes the object at fault.
type KeyFunc ¶
KeyFunc knows how to make a key from an object. Implementations should be deterministic.
type KeyLister ¶
type KeyLister interface {
ListKeys() []string
}
A KeyLister is anything that knows how to list its keys.
type KeyListerFunc ¶
type KeyListerFunc func() []string
KeyListerFunc adapts a raw function to be a KeyLister.
type ListWatch ¶
ListWatch knows how to list and watch a set of apiserver resources. It satisfies the ListerWatcher interface. It is a convenience function for users of NewReflector, etc. ListFunc and WatchFunc must not be nil
func NewListWatchFromClient ¶
func NewListWatchFromClient(c *client.Client, resource string, namespace string, fieldSelector fields.Selector) *ListWatch
NewListWatchFromClient creates a new ListWatch from the specified client, resource, namespace and field selector.
type ListerWatcher ¶
type ListerWatcher interface { // List should return a list type object; the Items field will be extracted, and the // ResourceVersion field will be used to start the watch in the right place. List() (runtime.Object, error) // Watch should begin a watch at the specified version. Watch(resourceVersion string) (watch.Interface, error) }
ListerWatcher is any object that knows how to perform an initial list and start a watch on a resource.
type Poller ¶
type Poller struct {
// contains filtered or unexported fields
}
Poller is like Reflector, but it periodically polls instead of watching. This is intended to be a workaround for api objects that don't yet support watching.
func NewPoller ¶
NewPoller constructs a new poller. Note that polling probably doesn't make much sense to use along with the FIFO queue. The returned Poller will call getFunc and sync the objects in 'store' with the returned Enumerator, waiting 'period' between each call. It probably only makes sense to use a poller if you're treating the store as read-only.
type Queue ¶
type Queue interface { Store // Pop blocks until it has something to return. Pop() interface{} // AddIfNotPresent adds a value previously // returned by Pop back into the queue as long // as nothing else (presumably more recent) // has since been added. AddIfNotPresent(interface{}) error }
Queue is exactly like a Store, but has a Pop() method too.
type Reflector ¶
type Reflector struct {
// contains filtered or unexported fields
}
Reflector watches a specified resource and causes all changes to be reflected in the given store.
func NewReflector ¶
func NewReflector(lw ListerWatcher, expectedType interface{}, store Store, resyncPeriod time.Duration) *Reflector
NewReflector creates a new Reflector object which will keep the given store up to date with the server's contents for the given resource. Reflector promises to only put things in the store that have the type of expectedType. If resyncPeriod is non-zero, then lists will be executed after every resyncPeriod, so that you can use reflectors to periodically process everything as well as incrementally processing the things that change.
func (*Reflector) LastSyncResourceVersion ¶
LastSyncResourceVersion is the resource version observed when last sync with the underlying store The value returned is not synchronized with access to the underlying store and is not thread-safe
type Store ¶
type Store interface { Add(obj interface{}) error Update(obj interface{}) error Delete(obj interface{}) error List() []interface{} ListKeys() []string Get(obj interface{}) (item interface{}, exists bool, err error) GetByKey(key string) (item interface{}, exists bool, err error) // Replace will delete the contents of the store, using instead the // given list. Store takes ownership of the list, you should not reference // it after calling this function. Replace([]interface{}) error }
Store is a generic object storage interface. Reflector knows how to watch a server and update a store. A generic store is provided, which allows Reflector to be used as a local caching system, and an LRU store, which allows Reflector to work like a queue of items yet to be processed.
Store makes no assumptions about stored object identity; it is the responsibility of a Store implementation to provide a mechanism to correctly key objects and to define the contract for obtaining objects by some arbitrary key type.
func NewFakeExpirationStore ¶
type StoreToNodeLister ¶
type StoreToNodeLister struct {
Store
}
StoreToNodeLister makes a Store have the List method of the client.NodeInterface The Store must contain (only) Nodes.
func (*StoreToNodeLister) GetNodeInfo ¶
func (s *StoreToNodeLister) GetNodeInfo(id string) (*api.Node, error)
TODO Move this back to scheduler as a helper function that takes a Store, rather than a method of StoreToNodeLister. GetNodeInfo returns cached data for the minion 'id'.
type StoreToPodLister ¶
type StoreToPodLister struct {
Store
}
StoreToPodLister makes a Store have the List method of the client.PodInterface The Store must contain (only) Pods.
Example: s := cache.NewStore() lw := cache.ListWatch{Client: c, FieldSelector: sel, Resource: "pods"} r := cache.NewReflector(lw, &api.Pod{}, s).Run() l := StoreToPodLister{s} l.List()
func (*StoreToPodLister) Exists ¶
func (s *StoreToPodLister) Exists(pod *api.Pod) (bool, error)
Exists returns true if a pod matching the namespace/name of the given pod exists in the store.
func (*StoreToPodLister) List ¶
Please note that selector is filtering among the pods that have gotten into the store; there may have been some filtering that already happened before that.
TODO: converge on the interface in pkg/client.
func (*StoreToPodLister) Pods ¶
func (s *StoreToPodLister) Pods(namespace string) storePodsNamespacer
Pods is taking baby steps to be more like the api in pkg/client
type StoreToServiceLister ¶
type StoreToServiceLister struct {
Store
}
StoreToServiceLister makes a Store that has the List method of the client.ServiceInterface The Store must contain (only) Services.
func (*StoreToServiceLister) GetPodServices ¶
TODO: Move this back to scheduler as a helper function that takes a Store, rather than a method of StoreToServiceLister.
func (*StoreToServiceLister) List ¶
func (s *StoreToServiceLister) List() (services api.ServiceList, err error)
type TTLPolicy ¶
type TTLPolicy struct { // >0: Expire entries with an age > ttl // <=0: Don't expire any entry Ttl time.Duration // Clock used to calculate ttl expiration Clock util.Clock }
TTLPolicy implements a ttl based ExpirationPolicy.
type ThreadSafeStore ¶
type ThreadSafeStore interface { Add(key string, obj interface{}) Update(key string, obj interface{}) Delete(key string) Get(key string) (item interface{}, exists bool) List() []interface{} ListKeys() []string Replace(map[string]interface{}) Index(indexName string, obj interface{}) ([]interface{}, error) }
ThreadSafeStore is an interface that allows concurrent access to a storage backend. TL;DR caveats: you must not modify anything returned by Get or List as it will break the indexing feature in addition to not being thread safe.
The guarantees of thread safety provided by List/Get are only valid if the caller treats returned items as read-only. For example, a pointer inserted in the store through `Add` will be returned as is by `Get`. Multiple clients might invoke `Get` on the same key and modify the pointer in a non-thread-safe way. Also note that modifying objects stored by the indexers (if any) will *not* automatically lead to a re-index. So it's not a good idea to directly modify the objects returned by Get/List, in general.
func NewThreadSafeStore ¶
func NewThreadSafeStore(indexers Indexers, indices Indices) ThreadSafeStore
type UndeltaStore ¶
type UndeltaStore struct { Store PushFunc func([]interface{}) }
UndeltaStore listens to incremental updates and sends complete state on every change. It implements the Store interface so that it can receive a stream of mirrored objects from Reflector. Whenever it receives any complete (Store.Replace) or incremental change (Store.Add, Store.Update, Store.Delete), it sends the complete state by calling PushFunc. It is thread-safe. It guarantees that every change (Add, Update, Replace, Delete) results in one call to PushFunc, but sometimes PushFunc may be called twice with the same values. PushFunc should be thread safe.
func NewUndeltaStore ¶
func NewUndeltaStore(pushFunc func([]interface{}), keyFunc KeyFunc) *UndeltaStore
NewUndeltaStore returns an UndeltaStore implemented with a Store.
func (*UndeltaStore) Add ¶
func (u *UndeltaStore) Add(obj interface{}) error
func (*UndeltaStore) Delete ¶
func (u *UndeltaStore) Delete(obj interface{}) error
func (*UndeltaStore) Replace ¶
func (u *UndeltaStore) Replace(list []interface{}) error
func (*UndeltaStore) Update ¶
func (u *UndeltaStore) Update(obj interface{}) error