bootstrap

package
v0.15.0-rc.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 2, 2020 License: Apache-2.0 Imports: 32 Imported by: 18

Documentation

Overview

Package bootstrap is a generated GoMock package.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Bootstrapper 

type Bootstrapper interface {
	// String returns the name of the bootstrapper
	String() string

	// Bootstrap performs bootstrapping of series data and index metadata
	// for the  given time ranges, returning the bootstrapped series data
	// and index blocks for the time ranges it's unable to fulfill in parallel.
	// A bootstrapper should only return an error should it want to entirely
	// cancel the bootstrapping of the node, i.e. non-recoverable situation
	// like not being able to read from the filesystem.
	Bootstrap(namespaces Namespaces) (NamespaceResults, error)
}

Bootstrapper is the interface for different bootstrapping mechanisms. Note that a bootstrapper can and will be reused so it is important to not rely on state stored in the bootstrapper itself with the mindset that it will always be set to default values from the constructor.

type BootstrapperProvider 

type BootstrapperProvider interface {
	// String returns the name of the bootstrapper.
	String() string

	// Provide constructs a bootstrapper.
	Provide() (Bootstrapper, error)
}

BootstrapperProvider constructs a bootstrapper.

type CheckoutSeriesResult added in v0.15.0 

type CheckoutSeriesResult struct {
	// Series is the series for the checkout operation.
	Series series.DatabaseSeries
	// Shard is the shard for the series.
	Shard uint32
	// UniqueIndex is the unique index for the series.
	UniqueIndex uint64
}

CheckoutSeriesResult is the result of a checkout series operation.

type DecodedBlockMap added in v0.15.0 

type DecodedBlockMap map[string]DecodedValues

DecodedBlockMap is a map of decoded datapoints per series ID.

func (DecodedBlockMap) VerifyEquals added in v0.15.0 

func (m DecodedBlockMap) VerifyEquals(other DecodedBlockMap) error

VerifyEquals verifies that two DecodedBlockMap are equal; errors otherwise.

type DecodedNamespaceMap added in v0.15.0 

type DecodedNamespaceMap map[string]DecodedBlockMap

DecodedNamespaceMap is a map of decoded blocks per namespace ID.

type DecodedValues added in v0.15.0 

type DecodedValues []series.DecodedTestValue

DecodedValues is a slice of series datapoints.

type MockBootstrapper 

type MockBootstrapper struct {
	// contains filtered or unexported fields
}

MockBootstrapper is a mock of Bootstrapper interface

func NewMockBootstrapper 

func NewMockBootstrapper(ctrl *gomock.Controller) *MockBootstrapper

NewMockBootstrapper creates a new mock instance

func (*MockBootstrapper) Bootstrap added in v0.15.0 

func (m *MockBootstrapper) Bootstrap(namespaces Namespaces) (NamespaceResults, error)

Bootstrap mocks base method

func (*MockBootstrapper) EXPECT 

func (m *MockBootstrapper) EXPECT() *MockBootstrapperMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockBootstrapper) String 

func (m *MockBootstrapper) String() string

String mocks base method

type MockBootstrapperMockRecorder 

type MockBootstrapperMockRecorder struct {
	// contains filtered or unexported fields
}

MockBootstrapperMockRecorder is the mock recorder for MockBootstrapper

func (*MockBootstrapperMockRecorder) Bootstrap added in v0.15.0 

func (mr *MockBootstrapperMockRecorder) Bootstrap(namespaces interface{}) *gomock.Call

Bootstrap indicates an expected call of Bootstrap

func (*MockBootstrapperMockRecorder) String 

func (mr *MockBootstrapperMockRecorder) String() *gomock.Call

String indicates an expected call of String

type MockBootstrapperProvider 

type MockBootstrapperProvider struct {
	// contains filtered or unexported fields
}

MockBootstrapperProvider is a mock of BootstrapperProvider interface

func NewMockBootstrapperProvider 

func NewMockBootstrapperProvider(ctrl *gomock.Controller) *MockBootstrapperProvider

NewMockBootstrapperProvider creates a new mock instance

func (*MockBootstrapperProvider) EXPECT 

func (m *MockBootstrapperProvider) EXPECT() *MockBootstrapperProviderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockBootstrapperProvider) Provide 

func (m *MockBootstrapperProvider) Provide() (Bootstrapper, error)

Provide mocks base method

func (*MockBootstrapperProvider) String 

func (m *MockBootstrapperProvider) String() string

String mocks base method

type MockBootstrapperProviderMockRecorder 

type MockBootstrapperProviderMockRecorder struct {
	// contains filtered or unexported fields
}

MockBootstrapperProviderMockRecorder is the mock recorder for MockBootstrapperProvider

func (*MockBootstrapperProviderMockRecorder) Provide 

func (mr *MockBootstrapperProviderMockRecorder) Provide() *gomock.Call

Provide indicates an expected call of Provide

func (*MockBootstrapperProviderMockRecorder) String 

func (mr *MockBootstrapperProviderMockRecorder) String() *gomock.Call

String indicates an expected call of String

type MockNamespaceDataAccumulator added in v0.15.0 

type MockNamespaceDataAccumulator struct {
	// contains filtered or unexported fields
}

MockNamespaceDataAccumulator is a mock of NamespaceDataAccumulator interface

func NewMockNamespaceDataAccumulator added in v0.15.0 

func NewMockNamespaceDataAccumulator(ctrl *gomock.Controller) *MockNamespaceDataAccumulator

NewMockNamespaceDataAccumulator creates a new mock instance

func (*MockNamespaceDataAccumulator) CheckoutSeriesWithLock added in v0.15.0 

func (m *MockNamespaceDataAccumulator) CheckoutSeriesWithLock(shardID uint32, id ident.ID, tags ident.TagIterator) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithLock mocks base method

func (*MockNamespaceDataAccumulator) CheckoutSeriesWithoutLock added in v0.15.0 

func (m *MockNamespaceDataAccumulator) CheckoutSeriesWithoutLock(shardID uint32, id ident.ID, tags ident.TagIterator) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithoutLock mocks base method

func (*MockNamespaceDataAccumulator) Close added in v0.15.0 

func (m *MockNamespaceDataAccumulator) Close() error

Close mocks base method

func (*MockNamespaceDataAccumulator) EXPECT added in v0.15.0 

func (m *MockNamespaceDataAccumulator) EXPECT() *MockNamespaceDataAccumulatorMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

type MockNamespaceDataAccumulatorMockRecorder added in v0.15.0 

type MockNamespaceDataAccumulatorMockRecorder struct {
	// contains filtered or unexported fields
}

MockNamespaceDataAccumulatorMockRecorder is the mock recorder for MockNamespaceDataAccumulator

func (*MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithLock added in v0.15.0 

func (mr *MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithLock(shardID, id, tags interface{}) *gomock.Call

CheckoutSeriesWithLock indicates an expected call of CheckoutSeriesWithLock

func (*MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithoutLock added in v0.15.0 

func (mr *MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithoutLock(shardID, id, tags interface{}) *gomock.Call

CheckoutSeriesWithoutLock indicates an expected call of CheckoutSeriesWithoutLock

func (*MockNamespaceDataAccumulatorMockRecorder) Close added in v0.15.0 

func (mr *MockNamespaceDataAccumulatorMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

type MockProcess 

type MockProcess struct {
	// contains filtered or unexported fields
}

MockProcess is a mock of Process interface

func NewMockProcess 

func NewMockProcess(ctrl *gomock.Controller) *MockProcess

NewMockProcess creates a new mock instance

func (*MockProcess) EXPECT 

func (m *MockProcess) EXPECT() *MockProcessMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockProcess) Run 

func (m *MockProcess) Run(start time.Time, namespaces []ProcessNamespace) (NamespaceResults, error)

Run mocks base method

type MockProcessMockRecorder 

type MockProcessMockRecorder struct {
	// contains filtered or unexported fields
}

MockProcessMockRecorder is the mock recorder for MockProcess

func (*MockProcessMockRecorder) Run 

func (mr *MockProcessMockRecorder) Run(start, namespaces interface{}) *gomock.Call

Run indicates an expected call of Run

type MockProcessOptions 

type MockProcessOptions struct {
	// contains filtered or unexported fields
}

MockProcessOptions is a mock of ProcessOptions interface

func NewMockProcessOptions 

func NewMockProcessOptions(ctrl *gomock.Controller) *MockProcessOptions

NewMockProcessOptions creates a new mock instance

func (*MockProcessOptions) CacheSeriesMetadata 

func (m *MockProcessOptions) CacheSeriesMetadata() bool

CacheSeriesMetadata mocks base method

func (*MockProcessOptions) EXPECT 

func (m *MockProcessOptions) EXPECT() *MockProcessOptionsMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockProcessOptions) Origin added in v0.4.6 

func (m *MockProcessOptions) Origin() topology.Host

Origin mocks base method

func (*MockProcessOptions) SetCacheSeriesMetadata 

func (m *MockProcessOptions) SetCacheSeriesMetadata(value bool) ProcessOptions

SetCacheSeriesMetadata mocks base method

func (*MockProcessOptions) SetOrigin added in v0.4.6 

func (m *MockProcessOptions) SetOrigin(value topology.Host) ProcessOptions

SetOrigin mocks base method

func (*MockProcessOptions) SetTopologyMapProvider added in v0.4.6 

func (m *MockProcessOptions) SetTopologyMapProvider(value topology.MapProvider) ProcessOptions

SetTopologyMapProvider mocks base method

func (*MockProcessOptions) TopologyMapProvider added in v0.4.6 

func (m *MockProcessOptions) TopologyMapProvider() topology.MapProvider

TopologyMapProvider mocks base method

func (*MockProcessOptions) Validate added in v0.4.4 

func (m *MockProcessOptions) Validate() error

Validate mocks base method

type MockProcessOptionsMockRecorder 

type MockProcessOptionsMockRecorder struct {
	// contains filtered or unexported fields
}

MockProcessOptionsMockRecorder is the mock recorder for MockProcessOptions

func (*MockProcessOptionsMockRecorder) CacheSeriesMetadata 

func (mr *MockProcessOptionsMockRecorder) CacheSeriesMetadata() *gomock.Call

CacheSeriesMetadata indicates an expected call of CacheSeriesMetadata

func (*MockProcessOptionsMockRecorder) Origin added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) Origin() *gomock.Call

Origin indicates an expected call of Origin

func (*MockProcessOptionsMockRecorder) SetCacheSeriesMetadata 

func (mr *MockProcessOptionsMockRecorder) SetCacheSeriesMetadata(value interface{}) *gomock.Call

SetCacheSeriesMetadata indicates an expected call of SetCacheSeriesMetadata

func (*MockProcessOptionsMockRecorder) SetOrigin added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) SetOrigin(value interface{}) *gomock.Call

SetOrigin indicates an expected call of SetOrigin

func (*MockProcessOptionsMockRecorder) SetTopologyMapProvider added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) SetTopologyMapProvider(value interface{}) *gomock.Call

SetTopologyMapProvider indicates an expected call of SetTopologyMapProvider

func (*MockProcessOptionsMockRecorder) TopologyMapProvider added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) TopologyMapProvider() *gomock.Call

TopologyMapProvider indicates an expected call of TopologyMapProvider

func (*MockProcessOptionsMockRecorder) Validate added in v0.4.4 

func (mr *MockProcessOptionsMockRecorder) Validate() *gomock.Call

Validate indicates an expected call of Validate

type MockProcessProvider 

type MockProcessProvider struct {
	// contains filtered or unexported fields
}

MockProcessProvider is a mock of ProcessProvider interface

func NewMockProcessProvider 

func NewMockProcessProvider(ctrl *gomock.Controller) *MockProcessProvider

NewMockProcessProvider creates a new mock instance

func (*MockProcessProvider) BootstrapperProvider 

func (m *MockProcessProvider) BootstrapperProvider() BootstrapperProvider

BootstrapperProvider mocks base method

func (*MockProcessProvider) EXPECT 

func (m *MockProcessProvider) EXPECT() *MockProcessProviderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockProcessProvider) Provide 

func (m *MockProcessProvider) Provide() (Process, error)

Provide mocks base method

func (*MockProcessProvider) SetBootstrapperProvider 

func (m *MockProcessProvider) SetBootstrapperProvider(bootstrapper BootstrapperProvider)

SetBootstrapperProvider mocks base method

type MockProcessProviderMockRecorder 

type MockProcessProviderMockRecorder struct {
	// contains filtered or unexported fields
}

MockProcessProviderMockRecorder is the mock recorder for MockProcessProvider

func (*MockProcessProviderMockRecorder) BootstrapperProvider 

func (mr *MockProcessProviderMockRecorder) BootstrapperProvider() *gomock.Call

BootstrapperProvider indicates an expected call of BootstrapperProvider

func (*MockProcessProviderMockRecorder) Provide 

func (mr *MockProcessProviderMockRecorder) Provide() *gomock.Call

Provide indicates an expected call of Provide

func (*MockProcessProviderMockRecorder) SetBootstrapperProvider 

func (mr *MockProcessProviderMockRecorder) SetBootstrapperProvider(bootstrapper interface{}) *gomock.Call

SetBootstrapperProvider indicates an expected call of SetBootstrapperProvider

type MockRunOptions 

type MockRunOptions struct {
	// contains filtered or unexported fields
}

MockRunOptions is a mock of RunOptions interface

func NewMockRunOptions 

func NewMockRunOptions(ctrl *gomock.Controller) *MockRunOptions

NewMockRunOptions creates a new mock instance

func (*MockRunOptions) CacheSeriesMetadata 

func (m *MockRunOptions) CacheSeriesMetadata() bool

CacheSeriesMetadata mocks base method

func (*MockRunOptions) EXPECT 

func (m *MockRunOptions) EXPECT() *MockRunOptionsMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockRunOptions) InitialTopologyState added in v0.4.4 

func (m *MockRunOptions) InitialTopologyState() *topology.StateSnapshot

InitialTopologyState mocks base method

func (*MockRunOptions) PersistConfig added in v0.4.4 

func (m *MockRunOptions) PersistConfig() PersistConfig

PersistConfig mocks base method

func (*MockRunOptions) SetCacheSeriesMetadata 

func (m *MockRunOptions) SetCacheSeriesMetadata(value bool) RunOptions

SetCacheSeriesMetadata mocks base method

func (*MockRunOptions) SetInitialTopologyState added in v0.4.4 

func (m *MockRunOptions) SetInitialTopologyState(value *topology.StateSnapshot) RunOptions

SetInitialTopologyState mocks base method

func (*MockRunOptions) SetPersistConfig added in v0.4.4 

func (m *MockRunOptions) SetPersistConfig(value PersistConfig) RunOptions

SetPersistConfig mocks base method

type MockRunOptionsMockRecorder 

type MockRunOptionsMockRecorder struct {
	// contains filtered or unexported fields
}

MockRunOptionsMockRecorder is the mock recorder for MockRunOptions

func (*MockRunOptionsMockRecorder) CacheSeriesMetadata 

func (mr *MockRunOptionsMockRecorder) CacheSeriesMetadata() *gomock.Call

CacheSeriesMetadata indicates an expected call of CacheSeriesMetadata

func (*MockRunOptionsMockRecorder) InitialTopologyState added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) InitialTopologyState() *gomock.Call

InitialTopologyState indicates an expected call of InitialTopologyState

func (*MockRunOptionsMockRecorder) PersistConfig added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) PersistConfig() *gomock.Call

PersistConfig indicates an expected call of PersistConfig

func (*MockRunOptionsMockRecorder) SetCacheSeriesMetadata 

func (mr *MockRunOptionsMockRecorder) SetCacheSeriesMetadata(value interface{}) *gomock.Call

SetCacheSeriesMetadata indicates an expected call of SetCacheSeriesMetadata

func (*MockRunOptionsMockRecorder) SetInitialTopologyState added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) SetInitialTopologyState(value interface{}) *gomock.Call

SetInitialTopologyState indicates an expected call of SetInitialTopologyState

func (*MockRunOptionsMockRecorder) SetPersistConfig added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) SetPersistConfig(value interface{}) *gomock.Call

SetPersistConfig indicates an expected call of SetPersistConfig

type MockSource 

type MockSource struct {
	// contains filtered or unexported fields
}

MockSource is a mock of Source interface

func NewMockSource 

func NewMockSource(ctrl *gomock.Controller) *MockSource

NewMockSource creates a new mock instance

func (*MockSource) AvailableData 

func (m *MockSource) AvailableData(ns namespace.Metadata, shardsTimeRanges result.ShardTimeRanges, runOpts RunOptions) (result.ShardTimeRanges, error)

AvailableData mocks base method

func (*MockSource) AvailableIndex 

func (m *MockSource) AvailableIndex(ns namespace.Metadata, shardsTimeRanges result.ShardTimeRanges, opts RunOptions) (result.ShardTimeRanges, error)

AvailableIndex mocks base method

func (*MockSource) EXPECT 

func (m *MockSource) EXPECT() *MockSourceMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockSource) Read added in v0.15.0 

func (m *MockSource) Read(namespaces Namespaces) (NamespaceResults, error)

Read mocks base method

type MockSourceMockRecorder 

type MockSourceMockRecorder struct {
	// contains filtered or unexported fields
}

MockSourceMockRecorder is the mock recorder for MockSource

func (*MockSourceMockRecorder) AvailableData 

func (mr *MockSourceMockRecorder) AvailableData(ns, shardsTimeRanges, runOpts interface{}) *gomock.Call

AvailableData indicates an expected call of AvailableData

func (*MockSourceMockRecorder) AvailableIndex 

func (mr *MockSourceMockRecorder) AvailableIndex(ns, shardsTimeRanges, opts interface{}) *gomock.Call

AvailableIndex indicates an expected call of AvailableIndex

func (*MockSourceMockRecorder) Read added in v0.15.0 

func (mr *MockSourceMockRecorder) Read(namespaces interface{}) *gomock.Call

Read indicates an expected call of Read

type Namespace added in v0.15.0 

type Namespace struct {
	// Metadata of the namespace being bootstrapped.
	Metadata namespace.Metadata
	// Shards is the shards for the namespace being bootstrapped.
	Shards []uint32
	// DataAccumulator is the data accumulator for the shards.
	DataAccumulator NamespaceDataAccumulator
	// Hooks is a set of namespace bootstrap hooks.
	Hooks NamespaceHooks
	// DataTargetRange is the data target bootstrap range.
	DataTargetRange TargetRange
	// IndexTargetRange is the index target bootstrap range.
	IndexTargetRange TargetRange
	// DataRunOptions are the options for the data bootstrap for this
	// namespace.
	DataRunOptions NamespaceRunOptions
	// IndexRunOptions are the options for the index bootstrap for this
	// namespace.
	IndexRunOptions NamespaceRunOptions
}

Namespace is a namespace that is being bootstrapped.

type NamespaceDataAccumulator added in v0.15.0 

type NamespaceDataAccumulator interface {
	// CheckoutSeriesWithoutLock retrieves a series for writing to
	// and when the accumulator is closed it will ensure that the
	// series is released.
	//
	// If indexing is not enabled, tags is still required, simply pass
	// ident.EmptyTagIterator.
	//
	// Returns the result, whether the node owns the specified shard, along with
	// an error if any. This allows callers to handle unowned shards differently
	// than other errors. If owned == false, err should not be nil.
	//
	// Note: Without lock variant does not perform any locking and callers
	// must ensure non-parallel access themselves, this helps avoid
	// overhead of the lock for the commit log bootstrapper which reads
	// in a single threaded manner.
	CheckoutSeriesWithoutLock(
		shardID uint32,
		id ident.ID,
		tags ident.TagIterator,
	) (result CheckoutSeriesResult, owned bool, err error)

	// CheckoutSeriesWithLock is the "with lock" version of
	// CheckoutSeriesWithoutLock.
	// Note: With lock variant performs locking and callers do not need
	// to be concerned about parallel access.
	CheckoutSeriesWithLock(
		shardID uint32,
		id ident.ID,
		tags ident.TagIterator,
	) (result CheckoutSeriesResult, owned bool, err error)

	// Close will close the data accumulator and will release
	// all series read/write refs.
	Close() error
}

NamespaceDataAccumulator is the namespace data accumulator. TODO(r): Consider rename this to a better name.

type NamespaceHooks added in v0.15.0 

type NamespaceHooks struct {
	// contains filtered or unexported fields
}

NamespaceHooks is a set of namespace bootstrap hooks.

func NewNamespaceHooks added in v0.15.0 

func NewNamespaceHooks(opts NamespaceHooksOptions) NamespaceHooks

NewNamespaceHooks returns a new set of bootstrap hooks.

func (NamespaceHooks) BootstrapSourceBegin added in v0.15.0 

func (h NamespaceHooks) BootstrapSourceBegin() error

BootstrapSourceBegin is a hook to call when a bootstrap source starts.

func (NamespaceHooks) BootstrapSourceEnd added in v0.15.0 

func (h NamespaceHooks) BootstrapSourceEnd() error

BootstrapSourceEnd is a hook to call when a bootstrap source ends.

type NamespaceHooksOptions added in v0.15.0 

type NamespaceHooksOptions struct {
	BootstrapSourceBegin func() error
	BootstrapSourceEnd   func() error
}

NamespaceHooksOptions is a set of hooks options.

type NamespaceMatcher added in v0.15.0 

type NamespaceMatcher struct {
	// Namespaces are the expected namespaces.
	Namespaces Namespaces
}

NamespaceMatcher is a matcher for namespaces.

func (NamespaceMatcher) Matches added in v0.15.0 

func (m NamespaceMatcher) Matches(x interface{}) bool

Matches returns whether x is a match.

func (NamespaceMatcher) String added in v0.15.0 

func (m NamespaceMatcher) String() string

String describes what the matcher matches.

type NamespaceResult added in v0.15.0 

type NamespaceResult struct {
	Metadata    namespace.Metadata
	Shards      []uint32
	DataResult  result.DataBootstrapResult
	IndexResult result.IndexBootstrapResult
}

NamespaceResult is the result of a bootstrap process for a given namespace.

type NamespaceResults added in v0.15.0 

type NamespaceResults struct {
	// Results is the result of a bootstrap process.
	Results *NamespaceResultsMap
}

NamespaceResults is the result of a bootstrap process.

func MergeNamespaceResults added in v0.15.0 

func MergeNamespaceResults(a, b NamespaceResults) NamespaceResults

MergeNamespaceResults merges two namespace results, this will mutate both a and b and return a merged copy of them reusing one of the results.

func NewNamespaceResults added in v0.15.0 

func NewNamespaceResults(
	namespaces Namespaces,
) NamespaceResults

NewNamespaceResults creates a namespace results map with an entry for each namespace spoecified by a namespaces map.

type NamespaceResultsMap added in v0.15.0 

type NamespaceResultsMap struct {
	// contains filtered or unexported fields
}

NamespaceResultsMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. NamespaceResultsMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewNamespaceResultsMap added in v0.15.0 

func NewNamespaceResultsMap(opts NamespaceResultsMapOptions) *NamespaceResultsMap

NewNamespaceResultsMap returns a new byte keyed map.

func (*NamespaceResultsMap) Contains added in v0.15.0 

func (m *NamespaceResultsMap) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*NamespaceResultsMap) Delete added in v0.15.0 

func (m *NamespaceResultsMap) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*NamespaceResultsMap) Get added in v0.15.0 

func (m *NamespaceResultsMap) Get(k ident.ID) (NamespaceResult, bool)

Get returns a value in the map for an identifier if found.

func (*NamespaceResultsMap) Iter added in v0.15.0 

func (m *NamespaceResultsMap) Iter() map[NamespaceResultsMapHash]NamespaceResultsMapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*NamespaceResultsMap) Len added in v0.15.0 

func (m *NamespaceResultsMap) Len() int

Len returns the number of map entries in the map.

func (*NamespaceResultsMap) Reallocate added in v0.15.0 

func (m *NamespaceResultsMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*NamespaceResultsMap) Reset added in v0.15.0 

func (m *NamespaceResultsMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*NamespaceResultsMap) Set added in v0.15.0 

func (m *NamespaceResultsMap) Set(k ident.ID, v NamespaceResult)

Set will set the value for an identifier.

func (*NamespaceResultsMap) SetUnsafe added in v0.15.0 

func (m *NamespaceResultsMap) SetUnsafe(k ident.ID, v NamespaceResult, opts NamespaceResultsMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type NamespaceResultsMapCopyFn added in v0.15.0 

type NamespaceResultsMapCopyFn func(ident.ID) ident.ID

NamespaceResultsMapCopyFn is the copy key function to execute when copying the key.

type NamespaceResultsMapEntry added in v0.15.0 

type NamespaceResultsMapEntry struct {
	// contains filtered or unexported fields
}

NamespaceResultsMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (NamespaceResultsMapEntry) Key added in v0.15.0 

func (e NamespaceResultsMapEntry) Key() ident.ID

Key returns the map entry key.

func (NamespaceResultsMapEntry) Value added in v0.15.0 

func (e NamespaceResultsMapEntry) Value() NamespaceResult

Value returns the map entry value.

type NamespaceResultsMapEqualsFn added in v0.15.0 

type NamespaceResultsMapEqualsFn func(ident.ID, ident.ID) bool

NamespaceResultsMapEqualsFn is the equals key function to execute when detecting equality of a key.

type NamespaceResultsMapFinalizeFn added in v0.15.0 

type NamespaceResultsMapFinalizeFn func(ident.ID)

NamespaceResultsMapFinalizeFn is the finalize key function to execute when finished with a key.

type NamespaceResultsMapHash added in v0.15.0 

type NamespaceResultsMapHash uint64

NamespaceResultsMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type NamespaceResultsMapHashFn added in v0.15.0 

type NamespaceResultsMapHashFn func(ident.ID) NamespaceResultsMapHash

NamespaceResultsMapHashFn is the hash function to execute when hashing a key.

type NamespaceResultsMapOptions added in v0.15.0 

type NamespaceResultsMapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

NamespaceResultsMapOptions provides options used when created the map.

type NamespaceResultsMapSetUnsafeOptions added in v0.15.0 

type NamespaceResultsMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

NamespaceResultsMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type NamespaceRunOptions added in v0.15.0 

type NamespaceRunOptions struct {
	// ShardTimeRanges are the time ranges for the shards that should be fulfilled
	// by the bootstrapper.
	ShardTimeRanges result.ShardTimeRanges
	// RunOptions are the run options for the bootstrap run.
	RunOptions RunOptions
}

NamespaceRunOptions are the run options for a bootstrap process run.

type Namespaces added in v0.15.0 

type Namespaces struct {
	// Namespaces are the namespaces being bootstrapped.
	Namespaces *NamespacesMap
}

Namespaces are a set of namespaces being bootstrapped.

func NewNamespaces added in v0.15.0 

func NewNamespaces(
	namespaces []ProcessNamespace,
) Namespaces

NewNamespaces returns a new set of bootstrappable namespaces.

func (Namespaces) Hooks added in v0.15.0 

func (n Namespaces) Hooks() NamespacesHooks

Hooks returns namespaces hooks for the set of namespaces.

type NamespacesHooks added in v0.15.0 

type NamespacesHooks struct {
	// contains filtered or unexported fields
}

NamespacesHooks is a helper to run hooks for a set of namespaces.

func (NamespacesHooks) BootstrapSourceBegin added in v0.15.0 

func (h NamespacesHooks) BootstrapSourceBegin() error

BootstrapSourceBegin is a hook to call when a bootstrap source starts.

func (NamespacesHooks) BootstrapSourceEnd added in v0.15.0 

func (h NamespacesHooks) BootstrapSourceEnd() error

BootstrapSourceEnd is a hook to call when a bootstrap source starts.

type NamespacesMap added in v0.15.0 

type NamespacesMap struct {
	// contains filtered or unexported fields
}

NamespacesMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. NamespacesMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewNamespacesMap added in v0.15.0 

func NewNamespacesMap(opts NamespacesMapOptions) *NamespacesMap

NewNamespacesMap returns a new byte keyed map.

func (*NamespacesMap) Contains added in v0.15.0 

func (m *NamespacesMap) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*NamespacesMap) Delete added in v0.15.0 

func (m *NamespacesMap) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*NamespacesMap) Get added in v0.15.0 

func (m *NamespacesMap) Get(k ident.ID) (Namespace, bool)

Get returns a value in the map for an identifier if found.

func (*NamespacesMap) Iter added in v0.15.0 

func (m *NamespacesMap) Iter() map[NamespacesMapHash]NamespacesMapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*NamespacesMap) Len added in v0.15.0 

func (m *NamespacesMap) Len() int

Len returns the number of map entries in the map.

func (*NamespacesMap) Reallocate added in v0.15.0 

func (m *NamespacesMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*NamespacesMap) Reset added in v0.15.0 

func (m *NamespacesMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*NamespacesMap) Set added in v0.15.0 

func (m *NamespacesMap) Set(k ident.ID, v Namespace)

Set will set the value for an identifier.

func (*NamespacesMap) SetUnsafe added in v0.15.0 

func (m *NamespacesMap) SetUnsafe(k ident.ID, v Namespace, opts NamespacesMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type NamespacesMapCopyFn added in v0.15.0 

type NamespacesMapCopyFn func(ident.ID) ident.ID

NamespacesMapCopyFn is the copy key function to execute when copying the key.

type NamespacesMapEntry added in v0.15.0 

type NamespacesMapEntry struct {
	// contains filtered or unexported fields
}

NamespacesMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (NamespacesMapEntry) Key added in v0.15.0 

func (e NamespacesMapEntry) Key() ident.ID

Key returns the map entry key.

func (NamespacesMapEntry) Value added in v0.15.0 

func (e NamespacesMapEntry) Value() Namespace

Value returns the map entry value.

type NamespacesMapEqualsFn added in v0.15.0 

type NamespacesMapEqualsFn func(ident.ID, ident.ID) bool

NamespacesMapEqualsFn is the equals key function to execute when detecting equality of a key.

type NamespacesMapFinalizeFn added in v0.15.0 

type NamespacesMapFinalizeFn func(ident.ID)

NamespacesMapFinalizeFn is the finalize key function to execute when finished with a key.

type NamespacesMapHash added in v0.15.0 

type NamespacesMapHash uint64

NamespacesMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type NamespacesMapHashFn added in v0.15.0 

type NamespacesMapHashFn func(ident.ID) NamespacesMapHash

NamespacesMapHashFn is the hash function to execute when hashing a key.

type NamespacesMapOptions added in v0.15.0 

type NamespacesMapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

NamespacesMapOptions provides options used when created the map.

type NamespacesMapSetUnsafeOptions added in v0.15.0 

type NamespacesMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

NamespacesMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type NamespacesTester added in v0.15.0 

type NamespacesTester struct {

	// Accumulators are the accumulators which incoming blocks get loaded into.
	// One per namespace.
	Accumulators []*TestDataAccumulator

	// Namespaces are the namespaces for this tester.
	Namespaces Namespaces
	// Results are the namespace results after bootstrapping.
	Results NamespaceResults
	// contains filtered or unexported fields
}

NamespacesTester is a utility to assist testing bootstrapping.

func BuildNamespacesTester added in v0.15.0 

func BuildNamespacesTester(
	t require.TestingT,
	runOpts RunOptions,
	ranges result.ShardTimeRanges,
	mds ...namespace.Metadata,
) NamespacesTester

BuildNamespacesTester builds a NamespacesTester.

func BuildNamespacesTesterWithReaderIteratorPool added in v0.15.0 

func BuildNamespacesTesterWithReaderIteratorPool(
	t require.TestingT,
	runOpts RunOptions,
	ranges result.ShardTimeRanges,
	iterPool encoding.MultiReaderIteratorPool,
	mds ...namespace.Metadata,
) NamespacesTester

BuildNamespacesTesterWithReaderIteratorPool builds a NamespacesTester with a given MultiReaderIteratorPool.

func (*NamespacesTester) DumpLoadedBlocks added in v0.15.0 

func (nt *NamespacesTester) DumpLoadedBlocks() DecodedNamespaceMap

DumpLoadedBlocks dumps any loaded blocks as decoded series per namespace.

func (*NamespacesTester) DumpWrites added in v0.15.0 

func (nt *NamespacesTester) DumpWrites() DecodedNamespaceMap

DumpWrites dumps the writes encountered for all namespaces.

func (*NamespacesTester) EnsureDumpAllForNamespace added in v0.15.0 

func (nt *NamespacesTester) EnsureDumpAllForNamespace(
	md namespace.Metadata,
) (DecodedBlockMap, error)

EnsureDumpAllForNamespace dumps all results for a single namespace, and fails if the namespace is not found. The results are unsorted; if sorted order is important for verification, they should be sorted afterwards.

func (*NamespacesTester) EnsureDumpLoadedBlocksForNamespace added in v0.15.0 

func (nt *NamespacesTester) EnsureDumpLoadedBlocksForNamespace(
	md namespace.Metadata,
) DecodedBlockMap

EnsureDumpLoadedBlocksForNamespace dumps all loaded blocks as decoded series, and fails if the namespace is not found.

func (*NamespacesTester) EnsureDumpReadersForNamespace added in v0.15.0 

func (nt *NamespacesTester) EnsureDumpReadersForNamespace(
	md namespace.Metadata,
) ReaderMap

EnsureDumpReadersForNamespace dumps the readers and their start times for a given namespace, and fails if the namespace is not found.

func (*NamespacesTester) EnsureDumpWritesForNamespace added in v0.15.0 

func (nt *NamespacesTester) EnsureDumpWritesForNamespace(
	md namespace.Metadata,
) DecodedBlockMap

EnsureDumpWritesForNamespace dumps the writes encountered for the given namespace, and fails if the namespace is not found.

func (*NamespacesTester) EnsureNoLoadedBlocks added in v0.15.0 

func (nt *NamespacesTester) EnsureNoLoadedBlocks()

EnsureNoLoadedBlocks ensures that no blocks have been loaded into any of this testers accumulators.

func (*NamespacesTester) EnsureNoWrites added in v0.15.0 

func (nt *NamespacesTester) EnsureNoWrites()

EnsureNoWrites ensures that no writes have been written into any of this testers accumulators.

func (*NamespacesTester) Finish added in v0.15.0 

func (nt *NamespacesTester) Finish()

Finish closes the namespaceTester and tests mocks for completion.

func (*NamespacesTester) ResultForNamespace added in v0.15.0 

func (nt *NamespacesTester) ResultForNamespace(id ident.ID) NamespaceResult

ResultForNamespace gives the result for the given namespace, and fails if the namespace is not found.

func (*NamespacesTester) TestBootstrapWith added in v0.15.0 

func (nt *NamespacesTester) TestBootstrapWith(b Bootstrapper)

TestBootstrapWith bootstraps the current Namespaces with the provided bootstrapper.

func (*NamespacesTester) TestReadWith added in v0.15.0 

func (nt *NamespacesTester) TestReadWith(s Source)

TestReadWith reads the current Namespaces with the provided bootstrap source.

func (*NamespacesTester) TestUnfulfilledForIDIsEmpty added in v0.15.0 

func (nt *NamespacesTester) TestUnfulfilledForIDIsEmpty(
	id ident.ID,
	useIndex bool,
)

TestUnfulfilledForIDIsEmpty ensures the given id has an empty unfulfilled range.

func (*NamespacesTester) TestUnfulfilledForNamespace added in v0.15.0 

func (nt *NamespacesTester) TestUnfulfilledForNamespace(
	md namespace.Metadata,
	ex result.ShardTimeRanges,
	exIdx result.ShardTimeRanges,
)

TestUnfulfilledForNamespace ensures the given namespace has the expected range flagged as unfulfilled.

func (*NamespacesTester) TestUnfulfilledForNamespaceIsEmpty added in v0.15.0 

func (nt *NamespacesTester) TestUnfulfilledForNamespaceIsEmpty(
	md namespace.Metadata,
)

TestUnfulfilledForNamespaceIsEmpty ensures the given namespace has an empty unfulfilled range.

type PersistConfig added in v0.4.4 

type PersistConfig struct {
	// If enabled bootstrappers are allowed to write out bootstrapped data
	// to disk on their own instead of just returning result in-memory.
	Enabled bool
	// If enabled, what type of persistence files should be generated during
	// the process.
	FileSetType persist.FileSetType
}

PersistConfig is the configuration for a bootstrap with persistence.

type Process 

type Process interface {
	// Run runs the bootstrap process, returning the bootstrap result and any error encountered.
	Run(start time.Time, namespaces []ProcessNamespace) (NamespaceResults, error)
}

Process represents the bootstrap process. Note that a bootstrap process can and will be reused so it is important to not rely on state stored in the bootstrap itself with the mindset that it will always be set to default values from the constructor.

type ProcessNamespace added in v0.15.0 

type ProcessNamespace struct {
	// Metadata of the namespace being bootstrapped.
	Metadata namespace.Metadata
	// Shards is the shards to bootstrap for the bootstrap process.
	Shards []uint32
	// DataAccumulator is the data accumulator for the shards.
	DataAccumulator NamespaceDataAccumulator
	// Hooks is a set of namespace bootstrap hooks.
	Hooks NamespaceHooks
}

ProcessNamespace is a namespace to pass to the bootstrap process.

type ProcessOptions 

type ProcessOptions interface {
	// SetCacheSeriesMetadata sets whether bootstrappers created by this
	// provider should cache series metadata between runs.
	SetCacheSeriesMetadata(value bool) ProcessOptions

	// CacheSeriesMetadata returns whether bootstrappers created by this
	// provider should cache series metadata between runs.
	CacheSeriesMetadata() bool

	// SetTopologyMapProvider sets the TopologyMapProvider.
	SetTopologyMapProvider(value topology.MapProvider) ProcessOptions

	// TopologyMapProvider returns the TopologyMapProvider.
	TopologyMapProvider() topology.MapProvider

	// SetOrigin sets the origin.
	SetOrigin(value topology.Host) ProcessOptions

	// Origin returns the origin.
	Origin() topology.Host

	// Validate validates that the ProcessOptions are correct.
	Validate() error
}

ProcessOptions is a set of options for a bootstrap provider.

func NewProcessOptions 

func NewProcessOptions() ProcessOptions

NewProcessOptions creates new bootstrap run options

type ProcessProvider 

type ProcessProvider interface {
	// SetBootstrapper sets the bootstrapper provider to use when running the
	// process.
	SetBootstrapperProvider(bootstrapper BootstrapperProvider)

	// Bootstrapper returns the current bootstrappe provider to use when
	// running the process.
	BootstrapperProvider() BootstrapperProvider

	// Provide constructs a bootstrap process.
	Provide() (Process, error)
}

ProcessProvider constructs a bootstrap process that can execute a bootstrap run.

func NewNoOpProcessProvider 

func NewNoOpProcessProvider() ProcessProvider

NewNoOpProcessProvider creates a no-op bootstrap process proivder.

func NewProcessProvider 

func NewProcessProvider(
	bootstrapperProvider BootstrapperProvider,
	processOpts ProcessOptions,
	resultOpts result.Options,
) (ProcessProvider, error)

NewProcessProvider creates a new bootstrap process provider.

type ReaderAtTime added in v0.15.0 

type ReaderAtTime struct {
	// Start is the block start time.
	Start time.Time
	// Reader is the block segment reader.
	Reader xio.SegmentReader
	// Tags is the list of tags in a basic string map format.
	Tags map[string]string
}

ReaderAtTime captures incoming block loads, including their start times and tags.

type ReaderMap added in v0.15.0 

type ReaderMap map[string]ReadersForID

ReaderMap is a map containing all gathered block segment readers.

type ReadersForID added in v0.15.0 

type ReadersForID []ReaderAtTime

ReadersForID is a slice of readers that share a series ID.

type RunOptions 

type RunOptions interface {
	// SetPersistConfig sets persistence configuration for this bootstrap.
	SetPersistConfig(value PersistConfig) RunOptions

	// PersistConfig returns the persistence configuration for this bootstrap.
	PersistConfig() PersistConfig

	// SetCacheSeriesMetadata sets whether bootstrappers created by this
	// provider should cache series metadata between runs.
	SetCacheSeriesMetadata(value bool) RunOptions

	// CacheSeriesMetadata returns whether bootstrappers created by this
	// provider should cache series metadata between runs.
	CacheSeriesMetadata() bool

	// SetInitialTopologyState sets the initial topology state as it was
	// measured before the bootstrap process began.
	SetInitialTopologyState(value *topology.StateSnapshot) RunOptions

	// InitialTopologyState returns the initial topology as it was measured
	// before the bootstrap process began.
	InitialTopologyState() *topology.StateSnapshot
}

RunOptions is a set of options for a bootstrap run.

func NewRunOptions 

func NewRunOptions() RunOptions

NewRunOptions creates new bootstrap run options

type ShardTimeRangesMatcher added in v0.15.0 

type ShardTimeRangesMatcher struct {
	// Ranges are the expected ranges.
	Ranges result.ShardTimeRanges
}

ShardTimeRangesMatcher is a matcher for ShardTimeRanges.

func (ShardTimeRangesMatcher) Matches added in v0.15.0 

func (m ShardTimeRangesMatcher) Matches(x interface{}) bool

Matches returns whether x is a match.

func (ShardTimeRangesMatcher) String added in v0.15.0 

func (m ShardTimeRangesMatcher) String() string

String describes what the matcher matches.

type Source 

type Source interface {
	// AvailableData returns what time ranges are available for bootstrapping a given set of shards.
	AvailableData(
		ns namespace.Metadata,
		shardsTimeRanges result.ShardTimeRanges,
		runOpts RunOptions,
	) (result.ShardTimeRanges, error)

	// AvailableIndex returns what time ranges are available for bootstrapping.
	AvailableIndex(
		ns namespace.Metadata,
		shardsTimeRanges result.ShardTimeRanges,
		opts RunOptions,
	) (result.ShardTimeRanges, error)

	// Read returns series data and index metadata for a given set of shards
	// and specified time ranges and the time ranges it's unable to fulfill.
	// A bootstrapper source should only return an error should it want to
	// entirely cancel the bootstrapping of the node, i.e. non-recoverable
	// situation like not being able to read from the filesystem.
	Read(namespaces Namespaces) (NamespaceResults, error)
}

Source represents a bootstrap source. Note that a source can and will be reused so it is important to not rely on state stored in the source itself with the mindset that it will always be set to default values from the constructor.

type TargetRange 

type TargetRange struct {
	// Range is the time range to bootstrap for.
	Range xtime.Range

	// RunOptions is the bootstrap run options specific to the target range.
	RunOptions RunOptions
}

TargetRange is a bootstrap target range.

type TestDataAccumulator added in v0.15.0 

type TestDataAccumulator struct {
	sync.Mutex
	// contains filtered or unexported fields
}

TestDataAccumulator is a NamespaceDataAccumulator that captures any series inserts for examination.

func (*TestDataAccumulator) CheckoutSeriesWithLock added in v0.15.0 

func (a *TestDataAccumulator) CheckoutSeriesWithLock(
	shardID uint32,
	id ident.ID,
	tags ident.TagIterator,
) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithLock will retrieve a series for writing to, and when the accumulator is closed, it will ensure that the series is released (with lock).

func (*TestDataAccumulator) CheckoutSeriesWithoutLock added in v0.15.0 

func (a *TestDataAccumulator) CheckoutSeriesWithoutLock(
	shardID uint32,
	id ident.ID,
	tags ident.TagIterator,
) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithoutLock will retrieve a series for writing to, and when the accumulator is closed, it will ensure that the series is released (without lock).

func (*TestDataAccumulator) Close added in v0.15.0 

func (a *TestDataAccumulator) Close() error

Close is a no-op on the test accumulator.

func (*TestDataAccumulator) Release added in v0.15.0 

func (a *TestDataAccumulator) Release()

Release is a no-op on the test accumulator.

Source Files

View all Source files

Directories

Path Synopsis
commitlog
fs
peers
uninitialized

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.