config

package
v0.0.0-...-837f88b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 26, 2017 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package config defines a package used for managing the configuration of 0-Disk services

The config package is split up into 3 major parts: source for the different config source options, config API that sets up an API for each subconfig, giving each a Read and Watch function and composition API that sets up the same API combining certain subconfigs to enable direct access the complete config of the different storage types (NBD/Tlog), giving each a Read and Watch function as well.

No Write or Set function is provided as the 0-Disk should only read the config. However for testing purposes a stub source with setters is provided.

Source

Config sources are provided allowing to get the configuration from different sources, intended for different purposes. A Source interface is defined allowing for the config to interact with the different sources.

The etcd source (intended for production) allows for getting the config from an etcd(v3) cluster (https://github.com/coreos/etcd). An etcd source needs 1 or more endpoints (dailstrings) to connect to an etcd cluster. Each subconfig for a vdisk is stored in different keys on the etcd server, the keys for each subconfig can be found in the etcd section of the 0-Disk config documentation: https://github.com/zero-os/0-Disk/blob/master/docs/config.md#etcd. The values stored in those keys are expected to be valid subconfigs serialised in YAML format. In case a key's data was not found valid, the MarkInvalidKey function will be called, which will log an error as well as broadcast the error to the 0-Orchestrator. For updates of the config, the etcd source uses the etcd Watch API (https://coreos.com/etcd/docs/latest/learning/api.html#watch-api) for each etcd key and sends the updated config key value to the channel returned by the Watch function.

The file source (intended for development) allows for getting the config from a YAML file. A file source needs a file path to where the config file can be found. How the YAML file should be formatted can be found in the file section of the 0-Disk config documentation: https://github.com/zero-os/0-Disk/blob/master/docs/config.md#file. For updates of the config, the file source will listen for a SIGHUP signal, each config key being watched by the Watch function will then read the config file and send the current subconfig in the file to the channel returned by the Watch function.

The stub source (intended for testing) allows for tests to use a stubbed config where user has control of the data in the source, the stub source can be written to using the setters while other sources are not writable from 0-Disk, these setters also trigger a reload, sending the newly set values to the config keys that are being watched using the Watch function.

SourceConfig is a wrapper source type that switches between a file source or etcd source based on the provided input string. If it is a dailstring or list of dailstrings, it will assume that the source should be an etcd source. If it is an empty string, it will use the default file resource (config.yml) to create a file source. If it was none of the above it will asume the string is a path to a YAML file for a file source.

Config API

The config API allows to interact with the subconfigs found on the source. For each subconfig a Read and Watch function is provided.

The Read will read the current subconfig from the provided source and vdisk/cluster/server ID and returns it when valid.

The Watch function creates a goroutine that listens for updates from the provided source on the config key of the subconfig and provided vdisk/cluster/server ID. The go routine will close when the provided context is cancelled. Should the provided context be nil, it will set the internal context to Background and runs the goroutine for as long as the program remains running.

Composition API

The composition API provides Read and Watch functions for getting full NBD or Tlog storage server configuration, combining the subconfigs required for each storage type and they are used in exactly the same way as the config API.

The NBD storage consists of following subconfigs: Static vdisk config (VdiskStaticConfig), NBD vdisk config (VdiskNBDConfig), primary storage cluster config (StorageClusterConfig), template storage cluster config (StorageClusterConfig), slave storage cluster config (StorageClusterConfig)

The Tlog storage consists of following subconfigs: Static vdisk config (VdiskStaticConfig), Tlog vdisk config (VdiskTlogConfig), 0-Stor cluster config (ZeroStorClusterConfig), slave storage cluster config (StorageClusterConfig)

Some of the subconfigs are optional and are only used for certain storage types.

The Read function will return a full NBDStorageConfig/TlogStorageConfig from provided source, vdisk ID and static configuration if the configuration is valid.

The Watch function will update the subconfigs through their Watch functions as they are received from the source and send an updated storage server config to the channel returned fron the Watch function.

More Information

More information about the config module can be found in the 0-Disk config documentation: https://github.com/zero-os/0-Disk/blob/master/docs/config.md

Index

Constants

View Source 
const (
	VdiskTypeBoot  = propDeduped | propPersistent | propTlogSupport | propTemplateSupport
	VdiskTypeDB    = propPersistent | propTlogSupport | propTemplateSupport
	VdiskTypeCache = propPersistent
	VdiskTypeTmp   = propTemporary
)

valid vdisk types based on /docs/README.md#zero-os-0-disk

View Source 
const (
	KeyVdiskStaticStr     = "VdiskStatic"
	KeyVdiskNBDStr        = "VdiskNBD"
	KeyVdiskTlogStr       = "VdiskTlog"
	KeyClusterStorageStr  = "ClusterStorage"
	KeyClusterTlogStr     = "ClusterTlog"
	KeyNBDServerVdisksStr = "NBDServerVdisks"
)

KeyType string representations

Variables

View Source 
var (
	// ErrNilSource is returned from any config API
	// function which received a nil source as its parameter.
	ErrNilSource = errors.New("no config source given")

	// ErrContextDone is returned from any config watch function
	// where the context is already finished,
	// while the watch function is still being created.
	ErrContextDone = errors.New("context is already done")

	// ErrNilResource is returned when for a given config function
	// a nil resource was given by the user.
	ErrNilResource = errors.New("invalid nil resource given")

	// ErrNilID is returned when for a given config function
	// a nil string was given as the id.
	ErrNilID = errors.New("nil ID given")

	// ErrSourceUnavailable is returned when the given source
	// is unavailable due to any kind of critical failure.
	ErrSourceUnavailable = errors.New("config: source is unavailable")

	// ErrConfigUnavailable is returned when a requested config
	// could not be found in the given source.
	ErrConfigUnavailable = errors.New("config is not available in source")

	// ErrInvalidKey is returned when a config is requested using
	// a config key which is either invalid or not supported by
	// the given source.
	ErrInvalidKey = errors.New("config key is invalid or not supported")

	// ErrNilConfig is returned when the given config was nil
	ErrNilConfig = errors.New("config is nil")

	// ErrInvalidConfig is returned when the given config was invalid
	ErrInvalidConfig = errors.New("config is invalid")

	// ErrNilStorage is returned when a storage was nil while being required
	ErrNilStorage = errors.New("storage is nil while it is required")
)

Error constants

Functions

func ETCDKey 

func ETCDKey(id string, ktype KeyType) (string, error)

ETCDKey returns the etcd key for a given type and unique id.

func IsServiceAddress 

func IsServiceAddress(str string) bool

IsServiceAddress checks if a string is a valid service address, which means it's either a unix address or a dial string.

func NewInvalidConfigError 

func NewInvalidConfigError(err error) error

NewInvalidConfigError creates a new error with ErrInvalidConfig as Cause with the provided error as wrapped context

func ReadConfig 

func ReadConfig(source Source, id string, keyType KeyType) ([]byte, error)

ReadConfig returns the requested config as a byte slice from the given source.

func ValidateBlockSize 

func ValidateBlockSize(bs int64) bool

ValidateBlockSize allows you to validate a block size, returning true if the given block size is valid.

func ValidateNBDServerConfigs 

func ValidateNBDServerConfigs(source Source, serverID string) error

ValidateNBDServerConfigs validates all available NBD Vdisk Configurations, for a given NBD server config, using a given config source.

func ValidateTlogServerConfigs 

func ValidateTlogServerConfigs(source Source, serverID string) error

ValidateTlogServerConfigs validates all available Tlog Vdisk Configurations, for a given Tlog server config, using a given config source.

func WatchNBDStorageConfig 

func WatchNBDStorageConfig(ctx context.Context, source Source, vdiskID string) (<-chan NBDStorageConfig, error)

WatchNBDStorageConfig watches a given source for NBDStorageConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it.

func WatchNBDVdisksConfig 

func WatchNBDVdisksConfig(ctx context.Context, source Source, serverID string) (<-chan NBDVdisksConfig, error)

WatchNBDVdisksConfig watches a given source for NBDVdisksConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it. An error is returned in case the watcher couldn't be started.

func WatchStorageClusterConfig 

func WatchStorageClusterConfig(ctx context.Context, source Source, clusterID string) (<-chan StorageClusterConfig, error)

WatchStorageClusterConfig watches a given source for StorageClusterConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it. An error is returned in case the watcher couldn't be started.

func WatchTlogClusterConfig 

func WatchTlogClusterConfig(ctx context.Context, source Source, clusterID string) (<-chan TlogClusterConfig, error)

WatchTlogClusterConfig watches a given source for TlogClusterConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it. An error is returned in case the watcher couldn't be started.

func WatchTlogStorageConfig 

func WatchTlogStorageConfig(ctx context.Context, source Source, vdiskID string) (<-chan TlogStorageConfig, error)

WatchTlogStorageConfig watches a given source for TlogStorageConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it.

func WatchVdiskNBDConfig 

func WatchVdiskNBDConfig(ctx context.Context, source Source, vdiskID string) (<-chan VdiskNBDConfig, error)

WatchVdiskNBDConfig watches a given source for VdiskNBDConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it. An error is returned in case the watcher couldn't be started.

func WatchVdiskTlogConfig 

func WatchVdiskTlogConfig(ctx context.Context, source Source, vdiskID string) (<-chan VdiskTlogConfig, error)

WatchVdiskTlogConfig watches a given source for VdiskTlogConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it. An error is returned in case the watcher couldn't be started.

func WatchZeroStorClusterConfig 

func WatchZeroStorClusterConfig(ctx context.Context, source Source, clusterID string) (<-chan ZeroStorClusterConfig, error)

WatchZeroStorClusterConfig watches a given source for ZeroStorClusterConfig updates. Sends the initial config to the channel when created, as well as any future updated versions of that config, for as long as the given context allows it. An error is returned in case the watcher couldn't be started.

Types

type FileFormatCompleteConfig 

type FileFormatCompleteConfig struct {
	Vdisks           map[string]FileFormatVdiskConfig `yaml:"vdisks" valid:"required"`
	StorageClusters  map[string]StorageClusterConfig  `yaml:"storageClusters" valid:"required"`
	TlogClusters     map[string]TlogClusterConfig     `yaml:"tlogClusters" valid:"optional"`
	ZeroStorClusters map[string]ZeroStorClusterConfig `yaml:"zeroStorClusters" valid:"optional"`
}

FileFormatCompleteConfig is the YAML format struct used for a zerodisk config file.

func (*FileFormatCompleteConfig) NBDVdisksConfig 

func (cfg *FileFormatCompleteConfig) NBDVdisksConfig() (*NBDVdisksConfig, error)

NBDVdisksConfig returns the NBD Vdisks configuration embedded in the YAML zerodisk config file.

func (*FileFormatCompleteConfig) StorageClusterConfig 

func (cfg *FileFormatCompleteConfig) StorageClusterConfig(id string) (*StorageClusterConfig, error)

StorageClusterConfig returns the StorageCluster configuration embedded in the YAML zerodisk config file.

func (*FileFormatCompleteConfig) TlogClusterConfig 

func (cfg *FileFormatCompleteConfig) TlogClusterConfig(id string) (*TlogClusterConfig, error)

TlogClusterConfig returns the TlogCluster configuration embedded in the YAML zerodisk config file.

func (*FileFormatCompleteConfig) VdiskConfig 

func (cfg *FileFormatCompleteConfig) VdiskConfig(id string) (*FileFormatVdiskConfig, error)

VdiskConfig returns the full vdisk configuration embedded in the YAML zerodisk config file.

func (*FileFormatCompleteConfig) ZeroStorClusterConfig 

func (cfg *FileFormatCompleteConfig) ZeroStorClusterConfig(id string) (*ZeroStorClusterConfig, error)

ZeroStorClusterConfig returns ZeroStorCluster configuration embedded in the YAML zerodisk config file

type FileFormatVdiskConfig 

type FileFormatVdiskConfig struct {
	BlockSize       uint64    `yaml:"blockSize" valid:"required"`
	ReadOnly        bool      `yaml:"readOnly" valid:"optional"`
	Size            uint64    `yaml:"size" valid:"required"`
	VdiskType       VdiskType `yaml:"type" valid:"required"`
	TemplateVdiskID string    `yaml:"vdiskTemplateID" valid:"required"`

	NBD  *VdiskNBDConfig  `yaml:"nbd" valid:"optional"`
	Tlog *VdiskTlogConfig `yaml:"tlog" valid:"optional"`
}

FileFormatVdiskConfig is the YAML format struct used for all vdisk file-originated configurations.

func (*FileFormatVdiskConfig) NBDConfig 

func (cfg *FileFormatVdiskConfig) NBDConfig() (*VdiskNBDConfig, error)

NBDConfig returns the vdisk's NBD configuration embedded in the vdisk config file format.

func (*FileFormatVdiskConfig) StaticConfig 

func (cfg *FileFormatVdiskConfig) StaticConfig() (*VdiskStaticConfig, error)

StaticConfig returns the vdisk's Static configuration embedded in the vdisk config file format.

func (*FileFormatVdiskConfig) TlogConfig 

func (cfg *FileFormatVdiskConfig) TlogConfig() (*VdiskTlogConfig, error)

TlogConfig returns the vdisk's Tlog configuration embedded in the vdisk config file format.

type FormatValidator 

type FormatValidator interface {
	// Validate the parsed format,
	// such that deserialized content is only used if valid.
	Validate() error
}

FormatValidator defines the validator interface of each format struct. All Content is serialized in the YAML 1.2 format.

type IYOCredentials 

type IYOCredentials struct {
	Org       string `yaml:"org" valid:"required"`
	Namespace string `yaml:"namespace" valid:"required"`
	ClientID  string `yaml:"clientID" valid:"optional"`
	Secret    string `yaml:"secret" valid:"optional"`
}

IYOCredentials represents Itsyou.online credentials needed for 0-Stor namespacing More information about the namespacing of the 0-Stor: https://github.com/zero-os/0-stor/blob/master/specs/concept.md#namespaces-concept

type Key 

type Key struct {
	ID   string
	Type KeyType
}

Key defines the type and unique ID of a key, which is used to fetch a config value in

func (Key) String 

func (k Key) String() string

type KeyType 

type KeyType uint8

KeyType defines the type of a key, which together with the ID makes it unique. 

const (
	KeyVdiskStatic KeyType = iota
	KeyVdiskNBD
	KeyVdiskTlog
	KeyClusterStorage
	KeyClusterZeroStor
	KeyClusterTlog
	KeyNBDServerVdisks
)

All possible KeyType values.

func (KeyType) String 

func (kt KeyType) String() string

type NBDStorageConfig 

type NBDStorageConfig struct {
	StorageCluster         StorageClusterConfig
	TemplateStorageCluster *StorageClusterConfig
	SlaveStorageCluster    *StorageClusterConfig
}

NBDStorageConfig contains all information needed to store and use the (meta)data of a vdisk.

func ReadNBDStorageConfig 

func ReadNBDStorageConfig(source Source, vdiskID string) (*NBDStorageConfig, error)

ReadNBDStorageConfig reads, validates and returns the requested NBDStorageConfig from a given source, or returns an error in case something went wrong along the way. The NBDStorageConfig is composed out of several subconfigs, and thus reading this Config might require multiple roundtrips.

func (*NBDStorageConfig) Clone 

func (cfg *NBDStorageConfig) Clone() NBDStorageConfig

Clone this config

func (*NBDStorageConfig) Validate 

func (cfg *NBDStorageConfig) Validate() error

Validate all properties of this config, using the Storage Type information for the optional properties.

type NBDVdisksConfig 

type NBDVdisksConfig struct {
	Vdisks []string `yaml:"vdisks" valid:"required"`
}

NBDVdisksConfig contains a list of all Vdisks exposed by a given NBD Server.

func NewNBDVdisksConfig 

func NewNBDVdisksConfig(data []byte) (*NBDVdisksConfig, error)

NewNBDVdisksConfig creates a new NBDVdisksConfig from a given YAML slice.

func ReadNBDVdisksConfig 

func ReadNBDVdisksConfig(source Source, serverID string) (*NBDVdisksConfig, error)

ReadNBDVdisksConfig returns the requested NBDVdisksConfig from a given config source.

func (*NBDVdisksConfig) Validate 

func (cfg *NBDVdisksConfig) Validate() error

Validate implements FormatValidator.Validate.

type ServerConfig 

type ServerConfig struct {
	Address string `yaml:"address" valid:"serviceaddress,required"`
}

ServerConfig represents a generic server config with only a service address.

type Source 

type Source interface {
	// Get a content value as a YAML byte slice,
	// using a given Key.
	Get(key Key) ([]byte, error)
	// Watch a content value based on its Key
	// and receive its value each time it is updated,
	// using a returned channel.
	Watch(ctx context.Context, key Key) (<-chan []byte, error)
	// MarkInvalidKey marks an invalid key,
	// in the hope that it can be fixed in the source,
	// and it becomes a valid key soon.
	// Optionally a vdiskID can be given,
	// in case the invalid state is related to
	// it being used for a specific vdisk.
	MarkInvalidKey(key Key, vdiskID string)
	// SourceConfig returns this source configuration.
	// For etcd this means a comma seperated list of its etcd cluster addresses.
	SourceConfig() interface{}
	// returns the type of this source
	Type() string
}

Source defines a minimalistic API used to fetch configs either once, or watching a channel for updates.

func NewOnceSource 

func NewOnceSource(source Source) Source

NewOnceSource creates a config source, that fetches the key's value from the provided source and saves in memory if the key is requested again, the value from cache will be returned. WARNING: this source is only to be used for a config that should be static, calling the Watch function on this source will result in a panic.

type SourceCloser 

type SourceCloser interface {
	Source

	// Close the config source.
	Close() error
}

SourceCloser defines a Source which can and has to be closed by the user.

func ETCDV3Source 

func ETCDV3Source(endpoints []string) (SourceCloser, error)

ETCDV3Source creates a config source, where the configurations originate from an ETCD v3 cluster.

func FileSource 

func FileSource(path string) (SourceCloser, error)

FileSource creates a config source, where the configurations originate from a file on the local file system. WARNING: this is only to be used for development and testing purposes, it is by no means intended for production.

func NewSource 

func NewSource(config SourceConfig) (SourceCloser, error)

NewSource creates a new Source based on the given configuration. Make sure to close the returned Source to avoid any leaks.

type SourceConfig 

type SourceConfig struct {
	// The resource used to identify the specific config origins.
	// Type = file -> Resource defines file path.
	// Type = etcd -> Resource defines etcd endpoints.
	Resource interface{}
	// Defines the type of source to be read from,
	// and thus also what type of Resource this is.
	SourceType SourceType
}

SourceConfig is the configuration used to create a source based on a given type and resource.

func NewSourceConfig 

func NewSourceConfig(data string) (SourceConfig, error)

NewSourceConfig creates a new source config by implicitly infering the source type based on the given data, and based on it use the data as the config's resource.

func (*SourceConfig) Set 

func (cfg *SourceConfig) Set(value string) (err error)

Set implements flag.Value.Set

func (*SourceConfig) String 

func (cfg *SourceConfig) String() string

String implements Stringer.String

func (*SourceConfig) Type 

func (cfg *SourceConfig) Type() string

Type implements pflag.Value.Type

func (*SourceConfig) Validate 

func (cfg *SourceConfig) Validate() error

Validate this SourceConfig struct.

type SourceType 

type SourceType uint8

SourceType specifies the type of config source to retrieve the configuration from. 

const (
	// FileSourceType defines the file config resource
	// This is the default config resource,
	// as the file config is the only config resource with
	// a valid default resource.
	FileSourceType SourceType = 0
	// ETCDSourceType defines the etcd config resource
	ETCDSourceType SourceType = 1
)

func (*SourceType) Set 

func (st *SourceType) Set(str string) error

Set allows you to set this Config Source Type using a raw string. Options: {etcd, file}

func (SourceType) String 

func (st SourceType) String() string

String returns the name of the Config Source Type

func (*SourceType) Type 

func (st *SourceType) Type() string

Type returns the flag type for a SourceType.

type StorageClusterConfig 

type StorageClusterConfig struct {
	Servers []StorageServerConfig `yaml:"servers" valid:"required"`
}

StorageClusterConfig defines the config for a storageCluster. A storage cluster is composed out of multiple data storage servers, and a single (optional) metadata storage.

func NewStorageClusterConfig 

func NewStorageClusterConfig(data []byte) (*StorageClusterConfig, error)

NewStorageClusterConfig creates a new StorageClusterConfig from a given YAML slice.

func ReadStorageClusterConfig 

func ReadStorageClusterConfig(source Source, clusterID string) (*StorageClusterConfig, error)

ReadStorageClusterConfig returns the requested StorageClusterConfig from a given config source.

func (*StorageClusterConfig) Clone 

func (cfg *StorageClusterConfig) Clone() (clone StorageClusterConfig)

Clone implements Cloner.Clone

func (*StorageClusterConfig) Equal 

func (cfg *StorageClusterConfig) Equal(other StorageClusterConfig) bool

Equal checks if the 2 configs are equal. Note that the order of data storage servers matters, as this order defines where vdisk's data will end up being.

func (*StorageClusterConfig) Validate 

func (cfg *StorageClusterConfig) Validate() error

Validate implements FormatValidator.Validate.

type StorageServerConfig 

type StorageServerConfig struct {
	Address string `yaml:"address" valid:"optional"`
	// Database '0' is assumed, in case no value is given.
	Database int `yaml:"db" valid:"optional"`
	// State defines a storage server's state.
	// Depending on the state, the properties above become optional.
	State StorageServerState `yaml:"state" valid:"optional"`
}

StorageServerConfig defines the config for a storage server

func ParseCSStorageServerConfigStrings 

func ParseCSStorageServerConfigStrings(dialCSConfigString string) (configs []StorageServerConfig, err error)

ParseCSStorageServerConfigStrings allows you to parse a slice of raw dial config strings. Dial Config Strings are a simple format used to specify ardb connection configs easily as a command line argument. The format is as follows: `<ip>:<port>[@<db_index>][,<ip>:<port>[@<db_index>]]`, where the db_index is optional, and you can give multiple configs by seperating them with a comma. The parsing algorithm of this function is very forgiving, and returns an error only in case an invalid address is given.

func ParseStorageServerConfigString 

func ParseStorageServerConfigString(dialConfigString string) (config StorageServerConfig, err error)

ParseStorageServerConfigString allows you to parse a raw dial config string. Dial Config String are a simple format used to specify ardb connection configs easily as a command line argument. The format is as follows: `<ip>:<port>[@<db_index>]`, where the db_index is optional. The parsing algorithm of this function is very forgiving, and returns an error only in case an invalid address is given.

func (*StorageServerConfig) Equal 

func (cfg *StorageServerConfig) Equal(other StorageServerConfig) bool

Equal checks if the 2 configs are equal. Note that the order of data storage servers matters, as this order defines where vdisk's data will end up being.

func (StorageServerConfig) String 

func (cfg StorageServerConfig) String() string

String implements Stringer.String

func (*StorageServerConfig) Validate 

func (cfg *StorageServerConfig) Validate() error

Validate this Storage Server Config, returning an error in case this config is invalid.

type StorageServerState 

type StorageServerState int8

StorageServerState represents the states a storage server can be in 

const (
	StorageServerStateOnline StorageServerState = iota
	StorageServerStateOffline
	StorageServerStateRepair
	StorageServerStateRespread
	StorageServerStateRIP
	StorageServerStateUnknown StorageServerState = -1
)

Different types of public storage server states. Negative values can be used for custom/private states.

func (StorageServerState) MarshalYAML 

func (state StorageServerState) MarshalYAML() (interface{}, error)

MarshalYAML implements yaml.Marshaler.MarshalYAML

func (*StorageServerState) SetString 

func (state *StorageServerState) SetString(s string) error

SetString allows you to set this StorageServerState using the correct string representation.

func (StorageServerState) String 

func (state StorageServerState) String() string

String returns the storage type as a string value

func (*StorageServerState) UnmarshalYAML 

func (state *StorageServerState) UnmarshalYAML(unmarshal func(interface{}) error) (err error)

UnmarshalYAML implements yaml.Unmarshaler.UnmarshalYAML

type StorageType 

type StorageType uint8

StorageType represents the type of storage of a vdisk 

const (
	StorageNil     StorageType = 0
	StorageDeduped StorageType = 1 << iota
	StorageNonDeduped
	// StorageSemiDeduped is not used for now
	StorageSemiDeduped
)

Different types of storage

func (StorageType) String 

func (st StorageType) String() string

String returns the name of the storage type

func (StorageType) UInt8 

func (st StorageType) UInt8() uint8

UInt8 returns the storage type as an uint8 value

type StubSource 

type StubSource struct {
	// contains filtered or unexported fields
}

StubSource is a modified file source using an internal stored config in memory used for testing purposes only.

func NewStubSource 

func NewStubSource() *StubSource

NewStubSource create a new stub source, for testing purposes

func (*StubSource) Close 

func (s *StubSource) Close() error

Close implements SourceCloser.Close

func (*StubSource) Get 

func (s *StubSource) Get(key Key) ([]byte, error)

Get implements Source.Get

func (*StubSource) InvalidKey 

func (s *StubSource) InvalidKey() <-chan Key

InvalidKey can be used to get a channel to wait for an incoming invalid key.

func (*StubSource) MarkInvalidKey 

func (s *StubSource) MarkInvalidKey(key Key, vdiskID string)

MarkInvalidKey implements Source.MarkInvalidKey

func (*StubSource) SetPrimaryStorageCluster 

func (s *StubSource) SetPrimaryStorageCluster(vdiskID, clusterID string, cfg *StorageClusterConfig)

SetPrimaryStorageCluster is a utility function to set a primary storage cluster config, thread-safe.

func (*StubSource) SetSlaveStorageCluster 

func (s *StubSource) SetSlaveStorageCluster(vdiskID, clusterID string, cfg *StorageClusterConfig)

SetSlaveStorageCluster is a utility function to set a tlog storage cluster config, thread-safe.

func (*StubSource) SetStorageCluster 

func (s *StubSource) SetStorageCluster(clusterID string, cfg *StorageClusterConfig)

SetStorageCluster is a utility function to set a storage cluster config, thread-safe.

func (*StubSource) SetTemplateStorageCluster 

func (s *StubSource) SetTemplateStorageCluster(vdiskID, clusterID string, cfg *StorageClusterConfig)

SetTemplateStorageCluster is a utility function to set a template storage cluster config, thread-safe.

func (*StubSource) SetTlogCluster 

func (s *StubSource) SetTlogCluster(clusterID string, cfg *TlogClusterConfig)

SetTlogCluster is a utility function to set a tlog cluster config, thread-safe.

func (*StubSource) SetTlogServerCluster 

func (s *StubSource) SetTlogServerCluster(vdiskID, clusterID string, cfg *TlogClusterConfig)

SetTlogServerCluster is a utility function to set a tlog server cluster config, thread-safe.

func (*StubSource) SetTlogZeroStorCluster 

func (s *StubSource) SetTlogZeroStorCluster(vdiskID, clusterID string, cfg *ZeroStorClusterConfig)

SetTlogZeroStorCluster is a utility function to set a tlog storage zerostor config, thread-safe.

func (*StubSource) SetVdiskConfig 

func (s *StubSource) SetVdiskConfig(vdiskID string, cfg *VdiskStaticConfig)

SetVdiskConfig is a utility function to set a vdisk config, thread-safe.

func (*StubSource) SetZeroStorCluster 

func (s *StubSource) SetZeroStorCluster(clusterID string, cfg *ZeroStorClusterConfig)

SetZeroStorCluster is a utility function to set a 0-stor cluster config, thread-safe.

func (*StubSource) SourceConfig 

func (s *StubSource) SourceConfig() interface{}

SourceConfig implements Source.SourceConfig

func (*StubSource) Type 

func (s *StubSource) Type() string

Type implements Source.Type

func (*StubSource) Watch 

func (s *StubSource) Watch(ctx context.Context, key Key) (<-chan []byte, error)

Watch implements Source.Watch

type TlogClusterConfig 

type TlogClusterConfig struct {
	Servers []string `yaml:"servers" valid:"required"`
}

TlogClusterConfig defines the config for a Tlog Server Custer. A Tlog Server cluster is composed out of one or more Tlog servers.

func NewTlogClusterConfig 

func NewTlogClusterConfig(data []byte) (*TlogClusterConfig, error)

NewTlogClusterConfig creates a new TlogClusterConfig from a given YAML slice.

func ReadTlogClusterConfig 

func ReadTlogClusterConfig(source Source, clusterID string) (*TlogClusterConfig, error)

ReadTlogClusterConfig returns the requested TlogClusterConfig from a given config source.

func (*TlogClusterConfig) Clone 

func (cfg *TlogClusterConfig) Clone() TlogClusterConfig

Clone implements Cloner.Clone

func (*TlogClusterConfig) Validate 

func (cfg *TlogClusterConfig) Validate() error

Validate implements FormatValidator.Validate.

type TlogStorageConfig 

type TlogStorageConfig struct {
	ZeroStorCluster ZeroStorClusterConfig
}

TlogStorageConfig contains all information needed to store tlogserver-related (meta)data.

func ReadTlogStorageConfig 

func ReadTlogStorageConfig(source Source, vdiskID string) (*TlogStorageConfig, error)

ReadTlogStorageConfig reads, validates and returns the requested TlogStorageConfig from a given source, or returns an error in case something went wrong along the way. The TlogStorageConfig is composed out of several subconfigs, and thus reading this Config might require multiple roundtrips.

func (*TlogStorageConfig) Validate 

func (cfg *TlogStorageConfig) Validate() error

Validate the required properties of this config, using the VdiskType information.

type VdiskNBDConfig 

type VdiskNBDConfig struct {
	StorageClusterID         string `yaml:"storageClusterID" valid:"required"`
	TemplateStorageClusterID string `yaml:"templateStorageClusterID" valid:"optional"`
	SlaveStorageClusterID    string `yaml:"slaveStorageClusterID" valid:"optional"`
	TlogServerClusterID      string `yaml:"tlogServerClusterID" valid:"optional"`
}

VdiskNBDConfig represents the nbdserver-related information for a vdisk.

func NewVdiskNBDConfig 

func NewVdiskNBDConfig(data []byte) (*VdiskNBDConfig, error)

NewVdiskNBDConfig creates a new VdiskNBDConfig from a given YAML slice.

func ReadVdiskNBDConfig 

func ReadVdiskNBDConfig(source Source, vdiskID string) (*VdiskNBDConfig, error)

ReadVdiskNBDConfig returns the requested VdiskNBDConfig from a given config source.

func (*VdiskNBDConfig) Validate 

func (cfg *VdiskNBDConfig) Validate() error

Validate implements FormatValidator.Validate.

type VdiskStaticConfig 

type VdiskStaticConfig struct {
	BlockSize       uint64    `yaml:"blockSize" valid:"required"`
	ReadOnly        bool      `yaml:"readOnly" valid:"optional"`
	Size            uint64    `yaml:"size" valid:"required"`
	Type            VdiskType `yaml:"type" valid:"required"`
	TemplateVdiskID string    `yaml:"templateVdiskID" valid:"optional"`
}

VdiskStaticConfig represents the static info of a vdisk.

func NewVdiskStaticConfig 

func NewVdiskStaticConfig(data []byte) (*VdiskStaticConfig, error)

NewVdiskStaticConfig creates a new VdiskStaticConfig from a given YAML slice.

func ReadVdiskStaticConfig 

func ReadVdiskStaticConfig(source Source, vdiskID string) (*VdiskStaticConfig, error)

ReadVdiskStaticConfig returns the requested VdiskStaticConfig from a given config source.

func (*VdiskStaticConfig) Validate 

func (cfg *VdiskStaticConfig) Validate() error

Validate implements FormatValidator.Validate.

type VdiskTlogConfig 

type VdiskTlogConfig struct {
	ZeroStorClusterID string `yaml:"zeroStorClusterID" valid:"required"`
}

VdiskTlogConfig represents the tlogserver-related information for a vdisk.

func NewVdiskTlogConfig 

func NewVdiskTlogConfig(data []byte) (*VdiskTlogConfig, error)

NewVdiskTlogConfig creates a new VdiskTlogConfig from a given YAML slice.

func ReadVdiskTlogConfig 

func ReadVdiskTlogConfig(source Source, vdiskID string) (*VdiskTlogConfig, error)

ReadVdiskTlogConfig returns the requested VdiskTlogConfig from a given config source.

func (*VdiskTlogConfig) Validate 

func (cfg *VdiskTlogConfig) Validate() error

Validate implements FormatValidator.Validate.

type VdiskType 

type VdiskType uint8

VdiskType represents the type of a vdisk, and each valid bit defines a property of the vdisk, and its the different collections of valid bits that defines each valid and unique type.

func (VdiskType) MarshalYAML 

func (vdiskType VdiskType) MarshalYAML() (interface{}, error)

MarshalYAML implements yaml.Marshaler.MarshalYAML

func (*VdiskType) SetString 

func (vdiskType *VdiskType) SetString(s string) error

SetString allows you to set this VdiskType using the correct string representation

func (VdiskType) StorageType 

func (vdiskType VdiskType) StorageType() StorageType

StorageType returns the type of storage this vdisk uses

func (VdiskType) String 

func (vdiskType VdiskType) String() string

String returns the storage type as a string value

func (VdiskType) TemplateSupport 

func (vdiskType VdiskType) TemplateSupport() bool

TemplateSupport returns whether or not this vdisk supports a template server, to get the data in case the data isn't available on the normal (local) storage cluster.

func (VdiskType) TlogSupport 

func (vdiskType VdiskType) TlogSupport() bool

TlogSupport returns whether or not the data of this vdisk has to send to the tlog server, to log its transactions.

func (*VdiskType) UnmarshalYAML 

func (vdiskType *VdiskType) UnmarshalYAML(unmarshal func(interface{}) error) (err error)

UnmarshalYAML implements yaml.Unmarshaler.UnmarshalYAML

func (VdiskType) Validate 

func (vdiskType VdiskType) Validate() error

Validate this vdisk type

type ZeroStorClusterConfig 

type ZeroStorClusterConfig struct {
	IYO             IYOCredentials `yaml:"iyo" valid:"required"`
	MetadataServers []ServerConfig `yaml:"metadataServers" valid:"required"`
	DataServers     []ServerConfig `yaml:"dataServers" valid:"required"`
	// data shards (K) variable of the erasure encoding
	DataShards int `yaml:"dataShards" valid:"required"`
	// parity shards (M) variable of the erasure encoding
	ParityShards int `yaml:"parityShards" valid:"required"`
}

ZeroStorClusterConfig defines the config for a ZeroStor server cluster

func NewZeroStorClusterConfig 

func NewZeroStorClusterConfig(data []byte) (*ZeroStorClusterConfig, error)

NewZeroStorClusterConfig creates a new ZeroStorClusterConfig from a given YAML slice.

func ReadZeroStoreClusterConfig 

func ReadZeroStoreClusterConfig(source Source, clusterID string) (*ZeroStorClusterConfig, error)

ReadZeroStoreClusterConfig returns requested ZeroStorClusterConfig from a given config source

func (*ZeroStorClusterConfig) Clone 

func (cfg *ZeroStorClusterConfig) Clone() ZeroStorClusterConfig

Clone implements Cloner.Clone

func (*ZeroStorClusterConfig) Equal 

func (cfg *ZeroStorClusterConfig) Equal(other ZeroStorClusterConfig) bool

Equal checks if the 2 configs are equal.

func (*ZeroStorClusterConfig) Validate 

func (cfg *ZeroStorClusterConfig) Validate() error

Validate implements FormatValidator.Validate.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.