harvester

package module
Version: v0.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 9, 2021 License: Apache-2.0 Imports: 12 Imported by: 0

README

Harvester Running CI Coverage Status Go Report Card GoDoc GitHub release

Harvester is a configuration library which helps setting up and monitoring configuration values in order to dynamically reconfigure your application.

Configuration can be obtained from the following sources:

  • Seed values, are hard-coded values into your configuration struct
  • Environment values, are obtained from the environment
  • Flag values, are obtained from CLI flags with the form -flag=value
  • File internals in local storage. Only text files are supported, don't use it for binary.
  • Consul, which is used to get initial values and to monitor them for changes

The order is applied as it is listed above. Consul seeder and monitor are optional and will be used only if Harvester is created with the above components.

Harvester expects a go structure with tags which defines one or more of the above like the following:

type Config struct {
    IndexName      sync.String          `seed:"customers-v1"`
    CacheRetention sync.Int64           `seed:"86400" env:"ENV_CACHE_RETENTION_SECONDS"`
    LogLevel       sync.String          `seed:"DEBUG" flag:"loglevel"`
    Signature      sync.String          `file:"signature.txt"`
    Sandbox        sync.Bool            `seed:"true" env:"ENV_SANDBOX" consul:"/config/sandbox-mode"`
    AccessToken    sync.Secret          `seed:"defaultaccesstoken" env:"ENV_ACCESS_TOKEN" consul:"/config/access-token"`
    WorkDuration   sync.TimeDuration    `seed:"1s" env:"ENV_WORK_DURATION" consul:"/config/work-duration"`
    OpeningBalance sync.Float64         `seed:"0.0" env:"ENV_OPENING_BALANCE" redis:"opening-balance"`
}

The above defines the following fields:

  • IndexName, which will be seeded with the value customers-v1
  • CacheRetention, which will be seeded with the value 18, and if exists, overridden with whatever value the env var ENV_CACHE_RETENTION_SECONDS holds
  • LogLevel, which will be seeded with the value DEBUG, and if exists, overridden with whatever value the flag loglevel holds
  • Sandbox, which will be seeded with the value true, and if exists, overridden with whatever value the env var ENV_SANDBOX holds and then from Consul if the consul seeder and/or watcher are provided.
  • WorkDuration, which will be seeded with the value 1s, and if exists, overridden with whatever value the env var ENV_WORK_DURATION holds and then from Consul if the consul seeder and/or watcher are provided.
  • OpeningBalance, which will be seeded with the value 0.0, and if exists, overridden with whatever value the env var ENV_OPENING_BALANCE holds and then from Redis if the redis seeder and/or watcher are provided.

The fields have to be one of the types that the sync package supports in order to allow concurrent read and write to the fields. The following types are supported:

  • sync.String, allows for concurrent string manipulation
  • sync.Int64, allows for concurrent int64 manipulation
  • sync.Float64, allows for concurrent float64 manipulation
  • sync.Bool, allows for concurrent bool manipulation
  • sync.Secret, allows for concurrent secret manipulation. Secrets can only be strings
  • sync.TimeDuration, allows for concurrent time.duration manipulation.
  • sync.StringMap, allows for concurrent map[string]string manipulation.

For sensitive configuration (passwords, tokens, etc.) that shouldn't be printed in log, you can use the Secret flavor of sync types. If one of these is selected, then at harvester log instead of the real value the text *** will be displayed.

Harvester has a seeding phase and an optional monitoring phase.

Seeding phase

  • Apply the seed tag value, if present
  • Apply the value contained in the env var, if present
  • Apply the value contained in the file, if present
  • Apply the value returned from Consul, if present and harvester is setup to seed from consul
  • Apply the value contained in the CLI flags, if present

Conditions where seeding fails:

  • If at the end of the seeding phase one or more fields have not been seeded
  • If the seed value is invalid
Seeder

Harvester allows the creation of custom getters which are used by the seeder and implement the following interface:

type Getter interface {
    Get(key string) (string, error)
}

Seed and env tags are supported by default, the Consul getter has to be setup when creating a Harvester with the builder.

Monitoring phase (Consul only)

  • Monitor a key and apply if tag key matches (Consul and Redis)
  • Monitor a key-prefix and apply if tag key matches (Consul only)
Monitor

Harvester allows for dynamically changing the config value by monitoring a source. The following sources are available:

  • Consul, which supports monitoring for keys and key-prefixes.

This feature have to be setup when creating a Harvester with the builder.

Builder

The Harvester builder pattern is used to create a Harvester instance. The builder supports setting up:

  • Consul seed, for setting up seeding from Consul
  • Consul monitor, for setting up monitoring from Consul
  • Redis seed, for setting up seeding from Redis
  • Redis monitor, for setting up monitoring from Redis
    h, err := New(&cfg).
                WithConsulSeed("address", "dc", "token").
                WithConsulMonitor("address", "dc", "token").
                WithRedisSeed(redisClient).
                WithRedisMonitor(redisClient, 10*time.Millisecond).
                Create()

The above snippet set's up a Harvester instance with Consul and Redis seed and monitor.

Notification support

In order to be able to monitor the changes in the configuration we provide a way to notify when a change is happening via the builder.

    h, err := harvester.New(&cfg).WithNotification(chNotify).Create()
    ...

Consul

Consul has support for versioning (ModifyIndex) which allows us to change the value only if the version is higher than the one currently.

Examples

Head over to examples readme on how to use harvester

How to Contribute

See Contribution Guidelines.

Code of conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project and its community you agree to abide by those terms.

Documentation

Overview

Package harvester is a configuration library which helps setting up and monitoring configuration values in order to dynamically reconfigure your application.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder of a harvester instance.

func New

func New(cfg interface{}) *Builder

New constructor.

func (*Builder) Create

func (b *Builder) Create() (Harvester, error)

Create the harvester instance.

func (*Builder) WithConsulMonitor

func (b *Builder) WithConsulMonitor(addr, dataCenter, token string, timeout time.Duration) *Builder

WithConsulMonitor enables support for monitoring key/prefixes on ConsulLogger. It automatically parses the config and monitors every field found tagged with ConsulLogger.

func (*Builder) WithConsulSeed

func (b *Builder) WithConsulSeed(addr, dataCenter, token string, timeout time.Duration) *Builder

WithConsulSeed enables support for seeding values with consul.

func (*Builder) WithNotification added in v0.11.0

func (b *Builder) WithNotification(chNotify chan<- config.ChangeNotification) *Builder

WithNotification constructor.

func (*Builder) WithRedisMonitor added in v0.12.0

func (b *Builder) WithRedisMonitor(client redis.UniversalClient, pollInterval time.Duration) *Builder

WithRedisMonitor enables support for monitoring keys in Redis. It automatically parses the config and monitors every field found tagged with ConsulLogger.

func (*Builder) WithRedisSeed added in v0.12.0

func (b *Builder) WithRedisSeed(client redis.UniversalClient) *Builder

WithRedisSeed enables support for seeding values with redis.

type Harvester

type Harvester interface {
	Harvest(ctx context.Context) error
}

Harvester interface.

type Monitor

type Monitor interface {
	Monitor(ctx context.Context) error
}

Monitor defines a interface for monitoring configuration changes from various sources.

type Seeder

type Seeder interface {
	Seed(cfg *config.Config) error
}

Seeder interface for seeding initial values of the configuration.

Directories

Path Synopsis
Package change supports change management of config values.
Package change supports change management of config values.
Package config handles config struct parsing.
Package config handles config struct parsing.
examples
Package log handles logging capabilities of harvester.
Package log handles logging capabilities of harvester.
Package monitor handles config value monitoring and changing.
Package monitor handles config value monitoring and changing.
consul
Package consul handles the monitor capabilities of harvester using ConsulLogger.
Package consul handles the monitor capabilities of harvester using ConsulLogger.
redis
Package redis handles the monitor capabilities of harvester using redis.
Package redis handles the monitor capabilities of harvester using redis.
Package seed handles seeding config values.
Package seed handles seeding config values.
consul
Package consul handles seeding capabilities with consul.
Package consul handles seeding capabilities with consul.
redis
Package redis handles seeding capabilities with redis.
Package redis handles seeding capabilities with redis.
Package sync handles synchronized read and write access to config values.
Package sync handles synchronized read and write access to config values.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL