README

This came from https://github.com/ldemailly/go-flagz, a fork of the code originally on https://github.com/mwitkow/go-flagz and https://github.com/improbable-eng/go-flagz with initial changes to get the go modules to work, reduce boiler plate needed for configmap watcher, avoid panic when there is extra whitespace, make the watcher work with regular files and relative paths and switched to standard golang flags. And now further changes, simplification, etc... as part of fortio.

Thanks to @mwitkow for having created this originally.

Fortio Dynamic Flags (was Go FlagZ)

Apache 2.0 License

Dynamic, thread-safe flag variables that can be modified at runtime through files, URL endpoint, or Kubernetes configmap changes.

For a similar project for JVM languages (Java, scala) see java-flagz

This sounds crazy. Why?

File-based or command-line configuration can only be changed when a service restarts. Dynamic flags provide flexibility in normal operations and emergencies. Two examples:

  • A new feature launches that you want to A/B test. You want to gradually enable it for a certain fraction of user requests (1%, 5%, 20%, 50%, 100%) without the need to restart servers.
  • Your service is getting overloaded and you want to disable certain costly features. You can't afford restarting because you'd lose important capacity.

All of this can be done simultaneously across a whole shard of your services.

Features

  • compatible with standard go flag package
  • dynamic flag that are thread-safe and efficient:
    • DynBool
    • DynInt64
    • DynFloat64
    • DynString
    • DynDuration
    • DynStringSlice
    • DynStringSet
    • DynJSON - a flag that takes an arbitrary JSON struct
  • validator functions for each flag, allows the user to provide checks for newly set values
  • notifier functions allow user code to be subscribed to flag changes
  • Kubernetes ConfigMap watcher, see configmap/README.md.
  • a HandlerFunc endpoint.ListFlags that allows for easy inspection of the service's runtime configuration
  • a HandlerFunc endpoint.SetFlag that let's you update the flag values

Here's a teaser of the debug endpoint:

Status Endpoint

Examples

Declare a single flag.FlagSet in some public package (e.g. common.SharedFlagSet) that you'll use throughout your server or stick to flag.CommandLine default flagset for your binary.

Dynamic JSON flag with a validator and notifier
var (
  limitsConfigFlag = dflag.DynJSON(
    common.SharedFlagSet, 
    "rate_limiting_config", 
    &rateLimitConfig{ DefaultRate: 10, Policy: "allow"},
    "Config for service's rate limit",
  ).WithValidator(rateLimitConfigValidator).WithNotifier(onRateLimitChange)
)

This declares a JSON flag of type rateLimitConfig with a default value. Whenever the config changes (statically or dynamically) the rateLimitConfigValidator will be called. If it returns no errors, the flag will be updated and onRateLimitChange will be called with both old and new, allowing the rate-limit mechanism to re-tune.

Dynamic feature flags

var (
  featuresFlag = dflag.DynStringSlice(common.SharedFlagSet, "enabled_features", []string{"fast_index"}, "list of enabled feature markers")
)
...
func MyHandler(resp http.ResponseWriter, req *http.Request) {
   ...
   if existsInStringSlice("fast_index", featuresFlag.Get()) {
     doFastIndex(req)
   }
   ...
}

All access to featuresFlag, which is a []string flag, is synchronised across go-routines using atomic pointer swaps.

Complete example

See a http server complete example.

Status

This code is production quality. It's been running happily in production in its earlier incarnation at Improbable for years and now everywhere fortio runs.

License

dflag (was go-flagz) is released under the Apache 2.0 license. See the LICENSE file for details.

Expand ▾ Collapse ▴

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ChecksumFlagSet

func ChecksumFlagSet(flagSet *flag.FlagSet, flagFilter func(flag *flag.Flag) bool) []byte

    ChecksumFlagSet will generate a FNV of the *set* values in a FlagSet.

    func IsFlagDynamic

    func IsFlagDynamic(f *flag.Flag) bool

      IsFlagDynamic returns whether the given Flag has been created in a Dynamic mode.

      func ReadFileFlags

      func ReadFileFlags(flagSet *flag.FlagSet) error

        ReadFileFlags parses the flagset to discover all "fileread" flags and evaluates them.

        By reading and evaluating it means: attempts to read the file and set the value.

        func ValidateDynFloat64Range

        func ValidateDynFloat64Range(fromInclusive float64, toInclusive float64) func(float64) error

          ValidateDynFloat64Range returns a validator that checks if the float value is in range.

          func ValidateDynInt64Range

          func ValidateDynInt64Range(fromInclusive int64, toInclusive int64) func(int64) error

            ValidateDynInt64Range returns a validator function that checks if the integer value is in range.

            func ValidateDynStringMatchesRegex

            func ValidateDynStringMatchesRegex(matcher *regexp.Regexp) func(string) error

              ValidateDynStringMatchesRegex returns a validator function that checks all flag's values against regex.

              func ValidateDynStringSetMinElements

              func ValidateDynStringSetMinElements(count int) func(map[string]struct{}) error

                ValidateDynStringSetMinElements validates that the given string slice has at least x elements.

                func ValidateDynStringSliceMinElements

                func ValidateDynStringSliceMinElements(count int) func([]string) error

                  ValidateDynStringSliceMinElements validates that the given string slice has at least x elements.

                  Types

                  type DynBoolValue

                  type DynBoolValue struct {
                  	DynamicFlagValueTag
                  	// contains filtered or unexported fields
                  }

                    DynBoolValue is a flag-related `int64` value wrapper.

                    func DynBool

                    func DynBool(flagSet *flag.FlagSet, name string, value bool, usage string) *DynBoolValue

                      DynBool creates a `Flag` that represents `bool` which is safe to change dynamically at runtime.

                      func (*DynBoolValue) Get

                      func (d *DynBoolValue) Get() bool

                        Get retrieves the value in a thread-safe manner.

                        func (*DynBoolValue) IsBoolFlag

                        func (d *DynBoolValue) IsBoolFlag() bool

                          IsBoolFlag lets the flag parsing know that -flagname is enough to turn to true.

                          func (*DynBoolValue) Set

                          func (d *DynBoolValue) Set(input string) error

                            Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                            func (*DynBoolValue) String

                            func (d *DynBoolValue) String() string

                              String returns the canonical string representation of the type.

                              func (*DynBoolValue) Type

                              func (d *DynBoolValue) Type() string

                                Type is an indicator of what this flag represents.

                                func (*DynBoolValue) WithNotifier

                                func (d *DynBoolValue) WithNotifier(notifier func(oldValue bool, newValue bool))

                                  WithNotifier adds a function is called every time a new value is successfully set. Each notifier is executed in a new go-routine.

                                  func (*DynBoolValue) WithValidator

                                  func (d *DynBoolValue) WithValidator(validator func(bool) error)

                                    WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                    type DynDurationValue

                                    type DynDurationValue struct {
                                    	DynamicFlagValueTag
                                    	// contains filtered or unexported fields
                                    }

                                      DynDurationValue is a flag-related `time.Duration` value wrapper.

                                      func DynDuration

                                      func DynDuration(flagSet *flag.FlagSet, name string, value time.Duration, usage string) *DynDurationValue

                                        DynDuration creates a `Flag` that represents `time.Duration` which is safe to change dynamically at runtime.

                                        func (*DynDurationValue) Get

                                        func (d *DynDurationValue) Get() time.Duration

                                          Get retrieves the value in a thread-safe manner.

                                          func (*DynDurationValue) Set

                                          func (d *DynDurationValue) Set(input string) error

                                            Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                                            func (*DynDurationValue) String

                                            func (d *DynDurationValue) String() string

                                              String represents the canonical representation of the type.

                                              func (*DynDurationValue) WithNotifier

                                              func (d *DynDurationValue) WithNotifier(notifier func(oldValue time.Duration, newValue time.Duration)) *DynDurationValue

                                                WithNotifier adds a function is called every time a new value is successfully set. Each notifier is executed in a new go-routine.

                                                func (*DynDurationValue) WithValidator

                                                func (d *DynDurationValue) WithValidator(validator func(time.Duration) error) *DynDurationValue

                                                  WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                                  type DynFloat64Value

                                                  type DynFloat64Value struct {
                                                  	DynamicFlagValueTag
                                                  	// contains filtered or unexported fields
                                                  }

                                                    DynFloat64Value is a flag-related `float64` value wrapper.

                                                    func DynFloat64

                                                    func DynFloat64(flagSet *flag.FlagSet, name string, value float64, usage string) *DynFloat64Value

                                                      DynFloat64 creates a `Flag` that represents `float64` which is safe to change dynamically at runtime.

                                                      func (*DynFloat64Value) Get

                                                      func (d *DynFloat64Value) Get() float64

                                                        Get retrieves the value in a thread-safe manner.

                                                        func (*DynFloat64Value) Set

                                                        func (d *DynFloat64Value) Set(input string) error

                                                          Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                                                          func (*DynFloat64Value) String

                                                          func (d *DynFloat64Value) String() string

                                                            String returns the canonical string representation of the type.

                                                            func (*DynFloat64Value) WithNotifier

                                                            func (d *DynFloat64Value) WithNotifier(notifier func(oldValue float64, newValue float64)) *DynFloat64Value

                                                              WithNotifier adds a function is called every time a new value is successfully set. Each notifier is executed in a new go-routine.

                                                              func (*DynFloat64Value) WithValidator

                                                              func (d *DynFloat64Value) WithValidator(validator func(float64) error) *DynFloat64Value

                                                                WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                                                type DynInt64Value

                                                                type DynInt64Value struct {
                                                                	DynamicFlagValueTag
                                                                	// contains filtered or unexported fields
                                                                }

                                                                  DynInt64Value is a flag-related `int64` value wrapper.

                                                                  func DynInt64

                                                                  func DynInt64(flagSet *flag.FlagSet, name string, value int64, usage string) *DynInt64Value

                                                                    DynInt64 creates a `Flag` that represents `int64` which is safe to change dynamically at runtime.

                                                                    func (*DynInt64Value) Get

                                                                    func (d *DynInt64Value) Get() int64

                                                                      Get retrieves the value in a thread-safe manner.

                                                                      func (*DynInt64Value) Set

                                                                      func (d *DynInt64Value) Set(input string) error

                                                                        Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                                                                        func (*DynInt64Value) String

                                                                        func (d *DynInt64Value) String() string

                                                                          String returns the canonical string representation of the type.

                                                                          func (*DynInt64Value) WithNotifier

                                                                          func (d *DynInt64Value) WithNotifier(notifier func(oldValue int64, newValue int64)) *DynInt64Value

                                                                            WithNotifier adds a function is called every time a new value is successfully set. Each notifier is executed in a new go-routine.

                                                                            func (*DynInt64Value) WithValidator

                                                                            func (d *DynInt64Value) WithValidator(validator func(int64) error) *DynInt64Value

                                                                              WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                                                              type DynJSONValue

                                                                              type DynJSONValue struct {
                                                                              	DynamicFlagValueTag
                                                                              	// contains filtered or unexported fields
                                                                              }

                                                                                DynJSONValue is a flag-related JSON struct value wrapper.

                                                                                func DynJSON

                                                                                func DynJSON(flagSet *flag.FlagSet, name string, value interface{}, usage string) *DynJSONValue

                                                                                  DynJSON creates a `Flag` that is backed by an arbitrary JSON which is safe to change dynamically at runtime. The `value` must be a pointer to a struct that is JSON (un)marshallable. New values based on the default constructor of `value` type will be created on each update.

                                                                                  func (*DynJSONValue) Get

                                                                                  func (d *DynJSONValue) Get() interface{}

                                                                                    Get retrieves the value in its original JSON struct type in a thread-safe manner.

                                                                                    func (*DynJSONValue) IsJSON

                                                                                    func (d *DynJSONValue) IsJSON() bool

                                                                                      IsJSON always return true (method is present for the DynamicJSONFlagValue interface tagging).

                                                                                      func (*DynJSONValue) Set

                                                                                      func (d *DynJSONValue) Set(input string) error

                                                                                        Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                                                                                        func (*DynJSONValue) String

                                                                                        func (d *DynJSONValue) String() string

                                                                                          String returns the canonical string representation of the type.

                                                                                          func (*DynJSONValue) WithFileFlag

                                                                                          func (d *DynJSONValue) WithFileFlag(defaultPath string) (*DynJSONValue, *FileReadValue)

                                                                                            WithFileFlag adds an companion <name>_path flag that allows this value to be read from a file with dflag.ReadFileFlags.

                                                                                            This is useful for reading large JSON files as flags. If the companion flag's value (whether default or overwritten) is set to empty string, nothing is read.

                                                                                            Flag value reads are subject to notifiers and validators.

                                                                                            func (*DynJSONValue) WithNotifier

                                                                                            func (d *DynJSONValue) WithNotifier(notifier func(oldValue interface{}, newValue interface{})) *DynJSONValue

                                                                                              WithNotifier adds a function is called every time a new value is successfully set. Each notifier is executed in a new go-routine.

                                                                                              func (*DynJSONValue) WithValidator

                                                                                              func (d *DynJSONValue) WithValidator(validator func(interface{}) error) *DynJSONValue

                                                                                                WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                                                                                type DynStringSetValue

                                                                                                type DynStringSetValue struct {
                                                                                                	DynamicFlagValueTag
                                                                                                	// contains filtered or unexported fields
                                                                                                }

                                                                                                  DynStringSetValue is a flag-related `map[string]struct{}` value wrapper.

                                                                                                  func DynStringSet

                                                                                                  func DynStringSet(flagSet *flag.FlagSet, name string, value []string, usage string) *DynStringSetValue

                                                                                                    DynStringSet creates a `Flag` that represents `map[string]struct{}` which is safe to change dynamically at runtime. Unlike `pflag.StringSlice`, consecutive sets don't append to the slice, but override it.

                                                                                                    func (*DynStringSetValue) Contains

                                                                                                    func (d *DynStringSetValue) Contains(val string) bool

                                                                                                      Contains returns whether the specified string is in the flag.

                                                                                                      func (*DynStringSetValue) Get

                                                                                                      func (d *DynStringSetValue) Get() map[string]struct{}

                                                                                                        Get retrieves the value in a thread-safe manner.

                                                                                                        func (*DynStringSetValue) Set

                                                                                                        func (d *DynStringSetValue) Set(val string) error

                                                                                                          Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                                                                                                          func (*DynStringSetValue) String

                                                                                                          func (d *DynStringSetValue) String() string

                                                                                                            String represents the canonical representation of the type.

                                                                                                            func (*DynStringSetValue) WithNotifier

                                                                                                            func (d *DynStringSetValue) WithNotifier(notifier func(oldValue map[string]struct{},
                                                                                                            	newValue map[string]struct{})) *DynStringSetValue

                                                                                                              WithNotifier adds a function that is called every time a new value is successfully set. Each notifier is executed asynchronously in a new go-routine.

                                                                                                              func (*DynStringSetValue) WithValidator

                                                                                                              func (d *DynStringSetValue) WithValidator(validator func(map[string]struct{}) error) *DynStringSetValue

                                                                                                                WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                                                                                                type DynStringSliceValue

                                                                                                                type DynStringSliceValue struct {
                                                                                                                	DynamicFlagValueTag
                                                                                                                	// contains filtered or unexported fields
                                                                                                                }

                                                                                                                  DynStringSliceValue is a flag-related `time.Duration` value wrapper.

                                                                                                                  func DynStringSlice

                                                                                                                  func DynStringSlice(flagSet *flag.FlagSet, name string, value []string, usage string) *DynStringSliceValue

                                                                                                                    DynStringSlice creates a `Flag` that represents `[]string` which is safe to change dynamically at runtime. Unlike `pflag.StringSlice`, consecutive sets don't append to the slice, but override it.

                                                                                                                    func (*DynStringSliceValue) Get

                                                                                                                    func (d *DynStringSliceValue) Get() []string

                                                                                                                      Get retrieves the value in a thread-safe manner.

                                                                                                                      func (*DynStringSliceValue) Set

                                                                                                                      func (d *DynStringSliceValue) Set(val string) error

                                                                                                                        Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                                                                                                                        func (*DynStringSliceValue) String

                                                                                                                        func (d *DynStringSliceValue) String() string

                                                                                                                          String represents the canonical representation of the type.

                                                                                                                          func (*DynStringSliceValue) WithNotifier

                                                                                                                          func (d *DynStringSliceValue) WithNotifier(notifier func(oldValue []string, newValue []string)) *DynStringSliceValue

                                                                                                                            WithNotifier adds a function that is called every time a new value is successfully set. Each notifier is executed asynchronously in a new go-routine.

                                                                                                                            func (*DynStringSliceValue) WithValidator

                                                                                                                            func (d *DynStringSliceValue) WithValidator(validator func([]string) error) *DynStringSliceValue

                                                                                                                              WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                                                                                                              type DynStringValue

                                                                                                                              type DynStringValue struct {
                                                                                                                              	DynamicFlagValueTag
                                                                                                                              	// contains filtered or unexported fields
                                                                                                                              }

                                                                                                                                DynStringValue is a flag-related `time.Duration` value wrapper.

                                                                                                                                func DynString

                                                                                                                                func DynString(flagSet *flag.FlagSet, name string, value string, usage string) *DynStringValue

                                                                                                                                  DynString creates a `Flag` that represents `string` which is safe to change dynamically at runtime.

                                                                                                                                  func (*DynStringValue) Get

                                                                                                                                  func (d *DynStringValue) Get() string

                                                                                                                                    Get retrieves the value in a thread-safe manner.

                                                                                                                                    func (*DynStringValue) Set

                                                                                                                                    func (d *DynStringValue) Set(val string) error

                                                                                                                                      Set updates the value from a string representation in a thread-safe manner. This operation may return an error if the provided `input` doesn't parse, or the resulting value doesn't pass an optional validator. If a notifier is set on the value, it will be invoked in a separate go-routine.

                                                                                                                                      func (*DynStringValue) String

                                                                                                                                      func (d *DynStringValue) String() string

                                                                                                                                        String represents the canonical representation of the type.

                                                                                                                                        func (*DynStringValue) WithNotifier

                                                                                                                                        func (d *DynStringValue) WithNotifier(notifier func(oldValue string, newValue string)) *DynStringValue

                                                                                                                                          WithNotifier adds a function is called every time a new value is successfully set. Each notifier is executed in a new go-routine.

                                                                                                                                          func (*DynStringValue) WithSyncNotifier

                                                                                                                                          func (d *DynStringValue) WithSyncNotifier(notifier func(oldValue string, newValue string)) *DynStringValue

                                                                                                                                            WithSyncNotifier adds a function is called synchronously every time a new value is successfully set.

                                                                                                                                            func (*DynStringValue) WithValidator

                                                                                                                                            func (d *DynStringValue) WithValidator(validator func(string) error) *DynStringValue

                                                                                                                                              WithValidator adds a function that checks values before they're set. Any error returned by the validator will lead to the value being rejected. Validators are executed on the same go-routine as the call to `Set`.

                                                                                                                                              type DynamicFlagValue

                                                                                                                                              type DynamicFlagValue interface {
                                                                                                                                              	IsDynamicFlag() bool
                                                                                                                                              }

                                                                                                                                                DynamicFlagValue interface is a tag to know if a type is dynamic or not.

                                                                                                                                                type DynamicFlagValueTag

                                                                                                                                                type DynamicFlagValueTag struct{}

                                                                                                                                                  DynamicFlagValueTag is a struct all dynamic flag inherit for marking they are dynamic.

                                                                                                                                                  func (*DynamicFlagValueTag) IsDynamicFlag

                                                                                                                                                  func (*DynamicFlagValueTag) IsDynamicFlag() bool

                                                                                                                                                    IsDynamicFlag always returns true.

                                                                                                                                                    type DynamicJSONFlagValue

                                                                                                                                                    type DynamicJSONFlagValue interface {
                                                                                                                                                    	IsJSON() bool
                                                                                                                                                    }

                                                                                                                                                      DynamicJSONFlagValue is a tag interface for JSON dynamic flags.

                                                                                                                                                      type FileReadValue

                                                                                                                                                      type FileReadValue struct {
                                                                                                                                                      	DynamicFlagValueTag
                                                                                                                                                      	// contains filtered or unexported fields
                                                                                                                                                      }

                                                                                                                                                        FileReadValue is a flag that wraps another flag and makes it readable from a local file in the filesystem.

                                                                                                                                                        func FileReadFlag

                                                                                                                                                        func FileReadFlag(flagSet *flag.FlagSet, parentFlagName string, defaultFilePath string) *FileReadValue

                                                                                                                                                          FileReadFlag creates a `Flag` that allows you to pass a flag.

                                                                                                                                                          If defaultFilePath is non empty, the dflag.ReadFileFlags will expect the file to be there.

                                                                                                                                                          func (*FileReadValue) Set

                                                                                                                                                          func (f *FileReadValue) Set(path string) error

                                                                                                                                                            Set updates the value from a string representation of the file path.

                                                                                                                                                            func (*FileReadValue) String

                                                                                                                                                            func (f *FileReadValue) String() string

                                                                                                                                                            Directories

                                                                                                                                                            Path Synopsis
                                                                                                                                                            examples