env

README

ENV PACKAGE

This is a handy configuration package that populates a struct with a limited set of type values, and provides graceful management controls for controlled startup and shutdown sequences as well as an ability to run a program like a daemon start|stop enabled process or via an interactive shell or scripted shell.

The package is licensed under MIT license Copyright 2020 by zxdev, the author.

env package recognizes the following stuct field tags:

• env:"name:require,order,environ" - name to use for the field value configuration and when omitted a lowercase exported field name is used; flags are described below.
• default:"value" - sets a default value for the struct field; see supported types below.
• help:"description" - helpful description text about the struct field.

The tag:env Parser recognizes these special tag modifiers, and will the following effects:

• require - will ensure that a value has been provided to poputate the struct field or fail; setting a default tag meets this requirement.
• order - will read flagless args sequentialy from os.Args to populate the struct field in the native struct ordinal tag:order, however order args must always appear before flagged args and follow the struct ordianal tag:order; default tag value as well as regular command line flag values will also work.
• environ - will read from and mirror the updated struct field back to the os environment.

The env Parser only understands and supports the following basic types:

• string
• int
• int64
• uint
• uint64
• bool stated textually as true|false, yes|no, on|off, or 1|0.

More complex types support, such as time.Duration or time.Time, can be handled locally and converted from these basic type.

Sample use

See example folder too.

// Example struct
type Example struct {
  File  string `env:"require,order,environ" default:"sample.dat" help:"filename to use"`
  Block bool   `env:"B:require" help:"blocking flag"`
  X     int    `help:"x is int"`
  Y     int    `help:"y is int"`
  z     int
}

// Start as a Graceful basic method; lock-step
func (ex *Example) Start(ctx context.Context) {
  // any init code here
    <-ctx.Done()
    // shutdown code here
}

// Start as the Graceful closure method; non lock-step
func (ex *Example) Start(ctx context.Context) {
  go func(){
  // any init code here
  }()
  <-ctx.Done()
  // shutdown code here
}

func main() {

  var example Example
  go env.Shutdown()
  env.Init(&example)
  env.Summary(&example)
  
  env.Manager(&example)
  env.NewManager("mypath")
  env.Ready()

  // a service loop example
  if example.Block {
    for !exit {
      log.Println("...")
      time.Sleep(time.Second)
    }
    env.Stop()
  }

  // a once through example
  log.Println("...")
  env.Stop()

}

Order of operations

The order of the Parser operations for the sources follows a lowest-to-highest archtype pattern and will overload previous values when the source key is present in the current processing source, this allows defaults to be applied and overloaded by higher order sources:

• apply tag default:"value" from the struct; when exists
• apply key:value settings from /etc/{identity}/{identity}.conf; when exists
• based on tag:env:environ apply os.Environ matching KEY= settings; when exists
• apply os.Args command line switches; when present
• apply os.Args flagless values; when present
• based on tag:env:environ mirror value to os.Environ

Parser

Regardless of the key-value source for the order of operations, the Parser recognizes key value, key=value, or key:value formats as valid pairs, as shown below:

type params struct {
  A string `env:"environ"`
  B bool   `env:"environ"`
  C int
}

$ A=abc ./prog -a xyz -b:on -c=42

2020/10/08 10:13:11 |----------------------------------------|
2020/10/08 10:13:11 | DEVELOPMENT :::::::::::::::: event log |
2020/10/08 10:13:11 |-----//o--------------------------------|
2020/10/08 10:13:11                     development version
2020/10/08 10:13:11                     development build
2020/10/08 10:13:11                             pid 20628
2020/10/08 10:13:11 |---- configuration ---------o//---------|
2020/10/08 10:13:11   A              | xyz
2020/10/08 10:13:11   B              | true
2020/10/08 10:13:11   C              | 42
2020/10/08 10:13:11 |---- service ---------------o//---------|

os environment [ A=xyz B=string(true) ]

Types

• Dir - type provides Join,Create path methods
• Expire - path based file expiration manager
• Persist - manages temporary disk backed data

The shell type provides a clean way to build simple command line tools and utilities that can run in interactive shell, by file script, or as a simple command line (or all three depending on the configuration).

Using Fork instead of Init allows a program to operate normally as well as opearte like a start|stop daemon process. Fork wraps Init so it runs otherwise normally. If the pidPath Dir paramater is nil, the pid file reference will we written into the current directory as {Identity}.pid

Variables

• Mode - used internally with some informative state status
• Version - information, set by a builder.sh
• Build - information, set by builder.sh
• Identity - of the app, os.Args[0] by default
• Description - program description, license info, etc
• EtcPath, SrvPath, VarPath, TmpPath are base Dir type paths
• FileMode - os.FileMode, 0644 by default
• ExitStatusCode - use range [0,125]

Funcs

• Init() - initialize supports help|version and calls Parser to populate structs
• Summary() - summarize param structs with env tag
• Fork() - start|stop daemon wrapper around Init
• Shutdown() - waits for os.Interrupt or sys.SIGTERM termination signal
• Manger() - graceful manager wraps a func or graceful struct bootstrap
• Ready() - blocks until all Manger bootstraps have completed
• Stop() - sends signal for termination
• Context() - master context used within env package

Graceful Management

The purpose of writing graceful functions and interfaces is to assure that packages and services are completely initialized before allowing the application to being normal operation, and to cleanly shutdown the same packages and services before exiting to avoid data loss and provide assurance any other clean up tasks, such as persistence files, have completed.

All gracefully managed processes are go rountine wrapped so they will all start more or less at the same time.

// graceful func signature types
func(ctx context.Context) // basic context function
func() // basic function

// graceful struct method type suported
interface { Start(ctx context.Context) } // basic struct.Start context method
interface { Start() } // basic struct.Start method

A gracefully managed item that needs params passed just needs to return as a closure, however a closoure will block further execution until the function returns the graceful signaure. When lock-step initialization is desired, blocking is the desired behavior, however with a long or slow startup sequence to prevent lock-step wrap the closure head in a go func(){...}() to prevent blocking of other Manager initializations, see examples below:

// pokey; is lock-step bootstrap
func okey(d time.Duration) func(ctx context.Context) {
  time.Sleep(d)
 return func(ctx context.Context) {
  <-ctx.Done()
 }
}

// notPokey; not lock-step bootstrap
func notPokey(d time.Duration) func(ctx context.Context) {
 go func() {
  time.Sleep(d)
 }()
 return func(ctx context.Context) {
  <-ctx.Done()
 }
}

Version and Help Information

The package provides version information as well as a help configuration information drawn from the env:help tag when it is configured. Struct fields without the tag:help will not be displayed.

$ ./example help

 development
-----------------------
 version development
 build   development

 file            | string | filename to use [sample.dat] [require,order,environ]
 -service        | bool   | service loop flag [require]
 -x              | int    | x is int

Expiration manager

The file expiration manager, the Expire type, demonstrates the use of a graceful struct design and provides a configurable timebase automated directory cleanup service or process.

expire := env.NewExpire("path1","path2")
env.Manage(expire)

Considerations

• All structs passed to Init, Fork, Summary, and Parser MUST be passed by pointer reference or the env package WILL report a misconfiguration.

• All log output is directed to stdout by default. To redirect output to a log file call env.Init(), configure log.SetOutput(...), then proceed to call env.Summary() and such. See example/fork.go for an example.

• Use env.Shutdown() and env.Ready() for service based programs, and incorporate env.Stop() with once-and-done based programs to signal services to begin the shutdown sequence. See example/main.go for an example of each use case and configuration.

Documentation

Index

• Constants
• Variables
• func Context() context.Context
• func Fork(pidPath *Dir, cfg ...interface{})
• func IsDevelopment() bool
• func Manager(g interface{})
• func NewExpire(path ...string)
• func Parser(arg []string, path *string, cfg ...interface{})
• func Ready()
• func Shutdown()
• func Stop()
• func Summary(cfg ...interface{})
• type Dir
  • func (d Dir) Create(a ...string) string
  • func (d Dir) Join(a ...string) string
• type Expire
  • func (ex *Expire) Add(path string, age time.Duration) *Expire
  • func (ex *Expire) Expire() *Expire
  • func (ex *Expire) Report() *Expire
  • func (ex *Expire) Start(ctx context.Context)
• type Message
• type Persist
  • func (p Persist) Load(persist interface{}, age *time.Duration) bool
  • func (p Persist) Save(persist interface{}) bool
• type PersistMap
  • func (m *PersistMap) Add(k interface{})
  • func (m *PersistMap) Next(age time.Duration) func() (item interface{})
• type Shell
  • func (sh *Shell) AddNotice(notice string)
  • func (sh *Shell) Args(cfg ...interface{})
  • func (sh *Shell) Command(command, help string, action interface{})
  • func (sh *Shell) Run(cfg ...interface{})
  • func (sh *Shell) SetTimeout(d time.Duration)

Constants

const (
	MDevelopment mode = 1 << iota
	MSilent           // Silence summary output flag

	MShell  // Shell interactive interface flag
	MScript // Shell file script processor flag
	MCLI    // Shell command line interface flag

)

Variables

var (
	Mode mode // env operational settings and status

	Version string // Version information, set by a builder.sh
	Build   string // Build information, set by a builder.sh

	EtcPath Dir = "/etc" // env Dir type /etc path location
	SrvPath Dir = "/srv" // env Dir type /srv path location
	VarPath Dir = "/var" // env Dir type /var path location
	TmpPath Dir = "/tmp" // ent Dir type /tmp path location

	FileMode       os.FileMode = 0644 // env os.FileMode (default: 0644)
	ExitStatusCode int                // exit status code in range [0, 125] (default: 0)

	Identity    = filepath.Base(os.Args[0]) // Identity of app, as configured by env
	Description string                      // Brief description, license, copyright
)

var Development = func(flag *bool) {

	if flag == nil || *flag || runtime.GOOS != "linux" {
		Mode |= MDevelopment
	}

	if flag != nil && !*flag {
		Mode &^= MDevelopment
	}

	if Mode&MDevelopment == MDevelopment {
		Identity = "development"
		Version = Identity
		Build = Identity
		EtcPath = "_dev/etc"
		SrvPath = "_dev/srv"
		VarPath = "_dev/var"
		TmpPath = "_dev/tmp"
	}
}

Development will autodetect production as linux however pass a boolean value to manually override auto detection; called by Init()

var Init = func(cfg ...interface{}) {

	if Mode&mInit != mInit {
		Mode |= mInit

		if len(os.Args) > 1 {

			req := strings.TrimLeft(os.Args[1], "-")
			if req == "version" || req == "help" {

				var n = 18
				if len(Identity) > n {
					n = len(Identity)
				}
				if len(Version)+10 > n {
					n = len(Version) + 10
				}
				if len(Build)+10 > n {
					n = len(Build) + 10
				}
				fmt.Printf("\n %-s\n%s\n version %s\n build   %s\n",
					Identity, strings.Repeat("-", n+2), Version, Build)

				if req == "help" {
					if len(Description) > 0 {
						fmt.Printf("%s\n", Description)
					}
					fmt.Println()
					var order int
					for i := range cfg {

						v := reflect.Indirect(reflect.ValueOf(cfg[i]))
						for j := 0; j < v.NumField(); j++ {
							if name, flag, ok := nameFlag(v.Field(j), v.Type().Field(j)); ok {
								if help, ok := v.Type().Field(j).Tag.Lookup("help"); ok || len(flag) > 0 {
									if len(flag) > 0 {
										flag = " " + flag
										if strings.Contains(flag, "order") {
											order++
											flag = strings.Replace(flag, "order", "arg:"+strconv.Itoa((order)), 1)
										}
									}
									value, _ := v.Type().Field(j).Tag.Lookup("default")
									if len(value) > 0 {
										value = " (default: " + value + ")"
									}
									fmt.Printf("-%-15s %s%s\n%-17s[%s%s]\n\n",
										name, help, value, "", v.Type().Field(j).Type.String(), flag)
								}
							}
						}

					}

				}
				fmt.Println()
				os.Exit(0)
			}

		}

		Development(nil)

		conf := EtcPath.Join(Identity + ".conf")
		Parser(os.Args[1:], &conf, cfg...)
	}

}

Init provides version,build help, development detection and modifications, and uses the Parser to process and populate the cfg structs from tag:default valuse, os.Env, os.Args, and /etc/{identity}.conf values based on the cfg tag descriptions and settings

Functions

func Context

func Context() context.Context

Context returns the master graceful context.Context

func Fork

func Fork(pidPath *Dir, cfg ...interface{})

Fork is an Init wrapper that enables a program to run normally or like a daemon start|stop process; pidPath directory must exist and the user must have r/w file level permissions for proper operation; pass nil for default

func IsDevelopment

func IsDevelopment() bool

IsDevelopment reports the current MDevelopment mode state

func Manager

func Manager(g interface{})

Manager places the struct or func under graceful management and references it by name and requires the function sigature of 'func(ctx context.Context)', so any closure must return the graceful signature to operate properly as well.

A graceful struct must have a 'Start(ctx)' method with the proper graceful signature.

The use of any params passed to a struct or function that uses a closure will make the sequence run lock-step. To avoid this, wrap the closure head in a go routine.

func NewExpire

func NewExpire(path ...string)

NewExpire is an Expire wrapper that will create and start a file expiration manger under env.Manager control using default values applied to the direcory paths that are passed as paramaters.

Configure *Expire directly when custom settings are desired.

func Parser

func Parser(arg []string, path *string, cfg ...interface{})

Parser will process and set struct field values obtained from default values, then a conf file, then the os environment based on (tag:env: environ) flag, then arg values when non-nil, then os.Args, followed by any flagless os.Args, values (tag:env: order), with the value pushed back to the os environment when the (tag:env: environ) is present.

	tag: 'env' name:order,require,environ
 tag: 'default' value; string, bool, int, int64, uint, uint64
 tag: 'help' description

var Param struct {
	File `env:"order,require" default:"result" help:"result file"`
}

func Ready

func Ready()

Ready blocks until all Initializations have completed.

func Shutdown

func Shutdown()

Shutdown waits for a termination signal (os.Interrupt, syscall.SIGTERM) and then initiates graceful cleanup; only initalized and configured once

func Stop

func Stop()

Stop immediately signals all graceful controllers to begin their shutdown sequences and then terminates application execution

func Summary

func Summary(cfg ...interface{})

Summary of cfg settings; log

Types

type Dir

type Dir string

Dir type

func (Dir) Create

func (d Dir) Create(a ...string) string

Create appends a... and return an updated string; create the directory tree when it does not exist and return a string representation of the full composite path. A file is presumend when the last element contains any of the following ._- characters.

conf.VarPath.Create() -> /var/log
conf.VarPath.Create("insite") -> /var/log/insite
conf.VarPath.Create("insite.log") ->  /var/log/insite.log
conf.VarPath.Create("insite","insite.log") -> /var/log/insite/iniste.log

func (Dir) Join

func (d Dir) Join(a ...string) string

Join appends a... and returns an updated string; no directory tree creation

type Expire

type Expire struct {
	CheckOn time.Duration // frequency of checks (default: hourly)
	// contains filtered or unexported fields
}

Expire is a file expiration manager

func (*Expire) Add

func (ex *Expire) Add(path string, age time.Duration) *Expire

Add will register a directory path with customized age timeframe

func (*Expire) Expire

func (ex *Expire) Expire() *Expire

Expire will run the registered expiration processes

func (*Expire) Report

func (ex *Expire) Report() *Expire

Report toggles expiration reporting on/off (default: off)

func (*Expire) Start

func (ex *Expire) Start(ctx context.Context)

Start expire service manger to check for expired files periodically based on expire.CheckOn setting (default: check hourly, expire after 24hr)

type Message

type Message interface {
	Prefix() string
	Message() string
	Event() string
}

type Persist

type Persist string

Persist type; filename

func (Persist) Load

func (p Persist) Load(persist interface{}, age *time.Duration) bool

Load persist object from disk; specify expiration age

func (Persist) Save

func (p Persist) Save(persist interface{}) bool

Save persist object to disk; accepts anything

type PersistMap

type PersistMap map[interface{}]time.Time

PersistMap tracker type

func (*PersistMap) Add

func (m *PersistMap) Add(k interface{})

Add entry

func (*PersistMap) Next

func (m *PersistMap) Next(age time.Duration) func() (item interface{})

Next returns a valid unexpired item

type Shell

type Shell struct {
	// contains filtered or unexported fields
}

Shell configurator type

func (*Shell) AddNotice

func (sh *Shell) AddNotice(notice string)

AddNotice adds event notices

func (*Shell) Args

func (sh *Shell) Args(cfg ...interface{})

Args process shell line args (os.Args) and populates cfg structs

func (*Shell) Command

func (sh *Shell) Command(command, help string, action interface{})

Command adds a new command defination to the shell

func (*Shell) Run

func (sh *Shell) Run(cfg ...interface{})

Run starts the shell,script system and calls any graceful or non-graceful Start methods assoiciated with the cfg structs passed in

func (*Shell) SetTimeout

func (sh *Shell) SetTimeout(d time.Duration)

SetTimeout for inactivity automatic shell exit