modules

package
Version: v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 29, 2022 License: GPL-3.0 Imports: 13 Imported by: 31

Documentation

Overview

Package modules provides a full module and task management ecosystem to cleanly put all big and small moving parts of a service together.

Modules are started in a multi-stage process and may depend on other modules: - Go's init(): register flags - prep: check flags, register config variables - start: start actual work, access config - stop: gracefully shut down

**Workers** A simple function that is run by the module while catching panics and reporting them. Ideal for long running (possibly) idle goroutines. Can be automatically restarted if execution ends with an error.

**Tasks** Functions that take somewhere between a couple seconds and a couple minutes to execute and should be queued, scheduled or repeated.

**MicroTasks** Functions that take less than a second to execute, but require lots of resources. Running such functions as MicroTasks will reduce concurrent execution and shall improve performance.

Ideally, _any_ execution by a module is done through these methods. This will not only ensure that all panics are caught, but will also give better insights into how your service performs.

Index

Constants

View Source 
const (
	StatusDead      uint8 = 0 // not prepared, not started
	StatusPreparing uint8 = 1
	StatusOffline   uint8 = 2 // prepared, not started
	StatusStopping  uint8 = 3
	StatusStarting  uint8 = 4
	StatusOnline    uint8 = 5 // online and running
)

Module Status Values.

View Source 
const (
	FailureNone    uint8 = 0
	FailureHint    uint8 = 1
	FailureWarning uint8 = 2
	FailureError   uint8 = 3
)

Module Failure Status Values.

View Source 
const (
	DefaultBackoffDuration = 2 * time.Second
)

Default Worker Configuration.

Variables

View Source 
var (

	// ErrCleanExit is returned by Start() when the program is interrupted before starting. This can happen for example, when using the "--help" flag.
	ErrCleanExit = errors.New("clean exit requested")
)
View Source 
var (
	// ErrRestartNow may be returned (wrapped) by service workers to request an immediate restart.
	ErrRestartNow = errors.New("requested restart")
)
View Source 
var (
	// HelpFlag triggers printing flag.Usage. It's exported for custom help handling.
	HelpFlag bool
)

Functions

func DisableModuleManagement added in v0.15.2 

func DisableModuleManagement()

DisableModuleManagement disables module management and returns the module system to the default start/stop behavior.

func EnableModuleManagement added in v0.5.0 

func EnableModuleManagement(changeNotifyFn func(*Module)) bool

EnableModuleManagement enables the module management functionality within modules. The supplied notify function will be called whenever the status of a module changes. The affected module will be in the parameter. You will need to manually enable modules, else nothing will start. EnableModuleManagement returns true if changeNotifyFn has been set and it has been called for the first time.

Example: 

EnableModuleManagement(func(m *modules.Module) {
	// some module has changed ...
	// do what ever you like

	// Run the built-in module management
	modules.ManageModules()
})

func GetExitStatusCode added in v0.4.0 

func GetExitStatusCode() int

GetExitStatusCode waits for the shutdown to complete and then returns the previously set exit code.

func IsShuttingDown added in v0.5.0 

func IsShuttingDown() bool

IsShuttingDown returns whether the global shutdown is in progress.

func ManageModules added in v0.5.0 

func ManageModules() error

ManageModules triggers the module manager to react to recent changes of enabled modules.

func SetCmdLineOperation added in v0.9.4 

func SetCmdLineOperation(fn func() error)

SetCmdLineOperation sets a command line operation to be executed instead of starting the system. This is useful when functions need all modules to be prepared for a special operation.

func SetErrorReportingChannel added in v0.3.0 

func SetErrorReportingChannel(reportingChannel chan *ModuleError)

SetErrorReportingChannel sets the channel to report module errors through. By default only panics are reported, all other errors need to be manually wrapped into a *ModuleError and reported.

func SetEventSubscriptionFunc added in v0.10.0 

func SetEventSubscriptionFunc(fn func(moduleName, eventName string, internal bool, data interface{})) bool

SetEventSubscriptionFunc sets a function that is called for every event. This enabled the runtime package to expose events.

func SetExitStatusCode added in v0.4.0 

func SetExitStatusCode(n int)

SetExitStatusCode sets the exit code that the program shell return to the host after shutdown.

func SetFailureUpdateNotifyFunc added in v0.10.0 

func SetFailureUpdateNotifyFunc(fn func(moduleFailure uint8, id, title, msg string)) bool

SetFailureUpdateNotifyFunc sets a function that is called on every change of a module's failure status.

func SetGlobalPrepFn added in v0.5.0 

func SetGlobalPrepFn(fn func() error)

SetGlobalPrepFn sets a global prep function that is run before all modules. This can be used to pre-initialize modules, such as setting the data root or database path.

func SetGlobalShutdownFn added in v0.12.2 

func SetGlobalShutdownFn(fn func())

SetGlobalShutdownFn sets a global shutdown function that is called first when shutting down.

func SetMaxConcurrentMicroTasks added in v0.3.0 

func SetMaxConcurrentMicroTasks(n int)

SetMaxConcurrentMicroTasks sets the maximum number of microtasks that should be run concurrently. The modules system initializes it with GOMAXPROCS. The minimum is 2.

func SetStdErrReporting added in v0.4.0 

func SetStdErrReporting(on bool)

SetStdErrReporting controls error reporting to stderr.

func Shutdown 

func Shutdown() error

Shutdown stops all modules in the correct order.

func ShuttingDown 

func ShuttingDown() <-chan struct{}

ShuttingDown returns a channel read on the global shutdown signal.

func Start 

func Start() error

Start starts all modules in the correct order. In case of an error, it will automatically shutdown again.

Types

type Module 

type Module struct {
	sync.RWMutex

	Name string

	// stop
	Ctx context.Context
	// contains filtered or unexported fields
}

Module represents a module.

func Register 

func Register(name string, prep, start, stop func() error, dependencies ...string) *Module

Register registers a new module. The control functions `prep`, `start` and `stop` are technically optional. `stop` is called _after_ all added module workers finished.

func (*Module) Dependencies added in v0.5.0 

func (m *Module) Dependencies() []*Module

Dependencies returns the module's dependencies.

func (*Module) Disable added in v0.5.0 

func (m *Module) Disable() (changed bool)

Disable disables the module. Only has an effect if module management is enabled.

func (*Module) Enable added in v0.5.0 

func (m *Module) Enable() (changed bool)

Enable enables the module. Only has an effect if module management is enabled.

func (*Module) Enabled added in v0.5.0 

func (m *Module) Enabled() bool

Enabled returns whether or not the module is currently enabled.

func (*Module) EnabledAsDependency added in v0.7.1 

func (m *Module) EnabledAsDependency() bool

EnabledAsDependency returns whether or not the module is currently enabled as a dependency.

func (*Module) Error added in v0.5.0 

func (m *Module) Error(id, title, msg string)

Error sets failure status to error. The supplied failureID is for improved automatic handling within connected systems, the failureMsg is for humans. The given ID must be unique for the given title and message. A call to Hint(), Warning() or Error() with the same ID as the existing one will be ignored.

func (*Module) FailureStatus added in v0.5.0 

func (m *Module) FailureStatus() (failureStatus uint8, failureID, failureMsg string)

FailureStatus returns the current failure status, ID and message.

func (*Module) Hint added in v0.5.0 

func (m *Module) Hint(id, title, msg string)

Hint sets failure status to hint. This is a somewhat special failure status, as the module is believed to be working correctly, but there is an important module specific information to convey. The supplied failureID is for improved automatic handling within connected systems, the failureMsg is for humans. The given ID must be unique for the given title and message. A call to Hint(), Warning() or Error() with the same ID as the existing one will be ignored.

func (*Module) InjectEvent added in v0.5.1 

func (m *Module) InjectEvent(sourceEventName, targetModuleName, targetEventName string, data interface{}) error

InjectEvent triggers an event from a foreign module and executes all hook functions registered to that event.

func (*Module) IsStopping added in v0.5.0 

func (m *Module) IsStopping() bool

IsStopping returns whether the module has started shutting down. In most cases, you should use Stopping instead.

func (*Module) NewErrorMessage added in v0.3.0 

func (m *Module) NewErrorMessage(taskName string, err error) *ModuleError

NewErrorMessage creates a new, reportable, error message (including a stack trace).

func (*Module) NewInfoMessage added in v0.3.0 

func (m *Module) NewInfoMessage(message string) *ModuleError

NewInfoMessage creates a new, reportable, info message (including a stack trace).

func (*Module) NewPanicError added in v0.3.0 

func (m *Module) NewPanicError(taskName, taskType string, panicValue interface{}) *ModuleError

NewPanicError creates a new, reportable, panic error message (including a stack trace).

func (*Module) NewTask added in v0.3.0 

func (m *Module) NewTask(name string, fn func(context.Context, *Task) error) *Task

NewTask creates a new task with a descriptive name (non-unique), a optional deadline, and the task function to be executed. You must call one of Queue, QueuePrioritized, StartASAP, Schedule or Repeat in order to have the Task executed.

func (*Module) Online added in v0.5.0 

func (m *Module) Online() bool

Online returns whether the module is online.

func (*Module) OnlineSoon added in v0.5.0 

func (m *Module) OnlineSoon() bool

OnlineSoon returns whether the module is or is about to be online.

func (*Module) RegisterEvent added in v0.4.0 

func (m *Module) RegisterEvent(event string, expose bool)

RegisterEvent registers a new event to allow for registering hooks. The expose argument controls whether these events and the attached data may be received by external components via APIs. If not exposed, the database record that carries the event and it's data will be marked as secret and as a crown jewel. Enforcement is left to the database layer.

func (*Module) RegisterEventHook added in v0.4.0 

func (m *Module) RegisterEventHook(module string, event string, description string, fn func(context.Context, interface{}) error) error

RegisterEventHook registers a hook function with (another) modules' event. Whenever a hook is triggered and the receiving module has not yet fully started, hook execution will be delayed until the modules completed starting.

func (*Module) Resolve added in v0.5.0 

func (m *Module) Resolve(failureID string)

Resolve removes the failure state from the module if the given failureID matches the current failure ID. If the given failureID is an empty string, Resolve removes any failure state.

func (*Module) RunHighPriorityMicroTask added in v0.15.0 

func (m *Module) RunHighPriorityMicroTask(name string, fn func(context.Context) error) error

RunHighPriorityMicroTask starts a new MicroTask with high priority. The given function will be executed and panics caught. The call blocks until the given function finishes.

func (*Module) RunLowPriorityMicroTask added in v0.4.0 

func (m *Module) RunLowPriorityMicroTask(name string, maxDelay time.Duration, fn func(context.Context) error) error

RunLowPriorityMicroTask starts a new MicroTask with low priority. It will wait until a slot becomes available while respecting maxDelay. You can also set maxDelay to 0 to use the default value of 3 seconds. The given function will be executed and panics caught. The call blocks until the given function finishes.

func (*Module) RunMicroTask added in v0.4.0 

func (m *Module) RunMicroTask(name string, maxDelay time.Duration, fn func(context.Context) error) error

RunMicroTask starts a new MicroTask with medium priority. It will wait until a slot becomes available while respecting maxDelay. You can also set maxDelay to 0 to use the default value of 1 second. The given function will be executed and panics caught. The call blocks until the given function finishes.

func (*Module) RunWorker added in v0.3.0 

func (m *Module) RunWorker(name string, fn func(context.Context) error) error

RunWorker directly runs a generic worker that does not fit to be a Task or MicroTask, such as long running (and possibly mostly idle) sessions. A call to RunWorker blocks until the worker is finished.

func (*Module) SetEnabled added in v0.5.0 

func (m *Module) SetEnabled(enable bool) (changed bool)

SetEnabled sets the module to the desired enabled state. Only has an effect if module management is enabled.

func (*Module) SignalHighPriorityMicroTask added in v0.15.0 

func (m *Module) SignalHighPriorityMicroTask() (done func())

SignalHighPriorityMicroTask signals the start of a new MicroTask with high priority. The returned "done" function SHOULD be called when the task has finished and MUST be called in any case. Failing to do so will have devastating effects. You can safely call "done" multiple times; additional calls do nothing.

func (*Module) SignalLowPriorityMicroTask added in v0.15.0 

func (m *Module) SignalLowPriorityMicroTask(maxDelay time.Duration) (done func())

SignalLowPriorityMicroTask signals the start of a new MicroTask with low priority. The call will wait until a slot becomes available while respecting maxDelay. You can also set maxDelay to 0 to use the default value of 1 second. The returned "done" function SHOULD be called when the task has finished and MUST be called in any case. Failing to do so will have devastating effects. You can safely call "done" multiple times; additional calls do nothing.

func (*Module) SignalMicroTask added in v0.15.0 

func (m *Module) SignalMicroTask(maxDelay time.Duration) (done func())

SignalMicroTask signals the start of a new MicroTask with medium priority. The call will wait until a slot becomes available while respecting maxDelay. You can also set maxDelay to 0 to use the default value of 1 second. The returned "done" function SHOULD be called when the task has finished and MUST be called in any case. Failing to do so will have devastating effects. You can safely call "done" multiple times; additional calls do nothing.

func (*Module) StartCompleted added in v0.5.0 

func (m *Module) StartCompleted() <-chan struct{}

StartCompleted returns a channel read that triggers when the module has finished starting.

func (*Module) StartHighPriorityMicroTask added in v0.15.0 

func (m *Module) StartHighPriorityMicroTask(name string, fn func(context.Context) error)

StartHighPriorityMicroTask starts a new MicroTask with high priority. It will start immediately. The call starts a new goroutine and returns immediately. The given function will be executed and panics caught.

func (*Module) StartLowPriorityMicroTask added in v0.3.0 

func (m *Module) StartLowPriorityMicroTask(name string, maxDelay time.Duration, fn func(context.Context) error)

StartLowPriorityMicroTask starts a new MicroTask with low priority. The call starts a new goroutine and returns immediately. It will wait until a slot becomes available while respecting maxDelay. You can also set maxDelay to 0 to use the default value of 3 seconds. The given function will be executed and panics caught.

func (*Module) StartMicroTask added in v0.3.0 

func (m *Module) StartMicroTask(name string, maxDelay time.Duration, fn func(context.Context) error)

StartMicroTask starts a new MicroTask with medium priority. The call starts a new goroutine and returns immediately. It will wait until a slot becomes available while respecting maxDelay. You can also set maxDelay to 0 to use the default value of 1 second. The given function will be executed and panics caught.

func (*Module) StartServiceWorker added in v0.3.0 

func (m *Module) StartServiceWorker(name string, backoffDuration time.Duration, fn func(context.Context) error)

StartServiceWorker starts a generic worker, which is automatically restarted in case of an error. A call to StartServiceWorker runs the service-worker in a new goroutine and returns immediately. `backoffDuration` specifies how to long to wait before restarts, multiplied by the number of failed attempts. Pass `0` for the default backoff duration. For custom error remediation functionality, build your own error handling procedure using calls to RunWorker.

func (*Module) StartWorker added in v0.4.0 

func (m *Module) StartWorker(name string, fn func(context.Context) error)

StartWorker directly starts a generic worker that does not fit to be a Task or MicroTask, such as long running (and possibly mostly idle) sessions. A call to StartWorker starts a new goroutine and returns immediately.

func (*Module) Status added in v0.5.0 

func (m *Module) Status() uint8

Status returns the current module status.

func (*Module) Stopping added in v0.5.0 

func (m *Module) Stopping() <-chan struct{}

Stopping returns a channel read that triggers when the module has initiated the stop procedure.

func (*Module) TriggerEvent added in v0.4.0 

func (m *Module) TriggerEvent(event string, data interface{})

TriggerEvent executes all hook functions registered to the specified event.

func (*Module) Warning added in v0.5.0 

func (m *Module) Warning(id, title, msg string)

Warning sets failure status to warning. The supplied failureID is for improved automatic handling within connected systems, the failureMsg is for humans. The given ID must be unique for the given title and message. A call to Hint(), Warning() or Error() with the same ID as the existing one will be ignored.

type ModuleError added in v0.3.0 

type ModuleError struct {
	Message string

	ModuleName string
	TaskName   string
	TaskType   string // one of "worker", "task", "microtask" or custom
	Severity   string // one of "info", "error", "panic" or custom

	PanicValue interface{}
	StackTrace string
}

ModuleError wraps a panic, error or message into an error that can be reported.

func GetLastReportedError added in v0.9.4 

func GetLastReportedError() *ModuleError

GetLastReportedError returns the last reported module error.

func IsPanic added in v0.3.0 

func IsPanic(err error) (bool, *ModuleError)

IsPanic returns whether the given error is a wrapped panic by the modules package and additionally returns it, if true.

func (*ModuleError) Error added in v0.3.0 

func (me *ModuleError) Error() string

Error returns the string representation of the error.

func (*ModuleError) Format added in v0.9.4 

func (me *ModuleError) Format() string

Format returns the error formatted in key/value form.

func (*ModuleError) Report added in v0.3.0 

func (me *ModuleError) Report()

Report reports the error through the configured reporting channel.

type Task added in v0.3.0 

type Task struct {
	// contains filtered or unexported fields
}

Task is managed task bound to a module.

func (*Task) Cancel added in v0.3.0 

func (t *Task) Cancel()

Cancel cancels the current and any future execution of the Task. This is not reversible by any other functions.

func (*Task) MaxDelay added in v0.3.0 

func (t *Task) MaxDelay(maxDelay time.Duration) *Task

MaxDelay sets a maximum delay within the task should be executed from being queued. Scheduled tasks are queued when they are triggered. The default delay is 3 minutes.

func (*Task) Queue added in v0.3.0 

func (t *Task) Queue() *Task

Queue queues the Task for execution.

func (*Task) QueuePrioritized added in v0.10.0 

func (t *Task) QueuePrioritized() *Task

QueuePrioritized queues the Task for execution in the prioritized queue.

func (*Task) Repeat added in v0.3.0 

func (t *Task) Repeat(interval time.Duration) *Task

Repeat sets the task to be executed in endless repeat at the specified interval. First execution will be after interval. Minimum repeat interval is one minute. An interval of zero will disable repeating, but won't change the current schedule.

func (*Task) Schedule added in v0.3.0 

func (t *Task) Schedule(executeAt time.Time) *Task

Schedule schedules the task for execution at the given time. A zero time will remove cancel the scheduled execution.

func (*Task) StartASAP added in v0.3.0 

func (t *Task) StartASAP() *Task

StartASAP schedules the task to be executed next.

Source Files

View all

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL