log

package module
Version: v1.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 27, 2023 License: MIT Imports: 17 Imported by: 7

README

Logging with eluv-io/log-go

CodeQL

The package eluv-io/log-go makes logging super simple, efficient, and consistent.

log.Info("create account", "account_id", ID, "account_name", name)
log.Warn("failed to create account", err)

It is based on the following principles:

  • structured logging
  • levelled logging
  • package-based configuration

For sample code, see

Structured Logging

Pack dynamic information into discrete fields (key-value pairs), which can be used later for easy querying and subsequent processing (eluv-io/log-go provides a log handler that can emit log entries as JSON objects).

log.Debug("creating account", "account_id", ID, "account_name", name)
log.Info("account created", "account_id", ID, "account_name", name)
log.Warn("failed to create account", err)

With the text formatter, these statements produce the following output:

2018-03-02T15:23:04.317Z DEBUG creating account          account_id=123456 account_name=Test Account logger=/eluvio/log/sample
2018-03-02T15:23:04.317Z INFO  account created           account_id=456789 account_name=Another Test Account logger=/eluvio/log/sample
2018-03-02T15:23:04.317Z WARN  failed to create account  error=op [create account] kind [item already exists] account_id [123456] cause [EOF] logger=/eluvio/log/sample

A log call takes these arguments: a message, an optional error, and a list of fields (key-value pairs).

Argument Description
message The message is the first argument and should be regarded as a label for the operation or action that is being logged. Keep it short and concise - and do not include any dynamic information (which has to go into fields)!
error Optional. Errors can be logged without field name - they are automatically assigned to the "error" key.
fields The rest of the arguments are fields with a name and a value. Field names are self-describing and follow json naming conventions: all lower case with underscores. Prefer "account_id" over just "id" in order to avoid ambiguities during log post-processing.

Traditional logging libraries (such as golang's standard log package) promote the composition of string messages from all the information. This leads to inconsistent formatting, requires elaborate parsing and in general makes automatic processing cumbersome. Hence, do not use the following approach (actually, eluv-io/log-go does not offer such formatting methods...):

// bad, do not use!
log.Infof("Creating account %s with name %s", ID, name)
log.Debugf("uploading [%s] for [%s]", filename, user)
log.Warnf("upload failed %s for %s: %s", filename, user, err)
Levelled Logging

Use different log levels according to the importance of a log entry:

Level Description
FATAL An unrecoverable error. This stops the running process with os.Exit(1)!
ERROR A serious error. The application will subsequently run in a degraded state.
WARN A recoverable error, for example a failure to create a business object.
INFO Business object lifecycle events, external events, etc.
DEBUG Log statement helping understand the functioning of the system.
TRACE Everything else.

Levels are mainly used to suppress log events in order to keep log size small. The default log level is INFO. Hence, do not log important information in DEBUG.

Package-Based Configuration

Logging can be configured individually based on hierarchical names. Using the go package as name for the log instance allows per-package configuration.

In order to achieve this, each go package should define a package-local (unexported) variable called log in its main go source file:

import elog "github.com/eluv-io/log-go"

var log = elog.Get("/eluvio/log/sample")

This creates the log instance for this package. If there is a specific configuration for the package, it is used. Otherwise, it inherits the configuration from the parent instance (i.e. from a package above or the root logger).

In other go files of the same package, no import of eluv-io/log-go is needed, since they will use the variable log instead of the package import.

Hierarchical configuration is achieved with a configuration object that is used to create the global singleton instance:

config := &elog.Config{
    Level:   "debug",
    Handler: "text",
    Named: map[string]*elog.Config{
        "/eluvio/log/sample/sub": {
            Level:   "normal",
            Handler: "json",
        },
    },
}
elog.SetDefault(config)

This obviously only needs to be done once at application startup. The configuration struct can also be used to parse directly from JSON or YAML. See the log configuration sample

Log Handlers

The following log handlers are available:

text

A handler for text-based log files:

2018-03-02T15:23:04.317Z INFO  logging example           logger=/eluvio/log/sample
2018-03-02T15:23:04.317Z DEBUG creating account          account_id=123456 account_name=Test Account logger=/eluvio/log/sample
2018-03-02T15:23:04.317Z WARN  failed to create account  error=op [create account] kind [item already exists] account_id [123456] cause [EOF] logger=/eluvio/log/sample
2018-03-02T15:23:04.317Z DEBUG creating account          account_id=456789 account_name=Another Test Account logger=/eluvio/log/sample
2018-03-02T15:23:04.317Z INFO  account created           account_id=456789 account_name=Another Test Account logger=/eluvio/log/sample
console

A handler for output to the terminal with coloring:

   0.000 INFO  logging example           logger=/eluvio/log/sample
   0.010 DEBUG creating account          account_id=123456 account_name=Test Account logger=/eluvio/log/sample
   1.425 WARN  failed to create account  error=op [create account] kind [item already exists] account_id [123456] cause [EOF] logger=/eluvio/log/sample
   1.433 DEBUG creating account          account_id=456789 account_name=Another Test Account logger=/eluvio/log/sample
   1.565 INFO  account created           account_id=456789 account_name=Another Test Account logger=/eluvio/log/sample
json

A handler emitting json objects:

{"fields":{"args":["arg1","arg2","arg3"],"logger":"/eluvio/log/sample/sub"},"level":"info","timestamp":"2018-03-02T16:52:04.831614+01:00","message":"call to sub"}
{"fields":{"error":{"cause":"EOF","file":"/tmp/app-config.yaml","kind":"I/O error","op":"failed to parse config","stacktrace":"runtime/asm_amd64.s:2337: runtime.goexit:\n\truntime/proc.go:195: ...main:\n\tsample/log_sample.go:66: main.main:\n\teluvio/log/sample/sub/sub.go:17: eluvio/log/sample/sub.Call"},"logger":"/eluvio/log/sample/sub"},"level":"warn","timestamp":"2018-03-02T16:52:04.831737+01:00","message":"failed to read config, using defaults"}
{"fields":{"error":{"cause":{"cause":"EOF","file":"/tmp/app-config.yaml","kind":"I/O error","op":"failed to parse config","stacktrace":"runtime/asm_amd64.s:2337: runtime.goexit:\n\truntime/proc.go:195: ...main:\n\tsample/log_sample.go:66: main.main:\n\teluvio/log/sample/sub/sub.go:17: eluvio/log/sample/sub.Call"},"kind":"invalid","op":"configuration incomplete: timeout missing","stacktrace":"runtime/asm_amd64.s:2337: runtime.goexit:\n\truntime/proc.go:195: ...main:\n\tsample/log_sample.go:66: main.main:\n\teluvio/log/sample/sub/sub.go:22: eluvio/log/sample/sub.Call"},"logger":"/eluvio/log/sample/sub"},"level":"warn","timestamp":"2018-03-02T16:52:04.831799+01:00","message":"call failed"}

And pretty printed:

{
  "fields": {
    "args": [
      "arg1",
      "arg2",
      "arg3"
    ],
    "logger": "/eluvio/log/sample/sub"
  },
  "level": "info",
  "timestamp": "2018-03-02T16:52:04.831614+01:00",
  "message": "call to sub"
}
{
  "fields": {
    "error": {
      "cause": "EOF",
      "file": "/tmp/app-config.yaml",
      "kind": "I/O error",
      "op": "failed to parse config",
      "stacktrace": "runtime/asm_amd64.s:2337: runtime.goexit:\n\truntime/proc.go:195: ...main:\n\tsample/log_sample.go:66: main.main:\n\teluvio/log/sample/sub/sub.go:17: eluvio/log/sample/sub.Call"
    },
    "logger": "/eluvio/log/sample/sub"
  },
  "level": "warn",
  "timestamp": "2018-03-02T16:52:04.831737+01:00",
  "message": "failed to read config, using defaults"
}
{
  "fields": {
    "error": {
      "cause": {
        "cause": "EOF",
        "file": "/tmp/app-config.yaml",
        "kind": "I/O error",
        "op": "failed to parse config",
        "stacktrace": "runtime/asm_amd64.s:2337: runtime.goexit:\n\truntime/proc.go:195: ...main:\n\tsample/log_sample.go:66: main.main:\n\teluvio/log/sample/sub/sub.go:17: eluvio/log/sample/sub.Call"
      },
      "kind": "invalid",
      "op": "configuration incomplete: timeout missing",
      "stacktrace": "runtime/asm_amd64.s:2337: runtime.goexit:\n\truntime/proc.go:195: ...main:\n\tsample/log_sample.go:66: main.main:\n\teluvio/log/sample/sub/sub.go:22: eluvio/log/sample/sub.Call"
    },
    "logger": "/eluvio/log/sample/sub"
  },
  "level": "warn",
  "timestamp": "2018-03-02T16:52:04.831799+01:00",
  "message": "call failed"
}
discard

A handler that discards all output.

Logging to Files

In order to write logs to a file, with automatic roll-over based on size and/or time, configure it accordingly:

  "log": {
    "level": "debug",
    "formatter": "text",
    "file": {
      "filename": "/var/log/qfab.log",
      "maxsize": 10,
      "maxage": 0,
      "maxbackups": 2,
      "localtime": false,
      "compress": false
    }
  },

File logging is implemented by the 3rd-party library lumberjack. See their documentation for an explanation of all configuration parameters.

Documentation

Overview

Package log provides a minimal wrapper around an arbitrary third-party logging library.

The wrapper abstracts from a concrete logging implementation in order to allow painless and seamless switch-over to alternative implementations in the future. It provides the following functionality:

- Levelled logging through the Debug, Info, Warn, Error and Fatal log methods.

- Structured logging using named fields (i.e. key-value pairs) for simplified log parsing. Depending on the underlying log implementation, log entries can be written in json format, simplifying log parsing even further.

- Super-simple function signatures. All log functions take a message string and an arbitrary number of additional arguments interpreted as fields (key-value pairs) for the structured logging.

The log message should be static (i.e. it should not contain any dynamic content), so that log parsing remains simple. Any dynamic data should be added as additional fields.

Example: 

 // conventional plain-text log
 log.Infof("uploading %s for %s", filename, user)
 log.Warnf("upload failed %s for %s: %s", filename, user, err)

 // structured logging
log.Info("uploading", "file", filename, "user", user)
log.Warn("upload failed", "file", filename, "user", user, "error", err)

// same as above: an error can be logged without specifying a key ("error" will be used as key)
log.Warn("upload failed", "file", filename, "user", user, err)

Some background on structured logging: https://medium.com/@tjholowaychuk/apex-log-e8d9627f4a9a

Index

Constants

This section is empty.

Variables

View Source 
var Stdout = &LumberjackConfig{}

Stdout is a LumberjackConfig with an empty Filename that leads to logging to stdout.

Functions

func CloseLogFiles 

func CloseLogFiles()

func Debug 

func Debug(msg string, fields ...interface{})

Debug logs the given message at the Debug level.

func Error 

func Error(msg string, fields ...interface{})

Error logs the given message at the Error level.

func Fatal 

func Fatal(msg string, fields ...interface{})

Fatal logs the given message at the Fatal level.

func Info 

func Info(msg string, fields ...interface{})

Info logs the given message at the Info level.

func IsDebug 

func IsDebug() bool

IsDebug returns true if the logger logs in Debug level.

func IsError 

func IsError() bool

IsError returns true if the logger logs in Error level.

func IsFatal 

func IsFatal() bool

IsFatal returns true if the logger logs in Fatal level.

func IsInfo 

func IsInfo() bool

IsInfo returns true if the logger logs in Info level.

func IsTrace 

func IsTrace() bool

IsTrace returns true if the logger logs in Trace level.

func IsWarn 

func IsWarn() bool

IsWarn returns true if the logger logs in Warn level.

func NewLumberjackLogger 

func NewLumberjackLogger(c *LumberjackConfig) *lumberjack.Logger

func SetDefault 

func SetDefault(c *Config)

SetDefault sets the default configuration and creates the default log based on that configuration.

func SetMetrics 

func SetMetrics(m Metrics)

SetMetrics sets a new global Metrics instance.

func Trace 

func Trace(msg string, fields ...interface{})

Trace logs the given message at the Trace level.

func Warn 

func Warn(msg string, fields ...interface{})

Warn logs the given message at the Warn level.

Types

type Config 

type Config struct {
	// Level is the log level. Default: normal
	Level string `json:"level"`

	// Handler specifies the log handler to use. Default: json
	Handler string `json:"formatter"`

	// File specifies the log file settings. Default: nil (log to stdout)
	File *LumberjackConfig `json:"file,omitempty"`

	// Include go routine ID as 'gid' in logged fields
	GoRoutineID *bool `json:"go_routine_id,omitempty"`

	// Include caller info (file:line) as 'caller' in logged fields
	Caller *bool `json:"caller,omitempty"`

	// Named contains the configuration of named loggers.
	// Any nested "Named" elements are ignored.
	Named map[string]*Config `json:"named,omitempty"`
}

func NewConfig 

func NewConfig() *Config

NewConfig returns a new config instance, initialized with default values

func (*Config) InitDefaults 

func (c *Config) InitDefaults() *Config

type Log 

type Log struct {
	// contains filtered or unexported fields
}

Log is the logger wrapping an apex Log instance

func Get 

func Get(path string) *Log

Get returns the named logger for the given path. Loggers are organized in a hierarchy (tree) defined by their paths. Paths use a forward slash '/' as separator (e.g. /eluvio/util/json). Loggers inherit attributes of their parent loggers. A logger's path is added to every log entry as a field: logger=/eluvio/util/json

func New 

func New(c *Config) *Log

New creates a new root Logger

func Root 

func Root() *Log

Root retrieves the root logger - same as Get("/")

func (*Log) Call 

func (l *Log) Call(f func() error, msg string, log ...func(msg string, fields ...interface{}))

Call invokes the function and simply logs if an error occurs. Useful when deferring a call like io.Close: 

defer log.Call(reader.Close, "package.Func")

Optionally provide a log function: 

log.Call(reader.Close, "package.Func", mylog.Debug)

func (*Log) Debug 

func (l *Log) Debug(msg string, fields ...interface{})

Debug logs the given message at the Debug level.

func (*Log) Error 

func (l *Log) Error(msg string, fields ...interface{})

Error logs the given message at the Error level.

func (*Log) Fatal 

func (l *Log) Fatal(msg string, fields ...interface{})

Fatal logs the given message at the Fatal level.

func (*Log) Handler 

func (l *Log) Handler() apex.Handler

func (*Log) Info 

func (l *Log) Info(msg string, fields ...interface{})

Info logs the given message at the Info level.

func (*Log) IsDebug 

func (l *Log) IsDebug() bool

IsDebug returns true if the logger logs in Debug level.

func (*Log) IsError 

func (l *Log) IsError() bool

IsError returns true if the logger logs in Error level.

func (*Log) IsFatal 

func (l *Log) IsFatal() bool

IsFatal returns true if the logger logs in Fatal level.

func (*Log) IsInfo 

func (l *Log) IsInfo() bool

IsInfo returns true if the logger logs in Info level.

func (*Log) IsTrace 

func (l *Log) IsTrace() bool

IsTrace returns true if the logger logs in Trace level.

func (*Log) IsWarn 

func (l *Log) IsWarn() bool

IsWarn returns true if the logger logs in Warn level.

func (*Log) Level 

func (l *Log) Level() string

func (*Log) Name 

func (l *Log) Name() string

Name returns the name of this logger

func (*Log) SetDebug 

func (l *Log) SetDebug()

SetDebug sets the log level to Debug.

func (*Log) SetError 

func (l *Log) SetError()

SetError sets the log level to Error.

func (*Log) SetFatal 

func (l *Log) SetFatal()

SetFatal sets the log level to Fatal.

func (*Log) SetInfo 

func (l *Log) SetInfo()

SetInfo sets the log level to Info.

func (*Log) SetLevel 

func (l *Log) SetLevel(level string)

SetLevel sets the log level according to the given string.

func (*Log) SetWarn 

func (l *Log) SetWarn()

SetWarn sets the log level to Warn.

func (*Log) Trace 

func (l *Log) Trace(msg string, fields ...interface{})

Trace logs the given message at the Trace level.

func (*Log) Warn 

func (l *Log) Warn(msg string, fields ...interface{})

Warn logs the given message at the Warn level.

type LumberjackConfig 

type LumberjackConfig struct {
	// Filename is the file to write logs to.  Backup log files will be retained
	// in the same directory.  It uses <processname>-lumberjack.log in
	// os.TempDir() if empty.
	Filename string `json:"filename"`

	// MaxSize is the maximum size in megabytes of the log file before it gets
	// rotated. It defaults to 100 megabytes.
	MaxSize int `json:"maxsize"`

	// MaxAge is the maximum number of days to retain old log files based on the
	// timestamp encoded in their filename.  Note that a day is defined as 24
	// hours and may not exactly correspond to calendar days due to daylight
	// savings, leap seconds, etc. The default is not to remove old log files
	// based on age.
	MaxAge int `json:"maxage"`

	// MaxBackups is the maximum number of old log files to retain.  The default
	// is to retain all old log files (though MaxAge may still cause them to get
	// deleted.)
	MaxBackups int `json:"maxbackups"`

	// LocalTime determines if the time used for formatting the timestamps in
	// backup files is the computer's local time.  The default is to use UTC
	// time.
	LocalTime bool `json:"localtime"`

	// Compress determines if the rotated log files should be compressed
	// using gzip. The default is not to perform compression.
	Compress bool `json:"compress"`
}

type Metrics 

type Metrics interface {
	// FileCreated increments the counter for created log files
	FileCreated()
	// InstanceCreated increments the counter for created log objects
	InstanceCreated()
	// Error increments the counter for messages logged with Error level
	Error(logger string)
	// Warn increments the counter for messages logged with Warn level
	Warn(logger string)
	// Info increments the counter for messages logged with Info level
	Info(logger string)
	// Debug increments the counter for messages logged with Debug level
	Debug(logger string)
}

Metrics is the interface for collecting log metrics (counters for log calls).

Source Files

View all

Directories

Path Synopsis
handlers
console
Package console implements a development-friendly textual handler.
 Package console implements a development-friendly textual handler.
raw
Package raw is like handlers/text, but omits the "log level" field and prints the "raw" field without label on a separate line.
 Package raw is like handlers/text, but omits the "log level" field and prints the "raw" field without label on a separate line.
text
Package text implements a development-friendly textual handler.
 Package text implements a development-friendly textual handler.
config
sub

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL