logos

README

logos

Logging + Printing + Compromising

logos is a small and super-opinionated wrapper around go.uber.org/zap to:

• give users decent looking output
• give auditors structured logs to analyze
• give developers an easy way to print and log with the same function call

Usage

A call to a logos.Logger looks like:

logger.Infow(
    "Now we're logging :)",
    "key", "value",
    "otherkey", "othervalue",
)

This produces a stdout output of:

INFO: Now we're logging :)
  key: "value"
  otherkey: "othervalue"

Assuming NewZapSugaredLogger is used to create the logger, this logger.Infow call produces this log line:

{"_level":"INFO","_timestamp":"2021-06-08T22:16:29.161-0700","_caller":"logos/example_logos_test.go:21","_function":"github.com/bbkane/logos_test.Example","_msg":"Now we're logging :)","_pid":49721,"_version":"v1.0.0","key":"value","otherkey":"othervalue"}

Note that logos can wrap any zap.Logger, the provided NewZapSugaredLogger is only a convenience function.

See the pkg.go.dev docs for the exact API and example usage.

When to use

logos might be useful for small CLI apps that need logs

logos won't be useful for performance sensitive apps (no attention paid to allocation, unbuffered prints), apps designed to produce output for piping to another command, or apps producing deeply nested logs.

logos is imported by these open-source packages.

Philosophy

logos users (i.e., the author 😃) believe only 3 log levels are needed:

• DEBUG for information only needed by auditors looking at the logs
• ERROR for problems
• INFO for other information

Correspondingly, logos offers the following functions and destinations for their content:

	stderr	stdout	logfile
Errorw	x
Infow		x
Logger.Debugw			x
Logger.Errorw	x		x
Logger.Infow		x	x

The logger methods are a subset of zap.SugaredLogger.

In addition, logos offers Logger.Sync to sync the logs and Logger.LogOnPanic as an optional function to recover from a panic, log, and then panic again.

Analyzing JSON Logs

The great thing about structured JSON logs is you can use powerful tools to analyze them. Let's analyze the logs for my local installation of grabbit. grabbit stores its logs in ~/.config/grabbit.jsonl

Analyze as JSON

Select the last error using jq:

jq -s 'map(select(._level == "ERROR")) | reverse | limit(1;.[])' ~/.config/grabbit.jsonl

{
  "_level": "ERROR",
  "_timestamp": "2021-06-07T12:51:23.023-0700",
  "_caller": "grabbit/main.go:277",
  "_function": "main.grab",
  "_msg": "can't download image",
  "_pid": 33557,
  "_version": "4.0.6",
  "subreddit": "wallpapers",
  "post": "Your Name ( Kimi No Na Wa ) Screens [1080P upscaled to 4K]",
  "url": "https://www.reddit.com/gallery/nqwe27",
  "err": "urlFileName doesn't end in allowed extension: \"nqwe27\" , []string{\".jpg\", \".jpeg\", \".png\"}\n ",
  "errVerbose": "urlFileName doesn't end in allowed extension: \"nqwe27\" , []string{\".jpg\", \".jpeg\", \".png\"}\n \nmain.validateImageURL\n\t/home/runner/work/grabbit/grabbit/main.go:198\nmain.grab\n\t/home/runner/work/grabbit/grabbit/main.go:275\nmain.run\n\t/home/runner/work/grabbit/grabbit/main.go:416\nmain.main\n\t/home/runner/work/grabbit/grabbit/main.go:429\nruntime.main\n\t/opt/hostedtoolcache/go/1.16.2/x64/src/runtime/proc.go:225\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.16.2/x64/src/runtime/asm_amd64.s:1371"
}

Other tools (most of which I haven't tried) to analyze JSON include ax, fblog, jiq, jsonui , jql, and jid.

Analyze as CSV

I wrote a small Python script I call jsonl_to.py to convert line-delimited JSON (.jsonl) to CSV for analysis in Google Sheets or similar programs. Here's my grabbit log for perusal in Google Sheets.

That CSV was generated with:

jsonl_to.py -f csv ~/.config/grabbit.jsonl > ~/tmp.csv

Analyze as SQLite3

Of course, SQLite3 can also import CSVs, and then it's possible to analyze logs with any SQLite3 tool. SQLite3 tools I like are the SQLite3 shell, litecli, Beekeeper Studio, and DbGate.

jsonl_to.py -f csv ~/.config/grabbit.jsonl \
| sqlite3 ~/tmpgrabbitlogs.db '.import --csv /dev/stdin logs' 

History

logos began as a set of functions in grabbit, so I could have a log of failed image downloads to analyze. Eventually I extracted it into sugarkane to use in other apps. Finally, I reworked the API and released logos.

Documentation

Overview

Example

https://blog.golang.org/examples

package main

import (
	"github.com/bbkane/logos"
	"go.uber.org/zap"

	lumberjack "gopkg.in/natefinch/lumberjack.v2"
)

func main() {
	// See https://github.com/natefinch/lumberjack for more options
	var lumberjackLogger *lumberjack.Logger = &lumberjack.Logger{
		Filename: "/tmp/testlog.jsonl",
		MaxSize:  1, // megabytes
	}
	l := logos.NewLogger(
		logos.NewZapSugaredLogger(lumberjackLogger, zap.DebugLevel, "v1.0.0"),
	)
	defer l.Sync()
	l.LogOnPanic()
	l.Infow(
		"Now we're logging :)",
		"key", "value",
		"otherkey", "othervalue",
	)
}

Output:

INFO: Now we're logging :)
  key: "value"
  otherkey: "othervalue"

Index

• func Errorw(msg string, keysAndValues ...interface{})
• func Infow(msg string, keysAndValues ...interface{})
• func NewZapSugaredLogger(lumberjackLogger *lumberjack.Logger, lvl zapcore.LevelEnabler, ...) *zap.Logger
• type Logger
  • func NewLogger(zapLogger *zap.Logger) *Logger
  • func (l *Logger) Debugw(msg string, keysAndValues ...interface{})
  • func (l *Logger) Errorw(msg string, keysAndValues ...interface{})
  • func (l *Logger) Infow(msg string, keysAndValues ...interface{})
  • func (l *Logger) LogOnPanic()
  • func (l *Logger) Sync() error

Examples

• Package

Functions

func Errorw

func Errorw(msg string, keysAndValues ...interface{})

func Infow

func Infow(msg string, keysAndValues ...interface{})

func NewZapSugaredLogger

func NewZapSugaredLogger(lumberjackLogger *lumberjack.Logger, lvl zapcore.LevelEnabler, appVersion string) *zap.Logger

NewZapSugaredLogger builds a zap.SugaredLogger configured with settings I like If lumberjackLogger == nil, then it returns an No-op logger, which can be useful when you want to use the library, but not have a log file

Types

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

Logger is a very opinionated wrapper around a uber/zap sugared logger It's designed primarily to simultaneously print "pretty-enough" input for a user and useful enough info to a lumberjack logger It should really only be used with simple key/value pairs It's designed to be fairly easily swappable with the sugared logger

func NewLogger

func NewLogger(zapLogger *zap.Logger) *Logger

NewLogger builds a new Logger

func (*Logger) Debugw

func (l *Logger) Debugw(msg string, keysAndValues ...interface{})

Debugw prints keys and values only to the log, not to the user

func (*Logger) Errorw

func (l *Logger) Errorw(msg string, keysAndValues ...interface{})

Errorw prints a message and keys and values with INFO level

func (*Logger) Infow

func (l *Logger) Infow(msg string, keysAndValues ...interface{})

Infow prints a message and keys and values with INFO level

func (*Logger) LogOnPanic

func (l *Logger) LogOnPanic()

LogOnPanic tries to log a panic. It should be called at the start of each goroutine. See panic and recover docs

func (*Logger) Sync

func (l *Logger) Sync() error

Sync syncs the underlying logger