logos

README

logos

Logging + Printing + Compromising

logos is a small and super-opinionated wrapper around go.uber.org/zap to:

• give users decent looking output
• give auditors structured logs to analyze
• give developers an easy way to print and log with the same function call

Usage

A call to a logos.Logger looks like:

logger.Infow(
    "Now we're logging :)",
    "key", "value",
    "otherkey", "othervalue",
)

This produces a stdout output of:

INFO: Now we're logging :)
  key: "value"
  otherkey: "othervalue"

Assuming NewZapSugaredLogger is used to create the logger, this logger.Infow call produces this log line:

{"_level":"INFO","_timestamp":"2021-06-08T22:16:29.161-0700","_caller":"logos/example_logos_test.go:21","_function":"github.com/bbkane/logos_test.Example","_msg":"Now we're logging :)","_pid":49721,"_version":"v1.0.0","key":"value","otherkey":"othervalue"}

Note that logos can wrap any zap.Logger, the provided NewZapSugaredLogger is only a convenience function.

See the pkg.go.dev docs for the exact API and example usage.

When to use

logos might be useful for small CLI apps that need logs

logos won't be useful for performance sensitive apps (no attention paid to allocation, unbuffered prints), apps designed to produce output for piping to another command, or apps producing deeply nested logs.

logos is imported by these open-source packages.

Philosophy

logos users (i.e., the author 😃) believe only 3 log levels are needed:

• DEBUG for information only needed by auditors looking at the logs
• ERROR for problems
• INFO for other information

Correspondingly, logos offers the following functions and destinations for their content:

	stderr	stdout	logfile
Logger.Debugw			x
Logger.Errorw	x		x
Logger.Infow		x	x

The logger methods are a subset of zap.SugaredLogger.

In addition, logos offers Logger.Sync to sync the logs and Logger.LogOnPanic as an optional function to recover from a panic, log, and then panic again.

Analyzing JSON Logs

The great thing about structured JSON logs is you can use powerful tools to analyze them. Let's analyze the logs for my local installation of grabbit. grabbit stores its logs in ~/.config/grabbit.jsonl

Analyze as JSON

Select the last error using jq:

jq -s 'map(select(._level == "ERROR")) | reverse | limit(1;.[])' ~/.config/grabbit.jsonl

{
  "_level": "ERROR",
  "_timestamp": "2021-06-07T12:51:23.023-0700",
  "_caller": "grabbit/main.go:277",
  "_function": "main.grab",
  "_msg": "can't download image",
  "_pid": 33557,
  "_version": "4.0.6",
  "subreddit": "wallpapers",
  "post": "Your Name ( Kimi No Na Wa ) Screens [1080P upscaled to 4K]",
  "url": "https://www.reddit.com/gallery/nqwe27",
  "err": "urlFileName doesn't end in allowed extension: \"nqwe27\" , []string{\".jpg\", \".jpeg\", \".png\"}\n ",
  "errVerbose": "urlFileName doesn't end in allowed extension: \"nqwe27\" , []string{\".jpg\", \".jpeg\", \".png\"}\n \nmain.validateImageURL\n\t/home/runner/work/grabbit/grabbit/main.go:198\nmain.grab\n\t/home/runner/work/grabbit/grabbit/main.go:275\nmain.run\n\t/home/runner/work/grabbit/grabbit/main.go:416\nmain.main\n\t/home/runner/work/grabbit/grabbit/main.go:429\nruntime.main\n\t/opt/hostedtoolcache/go/1.16.2/x64/src/runtime/proc.go:225\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.16.2/x64/src/runtime/asm_amd64.s:1371"
}

Other tools (most of which I haven't tried) to analyze JSON include ax, fblog, jiq, jsonui , jql, and jid.

Analyze as CSV

I wrote a small Python script I call jsonl_to.py to convert line-delimited JSON (.jsonl) to CSV for analysis in Google Sheets or similar programs. Here's my grabbit log for perusal in Google Sheets.

That CSV was generated with:

jsonl_to.py -f csv ~/.config/grabbit.jsonl > ~/tmp.csv

Analyze as SQLite3

Of course, SQLite3 can also import CSVs, and then it's possible to analyze logs with any SQLite3 tool. SQLite3 tools I like are the SQLite3 shell, litecli, Beekeeper Studio, and DbGate.

jsonl_to.py -f csv ~/.config/grabbit.jsonl \
| sqlite3 ~/tmpgrabbitlogs.db '.import --csv /dev/stdin logs'

History

logos began as a set of functions in grabbit, so I could have a log of failed image downloads to analyze. Eventually I extracted it into sugarkane to use in other apps. Finally, I reworked the API and released logos.

Documentation

Overview

Example

https://blog.golang.org/examples

package main

import (
	"go.bbkane.com/gocolor"
	"go.bbkane.com/logos"
	"go.uber.org/zap"

	lumberjack "gopkg.in/natefinch/lumberjack.v2"
)

func main() {
	// See https://github.com/natefinch/lumberjack for more options
	var lumberjackLogger *lumberjack.Logger = &lumberjack.Logger{
		Filename:   "/tmp/testlog.jsonl",
		MaxSize:    1, // megabytes
		MaxAge:     0,
		MaxBackups: 0,
		LocalTime:  true,
		Compress:   false,
	}
	color, err := gocolor.Prepare(true)
	if err != nil {
		panic(err)
	}
	l := logos.New(
		logos.NewBBKaneZapLogger(lumberjackLogger, zap.DebugLevel, "v1.0.0"),
		color,
	)
	defer l.Sync()
	l.LogOnPanic()
	l.Infow(
		"Now we're logging :)",
		"key", "value",
		"otherkey", "othervalue",
	)
}

Index

• func NewBBKaneZapLogger(lumberjackLogger *lumberjack.Logger, lvl zapcore.LevelEnabler, ...) *zap.Logger
• func NewDeterministicZapLogger(w io.Writer) *zap.Logger
• type Logger
  • func New(logger *zap.Logger, color gocolor.Color, opts ...LoggerOpt) *Logger
  • func NewNop() *Logger
  • func (l *Logger) Debugw(msg string, keysAndValues ...interface{})
  • func (l *Logger) Errorw(msg string, keysAndValues ...interface{})
  • func (l *Logger) Infow(msg string, keysAndValues ...interface{})
  • func (l *Logger) LogOnPanic()
  • func (l *Logger) Sync() error
• type LoggerOpt
  • func WithStderr(stderr io.Writer) LoggerOpt
  • func WithStdout(stdout io.Writer) LoggerOpt

Examples

• Package

Functions

func NewBBKaneZapLogger

func NewBBKaneZapLogger(lumberjackLogger *lumberjack.Logger, lvl zapcore.LevelEnabler, appVersion string) *zap.Logger

NewBBKaneZapLogger builds a zap.SugaredLogger configured with settings I like. As a special case, if lumberjackLogger == nil, then returns zap.newNop

func NewDeterministicZapLogger

func NewDeterministicZapLogger(w io.Writer) *zap.Logger

NewDeterministicZapLogger saves only levels, names, and messages for testing purposes

Types

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

Logger is a very opinionated wrapper around a uber/zap sugared logger It's designed primarily to simultaneously print "pretty-enough" input for a user and useful enough info to a lumberjack logger It should really only be used with simple key/value pairs It's designed to be fairly easily swappable with the sugared logger

func New

func New(logger *zap.Logger, color gocolor.Color, opts ...LoggerOpt) *Logger

New builds a new Logger. color should be initialized.

func NewNop

func NewNop() *Logger

NewNop returns a no-op Logger. It never writes logs or prints

func (*Logger) Debugw

func (l *Logger) Debugw(msg string, keysAndValues ...interface{})

Debugw prints only to the log

func (*Logger) Errorw

func (l *Logger) Errorw(msg string, keysAndValues ...interface{})

Errorw prints to stderr and the log

func (*Logger) Infow

func (l *Logger) Infow(msg string, keysAndValues ...interface{})

Infow prints to stdout and the log

func (*Logger) LogOnPanic

func (l *Logger) LogOnPanic()

LogOnPanic tries to log a panic. It should be called at the start of each goroutine. See panic and recover docs

func (*Logger) Sync

func (l *Logger) Sync() error

Sync syncs the underlying logger

type LoggerOpt

type LoggerOpt func(*Logger)

LoggerOpt allows customizations to New

func WithStderr

func WithStderr(stderr io.Writer) LoggerOpt

WithStderr overrides stderr for the logger

func WithStdout

func WithStdout(stdout io.Writer) LoggerOpt

WithStdout overrides stdout for the logger