logging

README

logging

logging 是 go-util 新的日志模块。

目标：

• 统一结构化日志输出
• 统一 app log / access log 模型
• 支持 context 透传 logger
• 支持请求内持续追加字段/错误/消息
• 在请求结束时统一输出一条 access log
• 提供 net/http 和 gin middleware

旧 github.com/imdm/go-util/log 已废弃，新代码请统一使用 github.com/imdm/go-util/logging。

---

包结构

• github.com/imdm/go-util/logging
  • 核心 logger
  • context logger
  • request log aggregation
• github.com/imdm/go-util/logging/http
  • net/http access log middleware
• github.com/imdm/go-util/logging/gin
  • gin access log middleware

---

快速开始

初始化 logger

package main

import (
	"os"

	"github.com/imdm/go-util/logging"
)

func main() {
	logger := logging.MustNewLogger(logging.Config{
		Service:      "example-service",
		Env:          "prod",
		Version:      "v1.0.0",
		Level:        logging.InfoLevel,
		Format:       logging.AutoFormat,
		AppWriter:    os.Stdout,
		AccessWriter: os.Stdout,
		ErrorWriter:  os.Stderr,
		AddCaller:    true,
	})

	logger.Info("service started", logging.StringField("component", "bootstrap"))
}

默认行为

如果没有显式配置：

• Level: info
• Format: auto
• TimeZone: UTC
• AppWriter: stdout
• AccessWriter: stdout
• ErrorWriter: stderr
• Redactor: 默认脱敏器
• Observer: no-op

Format=auto 时：

• TTY -> console pretty
• 非 TTY -> JSON

---

完整可运行 Demo

仓库内提供了两个可直接运行的 demo：

• logging/examples/nethttpdemo
• logging/examples/gindemo

运行 net/http demo

go run ./logging/examples/nethttpdemo

默认监听：

:8080

试几个请求：

curl -i 'http://127.0.0.1:8080/healthz'
curl -i 'http://127.0.0.1:8080/orders?order_id=o_1001'
curl -i 'http://127.0.0.1:8080/orders?order_id=o_1001&fail=1'
curl -i 'http://127.0.0.1:8080/panic'

可以观察到：

• /healthz 被 skip，不输出 access log
• /orders 会输出一条 access log，并包含 request_fields / request_messages
• /orders?...&fail=1 会输出 500 的 error 级 access log
• /panic 会被 middleware recover，并输出 panic 相关字段

运行 gin demo

go run ./logging/examples/gindemo

默认监听：

:8081

试几个请求：

curl -i 'http://127.0.0.1:8081/healthz'
curl -i 'http://127.0.0.1:8081/accounts/42'
curl -i 'http://127.0.0.1:8081/accounts/42?fail=1'
curl -i 'http://127.0.0.1:8081/panic'

Gin demo 除了 request aggregation 外，还可以观察到：

• route 会优先记录为 /accounts/:id

---

输出模式

1. App Log

用于：

• 初始化日志
• 后台任务日志
• worker / cron / consumer 日志
• 需要即时输出的业务日志

示例：

workerLog := logger.WithComponent("settlement_worker")

workerLog.Info("worker started")
workerLog.Error(
	"settlement failed",
	logging.StringField("job_id", "job_123"),
	logging.ErrorField(err),
)

2. Access Log

用于：

• 每个 HTTP 请求结束时统一输出一条日志

access log 会自动带上这些标准字段：

• log_type=access
• request_id
• trace_id
• span_id
• method
• path
• route
• status_code
• latency_ms
• client_ip
• user_agent
• request_fields
• request_messages
• request_errors

其中：

• 业务追加字段进入 request_fields
• 业务追加消息进入 request_messages
• 业务追加错误进入 request_errors

---

基础 API

创建 logger

logger, err := logging.NewLogger(config)
logger := logging.MustNewLogger(config)

派生 logger

logger = logger.WithFields(
	logging.StringField("service", "billing"),
	logging.StringField("env", "prod"),
)

componentLog := logger.WithComponent("charge_worker")
namedLog := logger.WithName("charge_loop")

即时日志

logger.Debug("debug message")
logger.Info("info message")
logger.Warn("warn message")
logger.Error("error message", logging.ErrorField(err))
logger.Fatal("fatal message")

字段构造函数

已提供：

• StringField
• IntField
• Int64Field
• BoolField
• DurationField
• TimeField
• ErrorField
• AnyField
• StringSliceField

---

Context Logger

写入 context

ctx = logging.IntoContext(ctx, logger)

从 context 读取

log := logging.FromContext(ctx)

如果 context 中没有 logger：

• FromContext(ctx) 返回默认 logger
• MustFromContext(ctx) 直接 panic

---

请求聚合日志

这是和旧 log/ 最大的差别之一。

请求处理中不要到处打零散日志，而是优先把与本次请求相关的信息追加到 request log 上，最终由 middleware 统一输出一条 access log。

追加字段

log.AppendRequestLogField("order_id", orderID)
log.AppendRequestLogFields(
	logging.StringField("user_id", userID),
	logging.StringField("pay_channel", "wechat"),
)

追加错误

log.AppendRequestLogError(err)

追加消息

log.AppendRequestLogMessage("validate request failed")

覆盖本次请求 access log 级别

log.OverrideRequestLogLevel(logging.ErrorLevel)

注意

• 这些方法只有在“请求链路 logger”上才会生效
• 非请求链路调用时当前实现是 no-op

---

net/http 接入

package main

import (
	"net/http"
	"os"

	"github.com/imdm/go-util/logging"
	logginghttp "github.com/imdm/go-util/logging/http"
)

func main() {
	logger := logging.MustNewLogger(logging.Config{
		Service:      "example-service",
		Format:       logging.AutoFormat,
		AppWriter:    os.Stdout,
		AccessWriter: os.Stdout,
		ErrorWriter:  os.Stderr,
	})

	mux := http.NewServeMux()
	mux.HandleFunc("/orders/", func(w http.ResponseWriter, r *http.Request) {
		log := logging.MustFromContext(r.Context())
		log.AppendRequestLogField("path", r.URL.Path)
		log.AppendRequestLogMessage("request handled")
		w.WriteHeader(http.StatusOK)
		_, _ = w.Write([]byte("ok"))
	})

	handler := logginghttp.NewAccessLogMiddleware(logger, logginghttp.AccessLogConfig{
		RequestIDHeader: "X-Request-ID",
		RecoverPanic:    true,
		TrustProxyHeaders: true,
		SkipPaths:       []string{"/healthz"},
	})(mux)

	_ = http.ListenAndServe(":8080", handler)
}

AccessLogConfig

支持：

• RequestIDHeader
• SkipPaths
• SkipPathRegexps
• Skipper
• DefaultLevel
• ClientErrorLevel
• ServerErrorLevel
• PathLevels
• RecoverPanic
• LogPanicStack
• TrustProxyHeaders
• request aggregation 限额配置

默认级别映射

• <400 -> info
• 400-499 -> warn
• >=500 -> error
• panic recover -> error

net/http 版本当前没有 route pattern 能力，route 默认等于 path。

---

Gin 接入

package main

import (
	"net/http"
	"os"

	"github.com/gin-gonic/gin"

	"github.com/imdm/go-util/logging"
	logginggin "github.com/imdm/go-util/logging/gin"
	logginghttp "github.com/imdm/go-util/logging/http"
)

func main() {
	logger := logging.MustNewLogger(logging.Config{
		Service:      "example-service",
		Format:       logging.AutoFormat,
		AppWriter:    os.Stdout,
		AccessWriter: os.Stdout,
		ErrorWriter:  os.Stderr,
	})

	r := gin.New()
	r.Use(logginggin.NewAccessLogMiddleware(logger, logginghttp.AccessLogConfig{
		RequestIDHeader: "X-Request-ID",
		RecoverPanic:    true,
	}))

	r.GET("/accounts/:id", func(c *gin.Context) {
		log := logging.MustFromContext(c.Request.Context())
		log.AppendRequestLogField("account_id", c.Param("id"))
		log.AppendRequestLogMessage("account loaded")
		c.String(http.StatusOK, "ok")
	})

	_ = r.Run(":8080")
}

Gin 版本会优先把 route 记录成：

/accounts/:id

---

Redaction

默认 redactor 会自动脱敏这些 key：

• authorization
• token
• access_token
• refresh_token
• password
• secret
• cookie
• set-cookie

例如：

log.AppendRequestLogField("authorization", "Bearer xxx")

最终输出会被替换成：

***redacted***

如果需要自定义脱敏策略，可实现：

type Redactor interface {
	RedactField(key string, value interface{}) interface{}
}

---

Observer

可以注入 observer 接收日志输出回调：

type Observer interface {
	ObserveAppLog(level Level, logType string)
	ObserveAccessLog(level Level, statusCode int, route string, latency time.Duration)
}

如果不需要，默认是 no-op。

---

Sampling

Config.Sampling 可控制 app log 采样。

当前支持：

• Enabled
• Interval
• First
• Thereafter
• DisableForError
• DisableForAccess

示例：

logger := logging.MustNewLogger(logging.Config{
	Sampling: logging.SamplingConfig{
		Enabled:    true,
		Thereafter: 10,
	},
})

---

Default Logger

如果你需要全局默认 logger：

logging.SetDefaultLogger(logger)
defaultLogger := logging.DefaultLogger()

但推荐主路径仍然是显式构造并显式传递，不要把 default logger 当成唯一依赖方式。

---

和旧 log/ 的主要差别

新模块推荐做法

• 用 logging.Logger
• 用 context 透传
• 用 AppendRequestLog... 做请求聚合
• 用 logging/http 或 logging/gin 统一输出 access log

旧模块不再推荐的做法

• 包级 log.Info(...)
• Infof/Debugw/Errorw
• 本地 rotate file 作为主输出路径
• 一次请求打多条零散日志去拼上下文

---

推荐写法 / 禁止写法

这一节更偏团队接入规范。

推荐写法

1. 显式构造 logger，并在进程入口统一初始化

推荐：

logger := logging.MustNewLogger(logging.Config{
	Service:      "billing",
	Env:          "prod",
	Version:      "v1.2.3",
	Format:       logging.AutoFormat,
	Level:        logging.InfoLevel,
	AppWriter:    os.Stdout,
	AccessWriter: os.Stdout,
	ErrorWriter:  os.Stderr,
})

原因：

• 配置收敛
• base fields 稳定
• 输出策略统一

2. 请求链路里优先使用 MustFromContext

推荐：

func Handle(w http.ResponseWriter, r *http.Request) {
	log := logging.MustFromContext(r.Context())
	log.AppendRequestLogField("order_id", "o_123")
}

适用前提：

• 你明确知道链路已经挂了 logging/http 或 logging/gin middleware

3. 请求处理过程中优先“追加到 request log”，而不是重复即时打印

推荐：

log.AppendRequestLogField("account_id", accountID)
log.AppendRequestLogError(err)
log.AppendRequestLogMessage("load account failed")

适用场景：

• handler
• controller
• service
• 请求上下文内的 deeper layer

4. 后台任务、worker、cron 使用即时 app log

推荐：

workerLog := logger.WithComponent("settlement_worker")
workerLog.Info("worker started")
workerLog.Error("job failed", logging.ErrorField(err))

5. 用结构化字段表达上下文，而不是自己拼字符串

推荐：

logger.Info(
	"charge succeeded",
	logging.StringField("order_id", orderID),
	logging.Int64Field("amount", amount),
)

而不是：

logger.Info("charge succeeded: order_id=" + orderID)

6. 用 WithComponent / WithName 派生 logger

推荐：

repoLog := logger.WithComponent("order_repository")
jobLog := logger.WithName("settlement_loop")

这样比手动在每条日志里重复同一个字段更稳定。

---

禁止写法

1. 不要在新代码里继续使用旧 log/

禁止：

import "github.com/imdm/go-util/log"

应改为：

import "github.com/imdm/go-util/logging"

2. 不要在请求链路里自己手打 access log

禁止：

logger.Info("request done", logging.StringField("path", r.URL.Path))

来代替 middleware 的 access log。

原因：

• access log 会重复
• schema 不稳定
• request_id / latency / status_code 容易遗漏

3. 不要把一次请求拆成很多条“拼图式日志”

不推荐：

logger.Info("request start")
logger.Info("load account")
logger.Info("account loaded")
logger.Error("request failed")

如果这些信息都只是在描述同一次请求，优先用：

• AppendRequestLogField
• AppendRequestLogError
• AppendRequestLogMessage

4. 不要把 default logger 当成唯一依赖方式

DefaultLogger() 可以用，但不要让业务逻辑完全依赖全局隐式状态。

更推荐：

• 显式构造
• 显式注入
• context 透传

5. 不要把敏感信息原样追加进日志

虽然默认 redactor 会处理一部分常见 key，但仍然不建议主动记录：

• password
• token
• cookie
• authorization 原文
• 业务敏感个人信息

应优先：

• 不记
• 记摘要
• 记脱敏后的值

6. 不要把 request aggregation 用在非请求链路上

这些方法在非请求链路里当前是 no-op：

• AppendRequestLogField
• AppendRequestLogError
• AppendRequestLogMessage

所以后台任务、worker、初始化流程应该直接打 app log，而不是误以为这些追加会生效。

7. 不要把 Infof / Debugw 这种旧风格重新引回来

当前 logging 没有提供这些 API，目的就是防止重新回到：

• 非结构化字符串拼接
• 风格不统一
• schema 不稳定

---

一个简单判断原则

如果你在写的是：

• “这次请求最终想留下什么上下文”
  -> 用 request aggregation

如果你在写的是：

• “这个后台流程此刻发生了一个独立事件”
  -> 用 app log

如果你不确定：

• HTTP 请求链路，优先 request aggregation
• 非 HTTP 链路，优先 app log

---

当前实现范围

当前 logging/ 已实现：

• 核心结构化 logger
• context logger
• request aggregation
• net/http middleware
• gin middleware
• redaction
• observer
• sampling 基础能力

如果你要了解设计目标和实施边界，可继续看：

• ../DESIGN_logging_refactor.md
• ../IMPLEMENTATION_CHECKLIST_logging_refactor.md

Documentation

Overview

Package logging provides structured application logging, request-scoped log aggregation, and HTTP/Gin access log middleware.

Index

• func IntoContext(ctx context.Context, logger *Logger) context.Context
• func SetDefaultLogger(logger *Logger)
• type AccessLogEntry
• type Config
• type Field
  • func AnyField(key string, value interface{}) Field
  • func BoolField(key string, value bool) Field
  • func DurationField(key string, value time.Duration) Field
  • func ErrorField(err error) Field
  • func Int64Field(key string, value int64) Field
  • func IntField(key string, value int) Field
  • func StringField(key, value string) Field
  • func StringSliceField(key string, value []string) Field
  • func TimeField(key string, value time.Time) Field
• type Format
  • func ParseFormat(format string) (Format, error)
  • func (f Format) String() string
• type Level
  • func ParseLevel(level string) (Level, error)
  • func (l Level) String() string
• type Logger
  • func DefaultLogger() *Logger
  • func FromContext(ctx context.Context) *Logger
  • func MustFromContext(ctx context.Context) *Logger
  • func MustNewLogger(config Config) *Logger
  • func NewLogger(config Config) (*Logger, error)
  • func (l *Logger) AppendRequestLogError(err error)
  • func (l *Logger) AppendRequestLogField(key string, value interface{})
  • func (l *Logger) AppendRequestLogFields(fields ...Field)
  • func (l *Logger) AppendRequestLogMessage(message string)
  • func (l *Logger) Debug(message string, fields ...Field)
  • func (l *Logger) Error(message string, fields ...Field)
  • func (l *Logger) Fatal(message string, fields ...Field)
  • func (l *Logger) Info(message string, fields ...Field)
  • func (l *Logger) OverrideRequestLogLevel(level Level)
  • func (l *Logger) Sync() error
  • func (l *Logger) Warn(message string, fields ...Field)
  • func (l *Logger) WithComponent(component string) *Logger
  • func (l *Logger) WithFields(fields ...Field) *Logger
  • func (l *Logger) WithName(name string) *Logger
  • func (l *Logger) WithRequestLogAccumulator(accumulator *RequestLogAccumulator) *Logger
  • func (l *Logger) WriteAccessLog(entry AccessLogEntry)
• type Observer
  • func NopObserver() Observer
• type Redactor
  • func DefaultRedactor() Redactor
• type RequestLogAccumulator
  • func NewRequestLogAccumulator(limits RequestLogLimits) *RequestLogAccumulator
• type RequestLogLimits
• type SamplingConfig

Functions

func IntoContext

func IntoContext(ctx context.Context, logger *Logger) context.Context

IntoContext stores logger into ctx.

func SetDefaultLogger

func SetDefaultLogger(logger *Logger)

SetDefaultLogger sets the process default logger.

Types

type AccessLogEntry

type AccessLogEntry struct {
	LogType string

	Timestamp time.Time
	Level     Level

	Service  string
	Env      string
	Version  string
	Hostname string

	RequestID string
	TraceID   string
	SpanID    string

	Method     string
	Path       string
	Route      string
	StatusCode int
	Latency    time.Duration
	LatencyMS  int64

	ClientIP      string
	UserAgent     string
	Referer       string
	RequestBytes  int64
	ResponseBytes int64

	RequestFields   map[string]interface{}
	RequestMessages []string
	RequestErrors   []string

	RequestLogTruncated bool
	PanicRecovered      bool
}

AccessLogEntry is the final structured request log payload.

type Config

type Config struct {
	Service  string
	Env      string
	Version  string
	Hostname string

	Level     Level
	Format    Format
	TimeZone  *time.Location
	AddCaller bool

	AppWriter    io.Writer
	AccessWriter io.Writer
	ErrorWriter  io.Writer

	StaticFields []Field

	Redactor Redactor
	Observer Observer

	Sampling SamplingConfig
}

Config controls logger construction.

type Field

type Field struct {
	// contains filtered or unexported fields
}

Field is the public structured field representation.

func AnyField

func AnyField(key string, value interface{}) Field

AnyField builds a generic field.

func BoolField

func BoolField(key string, value bool) Field

BoolField builds a bool field.

func DurationField

func DurationField(key string, value time.Duration) Field

DurationField builds a duration field.

func ErrorField

func ErrorField(err error) Field

ErrorField builds an error field under the standard error key.

func Int64Field

func Int64Field(key string, value int64) Field

Int64Field builds an int64 field.

func IntField

func IntField(key string, value int) Field

IntField builds an int field.

func StringField

func StringField(key, value string) Field

StringField builds a string field.

func StringSliceField

func StringSliceField(key string, value []string) Field

StringSliceField builds a string slice field.

func TimeField

func TimeField(key string, value time.Time) Field

TimeField builds a time field.

type Format

type Format uint8

Format is the log output format.

const (
	AutoFormat Format = iota
	JSONFormat
	ConsoleFormat
)

func ParseFormat

func ParseFormat(format string) (Format, error)

ParseFormat parses a string format.

func (Format) String

func (f Format) String() string

String returns the lowercase format name.

type Level

type Level uint8

Level is the structured log severity.

const (
	UnspecifiedLevel Level = iota
	DebugLevel
	InfoLevel
	WarnLevel
	ErrorLevel
	FatalLevel
)

func ParseLevel

func ParseLevel(level string) (Level, error)

ParseLevel parses a string level.

func (Level) String

func (l Level) String() string

String returns the lowercase level name.

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

Logger is the public structured logger.

func DefaultLogger

func DefaultLogger() *Logger

DefaultLogger returns the process default logger.

func FromContext

func FromContext(ctx context.Context) *Logger

FromContext returns logger from ctx or the default logger.

func MustFromContext

func MustFromContext(ctx context.Context) *Logger

MustFromContext returns logger from ctx and panics when absent.

func MustNewLogger

func MustNewLogger(config Config) *Logger

MustNewLogger constructs a new logger or panics.

func NewLogger

func NewLogger(config Config) (*Logger, error)

NewLogger constructs a new logger.

func (*Logger) AppendRequestLogError

func (l *Logger) AppendRequestLogError(err error)

AppendRequestLogError appends an error to the current request log.

func (*Logger) AppendRequestLogField

func (l *Logger) AppendRequestLogField(key string, value interface{})

AppendRequestLogField appends a request field to the current request log.

func (*Logger) AppendRequestLogFields

func (l *Logger) AppendRequestLogFields(fields ...Field)

AppendRequestLogFields appends request fields to the current request log.

func (*Logger) AppendRequestLogMessage

func (l *Logger) AppendRequestLogMessage(message string)

AppendRequestLogMessage appends a message to the current request log.

func (*Logger) Debug

func (l *Logger) Debug(message string, fields ...Field)

Debug emits a debug app log.

func (*Logger) Error

func (l *Logger) Error(message string, fields ...Field)

Error emits an error app log.

func (*Logger) Fatal

func (l *Logger) Fatal(message string, fields ...Field)

Fatal emits a fatal app log and exits the process.

func (*Logger) Info

func (l *Logger) Info(message string, fields ...Field)

Info emits an info app log.

func (*Logger) OverrideRequestLogLevel

func (l *Logger) OverrideRequestLogLevel(level Level)

OverrideRequestLogLevel overrides the current request access log level.

func (*Logger) Sync

func (l *Logger) Sync() error

Sync flushes writer buffers when supported.

func (*Logger) Warn

func (l *Logger) Warn(message string, fields ...Field)

Warn emits a warn app log.

func (*Logger) WithComponent

func (l *Logger) WithComponent(component string) *Logger

WithComponent returns a derived logger with a component field.

func (*Logger) WithFields

func (l *Logger) WithFields(fields ...Field) *Logger

WithFields returns a derived logger with extra structured fields.

func (*Logger) WithName

func (l *Logger) WithName(name string) *Logger

WithName returns a derived logger with a name field.

func (*Logger) WithRequestLogAccumulator

func (l *Logger) WithRequestLogAccumulator(accumulator *RequestLogAccumulator) *Logger

WithRequestLogAccumulator returns a derived request-scoped logger.

func (*Logger) WriteAccessLog

func (l *Logger) WriteAccessLog(entry AccessLogEntry)

WriteAccessLog emits the final structured access log entry.

type Observer

type Observer interface {
	ObserveAppLog(level Level, logType string)
	ObserveAccessLog(level Level, statusCode int, route string, latency time.Duration)
}

Observer receives structured log emission callbacks.

func NopObserver

func NopObserver() Observer

NopObserver returns a no-op observer.

type Redactor

type Redactor interface {
	RedactField(key string, value interface{}) interface{}
}

Redactor redacts sensitive field values before output.

func DefaultRedactor

func DefaultRedactor() Redactor

DefaultRedactor returns the built-in redactor.

type RequestLogAccumulator

type RequestLogAccumulator struct {
	// contains filtered or unexported fields
}

RequestLogAccumulator aggregates per-request structured additions.

func NewRequestLogAccumulator

func NewRequestLogAccumulator(limits RequestLogLimits) *RequestLogAccumulator

NewRequestLogAccumulator returns a new request log accumulator.

type RequestLogLimits

type RequestLogLimits struct {
	MaxFieldCount   int
	MaxMessageCount int
	MaxErrorCount   int
}

RequestLogLimits bounds request aggregation growth.

type SamplingConfig

type SamplingConfig struct {
	Enabled          bool
	Interval         time.Duration
	First            int
	Thereafter       int
	DisableForError  bool
	DisableForAccess bool
}

SamplingConfig controls app log sampling.