zlog

package module
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2025 License: MIT Imports: 10 Imported by: 1

README

zlog

CI Status codecov Go Report Card CodeQL Go Reference License Go Version

Signal-based structured logging for Go that acknowledges different events need different handling.

A different approach to logging that uses semantic signals instead of severity levels. Route payment events to your audit system, errors to your alerts, and metrics to your time-series database - all through one simple API.

// Traditional logging forces everything into severity levels
log.Info("payment processed")  // Is this info or audit?
log.Error("rate limit hit")    // Is this error or metric?

// zlog uses signals to route events where they belong
ctx := context.Background()
zlog.Emit(ctx, PAYMENT_RECEIVED, "Payment processed",
    zlog.String("user_id", "123"),
    zlog.Float64("amount", 99.99))

zlog.RouteSignal(PAYMENT_RECEIVED, auditSink)   // → Audit trail
zlog.RouteSignal(PAYMENT_RECEIVED, metricsSink) // → Revenue metrics

Why zlog?

  • Signal-based routing: Events go where they belong, not into severity buckets
  • True structured logging: Type-safe fields with compile-time safety
  • Simple and fast: Sequential sink processing for predictable performance
  • Extensible: Easy to add custom sinks for any destination
  • Zero-allocation fields: Field constructors create no heap allocations
  • Simple: Clean API that's easy to understand and use
  • Built on pipz: Access to pipeline patterns like retries and fallbacks when needed

Installation

go get github.com/zoobzio/zlog

Requirements: Go 1.21+ (for generics)

Quick Start

Traditional Logging

For a familiar logging experience with structured fields:

import "github.com/zoobzio/zlog"

func main() {
    // Enable JSON logging to stderr
    zlog.EnableStandardLogging(zlog.INFO)
    
    // Use familiar log levels
    ctx := context.Background()
    zlog.Info(ctx, "Server starting", zlog.Int("port", 8080))
    zlog.Debug(ctx, "This won't show with INFO level")
    
    // Structured fields with type safety
    zlog.Error(ctx, "Connection failed",
        zlog.Err(err),
        zlog.String("host", "db.example.com"),
        zlog.Duration("timeout", 30*time.Second))
}
Signal-Based Routing

Define signals that match your application's events:

// Define domain-specific signals
const (
    PAYMENT_RECEIVED = zlog.Signal("PAYMENT_RECEIVED")
    PAYMENT_FAILED   = zlog.Signal("PAYMENT_FAILED")
    USER_LOGIN       = zlog.Signal("USER_LOGIN")
    CACHE_MISS       = zlog.Signal("CACHE_MISS")
)

// Create specialized sinks
auditSink := zlog.NewSink("audit", func(ctx context.Context, e zlog.Event) error {
    // Write to audit log with regulatory compliance formatting
    return auditWriter.WriteEvent(e)
})

alertSink := zlog.NewSink("alerts", func(ctx context.Context, e zlog.Event) error {
    if e.Signal == PAYMENT_FAILED {
        return slack.PostAlert(e.Message, e.Fields)
    }
    return nil
})

// Route signals to appropriate handlers (multiple sinks per signal)
zlog.RouteSignal(PAYMENT_RECEIVED, auditSink)
zlog.RouteSignal(PAYMENT_FAILED, auditSink, alertSink)  // Goes to both!
zlog.RouteSignal(USER_LOGIN, auditSink)
zlog.RouteSignal(CACHE_MISS, metricsSink)

// Emit events with meaning
ctx := context.Background()
zlog.Emit(ctx, PAYMENT_RECEIVED, "Payment processed successfully",
    zlog.String("user_id", userID),
    zlog.String("payment_id", paymentID),
    zlog.Float64("amount", amount),
    zlog.String("currency", "USD"))

Core Concepts

Signals vs Levels

Traditional logging makes you choose a "severity" for every event. But is a failed payment an ERROR or a WARN? Is a successful login INFO or DEBUG? These aren't severity decisions - they're routing decisions.

Signals let you say what happened, not how important it is:

// Instead of arguing about severity...
log.Warn("Payment declined")  // or is it Error? Info?

// Say what actually happened
zlog.Emit(ctx, PAYMENT_DECLINED, "Card declined", 
    zlog.String("reason", "insufficient_funds"))
Structured Fields

Type-safe field constructors prevent errors at compile time:

zlog.Info(ctx, "Request completed",
    zlog.String("method", "POST"),
    zlog.String("path", "/api/users"),
    zlog.Int("status", 201),
    zlog.Duration("latency", time.Since(start)),
    zlog.Time("timestamp", time.Now()),
    zlog.Err(err),  // nil-safe
    zlog.Data("user", user))  // arbitrary types
Multiple Sinks

Events can go to multiple destinations in a single call:

// Route errors to multiple handlers at once
zlog.RouteSignal(zlog.ERROR, fileSink, consoleSink, alertSink, metricsSink)

// Or add them separately - same effect
zlog.RouteSignal(zlog.ERROR, fileSink)     // Permanent record
zlog.RouteSignal(zlog.ERROR, consoleSink)  // Developer visibility
Sampling High-Volume Events

Reduce load while maintaining visibility with sampling:

// Sample 10% of cache hits (high volume)
cacheSink := metricsSink.WithSampling(0.1)
zlog.RouteSignal(CACHE_HIT, cacheSink)

// Sample 1% of API requests
apiSink := fileSink.WithSampling(0.01).WithAsync()
zlog.RouteSignal(API_REQUEST, apiSink)

// For statistical sampling use probabilistic mode
randomSink := debugSink.WithProbabilisticSampling(0.25) // 25% random sample
Creating Modules

Modules are just functions that set up routing. See log.go for the standard logging module:

// myapp/logging/siem.go
package logging

import (
    "github.com/zoobzio/zlog"
    "github.com/splunk/splunk-sdk-go"
)

var siemSink = zlog.NewSink("siem-forwarder", func(ctx context.Context, e zlog.Event) error {
    return splunk.Send(convertToSplunkEvent(e))
})

// EnableSIEMForwarding routes security events to your SIEM
func EnableSIEMForwarding(config SIEMConfig) error {
    if err := splunk.Connect(config); err != nil {
        return err
    }
    
    zlog.RouteSignal(zlog.SECURITY, siemSink)
    zlog.RouteSignal(zlog.AUDIT, siemSink)
    zlog.RouteSignal("INTRUSION_DETECTED", siemSink)
    zlog.RouteSignal("PRIVILEGE_ESCALATION", siemSink)
    
    return nil
}

Examples

Web Service
const (
    REQUEST_START    = zlog.Signal("REQUEST_START")
    REQUEST_COMPLETE = zlog.Signal("REQUEST_COMPLETE")
    AUTH_FAILED      = zlog.Signal("AUTH_FAILED")
)

func handler(w http.ResponseWriter, r *http.Request) {
    start := time.Now()
    
    zlog.Emit(ctx, REQUEST_START, "Handling request",
        zlog.String("method", r.Method),
        zlog.String("path", r.URL.Path),
        zlog.String("remote_addr", r.RemoteAddr))
    
    if !authenticate(r) {
        zlog.Emit(ctx, AUTH_FAILED, "Authentication failed",
            zlog.String("path", r.URL.Path),
            zlog.String("auth_header", r.Header.Get("Authorization")))
        http.Error(w, "Unauthorized", 401)
        return
    }
    
    // ... handle request ...
    
    zlog.Emit(ctx, REQUEST_COMPLETE, "Request completed",
        zlog.String("method", r.Method),
        zlog.String("path", r.URL.Path),
        zlog.Int("status", 200),
        zlog.Duration("latency", time.Since(start)))
}
Background Jobs
const (
    JOB_STARTED   = zlog.Signal("JOB_STARTED")
    JOB_COMPLETED = zlog.Signal("JOB_COMPLETED")
    JOB_FAILED    = zlog.Signal("JOB_FAILED")
    JOB_RETRY     = zlog.Signal("JOB_RETRY")
)

func processJob(job Job) {
    zlog.Emit(ctx, JOB_STARTED, "Processing job",
        zlog.String("job_id", job.ID),
        zlog.String("type", job.Type))
    
    for attempt := 0; attempt < maxRetries; attempt++ {
        if err := job.Execute(); err != nil {
            zlog.Emit(ctx, JOB_RETRY, "Job failed, retrying",
                zlog.String("job_id", job.ID),
                zlog.Int("attempt", attempt+1),
                zlog.Err(err))
            time.Sleep(backoff(attempt))
            continue
        }
        
        zlog.Emit(ctx, JOB_COMPLETED, "Job completed successfully",
            zlog.String("job_id", job.ID),
            zlog.Duration("duration", time.Since(start)))
        return
    }
    
    zlog.Emit(ctx, JOB_FAILED, "Job failed after all retries",
        zlog.String("job_id", job.ID),
        zlog.Int("attempts", maxRetries))
}

Advanced Capabilities with pipz

zlog is built on pipz, giving you access to sophisticated event processing when you need it:

// Start simple - basic routing
zlog.RouteSignal(PAYMENT_FAILED, alertSink)

// Add reliability when needed
reliableAudit := pipz.Retry("audit-write", 3, auditSink)
zlog.RouteSignal(PAYMENT_RECEIVED, reliableAudit)

// Or build complex processing pipelines
errorPipeline := pipz.NewSequence("error-handling",
    pipz.Apply("sanitize", removeSensitiveData),
    pipz.NewFallback("delivery",
        pipz.Retry("primary", 3, sendToElasticsearch),
        pipz.Apply("fallback", writeToLocalFile),
    ),
    pipz.Effect("metrics", updateErrorMetrics),
)
zlog.RouteSignal(ERROR, errorPipeline)

With pipz integration, you get:

  • Retry with backoff - Automatic retries for transient failures
  • Fallback chains - Primary/backup sink strategies
  • Circuit breakers - Protect against cascading failures
  • Concurrent processing - Fan-out to multiple sinks in parallel
  • Event transformation - Modify events before delivery
  • Conditional routing - Route based on event content
  • And much more - Full pipeline capabilities when you need them

The beauty is progressive complexity - use simple sinks for simple needs, tap into pipz power when you need sophisticated processing.

Design Philosophy

  1. Events have types, not severities: A payment failure isn't an "error level" - it's a payment failure that might need fraud detection, customer notification, and metric tracking.

  2. Structured data is primary: Messages are for humans, fields are for machines. Every event should include rich context.

  3. Multiple handlers are normal: Real events often need multiple actions. Sequential routing keeps it simple and fast.

  4. Simple things should be simple: You can use zlog like a traditional logger with EnableStandardLogging() and gradually adopt signals.

  5. Progressive complexity: Start with simple sequential processing. Add pipz pipelines when you need retries, concurrency, or transformations.

Performance

zlog is designed with performance in mind:

  • Zero-allocation field constructors: Field creation benchmarks show 0 allocations
  • Efficient routing: Direct dispatch to sinks without cloning
  • Sequential processing: Predictable performance without concurrency overhead
  • 95.9% test coverage: Comprehensive test suite
  • Benchmarked: See BENCHMARKS.md for detailed performance metrics

Questions & Answers

How is this different from traditional loggers?

Traditional loggers focus on severity (debug < info < warn < error). zlog focuses on event types. You don't filter by "level", you route by signal.

Can I use both signals and levels?

Yes! EnableStandardLogging() provides familiar level-based logging. You can mix approaches as needed.

What about log sampling/filtering?

Create a filtering sink using pipz capabilities:

samplingSink := pipz.NewSampler(0.1, actualSink) // 10% sampling
zlog.RouteSignal(HIGH_VOLUME_SIGNAL, samplingSink)

How do I rotate log files?

File rotation belongs in the sink, not the logger. Use a proper file sink like lumberjack or let your platform handle it (systemd, Docker, K8s).

Contributing

Contributions welcome! Please ensure:

  • Tests pass: go test ./...
  • Coverage maintained: go test -cover (currently 95.9%)
  • Benchmarks pass: go test -bench=.
  • Code is formatted: go fmt ./...
  • Lint passes: golangci-lint run

License

MIT License - see LICENSE file for details.

Documentation

Overview

Package zlog provides signal-based structured logging for Go applications.

Traditional logging forces you into severity levels (debug, info, warn, error), but real applications have diverse event types that need different handling: payment events need audit trails, security events need alerting, metrics need aggregation, and debug logs need filtering. zlog solves this with signals.

Core Concepts

Signals are simple strings that categorize events by their meaning, not severity. Instead of deciding if something is "info" or "warn", you emit events with meaningful signals like "PAYMENT_PROCESSED" or "CACHE_MISS".

Events flow through a routing system that delivers them to appropriate sinks based on their signal. Multiple sinks can process the same signal concurrently, enabling patterns like storing errors in files while also sending alerts.

Basic Usage

For traditional logging to stderr: 

zlog.EnableStandardLogging(zlog.INFO)
zlog.Info(context.Background(), "Application started", zlog.String("version", "1.0.0"))
zlog.Error(context.Background(), "Database connection failed", zlog.Err(err))

Signal-Based Routing

Define domain-specific signals and route them appropriately: 

const (
    PAYMENT_RECEIVED = zlog.Signal("PAYMENT_RECEIVED")
    FRAUD_DETECTED   = zlog.Signal("FRAUD_DETECTED")
)

// Hook payment events to audit sink
auditSink := zlog.NewSink("audit", handleAuditEvent)
zlog.Hook(PAYMENT_RECEIVED, auditSink)

// Hook fraud to multiple destinations
zlog.Hook(FRAUD_DETECTED, auditSink)
zlog.Hook(FRAUD_DETECTED, alertSink)
zlog.Hook(FRAUD_DETECTED, metricsSink)

// Emit domain events
zlog.Emit(context.Background(), PAYMENT_RECEIVED, "Payment processed",
    zlog.String("user_id", "123"),
    zlog.Float64("amount", 99.99),
)

Creating Modules

Modules are functions that configure routing for specific use cases. See log.go for the standard logging module example: 

var jsonSink = zlog.NewSink("json", formatJSON)

func EnableMyModule(config Config) {
    zlog.RouteSignal(SIGNAL1, jsonSink)
    zlog.RouteSignal(SIGNAL2, customSink)
}

Performance

zlog is designed for high-throughput applications: - Efficient field creation with minimal allocations - Lock-free event routing on the hot path - Concurrent sink processing with event cloning for isolation - Immutable events prevent data races between sinks

Built on github.com/zoobzio/pipz for advanced pipeline capabilities.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Debug 

func Debug(ctx context.Context, msg string, fields ...Field)

Debug emits a debug-level event for development and troubleshooting. Debug events are typically filtered out in production. 

zlog.Debug(ctx, "Cache lookup", zlog.String("key", cacheKey))

func Emit 

func Emit(ctx context.Context, signal Signal, msg string, fields ...Field)

Emit sends an event with the specified signal, message, and optional fields.

This is the primary logging function in zlog. Unlike traditional loggers that force you to choose a severity level, Emit lets you specify exactly what type of event this is through the signal parameter.

The signal determines how the event is routed - different sinks can be registered to handle different signals. Any string can be used as a signal, though constants are provided for common cases (INFO, ERROR, etc.).

Fields provide structured context using type-safe constructors: 

zlog.Emit(ctx, zlog.INFO, "User logged in",
    zlog.String("user_id", "123"),
    zlog.String("ip", request.RemoteAddr),
    zlog.Duration("session_duration", 30*time.Minute),
)

Emit automatically captures caller information (file, line, function) for debugging. Events are processed asynchronously - Emit returns immediately after routing the event to the appropriate sinks.

func EnableStandardLogging 

func EnableStandardLogging(level Signal)

EnableStandardLogging enables JSON output to stderr for standard log signals. The level parameter determines the minimum signal level that will be logged:

  • DEBUG: All signals (DEBUG, INFO, WARN, ERROR, FATAL)
  • INFO: INFO and above (INFO, WARN, ERROR, FATAL)
  • WARN: WARN and above (WARN, ERROR, FATAL)
  • ERROR: ERROR and above (ERROR, FATAL)
  • FATAL: Only FATAL

func Error 

func Error(ctx context.Context, msg string, fields ...Field)

Error emits an error event for failures that need attention. The application continues running but something failed. 

zlog.Error(ctx, "Failed to send email", zlog.Err(err), zlog.String("to", email))

func ExtractContext 

func ExtractContext(fields ...ContextField)

ExtractContext configures automatic extraction of values from context for the global logger.

This function registers fields that should be automatically extracted from the context and added to every log event emitted through the global API (Debug, Info, Warn, Error, Fatal, Emit).

Example: 

// Configure global context extraction for tracing
zlog.ExtractContext(
    zlog.ContextField{
        ContextKey: "trace-id",
        FieldName:  "trace_id",
        FieldType:  zlog.StringType,
    },
    zlog.ContextField{
        ContextKey: "request-id",
        FieldName:  "request_id",
        FieldType:  zlog.StringType,
    },
)

// Now all logs will automatically include trace_id and request_id if present
ctx := context.WithValue(context.Background(), "trace-id", "abc123")
ctx = context.WithValue(ctx, "request-id", "req456")
zlog.Info(ctx, "Processing request") // Will include trace_id="abc123" request_id="req456"

func Fatal 

func Fatal(ctx context.Context, msg string, fields ...Field)

Fatal emits a fatal event and terminates the application with os.Exit(1). Use this for unrecoverable errors that prevent the application from continuing. Fatal includes a 100ms delay before exiting to allow sinks to flush. 

zlog.Fatal(ctx, "Failed to connect to database", zlog.Err(err))

func Hook 

func Hook(signal Signal, sinks ...*Sink)

Hook registers one or more sinks to process events with the specified signal.

Multiple sinks can process the same signal - they run in parallel using fire-and-forget semantics. This provides optimal performance with automatic event cloning for safe concurrent processing: 

// Send errors to multiple destinations (processed in parallel)
zlog.Hook(zlog.ERROR, fileSink, alertSink, metricsSink)

// Or add them separately - same effect (all run in parallel)
zlog.Hook(zlog.ERROR, fileSink)      // Permanent storage
zlog.Hook(zlog.ERROR, alertSink)     // Team notifications
zlog.Hook(zlog.ERROR, metricsSink)   // Error rate tracking

// Hook business events
zlog.Hook(PAYMENT_RECEIVED, auditSink, analyticsSink)

Routes can be added at any time, even after events start flowing. There's no way to remove routes - design your signal strategy accordingly.

func HookAll 

func HookAll(sinks ...*Sink)

HookAll registers one or more sinks to process ALL events before signal routing.

These sinks run before the signal-based routing, allowing you to implement cross-cutting concerns like development logging, metrics collection, or audit trails that need to see every event: 

// Log everything to console in development
if isDev {
    consoleSink := zlog.NewConsoleSink(os.Stderr)
    zlog.HookAll(consoleSink)
}

// Collect metrics for all events
zlog.HookAll(metricsSink)

Global sinks run in the order they were registered, before any signal-specific routing occurs. They see every event emitted to the system.

func Info 

func Info(ctx context.Context, msg string, fields ...Field)

Info emits an informational event for normal operational messages. Use this for events that confirm normal operation. 

zlog.Info(ctx, "Server started", zlog.Int("port", 8080))

func RouteAll 

func RouteAll(sinks ...*Sink)

RouteAll is a backward-compatible alias for HookAll. Deprecated: Use HookAll instead.

func RouteSignal 

func RouteSignal(signal Signal, sinks ...*Sink)

RouteSignal is a backward-compatible alias for Hook. Deprecated: Use Hook instead.

func Warn 

func Warn(ctx context.Context, msg string, fields ...Field)

Warn emits a warning event for concerning but recoverable situations. Use this when something is wrong but the application can continue. 

zlog.Warn(ctx, "API rate limit approaching", zlog.Int("remaining", 100))

Types

type CallerInfo 

type CallerInfo struct {
	File     string
	Function string
	Line     int
}

CallerInfo contains the file, line, and function of the log call site.

type CircuitBreakerConfig 

type CircuitBreakerConfig struct {
	// FailureThreshold is the number of consecutive failures before opening.
	FailureThreshold int
	// SuccessThreshold is the number of successes in half-open before closing.
	SuccessThreshold int
	// ResetTimeout is how long to wait before trying half-open.
	ResetTimeout time.Duration
}

CircuitBreakerConfig configures circuit breaker behavior.

type CircuitState 

type CircuitState string

CircuitState represents the current state of a circuit breaker. This type is kept for backwards compatibility with tests and examples. 

const (
	// CircuitClosed allows requests through (normal operation).
	CircuitClosed CircuitState = "closed"
	// CircuitOpen blocks all requests (failure mode).
	CircuitOpen CircuitState = "open"
	// CircuitHalfOpen allows limited requests for testing.
	CircuitHalfOpen CircuitState = "half-open"
)

type ContextField 

type ContextField struct {
	// ContextKey is the key used to retrieve the value from context.
	// This can be a string, custom type, or any comparable value.
	ContextKey any

	// FieldName is the name of the field in the log output.
	FieldName string

	// FieldType specifies the expected type of the context value.
	// This ensures type-safe extraction and prevents runtime panics.
	FieldType FieldType
}

ContextField defines a value to extract from context into log fields.

This type is used to configure automatic extraction of values from context.Context into structured log fields. Common use cases include trace IDs, span IDs, request IDs, and user identifiers.

Example: 

// Configure extraction of trace and user IDs
logger.ExtractContext(
    zlog.ContextField{
        ContextKey: "trace-id",
        FieldName:  "trace_id",
        FieldType:  zlog.StringType,
    },
    zlog.ContextField{
        ContextKey: userIDKey{}, // Can use custom types as keys
        FieldName:  "user_id",
        FieldType:  zlog.StringType,
    },
)

type Event 

type Event[T any] struct {
	Time    time.Time
	Data    T
	Message string
	Signal  Signal
	Caller  CallerInfo
}

Event represents an immutable signal event that flows through sinks. The generic type T allows for different data payloads:

  • Log for the global logger with structured fields
  • Event[Order] for typed loggers with domain objects

func (Event[T]) Clone 

func (e Event[T]) Clone() Event[T]

Clone creates a copy of the Event. This implements the pipz.Cloner interface for use with pipz pipelines.

type Field 

type Field struct {
	// Value holds the actual data
	Value any `json:"value"`

	// Key identifies this field
	Key string `json:"key"`

	// Type indicates how to interpret Value
	Type FieldType `json:"type"`
}

Field represents a typed key-value pair for structured logging.

Fields provide type-safe structured data that can be processed by sinks. Unlike using map[string]interface{}, fields preserve type information and are created with zero allocations using the provided constructors.

Fields are immutable after creation and safe to share between goroutines.

func Bool 

func Bool(key string, value bool) Field

Bool creates a boolean field. 

zlog.Bool("success", true)
zlog.Bool("is_authenticated", user.IsAuthenticated())

func ByteString 

func ByteString(key string, value []byte) Field

ByteString creates a field for binary data.

The bytes are converted to a string for storage. Sinks typically encode this as base64 or hex when formatting. 

zlog.ByteString("request_body", body)
zlog.ByteString("hash", sha256.Sum256(data))

func Data 

func Data[T any](key string, value T) Field

Data creates a field for arbitrary structured data.

Use this for complex types that don't fit the standard field types. The value is stored as-is - sinks are responsible for serialization. 

zlog.Data("user", user)
zlog.Data("request_headers", req.Header)
zlog.Data("metrics", map[string]int{"hits": 42, "misses": 7})

func Duration 

func Duration(key string, value time.Duration) Field

Duration creates a time duration field. 

zlog.Duration("latency", time.Since(start))
zlog.Duration("timeout", 30*time.Second)

func Err 

func Err(err error) Field

Err creates an error field with key "error".

The error is stored as a string. If err is nil, the field value is nil. 

zlog.Error("Failed to connect", zlog.Err(err))
zlog.Info("Retry succeeded", zlog.Err(lastErr))

func Float64 

func Float64(key string, value float64) Field

Float64 creates a floating-point field. 

zlog.Float64("temperature", 98.6)
zlog.Float64("response_time", 1.234)

func Int 

func Int(key string, value int) Field

Int creates an integer field. 

zlog.Int("status_code", 200)
zlog.Int("retry_count", attempts)

func Int64 

func Int64(key string, value int64) Field

Int64 creates a 64-bit integer field. 

zlog.Int64("user_id", userID)
zlog.Int64("timestamp", time.Now().Unix())

func String 

func String(key, value string) Field

String creates a string field. 

zlog.String("user_id", "123")
zlog.String("method", request.Method)

func Strings 

func Strings(key string, value []string) Field

Strings creates a field for string slices. 

zlog.Strings("tags", []string{"api", "v2", "public"})
zlog.Strings("errors", validationErrors)

func Time 

func Time(key string, value time.Time) Field

Time creates a time.Time field. 

zlog.Time("created_at", user.CreatedAt)
zlog.Time("next_retry", time.Now().Add(backoff))

type FieldType 

type FieldType string

FieldType identifies how a Field's Value should be interpreted.

Using strings instead of iota allows sinks to handle types without importing zlog, making the system more extensible. Custom sinks can define their own types if needed. 

const (
	// StringType for string values.
	StringType FieldType = "string"

	// IntType for int values.
	IntType FieldType = "int"

	// Int64Type for int64 values.
	Int64Type FieldType = "int64"

	// Float64Type for float64 values.
	Float64Type FieldType = "float64"

	// BoolType for boolean values.
	BoolType FieldType = "bool"

	// ErrorType for error values (stored as strings).
	ErrorType FieldType = "error"

	// DurationType for time.Duration values.
	DurationType FieldType = "duration"

	// TimeType for time.Time values.
	TimeType FieldType = "time"

	// ByteStringType for []byte values (often base64 encoded).
	ByteStringType FieldType = "bytestring"

	// DataType for arbitrary structured data.
	DataType FieldType = "data"

	// StringsType for []string values.
	StringsType FieldType = "strings"
)

Standard field types cover common logging use cases. Sinks can use the Type field to handle values appropriately.

type Fields 

type Fields []Field

Fields represents a collection of Field values that can be cloned. This type is used as the data type for the global logger's Log.

func (Fields) Clone 

func (f Fields) Clone() Fields

Clone creates a deep copy of the Fields slice. This implements the pipz.Cloner interface for use with pipz pipelines.

type Log 

type Log = Event[Fields]

Log is the standard event type used by the global logger. It's an alias for Event[Fields] to provide a cleaner API.

func NewEvent 

func NewEvent(signal Signal, msg string, fields []Field) Log

NewEvent creates a new Event with the current timestamp.

This is primarily used internally by Emit() and the convenience functions. Most users should use those higher-level functions instead of creating events directly.

The fields parameter can be nil if no structured data is needed.

type Logger 

type Logger[T any] struct {
	// contains filtered or unexported fields
}

Logger provides typed event processing with signal-based routing.

Logger[T] processes Event[T] types through a pipeline with signal-based routing. This enables type-safe hooks and transformations while maintaining integration with the existing zlog ecosystem.

The Logger uses the same pipeline architecture as the global system:

  • Events flow through a root Sequence (for HookAll processors)
  • Signal-based routing via Switch (extracts from Event.Signal)
  • Parallel processing via Scaffold for multiple hooks per signal

Example usage: 

type Order struct {
    ID     string
    Amount float64
    Status string
}

func (o Order) Clone() Order {
    return Order{ID: o.ID, Amount: o.Amount, Status: o.Status}
}

orderLogger := zlog.NewLogger[Order]()

// Add typed hooks that work directly with Order events
auditHook := zlog.NewHook[Event[Order]]("audit", func(ctx context.Context, event Event[Order]) (Event[Order], error) {
    auditDB.Store(event.Data)
    return event, nil
})

orderLogger.Hook(ORDER_CREATED, auditHook)
orderLogger.Emit(ORDER_CREATED, "Order created", order)

func NewLogger 

func NewLogger[T any]() *Logger[T]

NewLogger creates a typed logger that processes Event[T] types.

The logger processes events through a pipeline with signal-based routing, similar to the global logger but with type safety for the event data.

Example: 

orderLogger := zlog.NewLogger[Order]()
orderLogger.Emit(ORDER_CREATED, "Order created", order)

func (*Logger[T]) Emit 

func (l *Logger[T]) Emit(ctx context.Context, signal Signal, message string, data T)

Emit creates an Event[T] and processes it through the logger pipeline.

The event flows through:

  1. HookAll processors (cross-cutting concerns)
  2. Signal-based routing and Hook processors

Example: 

orderLogger.Emit(ctx, ORDER_CREATED, "Order created", order)

func (*Logger[T]) ExtractContext 

func (l *Logger[T]) ExtractContext(fields ...ContextField) *Logger[T]

ExtractContext configures automatic extraction of values from context.

This method registers fields that should be automatically extracted from the context and added to every log event. This is particularly useful for adding tracing information, request IDs, or user identifiers to all logs.

The extraction happens at the beginning of the pipeline, before any other processing or routing occurs.

Example: 

// Extract trace and span IDs for distributed tracing
logger.ExtractContext(
    zlog.ContextField{
        ContextKey: "trace-id",
        FieldName:  "trace_id",
        FieldType:  zlog.StringType,
    },
    zlog.ContextField{
        ContextKey: "span-id",
        FieldName:  "span_id",
        FieldType:  zlog.StringType,
    },
)

Note: This method only works for Logger[Fields]. For typed loggers with custom data types, context extraction is not supported as we cannot modify arbitrary types.

func (*Logger[T]) Hook 

func (l *Logger[T]) Hook(signal Signal, hooks ...pipz.Chainable[Event[T]]) *Logger[T]

Hook registers one or more hooks to process events with the specified signal.

Multiple hooks can process the same signal - they run in parallel using fire-and-forget semantics for optimal performance. This provides the same routing behavior as the global system but with type safety. 

orderLogger.Hook("HIGH_VALUE", auditHook, metricsHook, alertHook)

Hooks can be added dynamically without stopping event flow.

func (*Logger[T]) HookAll 

func (l *Logger[T]) HookAll(hooks ...pipz.Chainable[Event[T]]) *Logger[T]

HookAll registers one or more hooks to process ALL events before signal routing.

These hooks run before the signal-based routing, allowing you to implement cross-cutting concerns that need to see every typed event: 

orderLogger.HookAll(validationHook, enrichmentHook)

Global hooks run in the order they were registered, before any signal-specific routing occurs. They see every event emitted to this logger.

func (*Logger[T]) Process 

func (l *Logger[T]) Process(ctx context.Context, event Event[T])

Process handles pre-built Event[T] types through the logger pipeline. This method does not capture caller info - it should already be in the event.

func (*Logger[T]) Watch 

func (l *Logger[T]) Watch() *Logger[T]

Watch configures this logger to forward all events to the global logger after processing through the typed pipeline.

This enables typed loggers to integrate with the global logging system while maintaining type safety for their own processing.

Example: 

orderLogger := NewLogger[Order]().Watch()
orderLogger.Emit(ORDER_CREATED, "Order created", order)
// Event flows through typed hooks, then to global logger

func (*Logger[T]) WithAsync 

func (l *Logger[T]) WithAsync() *Logger[T]

WithAsync makes the logger process events asynchronously. 

orderLogger.WithAsync()

func (*Logger[T]) WithFilter 

func (l *Logger[T]) WithFilter(predicate func(Event[T]) bool) *Logger[T]

WithFilter adds a filter to the logger pipeline that only allows events matching the predicate to continue processing. 

orderLogger.WithFilter(func(order Order) bool {
    return order.Amount > 100.0
})

func (*Logger[T]) WithRetry 

func (l *Logger[T]) WithRetry(attempts int) *Logger[T]

WithRetry adds retry capability to the logger pipeline. 

orderLogger.WithRetry(3)

func (*Logger[T]) WithTimeout 

func (l *Logger[T]) WithTimeout(timeout time.Duration) *Logger[T]

WithTimeout adds timeout protection to the logger pipeline. 

orderLogger.WithTimeout(5 * time.Second)

type RateLimiterConfig 

type RateLimiterConfig struct {
	// RequestsPerSecond is the sustained rate limit.
	RequestsPerSecond float64
	// BurstSize allows temporary spikes above the rate.
	BurstSize int
	// WaitForSlot determines if Process should block or error when limited.
	WaitForSlot bool
}

RateLimiterConfig configures rate limiting behavior.

type Signal 

type Signal string

Signal represents an event type in the logging system.

Unlike traditional severity levels, signals categorize events by their meaning rather than their importance. This enables sophisticated routing where different types of events can be handled by different systems.

While predefined signals are provided for compatibility with traditional logging, you are encouraged to define domain-specific signals: 

const (
    PAYMENT_RECEIVED = Signal("PAYMENT_RECEIVED")
    USER_REGISTERED  = Signal("USER_REGISTERED")
    CACHE_MISS       = Signal("CACHE_MISS")
)

Signals are just strings, making them easy to create and use. The routing system uses exact string matching to determine which sinks handle each signal. 

const (
	// DEBUG indicates detailed information for diagnosing problems.
	// Typically disabled in production.
	DEBUG Signal = "DEBUG"

	// INFO indicates informational messages about normal operation.
	INFO Signal = "INFO"

	// WARN indicates potentially harmful situations that deserve attention.
	WARN Signal = "WARN"

	// ERROR indicates error events that might still allow the application to continue.
	ERROR Signal = "ERROR"

	// FATAL indicates severe errors that will cause the application to exit.
	FATAL Signal = "FATAL"
)

Standard logging signals provide compatibility with traditional level-based logging. These signals have implicit severity ordering when used with EnableStandardLogging. 

const (
	// AUDIT events track user actions for compliance and forensics.
	// Route these to secure, tamper-proof storage.
	AUDIT Signal = "AUDIT"

	// SECURITY events indicate potential security issues.
	// Route these to security monitoring systems.
	SECURITY Signal = "SECURITY"

	// METRIC events carry measurement data for monitoring.
	// Route these to time-series databases or metrics aggregators.
	METRIC Signal = "METRIC"
)

Specialized signals for common use cases beyond traditional logging. These demonstrate how signals can represent domain concepts rather than severities.

type Sink 

type Sink struct {
	// contains filtered or unexported fields
}

Sink processes events routed by signal with composable capabilities.

Sinks are the extensibility point of zlog - they determine what happens to events after they're emitted. Common sink patterns include:

  • Writing to files or stdout/stderr
  • Sending to external services (Elasticsearch, Datadog, etc.)
  • Filtering or transforming events
  • Aggregating metrics
  • Triggering alerts

Multiple sinks can process the same signal concurrently. Each sink receives its own copy of events, preventing interference between sinks.

Sinks provide a fluent builder API for adding capabilities like retry, batching, filtering, and async processing. Each capability wraps the underlying processor with pipz primitives.

Example with capabilities: 

sink := zlog.NewSink("api", handler).
    WithRetry(3).
    WithTimeout(30 * time.Second)

func ConsoleJSONSink 

func ConsoleJSONSink(stdout bool) *Sink

ConsoleJSONSink outputs JSON-formatted logs to stdout/stderr for ALL signals.

Unlike stderrJSONSink which is designed for standard log levels, this sink captures every event regardless of signal type. It's ideal for development environments where you want complete visibility into all events.

By default, it writes to stderr. Pass true for stdout to write there instead.

Usage: 

// Route all events to stderr in development
if isDev {
    zlog.RouteAll(zlog.ConsoleJSONSink(false))
}

// Or to stdout
zlog.RouteAll(zlog.ConsoleJSONSink(true))

func NewSink 

func NewSink(name string, handler func(context.Context, Log) error) *Sink

NewSink creates a custom sink that processes events.

The name parameter identifies the sink in error messages and debugging output. The handler function is called for each event routed to this sink.

Example sink that writes to a file: 

fileSink := zlog.NewSink("file-writer", func(ctx context.Context, event zlog.Log) error {
    _, err := fmt.Fprintf(file, "[%s] %s: %s\n",
        event.Time.Format(time.RFC3339),
        event.Signal,
        event.Message)
    return err
})

Example sink that sends metrics: 

metricSink := zlog.NewSink("metrics", func(ctx context.Context, event zlog.Log) error {
    for _, field := range event.Data {
        if field.Key == "duration" {
            metrics.RecordDuration(event.Signal, field.Value.(time.Duration))
        }
    }
    return nil
})

Sinks should handle errors gracefully - returning an error doesn't affect other sinks or the application. Sinks run asynchronously after Emit returns.

The returned Sink can be enhanced with capabilities using the fluent API: 

sink := zlog.NewSink("example", handler).WithRetry(3)

func RateLimitedSink 

func RateLimitedSink(name string, requestsPerSecond float64, handler func(context.Context, Log) error) *Sink

RateLimitedSink creates a rate-limited sink with sensible defaults.

This is a convenience function that creates a sink with:

  • Specified requests per second sustained rate
  • Burst capacity equal to 2x the rate
  • Non-blocking mode (drops excess requests)

Example: 

sink := zlog.RateLimitedSink("api", 50, handler)
// Equivalent to:
// zlog.NewSink("api", handler).WithRateLimit(zlog.RateLimiterConfig{
//     RequestsPerSecond: 50,
//     BurstSize: 100,
//     WaitForSlot: false,
// })

func (Sink) Name 

func (s Sink) Name() pipz.Name

Name returns the name of the underlying processor.

func (Sink) Process 

func (s Sink) Process(ctx context.Context, event Log) (Log, error)

Process delegates to the underlying processor. This makes Sink implement pipz.Chainable[Log].

func (*Sink) WithAsync 

func (s *Sink) WithAsync() *Sink

WithAsync adds asynchronous processing to the sink.

The sink will process events in a background goroutine without blocking the caller. This is useful for slow sinks (external APIs, databases) that shouldn't block the main application flow.

Important characteristics:

  • Fire-and-forget: errors are not reported back to the caller
  • No buffering: each event spawns a new goroutine immediately
  • No backpressure: unlimited goroutines can be spawned
  • Fresh context: background processing uses context.Background()

Example usage: 

// Prevent slow API calls from blocking
asyncSink := zlog.NewSink("api", slowApiHandler).WithAsync()
zlog.RouteSignal(zlog.INFO, asyncSink)

// Combine with other adapters for robust async processing
robustSink := zlog.NewSink("external", handler).
    WithAsync().                    // Don't block the application
    WithRetry(3).                   // Retry failures in background
    WithTimeout(30 * time.Second)   // Timeout long operations

Warning: WithAsync provides no backpressure control. If events are produced faster than they can be processed, goroutines will accumulate. For high-volume scenarios, consider implementing a proper queuing system.

The original context is not propagated to avoid issues with short-lived contexts (e.g., HTTP request contexts) canceling background work.

func (*Sink) WithBackoff 

func (s *Sink) WithBackoff(maxAttempts int, baseDelay time.Duration) *Sink

WithBackoff adds retry with exponential backoff capability to the sink.

The sink will automatically retry failed operations with increasing delays between attempts. The delay starts at baseDelay and doubles after each failure, creating an exponential backoff pattern that prevents overwhelming failed services and allows time for transient issues to resolve.

This is more sophisticated than basic retry as it includes delays between attempts, making it ideal for external services that may be temporarily overloaded or rate-limited.

Example usage: 

// Retry API calls with exponential backoff
apiSink := zlog.NewSink("api", apiHandler).
    WithBackoff(5, 100*time.Millisecond)
zlog.RouteSignal(zlog.ERROR, apiSink)

// Combined with timeout for robust error handling
resilientSink := zlog.NewSink("external", handler).
    WithBackoff(3, time.Second).
    WithTimeout(30 * time.Second)

// Backoff delays: 1s, 2s, 4s (total wait: 7s plus processing time)
dbSink := zlog.NewSink("database", dbHandler).
    WithBackoff(4, time.Second)

The exponential backoff pattern (delay, 2*delay, 4*delay, ...) is widely used for handling rate limits, temporary service overload, and network congestion. The operation can be canceled via context during waits.

Total time can be significant with multiple retries. Plan accordingly when setting maxAttempts and baseDelay values.

func (*Sink) WithCircuitBreaker 

func (s *Sink) WithCircuitBreaker(config CircuitBreakerConfig) *Sink

WithCircuitBreaker adds circuit breaker protection to a sink using pipz.NewCircuitBreaker.

Circuit breaker prevents cascading failures by:

  • Opening after consecutive failures reach threshold
  • Blocking requests while open (fail-fast)
  • Transitioning to half-open for recovery testing
  • Closing after successful requests in half-open

Example: 

dbSink := zlog.NewSink("database", dbHandler).
    WithCircuitBreaker(zlog.CircuitBreakerConfig{
        FailureThreshold: 5,           // Open after 5 consecutive failures
        SuccessThreshold: 3,           // Close after 3 successes in half-open
        ResetTimeout: 30 * time.Second, // Try half-open after 30s
    })

func (*Sink) WithDefaultCircuitBreaker 

func (s *Sink) WithDefaultCircuitBreaker() *Sink

WithDefaultCircuitBreaker adds circuit breaker with sensible defaults.

Default configuration:

  • Opens after 5 consecutive failures
  • Closes after 2 consecutive successes in half-open state
  • Waits 30 seconds before attempting recovery

Example: 

sink := zlog.NewSink("fragile-api", handler).WithDefaultCircuitBreaker()

func (*Sink) WithFallback 

func (s *Sink) WithFallback(fallbackSink *Sink) *Sink

WithFallback adds fallback capability to the sink.

When the primary sink fails, the fallback sink will be tried automatically. This creates resilient processing chains that can recover from failures gracefully by switching to an alternative implementation.

Unlike retry which attempts the same operation multiple times, fallback switches to a completely different sink. This is valuable when you have multiple ways to accomplish the same goal.

Example usage: 

// Primary/backup service failover
primarySink := zlog.NewSink("primary-api", primaryHandler)
backupSink := zlog.NewSink("backup-api", backupHandler)
resilientSink := primarySink.WithFallback(backupSink)

zlog.RouteSignal(zlog.ERROR, resilientSink)

// Graceful degradation - try database, fall back to cache
dbSink := zlog.NewSink("database", dbHandler)
cacheSink := zlog.NewSink("cache", cacheHandler)
storageSink := dbSink.WithFallback(cacheSink)

// Can be chained with other capabilities
robustSink := primarySink.
    WithRetry(2).
    WithFallback(backupSink).
    WithTimeout(10 * time.Second)

If the primary sink succeeds, the fallback is never called. If the primary fails, the same event data is passed to the fallback sink. Both sinks receive identical event data for consistent processing.

func (*Sink) WithFilter 

func (s *Sink) WithFilter(predicate func(context.Context, Log) bool) *Sink

WithFilter adds conditional processing to the sink.

The sink will only process events that pass the predicate function. Events that don't match are silently skipped without calling the underlying sink handler. This is useful for creating specialized sinks that only care about specific types of events.

The predicate function receives the full event and should return true to process the event or false to skip it. This allows filtering on any aspect of the event: signal, message, fields, or metadata.

Example usage: 

// Only process ERROR events
errorSink := zlog.NewSink("errors", handler).
    WithFilter(func(ctx context.Context, e Log) bool {
        return e.Signal == zlog.ERROR
    })

// Only process high-value transactions
highValueSink := zlog.NewSink("big-money", handler).
    WithFilter(func(ctx context.Context, e Log) bool {
        for _, field := range e.Data {
            if field.Key == "amount" {
                if amount, ok := field.Value.(float64); ok {
                    return amount > 10000.0
                }
            }
        }
        return false
    })

// Only process events from specific source
internalSink := zlog.NewSink("internal", handler).
    WithFilter(func(ctx context.Context, e Log) bool {
        for _, field := range e.Data {
            if field.Key == "source" && field.Value == "internal" {
                return true
            }
        }
        return false
    })

// Chain with other capabilities
filteredRetrySink := zlog.NewSink("api", handler).
    WithFilter(func(ctx context.Context, e Log) bool {
        return e.Signal == zlog.ERROR
    }).
    WithRetry(3).
    WithTimeout(30 * time.Second)

Filtering is transparent to the rest of the pipeline - other sinks in the same signal route will still receive all events. Only this specific sink becomes selective about what it processes.

The predicate function should be fast since it's called for every event routed to this sink. Avoid expensive operations in the filter.

func (*Sink) WithProbabilisticSampling 

func (s *Sink) WithProbabilisticSampling(rate float64) *Sink

WithProbabilisticSampling returns a sink adapter that randomly samples events.

Unlike WithSampling which uses deterministic sampling, this uses random sampling. Each event has an independent probability of being processed.

This can be more appropriate when:

  • Events arrive in bursts (deterministic might miss entire bursts)
  • You need true statistical sampling
  • Event order is unpredictable

Example usage: 

// Randomly sample 25% of events
randomSink := debugSink.WithProbabilisticSampling(0.25)

func (*Sink) WithRateLimit 

func (s *Sink) WithRateLimit(config RateLimiterConfig) *Sink

WithRateLimit adds token bucket rate limiting to a sink using pipz.NewRateLimiter.

Token bucket algorithm provides:

  • Sustained rate limiting (requests per second)
  • Burst capacity for temporary spikes
  • Optional blocking until tokens available

Example: 

httpSink := httpsink.NewHTTPSink("https://api.example.com/logs").
    WithRateLimit(zlog.RateLimiterConfig{
        RequestsPerSecond: 100,  // Sustained rate
        BurstSize: 200,         // Allow bursts up to 200
        WaitForSlot: false,     // Don't block, fail fast
    })

func (*Sink) WithRetry 

func (s *Sink) WithRetry(attempts int) *Sink

WithRetry adds retry capability to the sink.

The sink will automatically retry failed operations up to the specified number of attempts. Retries are immediate without delay - for operations that need backoff between attempts, consider using pipz.NewBackoff directly.

Each retry receives the same event data. Retries stop immediately if the context is canceled, allowing for early termination during application shutdown or timeout scenarios.

Example usage: 

// Basic retry - try up to 3 times total
reliableSink := zlog.NewSink("api", apiHandler).WithRetry(3)
zlog.RouteSignal(zlog.ERROR, reliableSink)

// Chaining with other capabilities (future)
complexSink := zlog.NewSink("complex", handler).
    WithRetry(3).
    WithTimeout(30 * time.Second)

If all retry attempts fail, the last error is returned with attempt count information for debugging.

func (*Sink) WithSampling 

func (s *Sink) WithSampling(rate float64) *Sink

WithSampling returns a sink adapter that only processes a percentage of events.

This is useful for high-volume signals where you want to reduce load while still getting a representative sample. The sampling is deterministic based on a counter to ensure consistent sampling rates.

The rate parameter should be between 0.0 and 1.0:

  • 0.0 = no events pass through (why would you do this?)
  • 0.1 = 10% of events pass through
  • 0.5 = 50% of events pass through
  • 1.0 = all events pass through (no sampling)

Example usage: 

// Only process 10% of cache hit events to reduce metrics load
cacheSink := metricsSink.WithSampling(0.1)
zlog.RouteSignal(CACHE_HIT, cacheSink)

// Sample 1% of high-volume API logs
apiSink := fileSink.WithSampling(0.01).WithAsync()
zlog.RouteSignal(API_REQUEST, apiSink)

The sampling decision is made before the event reaches the sink, so filtered events have minimal performance impact.

func (*Sink) WithTimeout 

func (s *Sink) WithTimeout(duration time.Duration) *Sink

WithTimeout adds timeout capability to the sink.

The sink will enforce a hard timeout on event processing. If an operation takes longer than the specified duration, it will be canceled via context and a timeout error will be returned.

This is critical for preventing hung operations, meeting SLA requirements, and protecting against slow external services. The wrapped sink handler should respect context cancellation for immediate termination.

Example usage: 

// Prevent slow API calls from hanging
apiSink := zlog.NewSink("api", apiHandler).WithTimeout(5 * time.Second)
zlog.RouteSignal(zlog.ERROR, apiSink)

// Combined with retry for robust error handling
resilientSink := zlog.NewSink("db", dbHandler).
    WithRetry(3).
    WithTimeout(10 * time.Second)

// Order matters - this retries the entire timeout operation
retryThenTimeout := sink.WithRetry(3).WithTimeout(30 * time.Second)

// This times out each retry attempt individually
timeoutThenRetry := sink.WithTimeout(10 * time.Second).WithRetry(3)

If the timeout expires, the operation is canceled and a timeout error is returned. Operations that ignore context cancellation may continue running in the background even after timeout.

Source Files

View all Source files

Directories

Path Synopsis
sinks
console
file
http
loki module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.