log

README

Go Log Utilities

A Go library providing advanced logging utilities focused on log sampling and dynamic log level management, designed to integrate seamlessly with Google's glog library.

Features

• Log Sampling: Reduce log volume with intelligent sampling mechanisms
• Dynamic Log Level Management: Change log levels at runtime via HTTP endpoints
• Multiple Sampler Types: Counter-based, time-based, glog-level-based, and custom samplers
• Thread-Safe: All components are designed for concurrent use
• Extensible: Factory pattern and interface-based design for easy customization

Installation

go get github.com/bborbe/log

Quick Start

Basic Log Sampling

package main

import (
    "time"
    "github.com/bborbe/log"
    "github.com/golang/glog"
)

func main() {
    // Sample every 10th log entry
    modSampler := log.NewSampleMod(10)
    
    // Sample once every 10 seconds
    timeSampler := log.NewSampleTime(10 * time.Second)
    
    // Use in your logging code
    if modSampler.IsSample() {
        glog.V(2).Infof("This will be logged every 10th time")
    }
    
    if timeSampler.IsSample() {
        glog.V(2).Infof("This will be logged at most once every 10 seconds")
    }
}

Dynamic Log Level Management

package main

import (
    "context"
    "net/http"
    "time"
    
    "github.com/bborbe/log"
    "github.com/golang/glog"
    "github.com/gorilla/mux"
)

func main() {
    ctx := context.Background()
    
    // Create log level setter that auto-resets after 5 minutes
    logLevelSetter := log.NewLogLevelSetter(
        glog.Level(1), // default level
        5*time.Minute, // auto-reset duration
    )
    
    // Set up HTTP handler for dynamic log level changes
    router := mux.NewRouter()
    router.Handle("/debug/loglevel/{level}", 
        log.NewSetLoglevelHandler(ctx, logLevelSetter))
    
    http.ListenAndServe(":8080", router)
}

Now you can change log levels at runtime:

curl http://localhost:8080/debug/loglevel/4

Sampler Types

ModSampler

Samples every Nth log entry based on a counter:

sampler := log.NewSampleMod(100) // Sample every 100th log

TimeSampler

Samples based on time intervals:

sampler := log.NewSampleTime(30 * time.Second) // Sample at most once per 30 seconds

GlogLevelSampler

Samples based on glog verbosity levels:

sampler := log.NewSamplerGlogLevel(3) // Sample when glog level >= 3

ListSampler

Combines multiple samplers with OR logic:

sampler := log.SamplerList{
    log.NewSampleTime(10 * time.Second),
    log.NewSamplerGlogLevel(4),
}

FuncSampler

Create custom sampling logic:

sampler := log.SamplerFunc(func() bool {
    // Your custom sampling logic
    return shouldSample()
})

TrueSampler

Always samples (useful for testing or special cases):

sampler := log.SamplerTrue{}

Factory Pattern

Use the factory pattern for dependency injection:

// Use the default factory
factory := log.DefaultSamplerFactory
sampler := factory.Sampler()

// Or create a custom factory
customFactory := log.SamplerFactoryFunc(func() log.Sampler {
    return log.NewSampleMod(50)
})

HTTP Log Level Management

The library provides built-in HTTP handlers for runtime log level changes:

• Endpoint: GET/POST /debug/loglevel/{level}
• Auto-reset: Automatically reverts to default level after specified duration
• Thread-safe: Safe for concurrent access

Example integration with gorilla/mux:

router := mux.NewRouter()
logLevelSetter := log.NewLogLevelSetter(glog.Level(1), 5*time.Minute)
router.Handle("/debug/loglevel/{level}", 
    log.NewSetLoglevelHandler(context.Background(), logLevelSetter))

Development

Running Tests

make test

Code Generation (Mocks)

make generate

Full Development Workflow

make precommit  # Format, test, lint, and check

Testing Framework

The library uses:

• Ginkgo v2 for BDD-style testing
• Gomega for assertions
• Counterfeiter for mock generation

License

This project is licensed under the BSD 3-Clause License - see the LICENSE file for details.

Contributing

1. Fork the repository
2. Create your feature branch
3. Add tests for your changes
4. Run make precommit to ensure code quality
5. Submit a pull request

Dependencies

• glog - Core logging functionality
• gorilla/mux - HTTP routing for log level endpoints
• github.com/bborbe/time - Time utilities

Documentation

Index

• func NewSetLoglevelHandler(ctx context.Context, logLevelSetter LogLevelSetter) http.Handler
• type LogLevelSetter
  • func NewLogLevelSetter(defaultLoglevel glog.Level, autoResetDuration time.Duration) LogLevelSetter
• type LogLevelSetterFunc
  • func (l LogLevelSetterFunc) Set(ctx context.Context, logLevel glog.Level) error
• type MemoryMonitor
  • func NewMemoryMonitor(logInterval time.Duration) MemoryMonitor
• type Sampler
  • func NewSampleMod(mod uint64) Sampler
  • func NewSampleTime(duration stdtime.Duration) Sampler
  • func NewSamplerGlogLevel(level glog.Level) Sampler
  • func NewSamplerTrue() Sampler
• type SamplerFactory
• type SamplerFactoryFunc
  • func (s SamplerFactoryFunc) Sampler() Sampler
• type SamplerFunc
  • func (s SamplerFunc) IsSample() bool
• type SamplerList
  • func (s SamplerList) IsSample() bool

Functions

func NewSetLoglevelHandler

func NewSetLoglevelHandler(ctx context.Context, logLevelSetter LogLevelSetter) http.Handler

NewSetLoglevelHandler creates an HTTP handler for dynamically changing log levels via REST API. The handler expects a URL path variable named "level" containing the desired log level.

Usage with gorilla/mux:

router := mux.NewRouter()
logLevelSetter := log.NewLogLevelSetter(glog.Level(1), 5*time.Minute)
router.Handle("/debug/loglevel/{level}", log.NewSetLoglevelHandler(ctx, logLevelSetter))

Example HTTP requests:

GET  /debug/loglevel/4  - Set log level to 4
POST /debug/loglevel/2  - Set log level to 2

Parameters:

• ctx: Context for the log level setter operations
• logLevelSetter: The LogLevelSetter implementation to use for changing levels

Returns an http.Handler that can be registered with any HTTP router.

Types

type LogLevelSetter

type LogLevelSetter interface {
	// Set changes the current log level to the specified value.
	// The implementation may automatically reset to a default level after a timeout.
	Set(ctx context.Context, logLevel glog.Level) error
}

LogLevelSetter provides an interface for dynamically changing log levels at runtime. This is particularly useful for debugging production systems without restarts.

func NewLogLevelSetter

func NewLogLevelSetter(
	defaultLoglevel glog.Level,
	autoResetDuration time.Duration,
) LogLevelSetter

NewLogLevelSetter creates a new LogLevelSetter that automatically resets to the default log level after the specified duration.

Parameters:

• defaultLoglevel: The log level to reset to after the auto-reset duration
• autoResetDuration: How long to wait before automatically resetting the log level

The setter is thread-safe and can handle concurrent log level changes.

type LogLevelSetterFunc

type LogLevelSetterFunc func(ctx context.Context, logLevel glog.Level) error

LogLevelSetterFunc is a function type that implements the LogLevelSetter interface. It allows regular functions to be used as log level setters.

func (LogLevelSetterFunc) Set

func (l LogLevelSetterFunc) Set(ctx context.Context, logLevel glog.Level) error

Set implements the LogLevelSetter interface by calling the underlying function.

type MemoryMonitor

type MemoryMonitor interface {
	LogMemoryUsage(name string)
	LogMemoryUsageOnStart()
	LogMemoryUsageOnEnd()
}

MemoryMonitor provides memory usage monitoring functionality

func NewMemoryMonitor

func NewMemoryMonitor(logInterval time.Duration) MemoryMonitor

NewMemoryMonitor creates a new memory monitor that logs memory usage at specified intervals

type Sampler

type Sampler interface {
	// IsSample returns true if the current log entry should be emitted.
	// Implementations may use various strategies such as counters, time intervals,
	// or log levels to determine sampling behavior.
	IsSample() bool
}

Sampler defines an interface for log sampling decisions. It allows reducing log volume by selectively determining which log entries should be emitted.

Example usage:

sampler := log.NewSampleMod(10)
if sampler.IsSample() {
    glog.V(2).Infof("This message is sampled")
}

func NewSampleMod

func NewSampleMod(mod uint64) Sampler

NewSampleMod creates a counter-based sampler that samples every Nth log entry. It maintains an internal counter that increments on each IsSample() call, returning true when the counter is divisible by the modulus value.

Example:

sampler := log.NewSampleMod(10) // Sample every 10th log entry
for i := 0; i < 100; i++ {
    if sampler.IsSample() {
        glog.V(2).Infof("Log entry %d", i) // This will be called 10 times
    }
}

Parameters:

• mod: The modulus value for sampling (must be > 0). Every mod-th call will return true.

The sampler is thread-safe and can be used concurrently from multiple goroutines.

func NewSampleTime

func NewSampleTime(duration stdtime.Duration) Sampler

NewSampleTime creates a time-based sampler that limits log sampling to a maximum frequency. It ensures that IsSample() returns true at most once per the specified duration.

Example:

sampler := log.NewSampleTime(5 * time.Second) // Sample at most once every 5 seconds
for {
    if sampler.IsSample() {
        glog.V(2).Infof("This message appears at most once every 5 seconds")
    }
    time.Sleep(100 * time.Millisecond)
}

Parameters:

• duration: The minimum time interval between samples. Must be > 0.

The sampler is thread-safe and can be used concurrently from multiple goroutines. It uses github.com/bborbe/time for consistent time handling across the library.

func NewSamplerGlogLevel

func NewSamplerGlogLevel(level glog.Level) Sampler

NewSamplerGlogLevel creates a sampler that samples based on the current glog verbosity level. It returns true when the current glog verbosity is at or above the specified level. This allows for dynamic sampling based on the runtime log level configuration.

Example:

// Sample when glog verbosity is 3 or higher
sampler := log.NewSamplerGlogLevel(3)
if sampler.IsSample() {
    glog.V(2).Infof("This logs when -v=3 or higher is set")
}

// Combine with other samplers for conditional high-verbosity logging
conditionalSampler := log.SamplerList{
    log.NewSamplerGlogLevel(4),           // Always sample at high verbosity
    log.NewSampleTime(30 * time.Second),  // Otherwise, sample every 30 seconds
}

Parameters:

• level: The minimum glog verbosity level required for sampling

This sampler has no internal state and queries glog's current verbosity setting on each call, making it inherently thread-safe and responsive to runtime changes.

func NewSamplerTrue

func NewSamplerTrue() Sampler

NewSamplerTrue creates a sampler that always returns true. This is useful for testing, debugging, or situations where you want to temporarily disable sampling and log everything.

Example:

// For debugging, log everything regardless of other sampling rules
debugSampler := log.NewSamplerTrue()
if debugSampler.IsSample() {
    glog.V(4).Infof("Debug message - always logged")
}

// Use in tests to ensure all log paths are exercised
testSampler := log.NewSamplerTrue()

This sampler has no internal state and is inherently thread-safe.

type SamplerFactory

type SamplerFactory interface {
	// Sampler creates and returns a new Sampler instance.
	Sampler() Sampler
}

SamplerFactory provides a factory pattern for creating Sampler instances. This interface enables dependency injection and makes testing easier.

For testing, use log.DefaultSamplerFactory instead of mocking this interface.

var DefaultSamplerFactory SamplerFactory = SamplerFactoryFunc(func() Sampler {
	return SamplerList{
		NewSampleTime(10 * time.Second),
		NewSamplerGlogLevel(4),
	}
})

DefaultSamplerFactory provides a default sampler configuration that combines time-based sampling (every 10 seconds) with glog level sampling (level 4+).

type SamplerFactoryFunc

type SamplerFactoryFunc func() Sampler

SamplerFactoryFunc is a function type that implements the SamplerFactory interface. It allows regular functions to be used as sampler factories.

func (SamplerFactoryFunc) Sampler

func (s SamplerFactoryFunc) Sampler() Sampler

Sampler implements the SamplerFactory interface by calling the underlying function.

type SamplerFunc

type SamplerFunc func() bool

SamplerFunc is a function type that implements the Sampler interface. It allows regular functions to be used as samplers, enabling custom sampling logic.

Example:

// Custom sampler that samples based on random chance
randomSampler := log.SamplerFunc(func() bool {
    return rand.Float64() < 0.1 // 10% chance
})

// Custom sampler with external state
counter := 0
customSampler := log.SamplerFunc(func() bool {
    counter++
    return counter%3 == 0 // Every 3rd call
})

func (SamplerFunc) IsSample

func (s SamplerFunc) IsSample() bool

IsSample implements the Sampler interface by calling the underlying function.

type SamplerList

type SamplerList []Sampler

SamplerList combines multiple samplers using OR logic. It returns true if ANY of the contained samplers returns true. This allows for complex sampling strategies by combining different sampling methods.

Example:

// Sample if either time-based OR mod-based condition is met
sampler := log.SamplerList{
    log.NewSampleTime(30 * time.Second),  // At most once per 30 seconds
    log.NewSampleMod(100),                // Every 100th call
}

// Sample if high log level OR time interval OR random chance
complexSampler := log.SamplerList{
    log.NewSamplerGlogLevel(4),           // When glog level >= 4
    log.NewSampleTime(10 * time.Second),  // At most once per 10 seconds
    log.SamplerFunc(func() bool {         // 5% random chance
        return rand.Float64() < 0.05
    }),
}

The samplers are evaluated in order and the first one to return true causes the entire list to return true (short-circuit evaluation).

func (SamplerList) IsSample

func (s SamplerList) IsSample() bool

IsSample implements the Sampler interface using OR logic across all contained samplers. It returns true if any sampler in the list returns true.