rotatefile

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 21, 2026 License: MIT Imports: 19 Imported by: 0

README

Rotate File

GoDoc Go Report Card Unit-Tests GitHub tag

rotatefile is a lightweight Go library for log file rotation, cleanup and gzip compression.

rotatefile.Writer is a plain io.Writer, so it drops into the standard library log/slog, the standard log, zap, gookit/slog — any logger that writes to an io.Writer. The Go standard library has no built-in log rotation; this fills that gap.

中文说明请看 README.zh-CN.md

Features

  • Rotate by size and/or time (every hour / day / 30min / minute …)
  • Two rotate modes: rename (write to one file, rename on rotate) and create (write to a new dated file each period)
  • Cleanup old files by BackupNum (max count) and/or BackupTime (max age)
  • Compress rotated files with gzip
  • Customizable: filename for size-rotation, time clock, file permission
  • FilesClear — a standalone old-files cleaner, usable for any program's logs (even non-Go ones, e.g. PHP-FPM)
  • filecleaner CLI — a JSON-configured command-line cleaner built on FilesClear
  • Sub-package bufwrite — buffered writers, incl. LineWriter that keeps every write (one log line) intact
  • Tiny dependency surface: only github.com/gookit/goutil

Install

go get github.com/gookit/rotatefile

Quick Start

Create a rotating writer
package main

import "github.com/gookit/rotatefile"

func main() {
	w, err := rotatefile.NewConfig("testdata/app.log").Create()
	if err != nil {
		panic(err)
	}
	defer w.Close() // flush + close

	_, _ = w.Write([]byte("a log message\n"))
}
Common config options
w, err := rotatefile.NewConfig("testdata/app.log", func(c *rotatefile.Config) {
	c.MaxSize = 100 * rotatefile.OneMByte // rotate at 100MB (0 = disable size rotate)
	c.RotateTime = rotatefile.EveryDay    // also rotate daily (0 = disable time rotate)
	c.RotateMode = rotatefile.ModeRename  // or rotatefile.ModeCreate
	c.BackupNum = 30                      // keep at most 30 old files
	c.BackupTime = 24 * 7                 // and/or keep files up to a week (hours)
	c.Compress = true                     // gzip rotated files
}).Create()

See Config on GoDoc for the full list.

Use with the standard log/slog (Go 1.21+)
import (
	"log/slog"

	"github.com/gookit/rotatefile"
)

w, _ := rotatefile.NewConfig("testdata/app.log", func(c *rotatefile.Config) {
	c.MaxSize = 50 * rotatefile.OneMByte
	c.RotateTime = rotatefile.EveryDay
	c.BackupNum = 7
}).Create()

logger := slog.New(slog.NewJSONHandler(w, nil))
logger.Info("log via std slog", "key", "value")
Use with the standard log (or zap, etc.)
import (
	"log"

	"github.com/gookit/rotatefile"
)

w, _ := rotatefile.NewConfig("testdata/app.log").Create()
log.SetOutput(w)
log.Println("log message")

Any logger that accepts an io.Writer works the same way (e.g. zap via zapcore.AddSync(w)).

Buffered writing (bufwrite)
import (
	"github.com/gookit/rotatefile"
	"github.com/gookit/rotatefile/bufwrite"
)

w, _ := rotatefile.NewConfig("testdata/app.log").Create()

// LineWriter keeps each Write (one log line) intact - it won't split a record
// across a flush, so an external collector never reads a half-written line.
bw := bufwrite.NewLineWriter(w)
defer bw.Close() // flush + close

_, _ = bw.Write([]byte("a complete log line\n"))

Clean old files (FilesClear)

FilesClear cleans old/expired files by pattern, independent of the rotating writer. It can also run as a background daemon.

fc := rotatefile.NewFilesClear(func(c *rotatefile.CConfig) {
	c.AddPattern("/path/to/some*.log")
	c.BackupNum = 2
	c.BackupTime = 12 // 12 hours
})

// one-off clean
_ = fc.Clean()

// or run on a daemon
go fc.DaemonClean(nil)
// NOTE: stop the daemon before exit
// fc.StopDaemon()

See CConfig on GoDoc for clean options.

filecleaner CLI

cmd/filecleaner is a small command-line tool built on FilesClear. It cleans old/expired files by patterns, configured via a JSON file — handy for cron jobs or cleaning logs of non-Go programs.

go install github.com/gookit/rotatefile/cmd/filecleaner@latest

filecleaner -c filecleaner.json            # one-off clean
filecleaner --dry-run -c filecleaner.json  # print what would be removed, delete nothing
filecleaner --daemon  -c filecleaner.json  # run periodically until Ctrl+C

Config file — a jobs array, one retention policy per job:

{
  "jobs": [
    { "patterns": ["/var/log/app/*.log.*"], "backup_num": 20, "backup_time": 168, "time_unit": "1h" },
    { "patterns": ["/var/log/svc"], "recursive": true, "remove_empty_dir": true, "backup_time": 7, "time_unit": "24h" }
  ]
}

See cmd/filecleaner/README.md for all options.

License

MIT

Documentation

Overview

Package rotatefile provides simple file rotation, compression and cleanup.

Index

Examples

Constants

View Source 
const (
	// OneMByte size
	OneMByte uint64 = 1024 * 1024

	// DefaultMaxSize of a log file. default is 20M.
	DefaultMaxSize = 20 * OneMByte
	// DefaultBackNum default backup numbers for old files.
	DefaultBackNum uint = 20
	// DefaultBackTime default backup time for old files. default keeps a week.
	DefaultBackTime uint = 24 * 7
)

Variables

View Source 
var (
	// DefaultFilePerm perm and flags for create log file
	DefaultFilePerm os.FileMode = 0664
	// DefaultFileFlags for open log file
	DefaultFileFlags = os.O_CREATE | os.O_WRONLY | os.O_APPEND

	// DefaultTimeClockFn for create time
	DefaultTimeClockFn = ClockFn(func() time.Time {
		return time.Now()
	})
)

Functions

func WithCompress 

func WithCompress(c *Config)

WithCompress setting for compress

func WithDebugMode 

func WithDebugMode(c *Config)

WithDebugMode setting for debug mode

Types

type CConfig 

type CConfig struct {
	// BackupNum max number for keep old files.
	//
	// 0 is not limit, default is 20.
	BackupNum uint `json:"backup_num" yaml:"backup_num"`

	// BackupTime max time for keep old files, unit is TimeUnit.
	//
	// 0 is not limit, default is a week.
	BackupTime uint `json:"backup_time" yaml:"backup_time"`

	// Compress determines if the rotated log files should be compressed using gzip.
	// The default is not to perform compression.
	Compress bool `json:"compress" yaml:"compress"`

	// Patterns dir path with filename match patterns.
	//
	// eg: ["/tmp/error.log.*", "/path/to/info.log.*", "/path/to/dir/*"]
	Patterns []string `json:"patterns" yaml:"patterns"`

	// Recursive clean files in matched subdirectories too. default is false.
	//
	// NOTE: when enabled, BackupNum/BackupTime apply to all files collected
	// per pattern (including nested ones) as a single pool.
	Recursive bool `json:"recursive" yaml:"recursive"`

	// RemoveEmptyDir remove subdirectories that become empty after cleaning.
	// only takes effect together with Recursive. default is false.
	RemoveEmptyDir bool `json:"remove_empty_dir" yaml:"remove_empty_dir"`

	// TimeClock for clean files
	TimeClock Clocker

	// TimeUnit for BackupTime. default is hours: time.Hour
	TimeUnit time.Duration `json:"time_unit" yaml:"time_unit"`

	// CheckInterval for clean files on daemon run. default is 60s.
	CheckInterval time.Duration `json:"check_interval" yaml:"check_interval"`

	// IgnoreError ignore remove file error, continue to clean other files.
	IgnoreError bool `json:"ignore_error" yaml:"ignore_error"`

	// DryRun only print the files to be removed, do not actually remove them.
	DryRun bool `json:"dry_run" yaml:"dry_run"`
}

CConfig struct for clean files

func NewCConfig 

func NewCConfig() *CConfig

NewCConfig instance

func (*CConfig) AddDirPath 

func (c *CConfig) AddDirPath(dirPaths ...string) *CConfig

AddDirPath for clean, will auto append * for match all files

func (*CConfig) AddPattern 

func (c *CConfig) AddPattern(patterns ...string) *CConfig

AddPattern for clean. eg: "/tmp/error.log.*"

func (*CConfig) WithConfigFn 

func (c *CConfig) WithConfigFn(fns ...CConfigFunc) *CConfig

WithConfigFn for custom settings

type CConfigFunc 

type CConfigFunc func(c *CConfig)

CConfigFunc for clean config

type ClockFn 

type ClockFn func() time.Time

ClockFn func

func (ClockFn) Now 

func (fn ClockFn) Now() time.Time

Now implements the Clocker

type Clocker 

type Clocker interface {
	Now() time.Time
}

Clocker is the interface used for determine the current time

type Config 

type Config struct {
	// Filepath the log file path, will be rotating. eg: "logs/error.log"
	Filepath string `json:"filepath" yaml:"filepath"`

	// FilePerm for create log file. default DefaultFilePerm
	FilePerm os.FileMode `json:"file_perm" yaml:"file_perm"`

	// RotateMode for rotate file. default ModeRename
	RotateMode RotateMode `json:"rotate_mode" yaml:"rotate_mode"`

	// MaxSize file contents max size, unit is bytes.
	// If is equals zero, disable rotate file by size
	//
	// default see DefaultMaxSize
	MaxSize uint64 `json:"max_size" yaml:"max_size"`

	// RotateTime the file rotating interval time, unit is seconds.
	// If is equals zero, disable rotate file by time
	//
	// default: EveryHour
	RotateTime RotateTime `json:"rotate_time" yaml:"rotate_time"`

	// CloseLock use sync lock on writing contents, rotating file.
	//
	// default: false
	CloseLock bool `json:"close_lock" yaml:"close_lock"`

	// BackupNum max number for keep old files.
	//
	// 0 is not limit, default is DefaultBackNum
	BackupNum uint `json:"backup_num" yaml:"backup_num"`

	// BackupTime max time for keep old files, unit is hours.
	//
	// 0 is not limit, default is DefaultBackTime
	BackupTime uint `json:"backup_time" yaml:"backup_time"`

	// CleanOnClose determines if the rotated log files should be cleaned up when close.
	CleanOnClose bool `json:"clean_on_close" yaml:"clean_on_close"`

	// Compress determines if the rotated log files should be compressed using gzip.
	// The default is not to perform compression.
	Compress bool `json:"compress" yaml:"compress"`

	// RenameFunc you can custom-build filename for rotate file by size.
	//
	// Example:
	//
	//  c.RenameFunc = func(filepath string, rotateNum uint) string {
	// 		suffix := time.Now().Format("06010215")
	//
	// 		// eg: /tmp/error.log => /tmp/error.log.24032116_894136
	// 		return filepath + fmt.Sprintf(".%s_%d", suffix, rotateNum)
	//  }
	RenameFunc func(filePath string, rotateNum uint) string `json:"-" yaml:"-"`

	// TimeClock for a rotating file by time.
	TimeClock Clocker `json:"-" yaml:"-"`

	// DebugMode for debug on development.
	DebugMode bool `json:"debug_mode" yaml:"debug_mode"`
}

Config struct for rotate dispatcher

func EmptyConfigWith 

func EmptyConfigWith(fns ...ConfigFn) *Config

EmptyConfigWith new empty config with custom func

func NewConfig 

func NewConfig(filePath string, fns ...ConfigFn) *Config

NewConfig by file path, and can with custom setting

func NewConfigWith 

func NewConfigWith(fns ...ConfigFn) *Config

NewConfigWith custom func

func NewDefaultConfig 

func NewDefaultConfig() *Config

NewDefaultConfig instance

func (*Config) Create 

func (c *Config) Create() (*Writer, error)

Create new Writer by config

func (*Config) IsMode 

func (c *Config) IsMode(m RotateMode) bool

IsMode check rotate mode

func (*Config) With 

func (c *Config) With(fns ...ConfigFn) *Config

With more config setting func

type ConfigFn 

type ConfigFn func(c *Config)

ConfigFn for setting config

func WithBackupNum 

func WithBackupNum(num uint) ConfigFn

WithBackupNum setting for backup number

func WithFilepath 

func WithFilepath(logfile string) ConfigFn

WithFilepath setting

type FilesClear 

type FilesClear struct {
	// contains filtered or unexported fields
}

FilesClear multi files by time.

use for rotate and clear other program produce log files

func NewFilesClear 

func NewFilesClear(fns ...CConfigFunc) *FilesClear

NewFilesClear instance

func (*FilesClear) Clean 

func (r *FilesClear) Clean() error

Clean old files by config

func (*FilesClear) Config 

func (r *FilesClear) Config() *CConfig

Config get

func (*FilesClear) DaemonClean 

func (r *FilesClear) DaemonClean(onStop func())

DaemonClean daemon clean old files by config

NOTE: this method will block current goroutine

Usage: 

fc := rotatefile.NewFilesClear(nil)
fc.WithConfigFn(func(c *rotatefile.CConfig) {
	c.AddDirPath("./testdata")
})

wg := sync.WaitGroup{}
wg.Add(1)

// start daemon
go fc.DaemonClean(func() {
	wg.Done()
})

// wait for stop
wg.Wait()

func (*FilesClear) StopDaemon 

func (r *FilesClear) StopDaemon()

StopDaemon for stop daemon clean

func (*FilesClear) WithConfig 

func (r *FilesClear) WithConfig(cfg *CConfig) *FilesClear

WithConfig for custom set config

func (*FilesClear) WithConfigFn 

func (r *FilesClear) WithConfigFn(fns ...CConfigFunc) *FilesClear

WithConfigFn for custom settings

type MockClocker 

type MockClocker struct {
	// contains filtered or unexported fields
}

MockClocker mock clock for test.

NOTE: it is concurrency-safe, since the mocked time may be advanced by the test goroutine while a background goroutine (eg. async cleaner) reads it.

func NewMockClock 

func NewMockClock(datetime string) *MockClocker

NewMockClock create a mock time instance from datetime string.

func (*MockClocker) Add 

func (mt *MockClocker) Add(d time.Duration)

Add progresses time by the given duration.

func (*MockClocker) Datetime 

func (mt *MockClocker) Datetime() string

Datetime returns the current time in the format "2006-01-02 15:04:05".

func (*MockClocker) Now 

func (mt *MockClocker) Now() time.Time

Now get current time.

type RotateMode 

type RotateMode uint8

RotateMode for a rotated file. 0: rename, 1: create 

const (
	// ModeRename rotating file by rename.
	//
	// Example flow:
	//  - always write to "error.log"
	//  - rotating by rename it to "error.log.20201223"
	//  - then re-create "error.log"
	ModeRename RotateMode = iota

	// ModeCreate rotating file by create a new file.
	//
	// Example flow:
	//  - directly create a new file on each rotated time. eg: "error.log.20201223", "error.log.20201224"
	ModeCreate
)

func StringToRotateMode 

func StringToRotateMode(s string) (RotateMode, error)

StringToRotateMode convert string to RotateMode

func (RotateMode) MarshalJSON 

func (m RotateMode) MarshalJSON() ([]byte, error)

MarshalJSON implement the JSON Marshal interface encoding/json.Marshaler

func (RotateMode) String 

func (m RotateMode) String() string

String get string name

func (*RotateMode) UnmarshalJSON 

func (m *RotateMode) UnmarshalJSON(data []byte) error

UnmarshalJSON implement the JSON Unmarshal interface encoding/json.Unmarshaler

type RotateTime 

type RotateTime int

RotateTime for a rotating file. unit is seconds.

EveryDay:

  • "error.log.20201223"

EveryHour, Every30Min, EveryMinute:

  • "error.log.20201223_1500"
  • "error.log.20201223_1530"
  • "error.log.20201223_1523"
const (
	EveryMonth  RotateTime = 30 * timex.OneDaySec
	EveryDay    RotateTime = timex.OneDaySec
	EveryHour   RotateTime = timex.OneHourSec
	Every30Min  RotateTime = 30 * timex.OneMinSec
	Every15Min  RotateTime = 15 * timex.OneMinSec
	EveryMinute RotateTime = timex.OneMinSec
	EverySecond RotateTime = 1 // only use for tests
)

built in rotate time constants

func StringToRotateTime 

func StringToRotateTime(s string) (RotateTime, error)

StringToRotateTime parse and convert string to RotateTime

func (RotateTime) FirstCheckTime 

func (rt RotateTime) FirstCheckTime(now time.Time) time.Time

FirstCheckTime for a rotated file. - will automatically align the time from the start of each hour.

func (RotateTime) Interval 

func (rt RotateTime) Interval() int64

Interval get check interval time. unit is seconds.

func (RotateTime) MarshalJSON 

func (rt RotateTime) MarshalJSON() ([]byte, error)

MarshalJSON implement the JSON Marshal interface encoding/json.Marshaler

func (RotateTime) String 

func (rt RotateTime) String() string

String rotate type to string

func (RotateTime) TimeFormat 

func (rt RotateTime) TimeFormat() (suffixFormat string)

TimeFormat get log file suffix format

EveryDay:

  • "error.log.20201223"

EveryHour, Every30Min, EveryMinute:

  • "error.log.20201223_1500"
  • "error.log.20201223_1530"
  • "error.log.20201223_1523"

func (*RotateTime) UnmarshalJSON 

func (rt *RotateTime) UnmarshalJSON(data []byte) error

UnmarshalJSON implement the JSON Unmarshal interface encoding/json.Unmarshaler

type RotateWriter 

type RotateWriter interface {
	io.WriteCloser
	Clean() error
	Flush() error
	Rotate() error
	Sync() error
}

RotateWriter interface

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

Writer a flush, close, writer and support rotate file.

refer https://github.com/flike/golog/blob/master/filehandler.go

func NewWriter 

func NewWriter(c *Config) (*Writer, error)

NewWriter create rotate write with config and init it.

Example (On_other_logger) 
package main

import (
	"log"

	"github.com/gookit/rotatefile"
)

func main() {
	logFile := "testdata/another_logger.log"
	writer, err := rotatefile.NewConfig(logFile).Create()
	if err != nil {
		panic(err)
	}

	log.SetOutput(writer)
	log.Println("log message")
}

func NewWriterWith 

func NewWriterWith(fns ...ConfigFn) (*Writer, error)

NewWriterWith create a rotated writer with some settings.

func (*Writer) Clean 

func (d *Writer) Clean() (err error)

Clean old files by config

func (*Writer) Close 

func (d *Writer) Close() error

Close the writer. will sync data to disk, then close the file handle. and it will stop the async clean backups.

func (*Writer) Config 

func (d *Writer) Config() Config

Config gets the config

func (*Writer) Flush 

func (d *Writer) Flush() error

Flush sync data to disk. alias of Sync()

func (*Writer) MustClose 

func (d *Writer) MustClose()

MustClose the writer. alias of Close(), but will panic if has error.

func (*Writer) Rotate 

func (d *Writer) Rotate() error

Rotate the file by config and async clean backups

func (*Writer) Sync 

func (d *Writer) Sync() error

Sync data to disk.

func (*Writer) Write 

func (d *Writer) Write(p []byte) (n int, err error)

Write data to file. then check and do rotate file, then async clean backups

func (*Writer) WriteString 

func (d *Writer) WriteString(s string) (n int, err error)

WriteString implements the io.StringWriter

Source Files

View all Source files

Directories

Path Synopsis
Package bufwrite provides buffered io.Writer with sync and close methods.
 Package bufwrite provides buffered io.Writer with sync and close methods.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.