rotatelogs

README

file-rotatelogs

Provide an io.Writer that periodically rotates log files from within the application. Port of File::RotateLogs from Perl to Go.

WARNINGS

THIS PROJECT HAS BEEN ARCHIVED. IT WILL NOT RECEIVE UPDATES, THE AUTHOR DOES NOT WISH TO MAINTAIN OR SUPPORT IT. IN SHORT, DO NOT USE THIS PROJECT.

SYNOPSIS

import (
  "log"
  "net/http"

  apachelog "github.com/khan-lau/apache-logformat"
  rotatelogs "github.com/khan-lau/file-rotatelogs"
)

func main() {
  mux := http.NewServeMux()
  mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { ... })

  logf, err := rotatelogs.New(
    "/path/to/access_log.%Y%m%d%H%M",
    rotatelogs.WithLinkName("/path/to/access_log"),
    rotatelogs.WithMaxAge(24 * time.Hour),
    rotatelogs.WithRotationTime(time.Hour),
  )
  if err != nil {
    log.Printf("failed to create rotatelogs: %s", err)
    return
  }

  // Now you must write to logf. apache-logformat library can create
  // a http.Handler that only writes the approriate logs for the request
  // to the given handle
  http.ListenAndServe(":8080", apachelog.CombinedLog.Wrap(mux, logf))
}

DESCRIPTION

When you integrate this to into your app, it automatically write to logs that are rotated from within the app: No more disk-full alerts because you forgot to setup logrotate!

To install, simply issue a go get:

go get github.com/khan-lau/file-rotatelogs

It's normally expected that this library is used with some other logging service, such as the built-in log library, or loggers such as github.com/lestrrat-go/apache-logformat.

import(
  "log"
  "github.com/khan-lau/file-rotatelogs"
)

func main() {
  rl, _ := rotatelogs.New("/path/to/access_log.%Y%m%d%H%M")

  log.SetOutput(rl)

  /* elsewhere ... */
  log.Printf("Hello, World!")
}

OPTIONS

Pattern (Required)

The pattern used to generate actual log file names. You should use patterns using the strftime (3) format. For example:

  rotatelogs.New("/var/log/myapp/log.%Y%m%d")

Clock (default: rotatelogs.Local)

You may specify an object that implements the roatatelogs.Clock interface. When this option is supplied, it's used to determine the current time to base all of the calculations on. For example, if you want to base your calculations in UTC, you may specify rotatelogs.UTC

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithClock(rotatelogs.UTC),
  )

Location

This is an alternative to the WithClock option. Instead of providing an explicit clock, you can provide a location for you times. We will create a Clock object that produces times in your specified location, and configure the rotatelog to respect it.

LinkName (default: "")

Path where a symlink for the actual log file is placed. This allows you to always check at the same location for log files even if the logs were rotated

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithLinkName("/var/log/myapp/current"),
  )

  // Else where
  $ tail -f /var/log/myapp/current

Links that share the same parent directory with the main log path will get a special treatment: namely, linked paths will be RELATIVE to the main log file.

Main log file name	Link name	Linked path
/path/to/log.%Y%m%d	/path/to/log	log.YYYYMMDD
/path/to/log.%Y%m%d	/path/to/nested/log	../log.YYYYMMDD
/path/to/log.%Y%m%d	/foo/bar/baz/log	/path/to/log.YYYYMMDD

If not provided, no link will be written.

RotationTime (default: 86400 sec)

Interval between file rotation. By default logs are rotated every 86400 seconds. Note: Remember to use time.Duration values.

  // Rotate every hour
  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithRotationTime(time.Hour),
  )

MaxAge (default: 7 days)

Time to wait until old logs are purged. By default no logs are purged, which certainly isn't what you want. Note: Remember to use time.Duration values.

  // Purge logs older than 1 hour
  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithMaxAge(time.Hour),
  )

RotationCount (default: -1)

The number of files should be kept. By default, this option is disabled.

Note: MaxAge should be disabled by specifing WithMaxAge(-1) explicitly.

  // Purge logs except latest 7 files
  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithMaxAge(-1),
    rotatelogs.WithRotationCount(7),
  )

Handler (default: nil)

Sets the event handler to receive event notifications from the RotateLogs object. Currently only supported event type is FiledRotated

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithHandler(rotatelogs.HandlerFunc(func(e rotatelogs.Event) {
      if e.Type() != rotatelogs.FileRotatedEventType {
        return
      }

      // Do what you want with the data. This is just an idea:
      storeLogFileToRemoteStorage(e.(*rotatelogs.FileRotatedEvent).PreviousFile())
    })),
  )

ForceNewFile

Ensure a new file is created every time New() is called. If the base file name already exists, an implicit rotation is performed.

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.ForceNewFile(),
  )

Rotating files forcefully

If you want to rotate files forcefully before the actual rotation time has reached, you may use the Rotate() method. This method forcefully rotates the logs, but if the generated file name clashes, then a numeric suffix is added so that the new file will forcefully appear on disk.

For example, suppose you had a pattern of '%Y.log' with a rotation time of 86400 so that it only gets rotated every year, but for whatever reason you wanted to rotate the logs now, you could install a signal handler to trigger this rotation:

rl := rotatelogs.New(...)

signal.Notify(ch, syscall.SIGHUP)

go func(ch chan os.Signal) {
  <-ch
  rl.Rotate()
}()

And you will get a log file name in like 2018.log.1, 2018.log.2, etc.

Documentation

Overview

package rotatelogs is a port of File-RotateLogs from Perl (https://metacpan.org/release/File-RotateLogs), and it allows you to automatically rotate output files when you write to them according to the filename pattern that you can specify.

Index

• Variables
• type AgingFunc
• type Clock
• type Event
• type EventType
• type FileRotatedEvent
  • func (e *FileRotatedEvent) CurrentFile() string
  • func (e *FileRotatedEvent) PreviousFile() string
  • func (e *FileRotatedEvent) Type() EventType
• type Handler
• type HandlerFunc
  • func (h HandlerFunc) Handle(e Event)
• type LogFileInfo
• type NamingFunc
• type Option
  • func ForceNewFile() Option
  • func WithAgingFunc(f AgingFunc) Option
  • func WithClock(c Clock) Option
  • func WithHandler(h Handler) Option
  • func WithLinkName(s string) Option
  • func WithLocation(loc *time.Location) Option
  • func WithMaxAge(d time.Duration) Option
  • func WithNamingFunc(f NamingFunc) Option
  • func WithRotationCount(n uint) Option
  • func WithRotationSize(s int64) Option
  • func WithRotationTime(d time.Duration) Option
• type RotateLogs
  • func New(p string, options ...Option) (*RotateLogs, error)
  • func (rl *RotateLogs) Close() error
  • func (rl *RotateLogs) CurrentFileName() string
  • func (rl *RotateLogs) Rotate() error
  • func (rl *RotateLogs) Write(p []byte) (n int, err error)

Examples

• ForceNewFile

Variables

var Local = clockFn(time.Now)

Local is an object satisfying the Clock interface, which returns the current time in the local timezone

var UTC = clockFn(func() time.Time { return time.Now().UTC() })

UTC is an object satisfying the Clock interface, which returns the current time in UTC

Types

type AgingFunc

type AgingFunc func(files []LogFileInfo) []string

AgingFunc 自定义日志老化清理回调，决定哪些旧日志文件应被删除

• files: 所有匹配的日志文件（已过滤 _lock/_symlink）

返回值为要删除的文件路径列表 设置后将完全替代内置的 maxAge / rotationCount 清理逻辑

type Clock

type Clock interface {
	Now() time.Time
}

Clock is the interface used by the RotateLogs object to determine the current time

type Event

type Event interface {
	Type() EventType
}

type EventType

type EventType int

const (
	InvalidEventType EventType = iota
	FileRotatedEventType
)

type FileRotatedEvent

type FileRotatedEvent struct {
	// contains filtered or unexported fields
}

func (*FileRotatedEvent) CurrentFile

func (e *FileRotatedEvent) CurrentFile() string

func (*FileRotatedEvent) PreviousFile

func (e *FileRotatedEvent) PreviousFile() string

func (*FileRotatedEvent) Type

func (e *FileRotatedEvent) Type() EventType

type Handler

type Handler interface {
	Handle(Event)
}

type HandlerFunc

type HandlerFunc func(Event)

func (HandlerFunc) Handle

func (h HandlerFunc) Handle(e Event)

type LogFileInfo

type LogFileInfo struct {
	Path     string      // 文件完整路径
	FileInfo os.FileInfo // 文件元信息（ModTime、Size 等）
}

LogFileInfo 记录匹配到的日志文件路径及其文件信息， 供 AgingFunc 自定义老化清理逻辑使用

type NamingFunc

type NamingFunc func(baseFilename string, generation int) string

NamingFunc 自定义滚动日志文件命名规则

• baseFilename: strftime 模式生成的基础文件名（如 "/var/log/app.log"）
• generation: 代际编号，从 1 开始递增

返回值为该代际实际使用的完整文件名 若不设置，默认规则为 "文件名.代际.扩展名"（如 app.1.log）

type Option

type Option interface {
	Name() string
	Value() interface{}
}

Option is used to pass optional arguments to the RotateLogs constructor

func ForceNewFile

func ForceNewFile() Option

ForceNewFile ensures a new file is created every time New() is called. If the base file name already exists, an implicit rotation is performed

Example

package main

import (
	"fmt"
	"os"

	rotatelogs "github.com/khan-lau/file-rotatelogs"
)

func main() {
	logDir, err := os.MkdirTemp("", "rotatelogs_test")
	if err != nil {
		fmt.Println("could not create log directory ", err)

		return
	}
	logPath := fmt.Sprintf("%s/test.log", logDir)

	for i := 0; i < 2; i++ {
		writer, innerErr := rotatelogs.New(logPath, rotatelogs.ForceNewFile())
		if innerErr != nil {
			fmt.Println("Could not open log file ", innerErr)

			return
		}

		n, writeErr := writer.Write([]byte("test"))
		if writeErr != nil || n != 4 {
			fmt.Println("Write failed ", writeErr, " number written ", n)

			return
		}
		err = writer.Close()
		if err != nil {
			fmt.Println("Close failed ", err)

			return
		}
	}

	files, err := os.ReadDir(logDir)
	if err != nil {
		fmt.Println("ReadDir failed ", err)

		return
	}
	for _, file := range files {
		info, infoErr := file.Info()
		if infoErr != nil {
			fmt.Println("Info failed ", infoErr)
			continue
		}
		fmt.Println(file.Name(), info.Size())
	}

	err = os.RemoveAll(logDir)
	if err != nil {
		fmt.Println("RemoveAll failed ", err)

		return
	}
}

Output:
test.1.log 4
test.log 4

func WithAgingFunc

func WithAgingFunc(f AgingFunc) Option

WithAgingFunc 设置自定义日志老化清理回调，决定哪些旧日志文件应被删除。 设置后将完全替代内置的 maxAge / rotationCount 清理逻辑。

回调接收所有匹配的日志文件（含 os.FileInfo）， 返回要删除的文件路径列表。

func WithClock

func WithClock(c Clock) Option

WithClock creates a new Option that sets a clock that the RotateLogs object will use to determine the current time.

By default rotatelogs.Local, which returns the current time in the local time zone, is used. If you would rather use UTC, use rotatelogs.UTC as the argument to this option, and pass it to the constructor.

func WithHandler

func WithHandler(h Handler) Option

WithHandler creates a new Option that specifies the Handler object that gets invoked when an event occurs. Currently `FileRotated` event is supported

func WithLinkName

func WithLinkName(s string) Option

WithLinkName creates a new Option that sets the symbolic link name that gets linked to the current file name being used.

func WithLocation

func WithLocation(loc *time.Location) Option

WithLocation creates a new Option that sets up a "Clock" interface that the RotateLogs object will use to determine the current time.

This optin works by always returning the in the given location.

func WithMaxAge

func WithMaxAge(d time.Duration) Option

WithMaxAge creates a new Option that sets the max age of a log file before it gets purged from the file system.

func WithNamingFunc

func WithNamingFunc(f NamingFunc) Option

WithNamingFunc 设置自定义滚动日志文件命名规则。 当需要滚动日志且新文件名与已有文件冲突时，回调此函数。 参数为 strftime 模式生成的基础文件名和代际编号，返回代际文件名。

不设置时默认规则为："文件名.代际编号.扩展名"（如 app.log → app.1.log）。

func WithRotationCount

func WithRotationCount(n uint) Option

WithRotationCount creates a new Option that sets the number of files should be kept before it gets purged from the file system.

func WithRotationSize

func WithRotationSize(s int64) Option

WithRotationSize creates a new Option that sets the log file size between rotation.

func WithRotationTime

func WithRotationTime(d time.Duration) Option

WithRotationTime creates a new Option that sets the time between rotation.

type RotateLogs

type RotateLogs struct {
	// contains filtered or unexported fields
}

RotateLogs represents a log file that gets automatically rotated as you write to it.

func New

func New(p string, options ...Option) (*RotateLogs, error)

New creates a new RotateLogs object. A log filename pattern must be passed. Optional `Option` parameters may be passed

func (*RotateLogs) Close

func (rl *RotateLogs) Close() error

Close satisfies the io.Closer interface. You must call this method if you performed any writes to the object.

func (*RotateLogs) CurrentFileName

func (rl *RotateLogs) CurrentFileName() string

CurrentFileName returns the current file name that the RotateLogs object is writing to

func (*RotateLogs) Rotate

func (rl *RotateLogs) Rotate() error

Rotate forcefully rotates the log files. If the generated file name clash because file already exists, a numeric suffix of the form ".1", ".2", ".3" and so forth are appended to the end of the log file

Thie method can be used in conjunction with a signal handler so to emulate servers that generate new log files when they receive a SIGHUP

func (*RotateLogs) Write

func (rl *RotateLogs) Write(p []byte) (n int, err error)

Write satisfies the io.Writer interface. It writes to the appropriate file handle that is currently being used. If we have reached rotation time, the target file gets automatically rotated, and also purged if necessary.