rollingwriter

README

RollingWriter

RollingWriter is an auto rotate io.Writer implementation. It always works with logger.

New Version v2.0 is comeing out! Much more Powerfull and Efficient. Try it by follow the demo

it contains 2 separate patrs:

• Manager: decide when to rotate the file with policy RlingPolicy give out the rolling policy

  1. WithoutRolling: no rolling will happen
  2. TimeRolling: rolling by time
  3. VolumeRolling: rolling by file size

• IOWriter: impement the io.Writer and do the io write

  • Writer: not parallel safe writer
  • LockedWriter: parallel safe garented by lock
  • AsyncWtiter: parallel safe async writer

Features

• Auto rotate
• Parallel safe
• Implement go io.Writer
• Time rotate with corn style task schedual
• Volume rotate
• Max remain rolling files with auto clean

Quick Start

	writer, err := rollingwriter.NewWriterFromConfig(&config)
	if err != nil {
		panic(err)
	}

	writer.Write([]byte("hello, world"))

for more, goto demo to see more details

Contribute && TODO

• the Buffered writer need to be dcescuessed
• fix the FIXME

Now I am about to release the v1.0.0-prerelease with redesigned interface

Any new feature inneed pls put up an issue

Documentation

Index

• Constants
• Variables
• func AsynchronousWriterErrorChan(wr RollingWriter) (chan error, error)
• func LogFilePath(c *Config) string
• type AsynchronousWriter
  • func (w *AsynchronousWriter) Close() error
  • func (w *AsynchronousWriter) Write(b []byte) (int, error)
• type BufferWriter
• type Config
  • func NewDefaultConfig() Config
• type LockedWriter
  • func (w *LockedWriter) Close() error
  • func (w *LockedWriter) Write(b []byte) (int, error)
• type Manager
  • func NewManager(c *Config) (Manager, error)
• type Option
  • func WithAsynchronous() Option
  • func WithCompress() Option
  • func WithFileName(name string) Option
  • func WithLock() Option
  • func WithLogPath(path string) Option
  • func WithMaxRemain(max int) Option
  • func WithRollingPolicy(policy int) Option
  • func WithRollingTimePattern(pattern string) Option
  • func WithRollingVolumeSize(size string) Option
  • func WithTimeTagFormat(format string) Option
• type RollingWriter
  • func NewWriter(ops ...Option) (RollingWriter, error)
  • func NewWriterFromConfig(c *Config) (RollingWriter, error)
  • func NewWriterFromConfigFile(path string) (RollingWriter, error)
• type Writer
  • func (w *Writer) AutoRemove()
  • func (w *Writer) Close() error
  • func (w *Writer) CompressFile(oldfile *os.File, cmpname string) error
  • func (w *Writer) Reopen(file string) error
  • func (w *Writer) Write(b []byte) (int, error)

Constants

const (
	WithoutRolling = iota
	TimeRolling
	VolumeRolling
)

RollingPolicies giveout 3 policy for rolling.

Variables

var (
	// BufferSize defined the buffer size, by default 1M buffer will be allocate
	BufferSize = 0x100000
	// QueueSize defined the queue size for asynchronize write
	QueueSize = 1024
	// Precision defined the precision about the reopen operation condition
	// check duration within second
	Precision = 1
	// DefaultFileMode set the default open mode
	DefaultFileMode = os.FileMode(0644)
	// DefaultFileFlag set the default file flag
	DefaultFileFlag = os.O_RDWR | os.O_CREATE | os.O_APPEND

	// ErrInternal defined the internal error
	ErrInternal = errors.New("error internal")
	// ErrClosed defined write while ctx close
	ErrClosed = errors.New("error write on close")
	// ErrInvalidArgument defined the invalid argument
	ErrInvalidArgument = errors.New("error argument invalid")
)

Functions

func AsynchronousWriterErrorChan

func AsynchronousWriterErrorChan(wr RollingWriter) (chan error, error)

AsynchronousWriterErrorChan return the error channel for asyn writer

func LogFilePath

func LogFilePath(c *Config) string

LogFilePath return the absolute path on log file

Types

type AsynchronousWriter

type AsynchronousWriter struct {
	Writer
	// contains filtered or unexported fields
}

AsynchronousWriter provide a asynchronous writer with the writer to confirm the write

func (*AsynchronousWriter) Close

func (w *AsynchronousWriter) Close() error

Close lock and close the file

func (*AsynchronousWriter) Write

func (w *AsynchronousWriter) Write(b []byte) (int, error)

Only when the error channel is empty, otherwise nothing will write and the last error will be return return the error channel

type BufferWriter

type BufferWriter struct {
	Writer
	// contains filtered or unexported fields
}

BufferWriter provide a parallel safe bufferd writer TODO TBD

type Config

type Config struct {
	// LogPath defined the full path of log file directory.
	// there comes out 2 different log file:
	//
	// 1. the current log
	//	log file path is located here:
	//	[LogPath]/[FileName].log
	//
	// 2. the tuncated log file
	//	the tuncated log file is backup here:
	//	[LogPath]/[FileName].log.[TimeTag]
	//  if compressed true
	//	[LogPath]/[FileName].log.gz.[TimeTag]
	//
	// NOTICE: blank field will be ignored
	// By default I use '-' as separator, you can set it yourself
	TimeTagFormat string `json:"time_tag_format"`
	LogPath       string `json:"log_path"`
	FileName      string `json:"file_name"`
	// MaxRemain will auto clear the roling file list, set 0 will disable auto clean
	MaxRemain int `json:"max_remain"`

	// RollingPolicy give out the rolling policy
	// We got 3 policies(actually, 2):
	//
	//	1. WithoutRolling: no rolling will happen
	//	2. TimeRolling: rolling by time
	//	3. VolumeRolling: rolling by file size
	RollingPolicy      int    `json:"rolling_ploicy"`
	RollingTimePattern string `json:"rolling_time_pattern"`
	RollingVolumeSize  string `json:"rolling_volume_size"`

	// Compress will compress log file with gzip
	Compress bool `json:"compress"`
	// Asynchronous enable the asynchronous write
	// by default the writer will be synchronous
	Asynchronous bool `json:"asynchronous"`
	// Lock enable the lock for writer, the writer will guarantee parallel safity
	// NOTICE: this will take effect only when writer is synchronous
	Lock bool `json:"lock"`
}

Config give out the config for manager

func NewDefaultConfig

func NewDefaultConfig() Config

NewDefaultConfig return the default config

type LockedWriter

type LockedWriter struct {
	Writer
	// contains filtered or unexported fields
}

LockedWriter provide a synchronous writer with lock write operate will be guaranteed by lock

func (*LockedWriter) Close

func (w *LockedWriter) Close() error

Close lock and close the file

func (*LockedWriter) Write

func (w *LockedWriter) Write(b []byte) (int, error)

type Manager

type Manager interface {
	// Fire will return a string channel
	// while the rolling event occoured, new file name will generate
	Fire() chan string
	// Close the Manager
	Close()
}

Manager used to trigger rolling event.

func NewManager

func NewManager(c *Config) (Manager, error)

NewManager generate the Manager with config

type Option

type Option func(*Config)

Option defined config option

func WithAsynchronous

func WithAsynchronous() Option

WithAsynchronous enable the asynchronous write for writer

func WithCompress

func WithCompress() Option

WithCompress will auto compress the tuncated log file with gzip

func WithFileName

func WithFileName(name string) Option

WithFileName set the log file name

func WithLock

func WithLock() Option

WithLock will enable the lock in writer Writer will call write with the Lock to guarantee the parallel safe

func WithLogPath

func WithLogPath(path string) Option

WithLogPath set the log dir and auto create dir tree if the dir/path is not exist

func WithMaxRemain

func WithMaxRemain(max int) Option

WithMaxRemain enable the auto deletion for old file when exceed the given max value Bydefault -1 will disable the auto deletion

func WithRollingPolicy

func WithRollingPolicy(policy int) Option

WithRollingPolicy set the rolling policy

func WithRollingTimePattern

func WithRollingTimePattern(pattern string) Option

WithRollingTimePattern set the time rolling policy time pattern obey the Corn table style visit http://crontab.org/ for details

func WithRollingVolumeSize

func WithRollingVolumeSize(size string) Option

WithRollingVolumeSize set the rolling file truncation threshold size

func WithTimeTagFormat

func WithTimeTagFormat(format string) Option

WithTimeTagFormat set the TimeTag format string

type RollingWriter

type RollingWriter interface {
	io.Writer
	Close() error
}

RollingWriter implement the io writer

func NewWriter

func NewWriter(ops ...Option) (RollingWriter, error)

NewWriter generate the rollingWriter with given option

func NewWriterFromConfig

func NewWriterFromConfig(c *Config) (RollingWriter, error)

NewWriterFromConfig generate the rollingWriter with given config

func NewWriterFromConfigFile

func NewWriterFromConfigFile(path string) (RollingWriter, error)

NewWriterFromConfigFile generate the rollingWriter with given config file

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer provide a synchronous file writer if Lock is set true, write will be guaranteed by lock

func (*Writer) AutoRemove

func (w *Writer) AutoRemove()

AutoRemove will delete the oldest file

func (*Writer) Close

func (w *Writer) Close() error

Close the file and return

func (*Writer) CompressFile

func (w *Writer) CompressFile(oldfile *os.File, cmpname string) error

CompressFile compress log file write into .gz and remove source file

func (*Writer) Reopen

func (w *Writer) Reopen(file string) error

Reopen do the rotate, open new file and swap FD then trate the old FD

func (*Writer) Write

func (w *Writer) Write(b []byte) (int, error)