rollingwriter

package module
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 14, 2024 License: MIT Imports: 17 Imported by: 0

README

RollingWriter Build Status Go Report Card GoDoc codecov Awesome

RollingWriter is an auto rotate io.Writer implementation. It can works well with logger.

Awesome Go popular log helper

New Version v2.0 is comming out! Much more Powerfull and Efficient. Try it by follow the demo

RollingWriter contains 2 separate patrs:

  • Manager: decide when to rotate the file with policy. RlingPolicy give out the rolling policy

    • WithoutRolling: no rolling will happen
    • TimeRolling: rolling by time
    • VolumeRolling: rolling by file size

  • IOWriter: impement the io.Writer and do the io write

    • Writer: not parallel safe writer
    • LockedWriter: parallel safe garented by lock
    • AsyncWtiter: parallel safe async writer
    • BufferWriter: merge serval write into one file.Write()

Features

  • Auto rotate with multi rotate policies
  • Implement go io.Writer, provide parallel safe writer
  • Max remain rolling files with auto cleanup
  • Easy for user to implement your manager

Benchmark

goos: darwin
goarch: amd64
pkg: github.com/arthurkiller/rollingWriter
BenchmarkWrite-4                          300000              5952 ns/op               0 B/op          0 allocs/op
BenchmarkParallelWrite-4                  200000              7846 ns/op               0 B/op          0 allocs/op
BenchmarkAsynWrite-4                      200000              7917 ns/op           16324 B/op          1 allocs/op
BenchmarkParallelAsynWrite-4              200000              8632 ns/op           12513 B/op          1 allocs/op
BenchmarkLockedWrite-4                    200000              5829 ns/op               0 B/op          0 allocs/op
BenchmarkParallelLockedWrite-4            200000              7796 ns/op               0 B/op          0 allocs/op
BenchmarkBufferWrite-4                    200000              6943 ns/op            1984 B/op          4 allocs/op
BenchmarkParallelBufferWrite-4           1000000              1026 ns/op            7129 B/op          1 allocs/op
PASS
ok      github.com/arthurkiller/rollingWriter   14.867s

Quick Start

	writer, err := rollingwriter.NewWriterFromConfig(&config)
	if err != nil {
		panic(err)
	}

	writer.Write([]byte("hello, world"))

Want more? View demo for more details.

Any suggestion or new feature inneed, please put up an issue

Documentation

Index

Constants

View Source 
const (
	WithoutRolling = iota
	TimeRolling
	VolumeRolling
)

RollingPolicies giveout 3 policy for rolling.

Variables

View Source 
var (
	// BufferSize defined the buffer size, by default 1 KB buffer will be allocated
	BufferSize = 1024
	// QueueSize defined the queue size for asynchronize write
	QueueSize = 1024
	// Precision defined the precision about the reopen operation condition
	// check duration within second
	Precision = 1
	// DefaultFileMode set the default open mode rw-r--r-- by default
	DefaultFileMode = os.FileMode(0644)
	// DefaultFileFlag set the default file flag
	DefaultFileFlag = os.O_RDWR | os.O_CREATE | os.O_APPEND

	// ErrInternal defined the internal error
	ErrInternal = errors.New("error internal")
	// ErrClosed defined write while ctx close
	ErrClosed = errors.New("error write on close")
	// ErrInvalidArgument defined the invalid argument
	ErrInvalidArgument = errors.New("error argument invalid")
	// ErrQueueFull defined the queue full
	ErrQueueFull = errors.New("async log queue full")
)

Functions

func AsynchronousWriterErrorChan 

func AsynchronousWriterErrorChan(wr RollingWriter) (chan error, error)

AsynchronousWriterErrorChan return the error channel for asyn writer

func LogFilePath 

func LogFilePath(c *Config) (filepath string)

LogFilePath return the absolute path on log file

Types

type AsynchronousWriter 

type AsynchronousWriter struct {
	Writer
	// contains filtered or unexported fields
}

AsynchronousWriter provide a asynchronous writer with the writer to confirm the write

func (*AsynchronousWriter) Close 

func (w *AsynchronousWriter) Close() error

Close set closed and close the file once

func (*AsynchronousWriter) Write 

func (w *AsynchronousWriter) Write(b []byte) (int, error)

Only when the error channel is empty, otherwise nothing will write and the last error will be returned the error channel

type BufferWriter 

type BufferWriter struct {
	Writer
	// contains filtered or unexported fields
}

BufferWriter merge some write operations into one.

func (*BufferWriter) Close 

func (w *BufferWriter) Close() error

Close bufferWriter flush all buffered write then close file

func (*BufferWriter) Write 

func (w *BufferWriter) Write(b []byte) (int, error)

type Config 

type Config struct {
	// LogPath defined the full path of log file directory.
	// there comes out 2 different log file:
	//
	// 1. the current log
	//	log file path is located here:
	//	[LogPath]/[FileName].[FileExtension]
	//
	// 2. the tuncated log file
	//	the tuncated log file is backup here:
	//	[LogPath]/[FileName].[FileExtension].[TimeTag]
	//  if compressed true
	//	[LogPath]/[FileName].[FileExtension].gz.[TimeTag]
	//
	// NOTICE: blank field will be ignored
	// By default we using '-' as separator, you can set it yourself
	TimeTagFormat string `json:"time_tag_format"`
	LogPath       string `json:"log_path"`
	FileName      string `json:"file_name"`
	// FileExtension defines the log file extension. By default, it's 'log'
	FileExtension string `json:"file_extension"`
	// FileFormatter log file path formatter for the file start write
	// By default, append '.gz' suffix when Compress is true
	FileFormatter LogFileFormatter `json:"-"`
	// MaxRemain will auto clear the roling file list, set 0 will disable auto clean
	MaxRemain int `json:"max_remain"`

	// RollingPolicy give out the rolling policy
	// We got 3 policies(actually, 2):
	//
	//	1. WithoutRolling: no rolling will happen
	//	2. TimeRolling: rolling by time
	//	3. VolumeRolling: rolling by file size
	RollingPolicy      int    `json:"rolling_ploicy"`
	RollingTimePattern string `json:"rolling_time_pattern"`
	RollingVolumeSize  string `json:"rolling_volume_size"`

	// WriterMode in 4 modes below
	// 1. none 2. lock
	// 3. async 4. buffer
	WriterMode string `json:"writer_mode"`
	// BufferWriterThershould in Byte
	BufferWriterThershould int `json:"buffer_thershould"`
	// Compress will compress log file with gzip
	Compress bool `json:"compress"`

	// FilterEmptyBackup will not backup empty file if you set it true
	FilterEmptyBackup bool `json:"filter_empty_backup"`
}

Config give out the config for manager

func NewDefaultConfig 

func NewDefaultConfig() Config

NewDefaultConfig return the default config

type LockedWriter 

type LockedWriter struct {
	Writer
	sync.Mutex
}

LockedWriter provide a synchronous writer with lock write operate will be guaranteed by lock

func (*LockedWriter) Close 

func (w *LockedWriter) Close() error

Close lock and close the file

func (*LockedWriter) Write 

func (w *LockedWriter) Write(b []byte) (n int, err error)

type Locker 

type Locker struct {
	// contains filtered or unexported fields
}

Locker is a spinlock implementation.

A Locker must not be copied after first use.

func (*Locker) Lock 

func (l *Locker) Lock()

Lock will try to get thel lock and yield the processor.

func (*Locker) Unlock 

func (l *Locker) Unlock()

Unlock unlocks l.

type LogFileFormatter 

type LogFileFormatter func(time.Time) string

LogFileFormatter log file format function

type Manager 

type Manager interface {
	// Fire will return a string channel
	// while the rolling event occoured, new file name will generate
	Fire() chan string
	// Close the Manager
	Close()
}

Manager used to trigger rolling event.

func NewManager 

func NewManager(c *Config) (Manager, error)

NewManager generate the Manager with config

type Option 

type Option func(*Config)

Option defined config option

func WithAsynchronous 

func WithAsynchronous() Option

WithAsynchronous enable the asynchronous write for writer

func WithBuffer 

func WithBuffer() Option

WithBuffer will enable the buffer writer mode

func WithBufferThershould 

func WithBufferThershould(n int) Option

WithBufferThershould set buffer write thershould

func WithCompress 

func WithCompress() Option

WithCompress will auto compress the tuncated log file with gzip

func WithFileExtension 

func WithFileExtension(ext string) Option

WithFileExtension set the log file extension

func WithFileFormatter 

func WithFileFormatter(formatter LogFileFormatter) Option

WithFileFormatter set the log file formatter

func WithFileName 

func WithFileName(name string) Option

WithFileName set the log file name

func WithLock 

func WithLock() Option

WithLock will enable the lock in writer Writer will call write with the Lock to guarantee the parallel safe

func WithLogPath 

func WithLogPath(path string) Option

WithLogPath set the log dir and auto create dir tree if the dir/path is not exist

func WithMaxRemain 

func WithMaxRemain(max int) Option

WithMaxRemain enable the auto deletion for old file when exceed the given max value Bydefault -1 will disable the auto deletion

func WithRollingTimePattern 

func WithRollingTimePattern(pattern string) Option

WithRollingTimePattern set the time rolling policy time pattern obey the Corn table style visit http://crontab.org/ for details

func WithRollingVolumeSize 

func WithRollingVolumeSize(size string) Option

WithRollingVolumeSize set the rolling file truncation threshold size

func WithTimeTagFormat 

func WithTimeTagFormat(format string) Option

WithTimeTagFormat set the TimeTag format string

func WithoutRollingPolicy 

func WithoutRollingPolicy() Option

WithoutRolling set no rolling policy

type RollingWriter 

type RollingWriter interface {
	io.Writer
	Close() error
}

RollingWriter implement the io writer

func NewWriter 

func NewWriter(ops ...Option) (RollingWriter, error)

NewWriter generate the rollingWriter with given option

func NewWriterFromConfig 

func NewWriterFromConfig(c *Config) (RollingWriter, error)

NewWriterFromConfig generate the rollingWriter with given config

func NewWriterFromConfigFile 

func NewWriterFromConfigFile(path string) (RollingWriter, error)

NewWriterFromConfigFile generate the rollingWriter with given config file

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

Writer provide a synchronous file writer if Lock is set true, write will be guaranteed by lock

func (*Writer) Close 

func (w *Writer) Close() error

Close the file and return

func (*Writer) CompressFile 

func (w *Writer) CompressFile(oldfile *os.File, cmpname string) error

CompressFile compress log file write into .gz

func (*Writer) DoRemove 

func (w *Writer) DoRemove()

DoRemove will delete the oldest file

func (*Writer) Reopen 

func (w *Writer) Reopen(file string) error

Reopen do the rotate, open new file and swap FD then trate the old FD

func (*Writer) Write 

func (w *Writer) Write(b []byte) (int, error)

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.