lognile

README

Lognile

Lightweight File Log Collection Tool for Golang

Lognile is a lightweight log collection tool implemented in Go. It allows you to monitor log files and automatically handle new log entries.

Quick Start

1. Create a Configuration File: Create a file named config.yaml with the following content:

   # Paths to the log files to monitor
   pattern:
       - ./_log/php-fpm-*.log
       - ./_log/nginx-*.log
   
   # File to store log progress
   db: lognile.db
   

2. Create a Basic Application: Create a file named main.go with the following content to use Lognile:

   package main
   
   import (
       "log"
       "github.com/boyxp/lognile"
   )
   
   func main() {
       // Initialize Lognile with the configuration file
       lognile.Init("config.yaml", func(row map[string]string) {
           // Handle new log entries
           log.Println(row)
       })
   }
   

3. Run the Application: Use the following command to run your application:

   go run main.go
   

Explanation

• Configuration File (config.yaml):

  • pattern: Specifies the paths to the log files to monitor. Wildcards can be used to match multiple files.
  • db: Specifies the file used to store log progress, so it can resume from the last position after restarting.

• Application (main.go):

  • Import Packages: Import the log package for logging and the lognile package for log collection.
  • Initialize Lognile: Call lognile.Init with the path to config.yaml and a callback function to handle new log entries.
  • Callback Function: The callback function receives a map containing the new log entry. In this example, it simply logs the entry using log.Println.

Running the Application

• Start Monitoring: Once the application runs, it will start monitoring the specified log files for new entries.
• Handle Log Entries: New log entries will be passed to the callback function, where you can process them as needed.

Demo

Please see the demo for a demonstration of its actual performance.

Documentation

Index

• func Init(cfg string, callback func(log map[string]string))
• type Config
  • func (C *Config) Db() string
  • func (C *Config) Init(cfg string) *Config
  • func (C *Config) Pattern() map[string][]string
• type Lognile
  • func (L *Lognile) Exit()
  • func (L *Lognile) Init(cfg string, callback func(log map[string]string))
• type Offset
  • func (O *Offset) Load() sync.Map
  • func (O *Offset) Save(offset map[uint64]int64)
• type Reader
  • func (R *Reader) Close()
  • func (R *Reader) Name() string
  • func (R *Reader) Offset() int64
  • func (R *Reader) Read(wait bool, queue chan map[string]string)
  • func (R *Reader) Rename(filename string)

Functions

func Init

func Init(cfg string, callback func(log map[string]string))

Types

type Config

type Config struct {
	// contains filtered or unexported fields
}

func (*Config) Db

func (C *Config) Db() string

func (*Config) Init

func (C *Config) Init(cfg string) *Config

func (*Config) Pattern

func (C *Config) Pattern() map[string][]string

type Lognile

type Lognile struct {
	// contains filtered or unexported fields
}

func (*Lognile) Exit

func (L *Lognile) Exit()

func (*Lognile) Init

func (L *Lognile) Init(cfg string, callback func(log map[string]string))

type Offset

type Offset struct {
	// contains filtered or unexported fields
}

func (*Offset) Load

func (O *Offset) Load() sync.Map

func (*Offset) Save

func (O *Offset) Save(offset map[uint64]int64)

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

func (*Reader) Close

func (R *Reader) Close()

func (*Reader) Name

func (R *Reader) Name() string

func (*Reader) Offset

func (R *Reader) Offset() int64

func (*Reader) Read

func (R *Reader) Read(wait bool, queue chan map[string]string)

func (*Reader) Rename

func (R *Reader) Rename(filename string)