util

Documentation

Index

• Constants
• func Read(ctx context.Context, category Category, linesCh chan []byte) error
• func UnescapeLine(line []byte) []byte
• func Watch(ctx context.Context, category Category, outChan chan string) error
• type Category
  • func (c *Category) GetCurrentSymlink() string
  • func (c *Category) GetDir() string
• type Writer
  • func (w *Writer) Write(eventBytes []byte) (int, error)
  • func (w *Writer) WriteJson(data interface{}) error

Constants

const IN_OPERATION_CLEAN_INTERVAL = time.Second * 10

const READDIR_BUFFER_SIZE = 10000

const WATCH_RESULTS_BUFFER_SIZE = 100

Functions

func Read

func Read(ctx context.Context, category Category, linesCh chan []byte) error

Read starts to watch for category's files and stream their content to linesCh line by line it deletes fully sent files immediately (no confirm mechanics) @TODO implement some kind of confirm if error occurs, Read exits immediately, closing all background stuff read process can be cancelled by cancelling parent ctx keep in mind that Read blocks your current goroutine, so you need to use it in separate one if you need more efficient read (batching, etc), you can use Watcher along with UnescapeLine func

func UnescapeLine

func UnescapeLine(line []byte) []byte

func Watch

func Watch(ctx context.Context, category Category, outChan chan string) error

Watch starts to look through category's files and sends full file names that are ready to be processed to outChan if error occurs, Watch exits immediately, closing all background stuff watch process can be cancelled by cancelling parent ctx keep in mind that Watch blocks your current goroutine, so you need to use it separate one

Types

type Category

type Category struct {
	BaseDir string
	Name    string
}

Category is used to compute paths according to lsd format

func (*Category) GetCurrentSymlink

func (c *Category) GetCurrentSymlink() string

func (*Category) GetDir

func (c *Category) GetDir() string

type Writer

type Writer struct {
	Category Category
}

Writer is used for events and data structures streaming best way to send events to LSD it writes events into <BaseDir>/<category>/<yyyymmddhhmm>.log files it also guarantees consistency for long lines (>4k) provides escaping of line breaks inside events (single event will always be sent as single line) for compatibility with badoo php library, we have to use analog of PHP's addcslashes($message, "\\\n") for escaping of lines so it's not safe to send raw binary data (without any encoding)

func (*Writer) Write

func (w *Writer) Write(eventBytes []byte) (int, error)

func (*Writer) WriteJson

func (w *Writer) WriteJson(data interface{}) error