apexorc

README

apexorc

Output to ORC files from github.com/apex/log

Overview

This package provides a handler for github.com/apex/log that will cause Entry structs to be persisted to an ORC file. The following columns are present in the resulting ORC file:

• timestamp timestamp
• level string
• message string
• fields map<string,string>

Note that the map type in fields only supports strings so you will have to make a string representation of any data you want to store their. Note also that using apex's .WithError function is actually just a shortcut to creating a field called error, which is where you'll find any errors you use.

Additionally, a RotatingHandler is provided to allow for ORC log files to be rotated on demand. No scheduling or other mechanism is provided, only the infrastructure for log rotation itself. A typical strategy in UNIX like environments is to do rotation in response to a signal.

Examples

Simple logging to an ORC file:

package main

import (
    "errors"

    "github.com/apex/log"
    "github.com/avct/apexorc"
)

func main() {
    // Note that ORC files aren't created until the handler is closed or
    // rotated.  Prior to this log output goes to a plain text journal file
    // in the same directory.
    handler := NewHandler("mylog.orc")

    // It's important to close the handler when we're done!
    defer handler.Close()
    
    log.SetHandler(handler)
    
    err := errors.New("Ouch")
    log.WithError(err).Error("An Orc attacked")
}

Using a rotating handler with the provided numeric archiving algorithm

package main

import (
    "errors"

    "github.com/apex/log"
    "github.com/avct/apexorc"
)

func main() {
    handler := apexorc.NewRotatingHandler("mylog.orc", apexorc.NumericArchiveF)
    // It's important to close the handler when we're done! As we currently
    // don't support appending to an ORC file, we treat exiting a program as
    // a reason to rotate.
    defer handler.Rotate()
    
    log.SetHandler(handler)

    // Initially the log entries will be written to a plaintext journal file
    // in the same directory as the specified ORC path.
    err := errors.New("Ouch")
    log.WithError(err).Error("An Orc attacked")
    
    // When we rotate, the current journal file will be closed, and moved out
    // of the way.  A new journal file will be created.  The old journal file
    // will be converted to an ORC file in the background and placed at
    // mylog.orc.1 (any existing file with that name will be moved to
    /  mylog.orc.2, and others will shuffle out of the way in a similar
    // fashion).
    handler.Rotate()
    
    
    log.Info("This will get logged to a brand new journal file")
    
    // When the program exits, the deferred Rotate() will move mylog.orc to mylog.orc.1 and the previous mylog.orc.1 will be moved to mylog.orc.2.  
}

Documentation

Overview

Package apexorc provides a handler for logging via github.com/apex/log to an ORC file.

Index

• func IsCriticalRotationError(e error) bool
• func NumericArchiveF(oldPath string) error
• type ArchiveFunc
• type CloserHandler
• type CriticalRotationError
• type Handler
  • func NewHandler(path string) *Handler
  • func (h *Handler) Close() error
  • func (h *Handler) HandleLog(e *log.Entry) error
• type RotatingHandler
  • func NewRotatingHandler(path string, archiveF ArchiveFunc) (*RotatingHandler, error)
  • func (h *RotatingHandler) EnableAlwaysRemoveTempFiles()
  • func (h *RotatingHandler) HandleLog(e *log.Entry) error
  • func (h *RotatingHandler) Rotate() error

Functions

func IsCriticalRotationError

func IsCriticalRotationError(e error) bool

IsCriticalRotationError returns true if the error passed to it is a CriticalRotationErorr.

func NumericArchiveF

func NumericArchiveF(oldPath string) error

NumericArchiveF is an ArchiveFunc that archives historic log files with numeric suffixes. The lower the suffix the more recent the file. Older archived files are pushed back to higher-number suffixes as the new archives are created.

Types

type ArchiveFunc

type ArchiveFunc func(oldPath string) error

ArchiveFunc is a function type that is used to move an ORC log file from its current path to another location as part of the log rotation process. The function will be called with the path of the current log file, it is the responsibility of an implementation to ensure that files are moved non-destructively, but the RotatingHandler guarantees that the file will be closed before an ArchvieFunc is called, and that no attemp to log will be made until after it has completed its work.

type CloserHandler

type CloserHandler interface {
	log.Handler
	Close() error
}

CloserHandler is a specialisation of the github.com/apex/log.Handler interface to support a Close function.

type CriticalRotationError

type CriticalRotationError struct {
	// contains filtered or unexported fields
}

CriticalRotationError is a special kind of error that can be returned by the Rotate() function of a RotatingHandler. Should you encounter a CriticalRotationError you should assume that logging is no longer working and decide how to proceed.

You can check an error to see if it is a CriticalRotationError by passing it to IsCriticalRotationError:

type Handler

type Handler struct {
	// contains filtered or unexported fields
}

Handler complies with the github.com/apex/log.Handler interface and can be passed to github.com/apex/log.SetHandler

func NewHandler

func NewHandler(path string) *Handler

NewHandler returns a Handler which can log to an ORC file at the provided path.

func (*Handler) Close

func (h *Handler) Close() error

Close finalises the underlying ORC file.

func (*Handler) HandleLog

func (h *Handler) HandleLog(e *log.Entry) error

HandleLog recieves new log.Entrys and writes them to an ORC file or errors, as specified by the github.com/apex/log.Handler intefrace.

type RotatingHandler

type RotatingHandler struct {
	// contains filtered or unexported fields
}

RotatingHandler is a github.com/apex/log.Handler implementation that uses a subordinate Handler to log to an ORC file, but additionally supports on demand rotation of this file via a Rotate function. The RotatingHandler should only ever be constructed using the NewRotatingHandler function.

func NewRotatingHandler

func NewRotatingHandler(path string, archiveF ArchiveFunc) (*RotatingHandler, error)

NewRotatingHandler returns an instance of the RotatingHandler with a subordinate ORC Handler logging to the provided path. Should Rotate be called then the provided ArchiveFunc will be used to move the current ORC log file out of the way before creating a new one at the same path and continuing to handle log entries.

func (*RotatingHandler) EnableAlwaysRemoveTempFiles

func (h *RotatingHandler) EnableAlwaysRemoveTempFiles()

EnableAlwaysRemoveTempFiles ensures that we always remove temp files even if we were unable to convert to ORC

func (*RotatingHandler) HandleLog

func (h *RotatingHandler) HandleLog(e *log.Entry) error

HandleLog passes logging duty through to the subordinate ORC Handler.

func (*RotatingHandler) Rotate

func (h *RotatingHandler) Rotate() error

Rotate is a blocking call and will not return until an ORC file has been created. Logging will only be blocked for the earliest part of the process, but subsequent calls to Rotate will not complete until earlier ones have already completed.

The caller should check any returned error using IsCriticalRotationError. If a CriticalRotationError is returned, logging will no longer work as the handler will not be unlocked. It is the callers responsiblity to decide on a course of action at that point (when all else fails, panic).