agerotate

README

agerotate

Agerotate is go a package to delete files or other objects such as logs or backups based on age, according to a flexible schedule. Fewer numbers of older files are retained allowing you to strike your own balance of having sufficient history while keeping disk usage low.

filerotate

Included is a binary to do age rotation on files. The configuration syntax looks like this:

# Rotation for live FooDB dumps.
PATHGLOB:/var/foodb/dumps/*.bz2
# Time-range lines take the form RANGE:Maximum Age:Retention Interval
RANGE:72h:0		# Keep all files less than 72 hours
RANGE:336h:6h	# For files less than two weeks, keep one per six hours
RANGE:4320h:24h	# For files less than 180 days, keep one per day
# Everything older than 180 days gets deleted.

If your path includes a colon, such as on Windows, you can use the -fieldsep command line argument to specify that a different separator character will be used in your config.

DANGER WARNING DEATH AHEAD

It's critical to understand that this tool deletes data entirely unattended. It deletes data based on the age of the data. If new data items aren't being added, eventually filerotate will delete all of your data as it ages. You may want to wrap invocation of filerotate in a script that only runs filerotate if a minimum number of files exist.

When first creating a rotation config it's a good idea to test it on hardlinks to your real data. This allows you to perform dry runs safely.

# Set PATHGLOB in your config to /tmp/footest/*.bz2
$ SOURCEDATA=/var/foodb/dumps/*
$ TESTDIR=/tmp/footest/
$ for i in ${SOURCEDATA}; do ln $i ${TESTDIR}; done
$ filerotate -config /path/to/myconfig
# Examine the contents of /tmp/footest.
# If everything looks right:
$ rm -rf /tmp/footest
# If not:
$ rm -rf /tmp/footest/*
$ for i in ${SOURCEDATA}; do ln $i ${TESTDIR}; done
# Adjust your config, rerun filerotate.

Extending agerotate

You can extend agerotate to work with arbitrary data sources by providing an implementation of agerotate.Objects to enumerate the dataset. It must return each object as an implementation of agerotate.Object with Age(), ID(), and Delete() methods. agerotate.fileobject is a good reference.

Documentation

Overview

agerotate provides cleanup of timestamped objects with user-defined schedules. Objects are grouped into "buckets" based on their age. Each bucket has an age threshold and an age interval. If a bucket has an interval of two hours, roughly one object will be kept for every two hours. This allows the user to keep many objects that are recent and fewer objects that are older.

Index

• type ByAge
  • func (a ByAge) Len() int
  • func (a ByAge) Less(i, j int) bool
  • func (a ByAge) Swap(i, j int)
• type Object
• type Objects
• type ObjectsByAge
  • func (a ObjectsByAge) Len() int
  • func (a ObjectsByAge) Less(i, j int) bool
  • func (a ObjectsByAge) Swap(i, j int)
• type Range
  • func (r Range) String() string

Types

type ByAge

type ByAge []Range

ByAge implements sort.Interface to sort Range objects by Age, ascending.

func (ByAge) Len

func (a ByAge) Len() int

func (ByAge) Less

func (a ByAge) Less(i, j int) bool

func (ByAge) Swap

func (a ByAge) Swap(i, j int)

type Object

type Object interface {
	// Age returns the age of the object.
	Age() time.Duration
	// Delete attempts to remove the object.
	Delete() error
	// ID returns an identifier string that is intended to be unique.
	ID() string
}

Object is the interface objects implement to be managed by agerotate.

type Objects

type Objects interface {
	// ID returns an identifier string that is intended to be unique.
	ID() string
	// List retrieves all of the available Objects.
	List() ([]Object, error)
}

Objects is the interface for a container of Object objects.

type ObjectsByAge

type ObjectsByAge struct {
	O []Object
}

ObjectsByAge implements sort.Interface to sort Objects by Age, ascending.

func (ObjectsByAge) Len

func (a ObjectsByAge) Len() int

func (ObjectsByAge) Less

func (a ObjectsByAge) Less(i, j int) bool

func (ObjectsByAge) Swap

func (a ObjectsByAge) Swap(i, j int)

type Range

type Range struct {
	Age      time.Duration
	Interval time.Duration
}

Range identifies a set of items for rotation. Age specifies the youngest items that belong to the set. Interval defines the minimum age gap between items to keep.

func (Range) String

func (r Range) String() string

String profiles a human-readable string for a range.