storage

README

mohawk/storage

Mohawk is a metric data storage engine that uses a plugin architecture for data storage and a simple REST API as the primary interface.

Storage Plugins

Mohawk can use different storage plugins for different use cases. Different storage plugins may vary in speed, persistence and scale ability. Mohawk use a subset of Hawkular's REST API, inheriting Hawkular's ecosystem of clients and plugins.

Plugin Development

A storage plugin should implement the Storage interface. Each storage plugin is built for specific use case, with features that best suite this use case.

Implementation of a feature should not interfere with the storage plugin functionality, for example, a plugin built for speed may choose not to implement a feature that may slow it down.

Plugins that implement a subset of the interface, must fail silently for unimplemented requests.

For a starting template of a storage plugin, look at the storage example directory.

Plugins Comparison

• Example - a storage template.
• Sqlite - a file storage based storage.
• Memory - a memory storage based storage.
• Mongo - a cluster based storage.

Features

Plugin	Speed	Retention Limit	Scaleability	Storage
Example				No storage
Memory	Very Fast	7 days		Memory
Sqlite	Fast			Local File
Mongo	Fast		Cluster	Mongo DB

REST Endpoint Implementation

Plugin	Multi Tenancy	Read	Write	Update	Delete
Example		✔️
Memory	✔️	✔️	✔️	✔️
Sqlite	✔️	✔️	✔️	✔️	✔️
Mongo	✔️	✔️	✔️	✔️

Metrics List Implementation

Plugin	Filter by Tag RegEx	Last Values
Example
Memory	✔️	✔️
Sqlite	✔️
Mongo	✔️

Aggregation and Statistics Implementation

Plugin	Min	Max	First	Last	Avg	Median	Std	Sum	Count
Example					✔️				✔️
Memory	✔️	✔️	✔️	✔️	✔️			✔️	✔️
Sqlite	✔️	✔️			✔️			✔️	✔️
Mongo	✔️	✔️	✔️	✔️	✔️			✔️	✔️

Documentation

Overview

Package storage interface for metric data storage

Package storage interface for metric data storage

Index

• func ParseSec(t string) int64
• func ParseTags(tags string) map[string]string
• type DataItem
• type Item
  • func FilterItems(vs []Item, f func(Item) bool) []Item
• type StatItem
• type Storage
• type Tenant

Functions

func ParseSec

func ParseSec(t string) int64

ParseSec parse a time string into seconds, posible postfix - s, mn, h, d e.g. "2h" => 2 * 60 * 60

func ParseTags

func ParseTags(tags string) map[string]string

ParseTags takes a comma separeted key:value list string and returns a map[string]string

e.g.
"warm:kitty,soft:kitty" => {"warm": "kitty", "soft": "kitty"}

Types

type DataItem

type DataItem struct {
	Timestamp int64   `json:"timestamp" bson:"timestamp"`
	Value     float64 `json:"value" bson:"value"`
}

DataItem one metric data point

type Item

type Item struct {
	ID         string            `json:"id" bson:"_id"`
	Type       string            `json:"type" bson:"type"`
	Tags       map[string]string `json:"tags" bson:"tags"`
	LastValues []DataItem        `json:"data,omitempty" bson:"data,omitempty"`
}

Item one metric item

func FilterItems

func FilterItems(vs []Item, f func(Item) bool) []Item

FilterItems filters a list using a filter function

type StatItem

type StatItem struct {
	Start   int64   `json:"start"`
	End     int64   `json:"end"`
	Empty   bool    `json:"empty"`
	Samples int64   `json:"samples,omitempty"`
	Min     float64 `json:"min,omitempty"`
	Max     float64 `json:"max,omitempty"`
	First   float64 `json:"first,omitempty"`
	Last    float64 `json:"last,omitempty"`
	Avg     float64 `json:"avg,omitempty"`
	Median  float64 `json:"median,omitempty"`
	Std     float64 `json:"std,omitempty"`
	Sum     float64 `json:"sum,omitempty"`
}

StatItem one statistics data point

type Storage

type Storage interface {
	Name() string
	Help() string
	Open(options url.Values)
	GetTenants() ([]Tenant, error)
	GetItemList(tenant string, tags map[string]string) ([]Item, error)
	GetRawData(tenant string, id string, end int64, start int64, limit int64, order string) ([]DataItem, error)
	GetStatData(tenant string, id string, end int64, start int64, limit int64, order string, bucketDuration int64) ([]StatItem, error)
	PostRawData(tenant string, id string, t int64, v float64) error
	PutTags(tenant string, id string, tags map[string]string) error
	DeleteData(tenant string, id string, end int64, start int64) error
	DeleteTags(tenant string, id string, tags []string) error
}

Storage metric data interface

type Tenant

type Tenant struct {
	ID string `json:"id"`
}

Tenant just the tenant name