blue

package module
Version: v0.0.0-...-0ab8f12 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2016 License: MIT Imports: 6 Imported by: 0

README

#blue.Line blue logo

GoDoc Go Report Card Build Status Coverage Status

Generates influxdb line protocol from json objects.

WARNING:Do not use this in production

Features

  • Flexible:Allow custom functions to choose

    • The name of the measurement
    • Tags
    • Fields
    • Timestamps
  • Works on arbitrary json input.

  • Tags and Fields are sorted. This is per recommendation from influxdb documentation.

The steps taken when processing the json input

Flattening

The json object is flattened to key, value pair.This is important because influxdb line protocol is based on key, value semantics.

This brings up the problem of nested objects like arrays of objects etc. The best strategy I could come up with is to join the keys that point to a value.

For example

{
  "top":{
    "level":1
  }
}

That can easily be flattered to top_level=1 , where by we used underscore _ to join the keys. This rises another problem though maybe the names_of_keys_are_like_this so to make it flexible the user of this library can specify the joining strategy of the keys, whereby you can define a function of the form func(a,b string)string that will be used to join keys.

Filtering

After the json bject has been flattered to key, value pairs it is filtered to pic what the user thinks is important. The main components are,

  • Measurement name
  • Tags
  • Fields

The user of this library, has control on all the mentioned components. The Options struct looks like this.

type Options struct {
	//This is the function that joins the keys when flattering the json object.
	//The first argument is the top level key(although this might be not the
	//case for deeply nested objects) and the second is the current key.
	//
	// The returned string is the key that will be used. This is implementation
	// specific, you can do whatever the hell you want with this.
	KeyJoinFunc func(a, b string) string

	// Checks  if the key is a tag. If the second returned value is true then
	// the first returned value is going to be used as key.
	//
	// Something to consider here the key might be a joint keys for the deeply
	// nested objects. The keys will be supposedly joined by the KeyJoinFunc
	// implementation.
	IsTag func(string) (string, bool)

	// Checks if the aregumen( a key) belongs to a Field. Implementations should
	// return true if the key is a field and false otherwise.
	IsField func(string) bool

	//IsMeasurement this determines if the given key is a key with the
	//measurement name.
	//
	//It is up to the implementaion to return the measurement name, true. If the
	//second returned value is false then the key is assumend to be not
	//measurement name.
	//
	// If the Measurement field is set i.e not empty then this function is nver
	// going to be used.
	IsMeasurement func(string, interface{}) (string, bool)

	// Determines the timestamp of the measurement. Timestamp is set only once.
	IsTimeStamp func(string, interface{}) (time.Time, bool)
	Measurement string
}

Specifying the Measurement field will make the IsMeasurent function to be ignored.

Installation

go get github.com/gernest/blue

Usage


package main

import (
	"fmt"
	"log"
	"strings"

	"github.com/gernest/blue"
)

const src = `
{
	"timestamp":1434055562000000000,
	"tags":{
		"noun":"broke",
		"verb":"bankrupt"
	},
	"anger_level":80,
	"this":{
		"is":{
			"nested":{
				"value": 1
			}
		}
	}
}
`

func main() {
	o := blue.Options{
		KeyJoinFunc: func(a, b string) string {
			if a == "" {
				return b
			}
			return a + "_" + b
		},
		IsTag: func(key string) (string, bool) {
			prefix := "tags_"
			if strings.HasPrefix(key, prefix) {
				return strings.TrimPrefix(key, prefix), true
			}
			return "", false
		},
		IsField: func(key string) bool {
			return true
		},
		Measurement: "ordinary",
	}
	m, err := blue.Line([]byte(src), o)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(m)
	//ordinary,noun=broke,verb=bankrupt anger_level=80,this_is_nested_value=1 1434055562000000000
}

Author

Geofrey Ernest @gernesti

Logo was made by @dr-okra

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsField

func IsField(key string) bool

IsField checks whether the key is a field. This is always true except when the key is either "measurement" or "timestamp"

func IsMeasurement

func IsMeasurement(key string, value interface{}) (string, bool)

IsMeasurement checks if the key is measuremet.

func IsTag

func IsTag(key string) (string, bool)

IsTag checks if key is tag. This is always false.

func IsTimeStamp

func IsTimeStamp(key string, value interface{}) (time.Time, bool)

IsTimeStamp checks whether the given key is timestamp.For this function, timestamp is expected to be a field with key=="timestamp" and the value is a float64, which is a normal json number.

The value is also assument to be time in nanoseconds since the UNIX Epoch.

Types

type Context

type Context struct {
	C collector
	// contains filtered or unexported fields
}

Context is the procesing context for json input. It is where the key, value pairs are collected.

type Field

type Field Unit

Field is a Unit that represent influxbd field.

func (*Field) Line

func (f *Field) Line() string

Line returns string representation of the underlying Field.

type Fields

type Fields []*Field

Fields is a list of Fields.

func (Fields) Len

func (f Fields) Len() int

func (Fields) Less

func (f Fields) Less(i, j int) bool

func (Fields) Line

func (f Fields) Line() string

Line returns the string representation of the underlying fields that complies with influxdb line protocol.

func (Fields) Swap

func (f Fields) Swap(i, j int)

type Measurement

type Measurement struct {
	Name      string
	Tags      Tags
	Fields    Fields
	Timestamp time.Time
}

Measurement represent influxdb metric.

func Line

func Line(src []byte, opts Options) (*Measurement, error)

Line generates *Measurement object from the src. src is expected to be a valid json input.

func (*Measurement) String

func (m *Measurement) String() string

type Options

type Options struct {
	//This is the function that joins the keys when flattering the json object.
	//The first argument is the top level key(although this might be not the
	//case for deeply nested objects) and the second is the current key.
	//
	// The returned string is the key that will be used. This is implementation
	// specific, you can do whatever the hell you want with this.
	KeyJoinFunc func(a, b string) string

	// Checks  if the key is a tag. If the second returned value is true then
	// the first returned value is going to be used as key.
	//
	// Something to consider here the key might be a joint keys for the deeply
	// nested objects. The keys will be supposedly joined by the KeyJoinFunc
	// implementation.
	IsTag func(string) (string, bool)

	// Checks if the aregumen( a key) belongs to a Field. Implementations should
	// return true if the key is a field and false otherwise.
	IsField func(string) bool

	//IsMeasurement this determines if the given key is a key with the
	//measurement name.
	//
	//It is up to the implementaion to return the measurement name, true. If the
	//second returned value is false then the key is assumend to be not
	//measurement name.
	//
	// If the Measurement field is set i.e not empty then this function is nver
	// going to be used.
	IsMeasurement func(string, interface{}) (string, bool)

	// Determines the timestamp of the measurement. Timestamp is set only once.
	IsTimeStamp func(string, interface{}) (time.Time, bool)
	Measurement string
}

Options provides fine grained options for processing the json input.

type Tag

type Tag Unit

Tag is a Unit representing influxdb field.

func (Tag) Line

func (t Tag) Line() string

Line returns influxdb line protocl compliant string representation of the field.

type Tags

type Tags []*Tag

Tags is a list of Tag

func (Tags) Len

func (t Tags) Len() int

func (Tags) Less

func (t Tags) Less(i, j int) bool

func (Tags) Line

func (t Tags) Line() string

Line returns influxdb line protocol compliant string representation of the underlying tags.

func (Tags) Swap

func (t Tags) Swap(i, j int)

type Unit

type Unit struct {
	Key   string
	Value interface{}
}

Unit is a key value pair.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL