datatype

package
v0.9.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 28, 2018 License: Apache-2.0 Imports: 12 Imported by: 16

Documentation

Overview

Package datatype contains necessary logic to sanitise a JSON object coming from a reader. This package is subjected to change.

Index

Examples

Constants

View Source 
const (
	// BYTE amount is the same as is read.
	BYTE = 1.0
	// KILOBYTE divides the amount to kilobytes to show smaller value.
	KILOBYTE = 1024 * BYTE
	// MEGABYTE divides the amount to megabytes to show smaller value.
	MEGABYTE = 1024 * KILOBYTE
)

Variables

View Source 
var ErrUnidentifiedJason = errors.New("unidentified jason value")

ErrUnidentifiedJason is an error when the value is not identified. It happens when the value is not a string or a float64 types, or the container ends up empty.

View Source 
var TimeStampFormat = "2006-01-02T15:04:05.999999-07:00"

TimeStampFormat specifies the format that all timestamps should be formatted with.

Functions

func FloatInSlice added in v0.9.0 

func FloatInSlice(niddle float64, haystack []float64) bool

FloatInSlice returns true if niddle is in the haystack

func Uint64InSlice added in v0.9.0 

func Uint64InSlice(niddle uint64, haystack []uint64) bool

Uint64InSlice returns true if niddle is in the haystack

Types

type ByteType 

type ByteType struct {
	Key   string
	Value float64
}

ByteType represents a pair of key values in which the value represents bytes. It converts the value to MB.

func (ByteType) Equal added in v0.3.0 

func (b ByteType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal.

func (ByteType) Write added in v0.9.0 

func (b ByteType) Write(p io.Writer) (n int, err error)

Write includes both Key and Value. It turns the byte into Megabyte.

type Container 

type Container struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Container satisfies the DataContainer

func New added in v0.7.0 

func New(list []DataType) *Container

New returns a new container and populates it with the given list.

func (*Container) Add 

func (c *Container) Add(d ...DataType)

Add adds d to the list. You can pass it as many items you need to.

Example 
package main

import (
	"fmt"

	"github.com/arsham/expipe/datatype"
)

func main() {
	c := datatype.New(nil)
	firstElm := datatype.FloatType{}

	c.Add(firstElm)
	fmt.Println(c.Len())

	c.Add(datatype.FloatType{}, datatype.FloatType{})
	fmt.Println(c.Len())
}

Output:

1
3

func (*Container) Generate added in v0.9.0 

func (c *Container) Generate(p io.Writer, timestamp time.Time) (int, error)

Generate prepends a timestamp pair and value to the list, and generates a json object suitable for recording into a document store.

Example (Add) 
package main

import (
	"bytes"
	"fmt"
	"time"

	"github.com/arsham/expipe/datatype"
)

func main() {
	t := time.Date(2009, time.November, 10, 23, 0, 0, 0, time.UTC)

	c := datatype.Container{}
	elm := datatype.FloatType{Key: "key", Value: 66.6}
	c.Add(elm)

	w := new(bytes.Buffer)
	_, err := c.Generate(w, t)
	fmt.Println("With new element:", w.String())
	fmt.Println("Error:", err)

}

Output:

With new element: {"@timestamp":"2009-11-10T23:00:00+00:00","key":66.600000}
Error: <nil>
Example (Errors) 
t := time.Date(2009, time.November, 10, 23, 0, 0, 0, time.UTC)

c := datatype.Container{}
c.Add(new(badDataType))
w := new(bytes.Buffer)
_, err := c.Generate(w, t)
fmt.Println(errors.Cause(err))

Output:

DataType Error
Example (New) 
package main

import (
	"bytes"
	"fmt"
	"time"

	"github.com/arsham/expipe/datatype"
)

func main() {
	t := time.Date(2009, time.November, 10, 23, 0, 0, 0, time.UTC)

	c := datatype.Container{}
	w := new(bytes.Buffer)
	n, err := c.Generate(w, t)
	fmt.Println(err, n)
	fmt.Println("Contents:", w.String())
}

Output:

<nil> 42
Contents: {"@timestamp":"2009-11-10T23:00:00+00:00"}

func (*Container) Len 

func (c *Container) Len() int

Len returns the length of the data.

Example 
package main

import (
	"fmt"

	"github.com/arsham/expipe/datatype"
)

func main() {
	l := []datatype.DataType{datatype.FloatType{}, datatype.FloatType{}}
	c := datatype.New(l)
	fmt.Println(c.Len())
}

Output:

2

func (*Container) List 

func (c *Container) List() []DataType

List returns the data.

type DataContainer 

type DataContainer interface {
	List() []DataType
	Len() int
	Generate(p io.Writer, timestamp time.Time) (n int, err error)
}

DataContainer is an interface for holding a list of DataType. I'm aware of the container/list package, which is awesome, but I needed a simple interface to do this job. You should not update the list returning from List as it is a shared list and anyone can read from it. If you append to this list, there is a chance you are not referring to the same underlying array in memory. Generate writes to w applying the timestamp. It returns the amount of bytes it's written and an error if any of its contents report.

func JobResultDataTypes 

func JobResultDataTypes(b []byte, mapper Mapper) (DataContainer, error)

JobResultDataTypes generates a list of DataType and puts them inside the DataContainer. It returns errors if unmarshaling is unsuccessful or ErrUnidentifiedJason when the container ends up empty.

type DataType 

type DataType interface {
	Write(p io.Writer) (n int, err error)
	Equal(other DataType) bool
}

DataType represents a single paired data. The key of the json value is mapped to Key, and the value is to Value. Write includes both Key and Value. THe Equal method comparison of the value is not ordered.

type FloatListType 

type FloatListType struct {
	Key   string
	Value []float64
}

FloatListType represents a pair of key values that the value is a list of floats.

func (FloatListType) Equal added in v0.3.0 

func (fl FloatListType) Equal(other DataType) bool

Equal compares both keys and all values and returns true if they are equal. The values are checked in an unordered fashion.

func (FloatListType) Write added in v0.9.0 

func (fl FloatListType) Write(p io.Writer) (n int, err error)

Write includes both Key and Value.

type FloatType 

type FloatType struct {
	Key   string
	Value float64
}

FloatType represents a pair of key values that the value is a float64.

func (FloatType) Equal added in v0.3.0 

func (f FloatType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal.

func (FloatType) Write added in v0.9.0 

func (f FloatType) Write(p io.Writer) (n int, err error)

Write includes both Key and Value.

type GCListType 

type GCListType struct {
	Key   string
	Value []uint64
}

GCListType represents a pair of key values of GC list info.

func (GCListType) Equal added in v0.3.0 

func (flt GCListType) Equal(other DataType) bool

Equal is not implemented. You should iterate and check yourself. Equal compares both keys and values and returns true if they are equal.

func (GCListType) Write added in v0.9.0 

func (flt GCListType) Write(p io.Writer) (n int, err error)

Write includes both Key and Value.

type KiloByteType added in v0.3.0 

type KiloByteType struct {
	Key   string
	Value float64
}

KiloByteType represents a pair of key values in which the value represents bytes. It converts the value to KB.

func (KiloByteType) Equal added in v0.3.0 

func (k KiloByteType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal.

func (KiloByteType) Write added in v0.9.0 

func (k KiloByteType) Write(p io.Writer) (n int, err error)

Write includes both Key and Value.

type MapConvert added in v0.3.0 

type MapConvert struct {
	// contains filtered or unexported fields
}

MapConvert can produce output from GC string list and memory type input.

func DefaultMapper added in v0.3.0 

func DefaultMapper() *MapConvert

DefaultMapper returns a MapConvert object that is populated by the default mappings. The data is hard coded in the program, but you can provide your mapping file in your configuration file.

func MapsFromViper added in v0.3.0 

func MapsFromViper(v treeReader) *MapConvert

MapsFromViper reads from the map file and produces functions for conversion used in type decoder. It first reads from the default settings defined in the maps.yml in the same folder, then overrides with the user specified mappings.

func (*MapConvert) Copy added in v0.7.0 

func (m *MapConvert) Copy() Mapper

Copy returns a new copy of the Mapper.

func (*MapConvert) Values added in v0.3.0 

func (m *MapConvert) Values(prefix string, values map[string]*jason.Value) []DataType

Values returns a slice of DataTypes based on the given name/value inputs. It flattens the float list values, therefore you will get multiple values per input. If the name is found in memory_bytes map, it will return one of those, otherwise it will return a FloatType or StringType if can convert. It will return nil if the value is not one of above.

type MapConvertMock added in v0.3.0 

type MapConvertMock struct {
	GCTypes     []string
	MemoryTypes map[string]MemTypeMock
	ValuesFunc  func(prefix string, values map[string]*jason.Value) []DataType

	DefaultCovertor Mapper
	// contains filtered or unexported fields
}

MapConvertMock is the mocked version of MapConvert.

func (*MapConvertMock) Copy added in v0.7.0 

func (m *MapConvertMock) Copy() Mapper

Copy returns a new copy of the Mapper.

func (*MapConvertMock) Values added in v0.3.0 

func (m *MapConvertMock) Values(prefix string, values map[string]*jason.Value) []DataType

Values calls the ValuesFunc if exists, otherwise returns nil.

type Mapper added in v0.3.0 

type Mapper interface {
	Values(prefix string, values map[string]*jason.Value) []DataType
	Copy() Mapper
}

Mapper generates DataTypes based on the given name/value inputs. Values closes the channel once all input has been exhausted. You should always copy the mapper if you are using it concurrently.

type MegaByteType added in v0.3.0 

type MegaByteType struct {
	Key   string
	Value float64
}

MegaByteType represents a pair of key values in which the value represents bytes. It converts the value to MB.

func (MegaByteType) Equal added in v0.3.0 

func (m MegaByteType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal.

func (MegaByteType) Write added in v0.9.0 

func (m MegaByteType) Write(p io.Writer) (n int, err error)

Write includes both Key and Value.

type MemTypeMock added in v0.3.0 

type MemTypeMock struct {
	// contains filtered or unexported fields
}

MemTypeMock is the mocked version of memType.

func (MemTypeMock) IsByte added in v0.3.0 

func (m MemTypeMock) IsByte() bool

func (MemTypeMock) IsKiloByte added in v0.3.0 

func (m MemTypeMock) IsKiloByte() bool

func (MemTypeMock) IsMegaByte added in v0.3.0 

func (m MemTypeMock) IsMegaByte() bool

type StringType 

type StringType struct {
	Key   string
	Value string
}

StringType represents a pair of key values that the value is a string.

func (StringType) Equal added in v0.3.0 

func (s StringType) Equal(other DataType) bool

Equal compares both keys and values and returns true if they are equal.

func (StringType) Write added in v0.9.0 

func (s StringType) Write(p io.Writer) (n int, err error)

Write includes both Key and Value.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.