mapr

Documentation

Index

• Constants
• type AggregateOperation
• type AggregateSet
  • func NewAggregateSet() *AggregateSet
  • func (s *AggregateSet) Aggregate(key string, agg AggregateOperation, value string, clientAggregation bool) (err error)
  • func (s *AggregateSet) Merge(query *Query, set *AggregateSet) error
  • func (s *AggregateSet) Serialize(groupKey string, ch chan<- string, stop chan struct{})
  • func (s *AggregateSet) String() string
• type GlobalGroupSet
  • func NewGlobalGroupSet() *GlobalGroupSet
  • func (g *GlobalGroupSet) IsEmpty() bool
  • func (g *GlobalGroupSet) Merge(query *Query, group *GroupSet) error
  • func (g *GlobalGroupSet) MergeNoblock(query *Query, group *GroupSet) (bool, error)
  • func (g *GlobalGroupSet) NumSets() int
  • func (g *GlobalGroupSet) Result(query *Query) (string, int, error)
  • func (g *GlobalGroupSet) String() string
  • func (g *GlobalGroupSet) SwapOut() *GroupSet
  • func (g *GlobalGroupSet) WriteResult(query *Query) error
• type GroupSet
  • func NewGroupSet() *GroupSet
  • func (g *GroupSet) GetSet(groupKey string) *AggregateSet
  • func (g *GroupSet) InitSet()
  • func (g *GroupSet) Result(query *Query) (string, int, error)
  • func (g *GroupSet) Serialize(ch chan<- string, stop chan struct{})
  • func (g *GroupSet) String() string
  • func (g *GroupSet) WriteResult(query *Query) error
• type Query
  • func NewQuery(queryStr string) (*Query, error)
  • func (q Query) String() string
  • func (q *Query) WhereClause(fields map[string]string) bool
• type QueryOperation

Constants

const (
	UndefWhereType whereType = iota
	Field          whereType = iota
	String         whereType = iota
	Float          whereType = iota
)

The possible field types.

Types

type AggregateOperation

type AggregateOperation int

AggregateOperation is to specify the aggregate operation type.

const (
	UndefAggregateOperation AggregateOperation = iota
	Count                   AggregateOperation = iota
	Sum                     AggregateOperation = iota
	Min                     AggregateOperation = iota
	Max                     AggregateOperation = iota
	Last                    AggregateOperation = iota
	Avg                     AggregateOperation = iota
	Len                     AggregateOperation = iota
)

Aggregate operation types

type AggregateSet

type AggregateSet struct {
	Samples int
	FValues map[string]float64
	SValues map[string]string
}

AggregateSet represents aggregated key/value pairs from the MAPREDUCE log lines. These could be either string values or float values.

func NewAggregateSet

func NewAggregateSet() *AggregateSet

NewAggregateSet creates a new empty aggregate set.

func (*AggregateSet) Aggregate

func (s *AggregateSet) Aggregate(key string, agg AggregateOperation, value string, clientAggregation bool) (err error)

Aggregate data to the aggregate set.

func (*AggregateSet) Merge

func (s *AggregateSet) Merge(query *Query, set *AggregateSet) error

Merge one aggregate set into this one.

func (*AggregateSet) Serialize

func (s *AggregateSet) Serialize(groupKey string, ch chan<- string, stop chan struct{})

Serialize the aggregate set so it can be sent over the wire.

func (*AggregateSet) String

func (s *AggregateSet) String() string

String representation of aggregate set.

type GlobalGroupSet

type GlobalGroupSet struct {
	GroupSet
	// contains filtered or unexported fields
}

GlobalGroupSet is used on the dtail client to merge multiple group sets (one group set per remote server) to one single global group set.

func NewGlobalGroupSet

func NewGlobalGroupSet() *GlobalGroupSet

NewGlobalGroupSet creates a new empty global group set.

func (*GlobalGroupSet) IsEmpty

func (g *GlobalGroupSet) IsEmpty() bool

IsEmpty determines whether the global group set has any data in it.

func (*GlobalGroupSet) Merge

func (g *GlobalGroupSet) Merge(query *Query, group *GroupSet) error

Merge (blocking) a group set into the global group set.

func (*GlobalGroupSet) MergeNoblock

func (g *GlobalGroupSet) MergeNoblock(query *Query, group *GroupSet) (bool, error)

MergeNoblock merges (non-blocking) a group set into the global group set.

func (*GlobalGroupSet) NumSets

func (g *GlobalGroupSet) NumSets() int

NumSets determines the number of sets.

func (*GlobalGroupSet) Result

func (g *GlobalGroupSet) Result(query *Query) (string, int, error)

Result returns the result of the mapreduce aggregation as a string.

func (*GlobalGroupSet) String

func (g *GlobalGroupSet) String() string

String representation of the global group set.

func (*GlobalGroupSet) SwapOut

func (g *GlobalGroupSet) SwapOut() *GroupSet

SwapOut teturn the underlying group set and create a new empty one, so that the global group set is empty again and can aggregate new data.

func (*GlobalGroupSet) WriteResult

func (g *GlobalGroupSet) WriteResult(query *Query) error

WriteResult writes the result of a mapreduce aggregation to an outfile.

type GroupSet

type GroupSet struct {
	// contains filtered or unexported fields
}

GroupSet represents a map of aggregate sets. The group sets are requierd by the "group by" mapr clause, whereas the group set map keys are the values of the "group by" arguments. E.g. "group by $cid" would create one aggregate set and one map entry per customer id.

func NewGroupSet

func NewGroupSet() *GroupSet

NewGroupSet returns a new empty group set.

func (*GroupSet) GetSet

func (g *GroupSet) GetSet(groupKey string) *AggregateSet

GetSet gets a specific aggregate set from the group set.

func (*GroupSet) InitSet

func (g *GroupSet) InitSet()

InitSet makes the group set empty (initialize).

func (*GroupSet) Result

func (g *GroupSet) Result(query *Query) (string, int, error)

Result returns a nicely formated result of the query from the group set.

func (*GroupSet) Serialize

func (g *GroupSet) Serialize(ch chan<- string, stop chan struct{})

Serialize the group set (e.g. to send it over the wire).

func (*GroupSet) String

func (g *GroupSet) String() string

String representation of the group set.

func (*GroupSet) WriteResult

func (g *GroupSet) WriteResult(query *Query) error

WriteResult writes the result to an outfile.

type Query

type Query struct {
	Select       []selectCondition
	Table        string
	Where        []whereCondition
	GroupBy      []string
	OrderBy      string
	ReverseOrder bool
	GroupKey     string
	Interval     time.Duration
	Limit        int
	Outfile      string
	RawQuery     string
	// contains filtered or unexported fields
}

Query represents a parsed mapr query.

func NewQuery

func NewQuery(queryStr string) (*Query, error)

NewQuery returns a new mapreduce query.

func (Query) String

func (q Query) String() string

func (*Query) WhereClause

func (q *Query) WhereClause(fields map[string]string) bool

WhereClause interprets the where clause of the mapreduce query.

type QueryOperation

type QueryOperation int

QueryOperation determines the mapreduce operation.

const (
	UndefQueryOperation QueryOperation = iota
	StringEq            QueryOperation = iota
	StringNe            QueryOperation = iota
	StringContains      QueryOperation = iota
	FloatOperation      QueryOperation = iota
	FloatEq             QueryOperation = iota
	FloatNe             QueryOperation = iota
	FloatLt             QueryOperation = iota
	FloatLe             QueryOperation = iota
	FloatGt             QueryOperation = iota
	FloatGe             QueryOperation = iota
)

The possible mapreduce operation.s