event

Documentation

Overview

Package event aggregates MySQL log and Perfomance Schema events into query classes and calculates basic statistics for class metrics like max Query_time. Event aggregation into query classes is the foundation of MySQL log file and Performance Schema analysis.

An event is a query like "SELECT col FROM t WHERE id = 1", some metrics like Query_time (slow log) or SUM_TIMER_WAIT (Performance Schema), and other metadata like default database, timestamp, etc. Events are grouped into query classes by fingerprinting the query (see percona.com/go-mysql/query/), then checksumming the fingerprint which yields a 16-character hexadecimal value called the class ID. As events are added to a class, metric values are saved. When there are no more events, the class is finalized to compute the statistics for all metrics.

There are two types of classes: global and per-query. A global class contains all events added to it, regardless of fingerprint or class ID. This is used, for example, to aggregate all events in a log file. A per-query class contains events with the same fingerprint and class ID. This is only enforced by convention, so be careful not to mix events from different classes. This is used, for example, to aggregate unique queries in a log file, then sort the classes by some metric, like max Query_time, to find the slowest query relative to a global class for the same set of events.

Index

• Constants
• func Float64(v float64) *float64
• func Float64Value(v *float64) float64
• func Uint64(v uint64) *uint64
• func Uint64Value(v *uint64) uint64
• type Aggregator
  • func NewAggregator(samples bool, utcOffset time.Duration, outlierTime float64) *Aggregator
  • func (a *Aggregator) AddEvent(event *log.Event, id, user, host, db, server, fingerprint string)
  • func (a *Aggregator) Finalize() Result
• type BoolStats
• type Class
  • func NewClass(id, user, host, db, server, fingerprint string, sample bool) *Class
  • func (c *Class) AddClass(newClass *Class)
  • func (c *Class) AddEvent(e *log.Event, outlier bool)
  • func (c *Class) Finalize(rateLimit uint)
• type Example
• type Metrics
  • func NewMetrics() *Metrics
  • func (m *Metrics) AddEvent(e *log.Event, outlier bool)
  • func (m *Metrics) Finalize(rateLimit uint, totalQueries uint)
• type NumberStats
• type Result
• type TimeStats

Constants

const (
	// MaxExampleBytes defines to how many bytes truncate a query.
	MaxExampleBytes = 2 * 1024 * 10

	// TruncatedExampleSuffix is added to truncated query.
	TruncatedExampleSuffix = "..."
)

Functions

func Float64

func Float64(v float64) *float64

Float64 returns a pointer to the float64 value passed in.

func Float64Value

func Float64Value(v *float64) float64

Float64Value returns the value of the float64 pointer passed in or 0 if the pointer is nil.

func Uint64

func Uint64(v uint64) *uint64

Uint64 returns a pointer to the uint64 value passed in.

func Uint64Value

func Uint64Value(v *uint64) uint64

Uint64Value returns the value of the uint64 pointer passed in or 0 if the pointer is nil.

Types

type Aggregator

type Aggregator struct {
	// contains filtered or unexported fields
}

An Aggregator groups events by class ID. When there are no more events, a call to Finalize computes all metric statistics and returns a Result.

func NewAggregator

func NewAggregator(samples bool, utcOffset time.Duration, outlierTime float64) *Aggregator

NewAggregator returns a new Aggregator. outlierTime is https://www.percona.com/doc/percona-server/5.5/diagnostics/slow_extended_55.html#slow_query_log_always_write_time

func (*Aggregator) AddEvent

func (a *Aggregator) AddEvent(event *log.Event, id, user, host, db, server, fingerprint string)

AddEvent adds the event to the aggregator, automatically creating new classes as needed.

func (*Aggregator) Finalize

func (a *Aggregator) Finalize() Result

Finalize calculates all metric statistics and returns a Result. Call this function when done adding events to the aggregator.

type BoolStats

type BoolStats struct {
	Cnt uint64
	Sum uint64 // %true = Sum/Cnt
	// contains filtered or unexported fields
}

BoolStats are boolean-based metrics like QC_Hit and Filesort.

type Class

type Class struct {
	Id                   string // 32-character hex checksum of fingerprint
	User                 string
	Host                 string
	Db                   string
	Server               string
	LabelsKey            []string
	LabelsValue          []string
	Fingerprint          string   // canonical form of query: values replaced with "?"
	Metrics              *Metrics // statistics for each metric, e.g. max Query_time
	TotalQueries         uint     // total number of queries in class
	UniqueQueries        uint     // unique number of queries in class
	Example              *Example `json:",omitempty"` // sample query with max Query_time
	NumQueriesWithErrors float32
	ErrorsCode           []uint64
	ErrorsCount          []uint64
	// contains filtered or unexported fields
}

A Class represents all events with the same fingerprint and class ID. This is only enforced by convention, so be careful not to mix events from different classes.

func NewClass

func NewClass(id, user, host, db, server, fingerprint string, sample bool) *Class

NewClass returns a new Class for the class ID and fingerprint. If sample is true, the query with the greatest Query_time is saved.

func (*Class) AddClass

func (c *Class) AddClass(newClass *Class)

AddClass adds a Class to the current class. This is used with Performance Schema which returns pre-aggregated classes instead of events.

func (*Class) AddEvent

func (c *Class) AddEvent(e *log.Event, outlier bool)

AddEvent adds an event to the query class.

func (*Class) Finalize

func (c *Class) Finalize(rateLimit uint)

Finalize calculates all metric statistics. Call this function when done adding events to the class.

type Example

type Example struct {
	QueryTime float64 // Query_time
	Db        string  // Schema: <db> or USE <db>
	Query     string  // truncated to MaxExampleBytes
	Size      int     `json:",omitempty"` // Original size of query.
	Ts        string  `json:",omitempty"` // in MySQL time zone
}

A Example is a real query and its database, timestamp, and Query_time. If the query is larger than MaxExampleBytes, it is truncated and TruncatedExampleSuffix is appended.

type Metrics

type Metrics struct {
	TimeMetrics   map[string]*TimeStats   `json:",omitempty"`
	NumberMetrics map[string]*NumberStats `json:",omitempty"`
	BoolMetrics   map[string]*BoolStats   `json:",omitempty"`
}

Metrics encapsulate the metrics of an event like Query_time and Rows_sent.

func NewMetrics

func NewMetrics() *Metrics

NewMetrics returns a pointer to an initialized Metrics structure.

func (*Metrics) AddEvent

func (m *Metrics) AddEvent(e *log.Event, outlier bool)

AddEvent saves all the metrics of the event.

func (*Metrics) Finalize

func (m *Metrics) Finalize(rateLimit uint, totalQueries uint)

Finalize calculates the statistics of the added metrics. Call this function when done adding events.

type NumberStats

type NumberStats struct {
	Cnt uint64
	Sum uint64
	Min *uint64 `json:",omitempty"`
	P99 *uint64 `json:",omitempty"` // 99th percentile
	Max *uint64 `json:",omitempty"`
	// contains filtered or unexported fields
}

NumberStats are integer-based metrics like Rows_sent and Merge_passes.

type Result

type Result struct {
	Global    *Class            // all classes
	Class     map[string]*Class // keyed on class ID
	RateLimit uint
	Error     string
}

A Result contains a global class and per-ID classes with finalized metric statistics. The classes are keyed on class ID.

type TimeStats

type TimeStats struct {
	Cnt uint64
	Sum float64
	Min *float64 `json:",omitempty"`
	P99 *float64 `json:",omitempty"` // 99th percentile
	Max *float64 `json:",omitempty"`
	// contains filtered or unexported fields
}

TimeStats are microsecond-based metrics like Query_time and Lock_time.