bench

package
v0.0.0-...-abed277 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 19, 2019 License: BSD-3-Clause Imports: 13 Imported by: 0

Documentation

Overview

bench contains benchmarks and common utilities useful to benchmarks

In order to write new benchmarks, one must satisfy the Benchmark interface in bench.go. In order to use the benchmark from pi, it needs a new file under tools/cmd which defines a cobra.Cmd - look at an existing benchmark's file to see what needs to be done.

When writing a new benchmark, there are a few things to keep in mind other than just implementing the interface:

1. The benchmark should modify its own configuration in its Init method based on the agentNum it is given. How it modifies is specific to the benchmark, but the idea is that it should make sense to call the benchmark with the same configuration, but multiple different agent numbers, and it should do useful work each time (i.e. not just setting the same bits, or running the same queries).

2. The Init method should do everything that needs to be done to get the benchmark to a runnable state - all code in run should be the stuff that we actually want to time.

3. The Run method does not need to report the total runtime - that is collected by calling code.

Files:

1. client.go contains pilosa client code which is shared by many benchmarks

2. errgroup.go contains the ErrGroup implementation copied from golang.org/x/ so as not to pull in a bunch of useless deps.

3. stats.go contains useful code for gathering stats about a series of timed operations.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Prettify 

func Prettify(m map[string]interface{}) map[string]interface{}

Recursively replaces elements of ugly types with their pretty wrappers

Types

type BasicQueryBenchmark 

type BasicQueryBenchmark struct {
	Name       string `json:"name"`
	MinRowID   int64  `json:"min-row-id"`
	Iterations int    `json:"iterations"`
	NumArgs    int    `json:"num-args"`
	Query      string `json:"query"`
	Index      string `json:"index"`
	Field      string `json:"field"`

	Logger *log.Logger `json:"-"`
}

BasicQueryBenchmark runs a query multiple times with increasing row ids.

func NewBasicQueryBenchmark 

func NewBasicQueryBenchmark() *BasicQueryBenchmark

NewBasicQueryBenchmark returns a new instance of BasicQueryBenchmark.

func (*BasicQueryBenchmark) Run 

func (b *BasicQueryBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark.

type Benchmark 

type Benchmark interface {
	Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)
}

Benchmark is an interface run benchmark components. Benchmarks should Marshal to valid JSON so that their configuration may be recorded with their results.

type DiagonalSetBitsBenchmark 

type DiagonalSetBitsBenchmark struct {
	Name        string `json:"name"`
	MinRowID    int    `json:"min-row-id"`
	MinColumnID int    `json:"min-column-id"`
	Iterations  int    `json:"iterations"`
	Index       string `json:"index"`
	Field       string `json:"field"`

	Logger *log.Logger `json:"-"`
}

DiagonalSetBitsBenchmark sets bits with increasing column id and row id.

func NewDiagonalSetBitsBenchmark 

func NewDiagonalSetBitsBenchmark() *DiagonalSetBitsBenchmark

NewDiagonalSetBitsBenchmark returns a new instance of DiagonalSetBitsBenchmark.

func (*DiagonalSetBitsBenchmark) Run 

func (b *DiagonalSetBitsBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark.

type HostSetup 

type HostSetup struct {
	Hosts         []string
	ClientOptions []pilosa.ClientOption
}

type ImportBenchmark 

type ImportBenchmark struct {
	Name         string `json:"name"`
	MinRowID     int64  `json:"min-row-id"`
	MinColumnID  int64  `json:"min-column-id"`
	MaxRowID     int64  `json:"max-row-id"`
	MaxColumnID  int64  `json:"max-column-id"`
	Index        string `json:"index"`
	Field        string `json:"field"`
	Iterations   int64  `json:"iterations"`
	Seed         int64  `json:"seed"`
	Distribution string `json:"distribution"`
	BufferSize   int    `json:"-"`

	Logger *log.Logger `json:"-"`
}

func NewImportBenchmark 

func NewImportBenchmark() *ImportBenchmark

NewImportBenchmark returns a new instance of ImportBenchmark.

func (*ImportBenchmark) RecordIterator 

func (b *ImportBenchmark) RecordIterator(seed int64) *RecordIterator

func (*ImportBenchmark) Run 

func (b *ImportBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the Import benchmark

type ImportRangeBenchmark 

type ImportRangeBenchmark struct {
	Name         string `json:"name"`
	MinValue     int64  `json:"min-value"`
	MinColumnID  int64  `json:"min-column-id"`
	MaxValue     int64  `json:"max-value"`
	MaxColumnID  int64  `json:"max-column-id"`
	Index        string `json:"index"`
	Field        string `json:"field"`
	Row          string `json:"row"`
	Iterations   int64  `json:"iterations"`
	Seed         int64  `json:"seed"`
	Distribution string `json:"distribution"`
	BufferSize   int    `json:"-"`

	Logger *log.Logger `json:"-"`
}

func NewImportRangeBenchmark 

func NewImportRangeBenchmark() *ImportRangeBenchmark

NewImportRangeBenchmark returns a new instance of ImportRangeBenchmark.

func (*ImportRangeBenchmark) Run 

func (b *ImportRangeBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark.

func (*ImportRangeBenchmark) ValueIterator 

func (b *ImportRangeBenchmark) ValueIterator(seed int64) *ValueIterator

type NumStats 

type NumStats struct {

	// NumZero counts the number of values that have been added which
	// are zero. This is a cheap, simple, replacement for more
	// sophisticated tracking of the distribution of the data that
	// let's us know if (e.g.) we have a bunch of queries doing
	// nothing because we're querying empty rows or something.
	NumZero int64   `json:"num-zero"`
	Min     int64   `json:"min"`
	Max     int64   `json:"max"`
	Mean    int64   `json:"mean"`
	Total   int64   `json:"total"`
	Num     int64   `json:"num"`
	All     []int64 `json:"all"`
	SaveAll bool    `json:"-"`
	// contains filtered or unexported fields
}

NumStats object helps track stats. This and Stats (which was originally made specifically for time) should probably be unified.

func NewNumStats 

func NewNumStats() *NumStats

NewNumStats gets a NumStats object.

func (*NumStats) Add 

func (s *NumStats) Add(td int64)

Add adds a new value to the stats object.

func (*NumStats) Combine 

func (s *NumStats) Combine(other *NumStats)

type PrettyDuration 

type PrettyDuration time.Duration

wrapper type to force human-readable JSON output

func (PrettyDuration) MarshalJSON 

func (d PrettyDuration) MarshalJSON() ([]byte, error)

MarshalJSON returns a nicely formatted duration, instead of it just being treated like an int.

type QueryBenchmark 

type QueryBenchmark struct {
	Name       string `json:"name"`
	Query      string `json:"query"`
	Index      string `json:"index"`
	Iterations int    `json:"iterations"`

	Logger *log.Logger `json:"-"`
}

func NewQueryBenchmark 

func NewQueryBenchmark() *QueryBenchmark

func (*QueryBenchmark) Run 

func (b *QueryBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark.

type QueryGenerator 

type QueryGenerator struct {
	// contains filtered or unexported fields
}

QueryGenerator holds the configuration and state for randomly generating queries.

func NewQueryGenerator 

func NewQueryGenerator(index *pilosa.Index, field *pilosa.Field, seed int64) *QueryGenerator

NewQueryGenerator returns a new QueryGenerator.

func (*QueryGenerator) Random 

func (g *QueryGenerator) Random(maxN, depth, maxargs int, idmin, idmax uint64) pilosa.PQLQuery

Random returns a randomly generated query.

func (*QueryGenerator) RandomBitmapCall 

func (g *QueryGenerator) RandomBitmapCall(depth, maxargs int, idmin, idmax uint64) *pilosa.PQLRowQuery

RandomBitmapCall returns a randomly generate query which returns a bitmap.

func (*QueryGenerator) RandomRange 

func (g *QueryGenerator) RandomRange(numArg int, idmin, idmax uint64) *pilosa.PQLRowQuery

func (*QueryGenerator) RandomRangeQuery 

func (g *QueryGenerator) RandomRangeQuery(depth, maxargs int, idmin, idmax uint64) pilosa.PQLQuery

RandomRangeQuery returns a randomly generated sum or range query.

func (*QueryGenerator) RandomSum 

func (g *QueryGenerator) RandomSum(depth, maxargs int, idmin, idmax uint64) pilosa.PQLQuery

RandomSum returns a randomly generated sum query.

func (*QueryGenerator) RandomTopN 

func (g *QueryGenerator) RandomTopN(maxN, depth, maxargs int, idmin, idmax uint64) *pilosa.PQLRowQuery

RandomTopN returns a randomly generated TopN query.

func (*QueryGenerator) RangeCall 

func (g *QueryGenerator) RangeCall(idmin, idmax uint64) *pilosa.PQLRowQuery

type RandomQueryBenchmark 

type RandomQueryBenchmark struct {
	Name       string `json:"name"`
	MaxDepth   int    `json:"max-depth"`
	MaxArgs    int    `json:"max-args"`
	MaxN       int    `json:"max-n"`
	MinRowID   int64  `json:"min-row-id"`
	MaxRowID   int64  `json:"max-row-id"`
	Iterations int    `json:"iterations"`
	Seed       int64  `json:"seed"`
	Index      string `json:"index"`
	Field      string `json:"field"`

	Logger *log.Logger `json:"-"`
}

RandomQueryBenchmark queries randomly and deterministically based on a seed.

func NewRandomQueryBenchmark 

func NewRandomQueryBenchmark() *RandomQueryBenchmark

NewRandomQueryBenchmark returns a new instance of RandomQueryBenchmark.

func (*RandomQueryBenchmark) Run 

func (b *RandomQueryBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the RandomQuery benchmark

type RandomSetBenchmark 

type RandomSetBenchmark struct {
	Name          string `json:"name"`
	MinRowID      int64  `json:"min-row-id"`
	MaxRowID      int64  `json:"max-row-id"`
	MinColumnID   int64  `json:"min-column-id"`
	MaxColumnID   int64  `json:"max-column-id"`
	Iterations    int    `json:"iterations"`
	BatchSize     int    `json:"batch-size"`
	Seed          int64  `json:"seed"`
	NumAttrs      int    `json:"num-attrs"`
	NumAttrValues int    `json:"num-attr-values"`
	Index         string `json:"index"`
	Field         string `json:"field"`

	Logger *log.Logger `json:"-"`
}

RandomSetBenchmark sets bits randomly and deterministically based on a seed.

func NewRandomSetBenchmark 

func NewRandomSetBenchmark() *RandomSetBenchmark

NewRandomSetBenchmark returns a new instance of RandomSetBenchmark.

func (*RandomSetBenchmark) Run 

func (b *RandomSetBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark.

type RangeQueryBenchmark 

type RangeQueryBenchmark struct {
	Name       string `json:"name"`
	MaxDepth   int    `json:"max-depth"`
	MaxArgs    int    `json:"max-args"`
	MaxN       int    `json:"max-n"`
	MinRange   int64  `json:"min-range"`
	MaxRange   int64  `json:"max-range"`
	Iterations int    `json:"iterations"`
	Seed       int64  `json:"seed"`
	Frame      string `json:"frame"`
	Index      string `json:"index"`
	Field      string `json:"field"`
	QueryType  string `json:"type"`

	Logger *log.Logger `json:"-"`
}

RangeQueryBenchmark runs Range query randomly.

func NewRangeQueryBenchmark 

func NewRangeQueryBenchmark() *RangeQueryBenchmark

NewRangeQueryBenchmark returns a new instance of RangeQueryBenchmark.

func (*RangeQueryBenchmark) Run 

func (b *RangeQueryBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark.

type RecordIterator 

type RecordIterator struct {
	// contains filtered or unexported fields
}

func NewRecordIterator 

func NewRecordIterator() *RecordIterator

func (*RecordIterator) NextRecord 

func (itr *RecordIterator) NextRecord() (pilosa.Record, error)

type Result 

type Result struct {
	Stats         *Stats                  `json:"stats"`
	Responses     []*pilosa.QueryResponse `json:"responses"`
	Extra         map[string]interface{}  `json:"extra"`
	AgentNum      int                     `json:"agentnum"`
	PilosaVersion string                  `json:"pilosa-version"`
	Configuration interface{}             `json:"configuration"`

	// Error exists so that errors can be correctly marshalled to JSON. It is set using Result.err.Error()
	Error string `json:"error,omitempty"`
}

Results holds the output from the run of a benchmark - the Benchmark's Run() method may set Stats, Responses, and Extra, and the RunBenchmark helper function will set the Duration, AgentNum, PilosaVersion, and Configuration. Either may set Error if there is an error. The structure of Result assumes that most benchmarks will run multiple queries and track statistics about how long each one takes. The Extra field is for benchmarks which either do not fit this model, or want to return additional information not covered by Stats and Responses.

func NewResult 

func NewResult() *Result

NewResult intializes and returns a Result.

func (*Result) Add 

func (r *Result) Add(d time.Duration, resp *pilosa.QueryResponse)

Add adds the duration to the Result's Stats object. If resp is non-nil, it also adds it to the slice of responses.

type SliceWidthBenchmark 

type SliceWidthBenchmark struct {
	Name       string  `json:"name"`
	Index      string  `json:"index"`
	Field      string  `json:"field"`
	BitDensity float64 `json:"bit-density"`
	SliceWidth int64   `json:"slice-width"`
	SliceCount int64   `json:"slice-count"`

	Logger *log.Logger `json:"-"`
}

SliceWidthBenchmark helps importing data based on slice-width and data density. a single slice on query time.

func NewSliceWidthBenchmark 

func NewSliceWidthBenchmark() *SliceWidthBenchmark

NewSliceWidthBenchmark creates slice width benchmark.

func (*SliceWidthBenchmark) Run 

func (b *SliceWidthBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark to import data.

type Stats 

type Stats struct {
	Min     time.Duration   `json:"min"`
	Max     time.Duration   `json:"max"`
	Mean    time.Duration   `json:"mean"`
	Total   time.Duration   `json:"total-time"`
	Num     int64           `json:"num"`
	All     []time.Duration `json:"all"`
	SaveAll bool            `json:"-"`
	// contains filtered or unexported fields
}

Stats object helps track timing stats.

func NewStats 

func NewStats() *Stats

NewStats gets a Stats object.

func (*Stats) Add 

func (s *Stats) Add(td time.Duration)

Add adds a new time to the stats object.

func (*Stats) Combine 

func (s *Stats) Combine(other *Stats)

type TPSBenchmark 

type TPSBenchmark struct {
	Name        string   `json:"name"`
	Intersect   bool     `json:"intersect" help:"If true, include Intersect queries in benchmark."`
	Union       bool     `json:"union" help:"If true, include Union queries in benchmark."`
	Difference  bool     `json:"difference" help:"If true, include Difference queries in benchmark."`
	Xor         bool     `json:"xor" help:"If true, include XOR queries in benchmark."`
	Fields      []string `json:"fields" help:"Comma separated list of fields. If blank, use all fields in index schema."`
	MinRowID    int64    `json:"min-row-id" help:"Minimum row ID to use in queries."`
	MaxRowID    int64    `json:"max-row-id" help:"Max row ID to use in queries. If 0, determine max available."`
	Index       string   `json:"index" help:"Index to use. If blank, one is chosen randomly from the schema."`
	Concurrency int      `json:"concurrency" help:"Run this many goroutines concurrently." short:"y"`
	Iterations  int      `json:"iterations" help:"Each goroutine will perform this many queries."`

	Logger *log.Logger `json:"-"`
}

func NewTPSBenchmark 

func NewTPSBenchmark() *TPSBenchmark

func (*TPSBenchmark) Run 

func (b *TPSBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the benchmark.

type ValueIterator 

type ValueIterator struct {
	// contains filtered or unexported fields
}

func NewValueIterator 

func NewValueIterator() *ValueIterator

func (*ValueIterator) NextRecord 

func (itr *ValueIterator) NextRecord() (pilosa.Record, error)

type ZipfBenchmark 

type ZipfBenchmark struct {
	Name           string  `json:"name"`
	MinRowID       int64   `json:"min-row-id"`
	MinColumnID    int64   `json:"min-column-id"`
	MaxRowID       int64   `json:"max-row-id"`
	MaxColumnID    int64   `json:"max-column-id"`
	Iterations     int     `json:"iterations"`
	Seed           int64   `json:"seed"`
	Index          string  `json:"index"`
	Field          string  `json:"field"`
	RowExponent    float64 `json:"row-exponent"`
	RowRatio       float64 `json:"row-ratio"`
	ColumnExponent float64 `json:"column-exponent"`
	ColumnRatio    float64 `json:"column-ratio"`
	Operation      string  `json:"operation"`

	Logger *log.Logger `json:"-"`
}

ZipfBenchmark sets random bits according to the Zipf-Mandelbrot distribution. This distribution accepts two parameters, Exponent and Ratio, for both rows and columns. It also uses PermutationGenerator to permute IDs randomly.

func NewZipfBenchmark 

func NewZipfBenchmark() *ZipfBenchmark

NewZipfBenchmark returns a new instance of ZipfBenchmark.

func (*ZipfBenchmark) Run 

func (b *ZipfBenchmark) Run(ctx context.Context, client *pilosa.Client, agentNum int) (*Result, error)

Run runs the Zipf benchmark

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.