gompet

README

Gompet - Go Multi-purpose Performance Evaluation Tool

Gompet is a multi-purpose performance evaluation tool which can quickly send thousands of commands to servers using different clients and varying input patterns. Command templates enable easy comparison of different client protocols and/or client implementations with the same input data.

The tool has been optimized for maximum throughput, a single computer can easily send tens of thousands of commands per second using multiple parallel clients (given enough network bandwidth).

The tool reports the latency percentiles for the execution, counts of command results, as well as counts of different errors received (if any). The percentiles can also be reported at regular intervals for long running tests.

Gompet currently includes a standard HTTP client, optimized FastHTTP client and SQL client for PostgreSQL.

It is easy to add clients for new protocols, and utilize the input variable expansion, worker pool management and statistics reporting from the framework. The library github.com/jkarjala/gompet can also be imported to applications outside of this repo.

Installation

64 bit binary releases:

• Windows: gompet-windows-amd64.zip
• Linux: gompet-linux-amd64.zip
• Mac: gompet-darwin-amd64.zip

Installation from sources, assuming go executable and GOPATH/bin in your PATH:

go install -i github.com/jkarjala/gompet/...

General usage

The clients receive commands from command line (each argument is a command, use quotes if command has whitespace), or from a file with paramter -f (each line is a command). Use "-" as filename for stdin. The commands are distributed to clients in random order, therefore they should not depend on each other if multiple clients are used.

Alternatively, a command template with variables $1 - $9 can be given with -t option. In this case, the input file/stdin must contain tab-separated-values to be inserted in the variables in the template to construct the final command. This is very useful when testing different client protocols (e.g. http vs sql), the same variable input data can be used with different clients, only the template needs to change.

See data folder for a few input file examples. For best throughput, use files with more than 500 lines (like the number-word examples), otherwise the file open/close overhead skews the results.

The one letter options (and pprof) in usages below are implemented by the framework and thus common to all clients, the long options are specific to the clients.

By default the tool sends commands as quickly as it can using a single client. The number of parallel clients can be configured with the -c option. Optionally the request rate of each client can be limited to N commands/second using the -R option. The rate limit is accurate up to 500 on Windows and up to 4000 on Linux, increase the number of clients for more load. The load can be ramped up slowly with the -D option, a new client is started once per given duration.

HTTP and FastHTTP Clients

The gompet-http uses the standard Go http library, while the gompet-fasthttp uses the fasthttp library. Fasthttp is 20-50% faster but only supports HTTP/1 and reads the whole body in memory, while the gompet-http reads and discards body unless -v option is given.

gompet-http [options] ['cmd 1' 'cmd 2' ...]
  -D duration
        Delay start of each client by given duration, e.g 5s for 5 seconds
  -P    Report progress once a second
  -R int
        Rate limit each client to N queries/sec (accuracy depends on OS)
  -S int
        Show and reset percentiles every N seconds, 0 shows at end
  -auth string
        HTTP Authorization header
  -c int
        Number of parallel clients executing commands (default 1)
  -content-type string
        HTTP body content type (default "application/json")
  -d duration
        Test until given duration elapses, e.g 5m for 5 minutes
  -f string
        Input file name, stdin if '-'
  -pprof
        Enable pprof web server
  -r int
        Repeat the input N times, does not work with stdin (default 1)
  -t string
        Command template, $1-$9 refers to tab-separated columns in input
  -timeout int
        HTTP Client timeout in seconds (default 10)
  -v    Verbose logging

The Command syntax for http clients is:

HTTP-VERB URL Body-as-single-line-if-needed

Examples (after installation)

Run a single HTTP PUT command via command line with geompet-fasthttp and show verbose output:

gompet-fasthttp -v 'PUT http://httpbin.org/put { "some" : "put" }'

Run HTTP commands from http.txt with verbose output (start the gompet-httpserver first):

gompet-fasthttp -f testdata/http.txt -v

Run HTTP commands with template and the URLs from urls.tsv, using 20 parallel clients, each sending at rate of 50 requests/second, repeating the input file 2000 times, report statistics every second:

gompet-fasthttp -f testdata/urls.tsv -t 'GET $1' -c 20 -R 50 -r 2000 -S 1

SQL Client

gompet-sql [options] ['cmd 1' 'cmd 2' ...]
  -D duration
        Delay start of each client by given duration, e.g 5s for 5 seconds
  -P    Report progress once a second
  -R int
        Rate limit each client to N queries/sec (accuracy depends on OS)
  -S int
        Show and reset percentiles every N seconds, 0 shows at end
  -c int
        Number of parallel clients executing commands (default 1)
  -d duration
        Test until given duration elapses, e.g 5m for 5 minutes
  -discard
        Discard result set with mimimal memory allocation
  -driver string
        Database driver, 'postgres' or 'mysql'
  -f string
        Input file name, stdin if '-'
  -pprof
        Enable pprof web server
  -r int
        Repeat the input N times, does not work with stdin (default 1)
  -t string
        Command template, $1-$9 refers to tab-separated columns in input
  -tx int
        Batch N commands in one transaction, does not work with SELECTs
  -url string
        SQL Connect URL, e.g. postgres://user:pass@host/db?sslmode=disable
  -v    Verbose logging

The Connect URL syntax is

postgres://user:pass@hostname/db?sslmode=disable

The Command syntax for SQL client is a single SQL statement. A template with variables is used as a prepared statement instead of simple text substitution. Unlike SQL prepared statement, the template may contain multiple references to the same variable, they will be duplicated to make the prepared statement work.

Licence

Gompet Copyright 2019-2020 Jari Karjala.

Gompet is licensed under GNU General Public License v3.

Documentation

Overview

Package gompet (Go Multi-purpose Performance Evaluation Tool) provides a multi-core benchmarking skeleton for use in different benchmarking clients. See sub-folders for some example command line clients.

Index

• func ClientRoutine(id int, client Client)
• func CollectResults(results *Results)
• func FeedArgs(reader *csv.Reader) error
• func FeedCmds(reader io.Reader) error
• func FormatDecimals(v float64) string
• func LaunchClients(clientFactory ClientFactory) error
• func OpenInput() (io.Reader, *os.File)
• func Percentile(input []float64, percent float64) (percentile float64)
• func PrintMap(heading string, m map[string]int64)
• func PrintPercentile(input []float64, percent float64)
• func Run(clientFactory ClientFactory)
• func Setup()
• type Client
• type ClientConfig
• type ClientFactory
• type ClientInput
• type ClientResult
• type Results
  • func Exec(clientFactory ClientFactory) *Results
  • func NewResults(progress bool, periodicStats int) *Results
  • func (results *Results) PercentileRow() string
  • func (results *Results) PercentileRowHeader() string
  • func (results *Results) Report()
  • func (results *Results) Update(res *ClientResult)
• type VarTemplate
  • func Parse(str string) *VarTemplate
  • func (t *VarTemplate) Expand(args []string) string

Functions

func ClientRoutine

func ClientRoutine(id int, client Client)

ClientRoutine is the processing function for data processing

func CollectResults

func CollectResults(results *Results)

CollectResults listens for processing results and updates the results

func FeedArgs

func FeedArgs(reader *csv.Reader) error

FeedArgs feeds the clients with arguments to patch to template

func FeedCmds

func FeedCmds(reader io.Reader) error

FeedCmds feeds the clients with command lines

func FormatDecimals

func FormatDecimals(v float64) string

FormatDecimals returns the value as a string with "nice" number of decimals

func LaunchClients

func LaunchClients(clientFactory ClientFactory) error

LaunchClients creates clients and starts the go routines for data processing

func OpenInput

func OpenInput() (io.Reader, *os.File)

OpenInput opens the input file for reading

func Percentile

func Percentile(input []float64, percent float64) (percentile float64)

Percentile finds the value at given percent in the input which must be pre-sorted Adapted from https://github.com/montanaflynn/stats/blob/master/percentile.go

func PrintMap

func PrintMap(heading string, m map[string]int64)

PrintMap outputs heading and map key values in increasing alphabetical order of keys

func PrintPercentile

func PrintPercentile(input []float64, percent float64)

PrintPercentile outputs the percent's percentile from input

func Run

func Run(clientFactory ClientFactory)

Run function executes the commands and reports results

func Setup

func Setup()

Setup configures the execution from the command line flags

Types

type Client

type Client interface {
	RunCommand(in *ClientInput) *ClientResult
	Term()
}

Client is the interface the client must implement

type ClientConfig

type ClientConfig struct {
	ID         int
	Template   *VarTemplate
	NumClients int
	Verbose    bool
}

ClientConfig is passed to the client factory when a new instance is created

type ClientFactory

type ClientFactory func(config ClientConfig) (Client, error)

ClientFactory creates a client instance

type ClientInput

type ClientInput struct {
	Cmd  string   // command to execute, nil if template
	Args []string // variables values for template if cmd==nil
}

ClientInput is given to client once per input command

type ClientResult

type ClientResult struct {
	Res  string  // result, count of each separate value is reported
	Time float64 // execution time in seconds, percentiles are reported
	Err  error   // error result or nil, count of each error is reported (if any)
}

ClientResult is returned from client after processing one input line

type Results

type Results struct {
	Start         time.Time
	LastProgress  time.Time
	LastStats     time.Time
	Elapsed       float64
	PeriodicStats int
	Progress      bool
	Count         int64
	LastCount     int64
	Times         []float64
	Results       map[string]int64
	Errs          map[string]int64
}

Results collects overall statistics

func Exec

func Exec(clientFactory ClientFactory) *Results

Exec executes the commands, global flags must have been set up before this

func NewResults

func NewResults(progress bool, periodicStats int) *Results

NewResults returns a newly initialized Results

func (*Results) PercentileRow

func (results *Results) PercentileRow() string

PercentileRow formats a row of percentiles from results

func (*Results) PercentileRowHeader

func (results *Results) PercentileRowHeader() string

PercentileRowHeader returns the header for the stats rows

func (*Results) Report

func (results *Results) Report()

Report results to stdout

func (*Results) Update

func (results *Results) Update(res *ClientResult)

Update results from one Run

type VarTemplate

type VarTemplate struct {
	Pieces  []string        // fixed pieces
	Indices []int           // arg indices for variables between pieces
	Builder strings.Builder // builder for Expand functions
}

VarTemplate holds the pieces of the template in optimal format This is not safe for concurrent use!

func Parse

func Parse(str string) *VarTemplate

Parse parses the given string to an efficient template

func (*VarTemplate) Expand

func (t *VarTemplate) Expand(args []string) string

Expand constructs string with variables replaced with given arguments