main

command
v0.14.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 25, 2019 License: Apache-2.0 Imports: 22 Imported by: 0

README

Benchmarking

Setup m3db and coordinator:

  1. Run m3dbnode

     $ cd $GOPATH/src/github.com/m3db/m3/
     $ sudo ./bin/m3dbnode -f $GOPATH/src/github.com/m3db/m3/src/query/benchmark/benchmarker/main/m3dbnode-local-config.yaml
    
  2. Run m3coordinator

     $ cd $GOPATH/src/github.com/m3db/m3/src/query
     $ make services
     $ ./bin/m3coordinator --config.file=$GOPATH/src/github.com/m3db/m3/src/query/benchmark/benchmarker/main/m3coordinator-local-config.yml
    

This new benchmarker requires a list of files in opentsdb format. These can be generated using the influxdb-comparisons tool, and may need to be regenerated periodically if the data gets too stale:

  1. Pull down benraskin92's fork of influxdb-comparisons to generate data:

    $ git clone https://github.com/benraskin92/influxdb-comparisons.git $GOPATH/src/github.com/
    $ cd cmd/bulk_data_gen
    $ go build
    $ cd $GOPATH/src/github.com/m3db/m3/src/query/benchmark/benchmarker/main/
    $ ./data_gen.sh
    

    (NB: the path during these steps and/or in data_gen.sh may not be correct depending on $GOPATH setup) This should create a file named benchmark_opentsdb with around 500,000 generated metrics.

  2. Build the benchmarker

    $ cd $GOPATH/src/github.com/m3db/m3/src/query/benchmark/benchmarker/main/
    $ go build
    
  3. The first time you run the benchmarker, or any time you need to regenerate data (e.g. if generated data is too stale to be written to m3db), add the --regenerateData flag:

     $ ./benchmarker --regenerateData
    
  4. Otherwise, run

     $ ./benchmarker
    

Notable options for tweaking performance:

    $ --batch=1000 : signifies the number of metrics to send per batch (prometheus defaults to 100, the benchmarker defaults to 1000)
    $ --workers=2 : signifies the number of parallel workers that send encoded metrics to coordinator
    $ --cardinality : if this is set, cardinality for the input file is calculated instead of benchmarking
  1. Don't forget to delete all the generated junk before committing (sorry, will fix this)!

Documentation

The Go Gopher

There is no documentation for this package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL