README
¶
boomer 
Links
- Locust Website: locust.io
- Locust Documentation: docs.locust.io
- Boomer Documentation: boomer.readthedocs.io
Description
Boomer is a better load generator for locust, written in golang. It can spawn thousands of goroutines to run your code concurrently.
It will listen and report to the locust master automatically, your test results will be displayed on the master's web UI.
Use it as a library, not a general-purpose benchmarking tool.
Versioning
Boomer used to support all versions of locust, even if locust didn't keep backward compatibility.
Now boomer follows locust's versioning, and the master branch works with locust's master branch.
If locust introduces breaking changes, boomer will have a tagged version that works previous version of locust.
Install
# Install the master branch
$ go get github.com/myzhan/boomer@master
# Install a tagged version that works with locust 1.6.0
$ go get github.com/myzhan/boomer@v1.6.0
Build
Boomer use gomq by default, which is a pure Go implementation of the ZeroMQ protocol.
Because of the instability of gomq, you can switch to goczmq.
# use gomq
$ go build -o a.out main.go
# use goczmq
$ go build -tags 'goczmq' -o a.out main.go
If you fail to compile boomer with gomq, try to update gomq first.
$ go get -u github.com/zeromq/gomq
Examples(main.go)
This is a example of boomer's API. You can find more in the "examples" directory.
package main
import "time"
import "github.com/myzhan/boomer"
func foo(){
start := time.Now()
time.Sleep(100 * time.Millisecond)
elapsed := time.Since(start)
/*
Report your test result as a success, if you write it in locust, it will looks like this
events.request_success.fire(request_type="http", name="foo", response_time=100, response_length=10)
*/
boomer.RecordSuccess("http", "foo", elapsed.Nanoseconds()/int64(time.Millisecond), int64(10))
}
func bar(){
start := time.Now()
time.Sleep(100 * time.Millisecond)
elapsed := time.Since(start)
/*
Report your test result as a failure, if you write it in locust, it will looks like this
events.request_failure.fire(request_type="udp", name="bar", response_time=100, exception=Exception("udp error"))
*/
boomer.RecordFailure("udp", "bar", elapsed.Nanoseconds()/int64(time.Millisecond), "udp error")
}
func main(){
task1 := &boomer.Task{
Name: "foo",
// The weight is used to distribute goroutines over multiple tasks.
Weight: 10,
Fn: foo,
}
task2 := &boomer.Task{
Name: "bar",
Weight: 20,
Fn: bar,
}
boomer.Run(task1, task2)
}
Run
For debug purpose, you can run tasks without connecting to the master.
$ go build -o a.out main.go
./a.out --run-tasks foo,bar
Otherwise, start the master using the included dummy.py.
$ locust --master -f dummy.py
--max-rps means the max count that all the Task.Fn can be called in one second.
The result may be misleading if you call boomer.RecordSuccess() more than once in Task.Fn.
$ go build -o a.out main.go
$ ./a.out --max-rps 10000
If you want the RPS increase from zero to max-rps or infinity.
$ go build -o a.out main.go
# The default interval is 1 second
$ ./a.out --request-increase-rate 10
# Change the interval to 1 minute
# Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h"
$ ./a.out --request-increase-rate 10/1m
So far, dummy.py is necessary when starting a master, because locust needs such a file.
Don't worry, dummy.py has nothing to do with your test.
Profiling
You may think there are bottlenecks in your load generator, don't hesitate to do profiling.
Both CPU and memory profiling are supported.
It's not suggested to run CPU profiling and memory profiling at the same time.
CPU Profiling
# 1. run locust master.
# 2. run boomer with cpu profiling for 30 seconds.
$ go run main.go -cpu-profile cpu.pprof -cpu-profile-duration 30s
# 3. start test in the WebUI.
# 4. run pprof.
$ go tool pprof cpu.pprof
Type: cpu
Time: Nov 14, 2018 at 8:04pm (CST)
Duration: 30.17s, Total samples = 12.07s (40.01%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) web
Memory Profiling
# 1. run locust master.
# 2. run boomer with memory profiling for 30 seconds.
$ go run main.go -mem-profile mem.pprof -mem-profile-duration 30s
# 3. start test in the WebUI.
# 4. run pprof and try 'go tool pprof --help' to learn more.
$ go tool pprof -alloc_space mem.pprof
Type: alloc_space
Time: Nov 14, 2018 at 8:26pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Exporter
If you are not satisfied with the build-in web monitor in Locust, you can run prometheus_exporter.py instead of dummy.py as your master.
Try this
$ locust --master -f prometheus_exporter.py
Thanks to Prometheus and Grafana, you will get an awesome dashboard: Locust for Prometheus
Contributing
If you are enjoying boomer and willing to add new features to it, you are welcome.
Also, good examples are welcome!!!
License
Open source licensed under the MIT license (see LICENSE file for details).
Documentation
¶
Index ¶
- Constants
- Variables
- func GetCurrentCPUUsage() float64
- func MD5(slice ...string) string
- func Now() int64
- func RecordFailure(requestType, name string, responseTime int64, exception string)
- func RecordSuccess(requestType, name string, responseTime int64, responseLength int64)
- func Run(tasks ...*Task)
- func StartCPUProfile(file string, duration time.Duration) (err error)
- func StartMemoryProfile(file string, duration time.Duration) (err error)
- type Boomer
- func (b *Boomer) AddOutput(o Output)
- func (b *Boomer) EnableCPUProfile(cpuProfile string, duration time.Duration)
- func (b *Boomer) EnableMemoryProfile(memoryProfile string, duration time.Duration)
- func (b *Boomer) Quit()
- func (b *Boomer) RecordFailure(requestType, name string, responseTime int64, exception string)
- func (b *Boomer) RecordSuccess(requestType, name string, responseTime int64, responseLength int64)
- func (b *Boomer) Run(tasks ...*Task)
- func (b *Boomer) SetMode(mode Mode)
- func (b *Boomer) SetRateLimiter(rateLimiter RateLimiter)
- type ConsoleOutput
- type Mode
- type Output
- type PrometheusPusherOutput
- type RampUpRateLimiter
- type RateLimiter
- type StableRateLimiter
- type Task
- type TaskSet
- type WeighingTaskSet
Constants ¶
const ( EVENT_SPAWN = "boomer:spawn" EVENT_STOP = "boomer:stop" EVENT_QUIT = "boomer:quit" )
Variables ¶
var ErrParsingRampUpRate = errors.New("ratelimiter: invalid format of rampUpRate, try \"1\" or \"1/1s\"")
ErrParsingRampUpRate is the error returned if the format of rampUpRate is invalid.
var Events = EventBus.New()
Events is the global event bus instance.
Functions ¶
func GetCurrentCPUUsage ¶
func GetCurrentCPUUsage() float64
GetCurrentCPUUsage get current CPU usage
func RecordFailure ¶
RecordFailure reports a failure. It's a convenience function to use the defaultBoomer.
func RecordSuccess ¶
RecordSuccess reports a success. It's a convenience function to use the defaultBoomer.
func Run ¶
func Run(tasks ...*Task)
Run accepts a slice of Task and connects to a locust master. It's a convenience function to use the defaultBoomer.
func StartCPUProfile ¶
StartCPUProfile starts cpu profiling and save the results in file.
Types ¶
type Boomer ¶
type Boomer struct {
// contains filtered or unexported fields
}
A Boomer is used to run tasks. This type is exposed, so users can create and control a Boomer instance programmatically.
func NewStandaloneBoomer ¶
NewStandaloneBoomer returns a new Boomer, which can run without master.
func (*Boomer) EnableCPUProfile ¶
EnableCPUProfile will start cpu profiling after run.
func (*Boomer) EnableMemoryProfile ¶
EnableMemoryProfile will start memory profiling after run.
func (*Boomer) RecordFailure ¶
RecordFailure reports a failure.
func (*Boomer) RecordSuccess ¶
RecordSuccess reports a success.
func (*Boomer) SetRateLimiter ¶
func (b *Boomer) SetRateLimiter(rateLimiter RateLimiter)
SetRateLimiter allows user to use their own rate limiter. It must be called before the test is started.
type ConsoleOutput ¶
type ConsoleOutput struct {
}
ConsoleOutput is the default output for standalone mode.
func NewConsoleOutput ¶
func NewConsoleOutput() *ConsoleOutput
NewConsoleOutput returns a ConsoleOutput.
func (*ConsoleOutput) OnEvent ¶
func (o *ConsoleOutput) OnEvent(data map[string]interface{})
OnEvent will print to the console.
func (*ConsoleOutput) OnStart ¶
func (o *ConsoleOutput) OnStart()
OnStart of ConsoleOutput has nothing to do.
func (*ConsoleOutput) OnStop ¶
func (o *ConsoleOutput) OnStop()
OnStop of ConsoleOutput has nothing to do.
type Mode ¶
type Mode int
Mode is the running mode of boomer, both standalone and distributed are supported.
type Output ¶
type Output interface {
// OnStart will be call before the test starts.
OnStart()
// By default, each output receive stats data from runner every three seconds.
// OnEvent is responsible for dealing with the data.
OnEvent(data map[string]interface{})
// OnStop will be called before the test ends.
OnStop()
}
Output is primarily responsible for printing test results to different destinations such as consoles, files. You can write you own output and add to boomer. When running in standalone mode, the default output is ConsoleOutput, you can add more. When running in distribute mode, test results will be reported to master with or without an output. All the OnXXX function will be call in a separated goroutine, just in case some output will block. But it will wait for all outputs return to avoid data lost.
type PrometheusPusherOutput ¶
type PrometheusPusherOutput struct {
// contains filtered or unexported fields
}
PrometheusPusherOutput pushes boomer stats to Prometheus Pushgateway.
func NewPrometheusPusherOutput ¶
func NewPrometheusPusherOutput(gatewayURL, jobName string) *PrometheusPusherOutput
NewPrometheusPusherOutput returns a PrometheusPusherOutput.
func (*PrometheusPusherOutput) OnEvent ¶
func (o *PrometheusPusherOutput) OnEvent(data map[string]interface{})
OnEvent will push metric to Prometheus Pushgataway
func (*PrometheusPusherOutput) OnStart ¶
func (o *PrometheusPusherOutput) OnStart()
OnStart will register all prometheus metric collectors
func (*PrometheusPusherOutput) OnStop ¶
func (o *PrometheusPusherOutput) OnStop()
OnStop of PrometheusPusherOutput has nothing to do.
type RampUpRateLimiter ¶
type RampUpRateLimiter struct {
// contains filtered or unexported fields
}
A RampUpRateLimiter uses the token bucket algorithm. the threshold is updated according to the warm up rate. the bucket is refilled according to the refill period, no burst is allowed.
func NewRampUpRateLimiter ¶
func NewRampUpRateLimiter(maxThreshold int64, rampUpRate string, refillPeriod time.Duration) (rateLimiter *RampUpRateLimiter, err error)
NewRampUpRateLimiter returns a RampUpRateLimiter. Valid formats of rampUpRate are "1", "1/1s".
func (*RampUpRateLimiter) Acquire ¶
func (limiter *RampUpRateLimiter) Acquire() (blocked bool)
Acquire a token from the bucket, returns true if the bucket is exhausted.
func (*RampUpRateLimiter) Start ¶
func (limiter *RampUpRateLimiter) Start()
Start to refill the bucket periodically.
type RateLimiter ¶
type RateLimiter interface {
// Start is used to enable the rate limiter.
// It can be implemented as a noop if not needed.
Start()
// Acquire() is called before executing a task.Fn function.
// If Acquire() returns true, the task.Fn function will be executed.
// If Acquire() returns false, the task.Fn function won't be executed this time, but Acquire() will be called very soon.
// It works like:
// for {
// blocked := rateLimiter.Acquire()
// if !blocked {
// task.Fn()
// }
// }
// Acquire() should block the caller until execution is allowed.
Acquire() bool
// Stop is used to disable the rate limiter.
// It can be implemented as a noop if not needed.
Stop()
}
RateLimiter is used to put limits on task executions.
type StableRateLimiter ¶
type StableRateLimiter struct {
// contains filtered or unexported fields
}
A StableRateLimiter uses the token bucket algorithm. the bucket is refilled according to the refill period, no burst is allowed.
func NewStableRateLimiter ¶
func NewStableRateLimiter(threshold int64, refillPeriod time.Duration) (rateLimiter *StableRateLimiter)
NewStableRateLimiter returns a StableRateLimiter.
func (*StableRateLimiter) Acquire ¶
func (limiter *StableRateLimiter) Acquire() (blocked bool)
Acquire a token from the bucket, returns true if the bucket is exhausted.
func (*StableRateLimiter) Start ¶
func (limiter *StableRateLimiter) Start()
Start to refill the bucket periodically.
type Task ¶
type Task struct {
// The weight is used to distribute goroutines over multiple tasks.
Weight int
// Fn is called by the goroutines allocated to this task, in a loop.
Fn func()
Name string
}
Task is like the "Locust object" in locust, the python version. When boomer receives a start message from master, it will spawn several goroutines to run Task.Fn. But users can keep some information in the python version, they can't do the same things in boomer. Because Task.Fn is a pure function.
type TaskSet ¶
type TaskSet interface {
// Add a Task to the TaskSet.
AddTask(task *Task)
// Set the weight of the TaskSet.
SetWeight(weight int)
// Get the weight of the TaskSet.
GetWeight() (weight int)
// Run will pick up a Task from the TaskSet and run.
Run()
}
TaskSet is an experimental feature, the API is not stabilized. It needs to be more considered and tested.
type WeighingTaskSet ¶
type WeighingTaskSet struct {
// contains filtered or unexported fields
}
WeighingTaskSet is a implementation of the TaskSet interface. When a Task is added to WeighingTaskSet, it's weight is used to calculate the probability to be called.
func NewWeighingTaskSet ¶
func NewWeighingTaskSet() *WeighingTaskSet
NewWeighingTaskSet returns a new WeighingTaskSet.
func (*WeighingTaskSet) AddTask ¶
func (ts *WeighingTaskSet) AddTask(task *Task)
AddTask add a Task to the Weighing TaskSet. If the task's weight is <=0, it will be ignored.
func (*WeighingTaskSet) GetTask ¶
func (ts *WeighingTaskSet) GetTask(roll int) (task *Task)
GetTask returns a task in the task set.
func (*WeighingTaskSet) GetWeight ¶
func (ts *WeighingTaskSet) GetWeight() (weight int)
GetWeight returns the weight of the task set.
func (*WeighingTaskSet) Run ¶
func (ts *WeighingTaskSet) Run()
Run will pick up a task in the task set randomly and run. It can is used as a Task.Fn.
func (*WeighingTaskSet) SetWeight ¶
func (ts *WeighingTaskSet) SetWeight(weight int)
SetWeight sets the weight of the task set.
Source Files
Directories