loadtest

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 29, 2020 License: Apache-2.0 Imports: 26 Imported by: 0

README

Tendermint Load Testing Framework

tm-load-test comes with the ability to define your own load testing clients. A client is the part of the load tester that generates transactions specific to a particular ABCI application. By default, tm-load-test comes with support for the kvstore ABCI app, but what if you want to extend it to test your own ABCI app?

Requirements

To follow this guide, you'll need:

  • Go v1.12+

Creating a Custom ABCI Load Testing App

Step 1: Create your project

You'll effectively have to create your own load testing tool by importing the tm-load-test package into a new project.

mkdir -p /your/project/
cd /your/project
go mod init github.com/you/my-load-tester
Step 2: Create your load testing client

Create a client that generates transactions for your ABCI app. For an example, you can look at the kvstore client code. Put this in ./pkg/myabciapp/client.go

package myabciapp

import "github.com/interchainio/tm-load-test/pkg/loadtest"

// MyABCIAppClientFactory creates instances of MyABCIAppClient
type MyABCIAppClientFactory struct {}

// MyABCIAppClientFactory implements loadtest.ClientFactory
var _ loadtest.ClientFactory = (*MyABCIAppClientFactory)(nil)

// MyABCIAppClient is responsible for generating transactions. Only one client
// will be created per connection to the remote Tendermint RPC endpoint, and
// each client will be responsible for maintaining its own state in a
// thread-safe manner.
type MyABCIAppClient struct {}

// MyABCIAppClient implements loadtest.Client
var _ loadtest.Client = (*MyABCIAppClient)(nil)

func (f *MyABCIAppClientFactory) ValidateConfig(cfg loadtest.Config) error {
    // Do any checks here that you need to ensure that the load test 
    // configuration is compatible with your client.
    return nil
}

func (f *MyABCIAppClientFactory) NewClient(cfg loadtest.Config) (loadtest.Client, error) {
    return &MyABCIAppClient{}, nil
}

// GenerateTx must return the raw bytes that make up the transaction for your
// ABCI app. The conversion to base64 will automatically be handled by the 
// loadtest package, so don't worry about that. Only return an error here if you
// want to completely fail the entire load test operation.
func (c *MyABCIAppClient) GenerateTx() ([]byte, error) {
    return []byte("this is my transaction"), nil
}
Step 3: Create your CLI

Create your own CLI in ./cmd/my-load-tester/main.go:

package main

import (
    "github.com/interchainio/tm-load-test/pkg/loadtest"
    "github.com/you/my-load-tester/pkg/myabciapp"
)

func main() {
    if err := loadtest.RegisterClientFactory("my-abci-app-name", &myabciapp.MyABCIAppClientFactory{}); err != nil {
        panic(err)
    }
    // The loadtest.Run method will handle CLI argument parsing, errors, 
    // configuration, instantiating the load test and/or master/slave 
    // operations, etc. All it needs is to know which client factory to use for
    // its load testing.
    loadtest.Run(&loadtest.CLIConfig{
        AppName:              "my-load-tester",
        AppShortDesc:         "Load testing application for My ABCI App (TM)",
        AppLongDesc:          "Some long description on how to use the tool",
        DefaultClientFactory: "my-abci-app-name",
    })
}

For an example of very simple integration testing, you could do something similar to what's covered in integration_test.go.

Step 4: Build your CLI

Then build the executable:

go build -o ./build/my-load-tester ./cmd/my-load-tester/main.go
Step 5: Run your load test!

Then just follow the same instructions as for running the tm-load-test tool to run your own tool. It will use the same command line parameters.

Documentation

Index

Constants

View Source 
const (
	SelectSuppliedEndpoints   = "supplied"   // Select only the supplied endpoint(s) for load testing (the default).
	SelectDiscoveredEndpoints = "discovered" // Select newly discovered endpoints only (excluding supplied endpoints).
	SelectAnyEndpoints        = "any"        // Select from any of supplied and/or discovered endpoints.
)
View Source 
const CLIVersion = "v0.9.0"

CLIVersion must be manually updated as new versions are released.

View Source 
const KVStoreClientIDLen int = 5 // Allows for 6,471,002 random client IDs (62C5)

The Tendermint common.RandStr method can effectively generate human-readable (alphanumeric) strings from a set of 62 characters. We aim here with the KVStore client to generate unique client IDs as well as totally unique keys for all transactions. Values are not so important.

Variables

This section is empty.

Functions

func ExecuteStandalone added in v0.7.1 

func ExecuteStandalone(cfg Config) error

ExecuteStandalone will run a standalone (non-master/slave) load test.

func RegisterClientFactory 

func RegisterClientFactory(name string, factory ClientFactory) error

RegisterClientFactory allows us to programmatically register different client factories to easily switch between different ones at runtime.

func Run 

func Run(cli *CLIConfig)

Run must be executed from your `main` function in your Go code. This can be used to fast-track the construction of your own load testing tool for your Tendermint ABCI application.

Types

type AggregateStats added in v0.9.0 

type AggregateStats struct {
	TotalTxs         int     // The total number of transactions sent.
	TotalTimeSeconds float64 // The total time taken to send `TotalTxs` transactions.
	TotalBytes       int64   // The cumulative number of bytes sent as transactions.

	// Computed statistics
	AvgTxRate   float64 // The rate at which transactions were submitted (tx/sec).
	AvgDataRate float64 // The rate at which data was transmitted in transactions (bytes/sec).
}

func (*AggregateStats) Compute added in v0.9.0 

func (s *AggregateStats) Compute()

func (*AggregateStats) String added in v0.9.0 

func (s *AggregateStats) String() string

type CLIConfig added in v0.4.0 

type CLIConfig struct {
	AppName              string
	AppShortDesc         string
	AppLongDesc          string
	DefaultClientFactory string
}

CLIConfig allows developers to customize their own load testing tool.

type Client 

type Client interface {
	// GenerateTx must generate a raw transaction to be sent to the relevant
	// broadcast_tx method for a given endpoint.
	GenerateTx() ([]byte, error)
}

Client generates transactions to be sent to a specific endpoint.

type ClientFactory 

type ClientFactory interface {
	// ValidateConfig must check whether the given configuration is valid for
	// our specific client factory.
	ValidateConfig(cfg Config) error

	// NewClient must instantiate a new load testing client, or produce an error
	// if that process fails.
	NewClient(cfg Config) (Client, error)
}

ClientFactory produces load testing clients.

type Config 

type Config struct {
	ClientFactory        string   `json:"client_factory"`         // Which client factory should we use for load testing?
	Connections          int      `json:"connections"`            // The number of WebSockets connections to make to each target endpoint.
	Time                 int      `json:"time"`                   // The total time, in seconds, for which to handle the load test.
	SendPeriod           int      `json:"send_period"`            // The period (in seconds) at which to send batches of transactions.
	Rate                 int      `json:"rate"`                   // The number of transactions to generate, per send period.
	Size                 int      `json:"size"`                   // The desired size of each generated transaction, in bytes.
	Count                int      `json:"count"`                  // The maximum number of transactions to send. Set to -1 for unlimited.
	BroadcastTxMethod    string   `json:"broadcast_tx_method"`    // The broadcast_tx method to use (can be "sync", "async" or "commit").
	Endpoints            []string `json:"endpoints"`              // A list of the Tendermint node endpoints to which to connect for this load test.
	EndpointSelectMethod string   `json:"endpoint_select_method"` // The method by which to select endpoints for load testing.
	ExpectPeers          int      `json:"expect_peers"`           // The minimum number of peers to expect before starting a load test. Set to 0 by default (no minimum).
	MaxEndpoints         int      `json:"max_endpoints"`          // The maximum number of endpoints to use for load testing. Set to 0 by default (no maximum).
	MinConnectivity      int      `json:"min_connectivity"`       // The minimum number of peers to which each peer must be connected before starting the load test. Set to 0 by default (no minimum).
	PeerConnectTimeout   int      `json:"peer_connect_timeout"`   // The maximum time to wait (in seconds) for all peers to connect, if ExpectPeers > 0.
	StatsOutputFile      string   `json:"stats_output_file"`      // Where to store the final aggregate statistics file (in CSV format).
	NoTrapInterrupts     bool     `json:"no_trap_interrupts"`     // Should we avoid trapping Ctrl+Break? Only relevant for standalone execution mode.
}

Config represents the configuration for a single client (i.e. standalone or slave).

func (Config) MaxTxsPerEndpoint added in v0.9.0 

func (c Config) MaxTxsPerEndpoint() uint64

MaxTxsPerEndpoint estimates the maximum number of transactions that this configuration would generate for a single endpoint.

func (Config) ToJSON added in v0.4.0 

func (c Config) ToJSON() string

func (Config) Validate 

func (c Config) Validate() error

type KVStoreClient added in v0.4.0 

type KVStoreClient struct {
	// contains filtered or unexported fields
}

KVStoreClient generates arbitrary transactions (random key=value pairs) to be sent to the kvstore ABCI application. The keys are structured as follows:

`[client_id][tx_id]=[tx_id]`

where each value (`client_id` and `tx_id`) is padded with 0s to meet the transaction size requirement.

func (*KVStoreClient) GenerateTx added in v0.4.0 

func (c *KVStoreClient) GenerateTx() ([]byte, error)

type KVStoreClientFactory added in v0.4.0 

type KVStoreClientFactory struct{}

KVStoreClientFactory creates load testing clients to interact with the built-in Tendermint kvstore ABCI application.

func NewKVStoreClientFactory added in v0.4.0 

func NewKVStoreClientFactory() *KVStoreClientFactory

func (*KVStoreClientFactory) NewClient added in v0.4.0 

func (f *KVStoreClientFactory) NewClient(cfg Config) (Client, error)

func (*KVStoreClientFactory) ValidateConfig added in v0.9.0 

func (f *KVStoreClientFactory) ValidateConfig(cfg Config) error

type Master 

type Master struct {
	// contains filtered or unexported fields
}

Master is a WebSockets server that allows slaves to connect to it to obtain configuration information. It does nothing but coordinate load testing amongst the slaves.

func NewMaster 

func NewMaster(cfg *Config, masterCfg *MasterConfig) *Master

func (*Master) ReceiveSlaveUpdate added in v0.4.0 

func (m *Master) ReceiveSlaveUpdate(msg slaveMsg)

func (*Master) RegisterRemoteSlave added in v0.4.0 

func (m *Master) RegisterRemoteSlave(rs *remoteSlave) error

func (*Master) Run added in v0.2.0 

func (m *Master) Run() error

Run will execute the master's operations in a blocking manner, returning any error that causes one of the slaves or the master to fail.

func (*Master) UnregisterRemoteSlave added in v0.4.0 

func (m *Master) UnregisterRemoteSlave(id string, err error)

type MasterConfig 

type MasterConfig struct {
	BindAddr            string `json:"bind_addr"`       // The "host:port" to which to bind the master node to listen for incoming slaves.
	ExpectSlaves        int    `json:"expect_slaves"`   // The number of slaves to expect before starting the load test.
	SlaveConnectTimeout int    `json:"connect_timeout"` // The number of seconds to wait for all slaves to connect.
	ShutdownWait        int    `json:"shutdown_wait"`   // The number of seconds to wait at shutdown (while keeping the HTTP server running - primarily to allow Prometheus to keep polling).
	LoadTestID          int    `json:"load_test_id"`    // An integer greater than 0 that will be exposed via a Prometheus gauge while the load test is underway.
}

MasterConfig is the configuration options specific to a master node.

func (MasterConfig) ToJSON added in v0.4.0 

func (c MasterConfig) ToJSON() string

func (MasterConfig) Validate 

func (c MasterConfig) Validate() error

type Slave 

type Slave struct {
	// contains filtered or unexported fields
}

Slave is a WebSockets client that interacts with the Master node to (1) fetch its configuration, (2) execute a load test, and (3) report back to the master node regularly on its progress.

func NewSlave 

func NewSlave(cfg *SlaveConfig) (*Slave, error)

func (*Slave) Config added in v0.4.0 

func (s *Slave) Config() Config

func (*Slave) ID added in v0.4.0 

func (s *Slave) ID() string

func (*Slave) Run added in v0.2.0 

func (s *Slave) Run() error

Run executes the primary event loop for this slave.

type SlaveConfig 

type SlaveConfig struct {
	ID                   string `json:"id"`              // A unique ID for this slave instance. Will show up in the metrics reported by the master for this slave.
	MasterAddr           string `json:"master_addr"`     // The address at which to find the master node.
	MasterConnectTimeout int    `json:"connect_timeout"` // The maximum amount of time, in seconds, to allow for the master to become available.
}

SlaveConfig is the configuration options specific to a slave node.

func (SlaveConfig) ToJSON added in v0.4.0 

func (c SlaveConfig) ToJSON() string

func (SlaveConfig) Validate 

func (c SlaveConfig) Validate() error

type Transactor added in v0.4.0 

type Transactor struct {
	// contains filtered or unexported fields
}

Transactor represents a single wire-level connection to a Tendermint RPC endpoint, and this is responsible for sending transactions to that endpoint.

func NewTransactor added in v0.4.0 

func NewTransactor(remoteAddr string, config *Config) (*Transactor, error)

NewTransactor initiates a WebSockets connection to the given host address. Must be a valid WebSockets URL, e.g. "ws://host:port/websocket"

func (*Transactor) Cancel added in v0.4.0 

func (t *Transactor) Cancel()

Cancel will indicate to the transactor that it must stop, but does not wait until it has completely stopped. To wait, call the Transactor.Wait() method.

func (*Transactor) GetTxBytes added in v0.9.0 

func (t *Transactor) GetTxBytes() int64

GetTxBytes returns the cumulative total number of bytes (as transactions) sent thus far by this transactor.

func (*Transactor) GetTxCount added in v0.4.0 

func (t *Transactor) GetTxCount() int

GetTxCount returns the total number of transactions sent thus far by this transactor.

func (*Transactor) GetTxRate added in v0.4.0 

func (t *Transactor) GetTxRate() float64

GetTxRate returns the average number of transactions per second sent by this transactor over the duration of its operation.

func (*Transactor) SetProgressCallback added in v0.4.0 

func (t *Transactor) SetProgressCallback(id int, interval time.Duration, callback func(int, int, int64))

func (*Transactor) Start added in v0.4.0 

func (t *Transactor) Start()

Start kicks off the transactor's operations in separate goroutines (one for reading from the WebSockets endpoint, and one for writing to it).

func (*Transactor) Wait added in v0.4.0 

func (t *Transactor) Wait() error

Wait will block until the transactor terminates.

type TransactorGroup added in v0.4.0 

type TransactorGroup struct {
	// contains filtered or unexported fields
}

TransactorGroup allows us to encapsulate the management of a group of transactors.

func NewTransactorGroup added in v0.4.0 

func NewTransactorGroup() *TransactorGroup

func (*TransactorGroup) Add added in v0.4.0 

func (g *TransactorGroup) Add(remoteAddr string, config *Config) error

Add will instantiate a new Transactor with the given parameters. If instantiation fails it'll automatically shut down and close all other transactors, returning the error.

func (*TransactorGroup) AddAll added in v0.4.0 

func (g *TransactorGroup) AddAll(cfg *Config) error

func (*TransactorGroup) Cancel added in v0.4.0 

func (g *TransactorGroup) Cancel()

Cancel signals to all transactors to stop their operations.

func (*TransactorGroup) SetProgressCallback added in v0.4.0 

func (g *TransactorGroup) SetProgressCallback(interval time.Duration, callback func(*TransactorGroup, int, int64))

func (*TransactorGroup) Start added in v0.4.0 

func (g *TransactorGroup) Start()

Start will handle through all transactors and start them.

func (*TransactorGroup) Wait added in v0.4.0 

func (g *TransactorGroup) Wait() error

Wait will wait for all transactors to complete, returning the first error we encounter.

func (*TransactorGroup) WriteAggregateStats added in v0.7.1 

func (g *TransactorGroup) WriteAggregateStats(filename string) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.