Documentation

Overview

Package simulation implements a full fledged Cosmos SDK application used for executing simulation test suites.

Simulation App

The SimApp type defines an application used for running extensive simulation testing suites. It contains all core modules, including governance, staking, slashing, and distribution.

Simulation is executed with various inputs including the number of blocks to simulate, the block size, whether the app should commit or not, the invariant checking period, and a seed which is used as a source of pseudo-randomness.

In addition to the various inputs, simulation runs mainly in three modes:

1. Completely random where the initial state, module parameters and simulation parameters are pseudo-randomly generated.

2. From a genesis file where the initial state and the module parameters are defined. This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested.

3. From a params file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated.

The simulation test suite also supports testing determinism and import/export functionality.

Randomness

Currently, simulation uses a single seed (integer) as a source for a PRNG by which all random operations are executed from. Any call to the PRNG changes all future operations as the internal state of the PRNG is modified. For example, if a new message type is created and needs to be simulated, the new introduced PRNG call will change all subsequent operations.

This may can often be problematic when testing fixes to simulation faults. One current solution to this is to use a params file as mentioned above. In the future the simulation suite is expected to support a series of PRNGs that can be used uniquely per module and simulation component so that they will not effect each others state execution outcome.

Usage

To execute a completely pseudo-random simulation:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-v -timeout 24h

To execute simulation from a genesis file:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-Genesis=/path/to/genesis.json \
 	-v -timeout 24h

To execute simulation from a simulation params file:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-Params=/path/to/params.json \
	-v -timeout 24h

To export the simulation params to a file at a given block height:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportParamsPath=/path/to/params.json \
	-ExportParamsHeight=50 \
	 -v -timeout 24h

To export the simulation app state (i.e genesis) to a file:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportStatePath=/path/to/genesis.json \
	 v -timeout 24h

Params

Params that are provided to simulation from a JSON file are used to used to set both module parameters and simulation parameters. See sim_test.go for the full set of parameters that can be provided.

Index

Constants

View Source
const (
	BeginBlockEntryKind = "begin_block"
	EndBlockEntryKind   = "end_block"
	MsgEntryKind        = "msg"
	QueuedMsgEntryKind  = "queued_msg"
)

    entry kinds for use within OperationEntry

    Variables

    This section is empty.

    Functions

    func DeriveRand

    func DeriveRand(r *rand.Rand) *rand.Rand

      DeriveRand derives a new Rand deterministically from another random source. Unlike rand.New(rand.NewSource(seed)), the result is "more random" depending on the source and state of r.

      NOTE: not crypto safe.

      func GetMemberOfInitialState

      func GetMemberOfInitialState(r *rand.Rand, weights []int) int

        GetMemberOfInitialState takes an initial array of weights, of size n. It returns a weighted random number in [0,n).

        func RandIntBetween

        func RandIntBetween(r *rand.Rand, min, max int) int

          RandIntBetween returns a random int between two numbers inclusively.

          func RandPositiveInt

          func RandPositiveInt(r *rand.Rand, max sdk.Int) (sdk.Int, error)

            RandPositiveInt get a rand positive sdk.Int

            func RandStringOfLength

            func RandStringOfLength(r *rand.Rand, n int) string

              RandStringOfLength generates a random string of a particular length

              func RandSubsetCoins

              func RandSubsetCoins(r *rand.Rand, coins sdk.Coins) sdk.Coins

                returns random subset of the provided coins will return at least one coin unless coins argument is empty or malformed i.e. 0 amt in coins

                func RandTimestamp

                func RandTimestamp(r *rand.Rand) time.Time

                  RandTimestamp generates a random timestamp

                  func RandomAmount

                  func RandomAmount(r *rand.Rand, max sdk.Int) sdk.Int

                    RandomAmount generates a random amount Note: The range of RandomAmount includes max, and is, in fact, biased to return max as well as 0.

                    func RandomDecAmount

                    func RandomDecAmount(r *rand.Rand, max sdk.Dec) sdk.Dec

                      RandomDecAmount generates a random decimal amount Note: The range of RandomDecAmount includes max, and is, in fact, biased to return max as well as 0.

                      func RandomFees

                      func RandomFees(r *rand.Rand, ctx sdk.Context, spendableCoins sdk.Coins) (sdk.Coins, error)

                        RandomFees returns a random fee by selecting a random coin denomination and amount from the account's available balance. If the user doesn't have enough funds for paying fees, it returns empty coins.

                        func RandomRequestBeginBlock

                        func RandomRequestBeginBlock(r *rand.Rand, params Params,
                        	validators mockValidators, pastTimes []time.Time,
                        	pastVoteInfos [][]abci.VoteInfo,
                        	event func(route, op, evResult string), header abci.Header) abci.RequestBeginBlock

                          RandomRequestBeginBlock generates a list of signing validators according to the provided list of validators, signing fraction, and evidence fraction

                          Types

                          type Account

                          type Account struct {
                          	PrivKey crypto.PrivKey
                          	PubKey  crypto.PubKey
                          	Address sdk.AccAddress
                          }

                            Account contains a privkey, pubkey, address tuple eventually more useful data can be placed in here. (e.g. number of coins)

                            func FindAccount

                            func FindAccount(accs []Account, address sdk.Address) (Account, bool)

                              FindAccount iterates over all the simulation accounts to find the one that matches the given address

                              func RandomAcc

                              func RandomAcc(r *rand.Rand, accs []Account) (Account, int)

                                RandomAcc picks and returns a random account from an array and returs its position in the array.

                                func RandomAccounts

                                func RandomAccounts(r *rand.Rand, n int) []Account

                                  RandomAccounts generates n random accounts

                                  func (Account) Equals

                                  func (acc Account) Equals(acc2 Account) bool

                                    Equals returns true if two accounts are equal

                                    type AppParams

                                    type AppParams map[string]json.RawMessage

                                      AppParams defines a flat JSON of key/values for all possible configurable simulation parameters. It might contain: operation weights, simulation parameters and flattened module state parameters (i.e not stored under it's respective module name).

                                      func (AppParams) GetOrGenerate

                                      func (sp AppParams) GetOrGenerate(cdc *codec.Codec, key string, ptr interface{}, r *rand.Rand, ps ParamSimulator)

                                        GetOrGenerate attempts to get a given parameter by key from the AppParams object. If it exists, it'll be decoded and returned. Otherwise, the provided ParamSimulator is used to generate a random value or default value (eg: in the case of operation weights where Rand is not used).

                                        type AppStateFn

                                        type AppStateFn func(r *rand.Rand, accs []Account, config Config) (
                                        	appState json.RawMessage, accounts []Account, chainId string, genesisTimestamp time.Time,
                                        )

                                          AppStateFn returns the app state json bytes and the genesis accounts

                                          type Config

                                          type Config struct {
                                          	GenesisFile string // custom simulation genesis file; cannot be used with params file
                                          	ParamsFile  string // custom simulation params file which overrides any random params; cannot be used with genesis
                                          
                                          	ExportParamsPath   string // custom file path to save the exported params JSON
                                          	ExportParamsHeight int    //height to which export the randomly generated params
                                          	ExportStatePath    string //custom file path to save the exported app state JSON
                                          	ExportStatsPath    string // custom file path to save the exported simulation statistics JSON
                                          
                                          	Seed               int64  // simulation random seed
                                          	InitialBlockHeight int    // initial block to start the simulation
                                          	NumBlocks          int    // number of new blocks to simulate from the initial block height
                                          	BlockSize          int    // operations per block
                                          	ChainID            string // chain-id used on the simulation
                                          
                                          	Lean   bool // lean simulation log output
                                          	Commit bool // have the simulation commit
                                          
                                          	OnOperation   bool // run slow invariants every operation
                                          	AllInvariants bool // print all failed invariants if a broken invariant is found
                                          }

                                            Config contains the necessary configuration flags for the simulator

                                            type ContentSimulatorFn

                                            type ContentSimulatorFn func(r *rand.Rand, ctx sdk.Context, accs []Account) govtypes.Content

                                              ContentSimulatorFn defines a function type alias for generating random proposal content.

                                              type DummyLogWriter

                                              type DummyLogWriter struct{}

                                                _____________________ dummy log writter

                                                func (*DummyLogWriter) AddEntry

                                                func (lw *DummyLogWriter) AddEntry(_ OperationEntry)

                                                  do nothing

                                                  func (*DummyLogWriter) PrintLogs

                                                  func (lw *DummyLogWriter) PrintLogs()

                                                    do nothing

                                                    type EventStats

                                                    type EventStats map[string]map[string]map[string]int

                                                      EventStats defines an object that keeps a tally of each event that has occurred during a simulation.

                                                      func NewEventStats

                                                      func NewEventStats() EventStats

                                                        NewEventStats creates a new empty EventStats object

                                                        func (EventStats) ExportJSON

                                                        func (es EventStats) ExportJSON(path string)

                                                          ExportJSON saves the event stats as a JSON file on a given path

                                                          func (EventStats) Print

                                                          func (es EventStats) Print(w io.Writer)

                                                            Print the event stats in JSON format.

                                                            func (EventStats) Tally

                                                            func (es EventStats) Tally(route, op, evResult string)

                                                              Tally increases the count of a simulation event.

                                                              type FutureOperation

                                                              type FutureOperation struct {
                                                              	BlockHeight int
                                                              	BlockTime   time.Time
                                                              	Op          Operation
                                                              }

                                                                FutureOperation is an operation which will be ran at the beginning of the provided BlockHeight. If both a BlockHeight and BlockTime are specified, it will use the BlockHeight. In the (likely) event that multiple operations are queued at the same block height, they will execute in a FIFO pattern.

                                                                type LogWriter

                                                                type LogWriter interface {
                                                                	AddEntry(OperationEntry)
                                                                	PrintLogs()
                                                                }

                                                                  log writter

                                                                  func NewLogWriter

                                                                  func NewLogWriter(testingmode bool) LogWriter

                                                                    LogWriter - return a dummy or standard log writer given the testingmode

                                                                    type Operation

                                                                    type Operation func(r *rand.Rand, app *baseapp.BaseApp,
                                                                    	ctx sdk.Context, accounts []Account, chainID string) (
                                                                    	OperationMsg OperationMsg, futureOps []FutureOperation, err error)

                                                                      Operation runs a state machine transition, and ensures the transition happened as expected. The operation could be running and testing a fuzzed transaction, or doing the same for a message.

                                                                      For ease of debugging, an operation returns a descriptive message "action", which details what this fuzzed state machine transition actually did.

                                                                      Operations can optionally provide a list of "FutureOperations" to run later These will be ran at the beginning of the corresponding block.

                                                                      type OperationEntry

                                                                      type OperationEntry struct {
                                                                      	EntryKind string          `json:"entry_kind" yaml:"entry_kind"`
                                                                      	Height    int64           `json:"height" yaml:"height"`
                                                                      	Order     int64           `json:"order" yaml:"order"`
                                                                      	Operation json.RawMessage `json:"operation" yaml:"operation"`
                                                                      }

                                                                        OperationEntry - an operation entry for logging (ex. BeginBlock, EndBlock, XxxMsg, etc)

                                                                        func BeginBlockEntry

                                                                        func BeginBlockEntry(height int64) OperationEntry

                                                                          BeginBlockEntry - operation entry for begin block

                                                                          func EndBlockEntry

                                                                          func EndBlockEntry(height int64) OperationEntry

                                                                            EndBlockEntry - operation entry for end block

                                                                            func MsgEntry

                                                                            func MsgEntry(height, order int64, opMsg OperationMsg) OperationEntry

                                                                              MsgEntry - operation entry for standard msg

                                                                              func NewOperationEntry

                                                                              func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry

                                                                                NewOperationEntry creates a new OperationEntry instance

                                                                                func QueuedMsgEntry

                                                                                func QueuedMsgEntry(height int64, opMsg OperationMsg) OperationEntry

                                                                                  QueuedMsgEntry creates an operation entry for a given queued message.

                                                                                  func (OperationEntry) MustMarshal

                                                                                  func (oe OperationEntry) MustMarshal() json.RawMessage

                                                                                    MustMarshal marshals the operation entry, panic on error.

                                                                                    type OperationMsg

                                                                                    type OperationMsg struct {
                                                                                    	Route   string          `json:"route" yaml:"route"`     // msg route (i.e module name)
                                                                                    	Name    string          `json:"name" yaml:"name"`       // operation name (msg Type or "no-operation")
                                                                                    	Comment string          `json:"comment" yaml:"comment"` // additional comment
                                                                                    	OK      bool            `json:"ok" yaml:"ok"`           // success
                                                                                    	Msg     json.RawMessage `json:"msg" yaml:"msg"`         // JSON encoded msg
                                                                                    }

                                                                                      OperationMsg - structure for operation output

                                                                                      func NewOperationMsg

                                                                                      func NewOperationMsg(msg sdk.Msg, ok bool, comment string) OperationMsg

                                                                                        NewOperationMsg - create a new operation message from sdk.Msg

                                                                                        func NewOperationMsgBasic

                                                                                        func NewOperationMsgBasic(route, name, comment string, ok bool, msg []byte) OperationMsg

                                                                                          NewOperationMsgBasic creates a new operation message from raw input.

                                                                                          func NoOpMsg

                                                                                          func NoOpMsg(route string) OperationMsg

                                                                                            NoOpMsg - create a no-operation message

                                                                                            func (OperationMsg) LogEvent

                                                                                            func (om OperationMsg) LogEvent(eventLogger func(route, op, evResult string))

                                                                                              LogEvent adds an event for the events stats

                                                                                              func (OperationMsg) MustMarshal

                                                                                              func (om OperationMsg) MustMarshal() json.RawMessage

                                                                                                MustMarshal Marshals the operation msg, panic on error

                                                                                                func (OperationMsg) String

                                                                                                func (om OperationMsg) String() string

                                                                                                  log entry text for this operation msg

                                                                                                  type OperationQueue

                                                                                                  type OperationQueue map[int][]Operation

                                                                                                    OperationQueue defines an object for a queue of operations

                                                                                                    func NewOperationQueue

                                                                                                    func NewOperationQueue() OperationQueue

                                                                                                      NewOperationQueue creates a new OperationQueue instance.

                                                                                                      type ParamChange

                                                                                                      type ParamChange struct {
                                                                                                      	Subspace string
                                                                                                      	Key      string
                                                                                                      	SimValue SimValFn
                                                                                                      }

                                                                                                        ParamChange defines the object used for simulating parameter change proposals

                                                                                                        func NewSimParamChange

                                                                                                        func NewSimParamChange(subspace, key string, simVal SimValFn) ParamChange

                                                                                                          NewSimParamChange creates a new ParamChange instance

                                                                                                          func (ParamChange) ComposedKey

                                                                                                          func (spc ParamChange) ComposedKey() string

                                                                                                            ComposedKey creates a new composed key for the param change proposal

                                                                                                            type ParamSimulator

                                                                                                            type ParamSimulator func(r *rand.Rand)

                                                                                                              ParamSimulator creates a parameter value from a source of random number

                                                                                                              type Params

                                                                                                              type Params struct {
                                                                                                              	PastEvidenceFraction      float64
                                                                                                              	NumKeys                   int
                                                                                                              	EvidenceFraction          float64
                                                                                                              	InitialLivenessWeightings []int
                                                                                                              	LivenessTransitionMatrix  TransitionMatrix
                                                                                                              	BlockSizeTransitionMatrix TransitionMatrix
                                                                                                              }

                                                                                                                Params define the parameters necessary for running the simulations

                                                                                                                func RandomParams

                                                                                                                func RandomParams(r *rand.Rand) Params

                                                                                                                  RandomParams returns random simulation parameters

                                                                                                                  func SimulateFromSeed

                                                                                                                  func SimulateFromSeed(
                                                                                                                  	tb testing.TB, w io.Writer, app *baseapp.BaseApp,
                                                                                                                  	appStateFn AppStateFn, ops WeightedOperations,
                                                                                                                  	blackListedAccs map[string]bool, config Config,
                                                                                                                  ) (stopEarly bool, exportedParams Params, err error)

                                                                                                                    SimulateFromSeed tests an application by running the provided operations, testing the provided invariants, but using the provided config.Seed. TODO: split this monster function up

                                                                                                                    type SimValFn

                                                                                                                    type SimValFn func(r *rand.Rand) string

                                                                                                                      SimValFn function to generate the randomized parameter change value

                                                                                                                      type StandardLogWriter

                                                                                                                      type StandardLogWriter struct {
                                                                                                                      	OpEntries []OperationEntry `json:"op_entries" yaml:"op_entries"`
                                                                                                                      }

                                                                                                                        log writter

                                                                                                                        func (*StandardLogWriter) AddEntry

                                                                                                                        func (lw *StandardLogWriter) AddEntry(opEntry OperationEntry)

                                                                                                                          add an entry to the log writter

                                                                                                                          func (*StandardLogWriter) PrintLogs

                                                                                                                          func (lw *StandardLogWriter) PrintLogs()

                                                                                                                            PrintLogs - print the logs to a simulation file

                                                                                                                            type TransitionMatrix

                                                                                                                            type TransitionMatrix struct {
                                                                                                                            	// contains filtered or unexported fields
                                                                                                                            }

                                                                                                                              TransitionMatrix is _almost_ a left stochastic matrix. It is technically not one due to not normalizing the column values. In the future, if we want to find the steady state distribution, it will be quite easy to normalize these values to get a stochastic matrix. Floats aren't currently used as the default due to non-determinism across architectures

                                                                                                                              func CreateTransitionMatrix

                                                                                                                              func CreateTransitionMatrix(weights [][]int) (TransitionMatrix, error)

                                                                                                                                CreateTransitionMatrix creates a transition matrix from the provided weights. TODO: Provide example usage

                                                                                                                                func (TransitionMatrix) NextState

                                                                                                                                func (t TransitionMatrix) NextState(r *rand.Rand, i int) int

                                                                                                                                  NextState returns the next state randomly chosen using r, and the weightings provided in the transition matrix.

                                                                                                                                  type WeightedOperation

                                                                                                                                  type WeightedOperation struct {
                                                                                                                                  	Weight int
                                                                                                                                  	Op     Operation
                                                                                                                                  }

                                                                                                                                    WeightedOperation is an operation with associated weight. This is used to bias the selection operation within the simulator.

                                                                                                                                    func NewWeightedOperation

                                                                                                                                    func NewWeightedOperation(weight int, op Operation) WeightedOperation

                                                                                                                                      NewWeightedOperation creates a new WeightedOperation instance

                                                                                                                                      type WeightedOperations

                                                                                                                                      type WeightedOperations []WeightedOperation

                                                                                                                                        WeightedOperations is the group of all weighted operations to simulate.

                                                                                                                                        type WeightedProposalContent

                                                                                                                                        type WeightedProposalContent struct {
                                                                                                                                        	AppParamsKey       string             // key used to retrieve the value of the weight from the simulation application params
                                                                                                                                        	DefaultWeight      int                // default weight
                                                                                                                                        	ContentSimulatorFn ContentSimulatorFn // content simulator function
                                                                                                                                        }

                                                                                                                                          WeightedProposalContent defines a common struct for proposal contents defined by external modules (i.e outside gov)