vector

README

Visor Vector How-To

You can use the visor vector command to create and execute test vectors with Visor. Vectors serve as a set of reproducible scenarios that allow developers to ensure their changes haven't broken Visor. All that's required is a Lotus repo.

How to build a vector

To build a test vector, use the vector build command. The example below creates a test vector file named "blocks_471000-471010.json".

visor --repo-read-only=false --repo="~/.lotus" vector build --tasks=blocks --from=471000 --to=471010 --vector-file="./blocks_471000-471010.json" --vector-desc="block models from epoch 471000 to 471010"

The file contains information about the version of Visor used to build it:

$ cat blocks_471000-471010.json | jq ".metadata"
{
 "version": "v0.4.0+rc2-55-g17c8329-dirty",
 "description": "block models from epoch 471000 to 471010",
 "network": "mainnet",
 "time": 1613515320
}

the commands to produce it:

$ cat blocks_471000-471010.json | jq ".parameters"
{
 "from": 471000,
 "to": 471010,
 "tasks": [
   "blocks"
 ],
 "actor-address": ""
}

the extracted chain state as a base64 encoded CAR file:

$ cat blocks_471000-471010.json | jq ".car"
<lot_of_data>

and the models extracted:

$ cat blocks_471000-471010.json | jq ".expected[].block_headers"
[
 {
   "Height": 471010,
   "Cid": "bafy2bzacecq2fysiktcuc5r762hvuxstwv4qzx645ld4h66spenheckyuko2y",
   "Miner": "f02770",
   "ParentWeight": "10230517226",
   "ParentBaseFee": "3275882399",
   "ParentStateRoot": "bafy2bzacecsckqgj6ufbefj7bx6odzvvbftky77py5x2th3wrqx3yvpksyuc6",
   "WinCount": 1,
   "Timestamp": 1612436700,
   "ForkSignaling": 0
 },
 {
   "Height": 471010,
   "Cid": "bafy2bzacecxmra4cdbqq7lxao7e2gmxvf7ef3o464et7ejykcn24itps7vnau",
   "Miner": "f02775",
   "ParentWeight": "10230517226",
   "ParentBaseFee": "3275882399",
   "ParentStateRoot": "bafy2bzacecsckqgj6ufbefj7bx6odzvvbftky77py5x2th3wrqx3yvpksyuc6",
   "WinCount": 1,
   "Timestamp": 1612436700,
   "ForkSignaling": 0
 },
 ...

How to execute a vector

To execute a vector, use the vector execute command. The example below executes a test vector file named "blocks_471000-471010.json".

$ ./visor vector execute --vector-file=blocks_471000-471010.json
2021-02-16T16:32:29.225-0800   INFO   vector vector/runner.go:146 Validate Model block_headers: Passed
2021-02-16T16:32:29.230-0800   INFO   vector vector/runner.go:146 Validate Model block_parents: Passed
2021-02-16T16:32:29.230-0800   INFO   vector vector/runner.go:146 Validate Model drand_block_entries: Passed

When executing the vector file Visor uses the data in the car field as its data source, executes the commands in the parameters field, then validates the data returned from the execution matches the expected models in the vector file.

How to save a vector

Vector files are stored on the IPFS network, and a list of their hashes is kept in the VECTOR_MANIFEST file. Storing the hashes in the git repo and the files in IPFS helps keep our repo compact. The example below will demonstrate how to save a vector file to run as a part of CI. We will use the aforementioned blocks_471000-471010.json in this example.

$ ipfs add blocks_471000-471010.json
added QmPTKSr1uwExGTVByDUoVm4CfrjU34oNV6gfvWvufaN9h1 blocks_471000-471010.json
 146.43 KiB / 146.43 KiB [==] 100.00%

next, add the hash to the manifest:

$ echo QmPTKSr1uwExGTVByDUoVm4CfrjU34oNV6gfvWvufaN9h1 >> ./vector/VECTOR_MANIFEST

then make the vector deps:

$ make deps
cd ./vector; ./fetch_vectors.sh
Fetching QmPTKSr1uwExGTVByDUoVm4CfrjU34oNV6gfvWvufaN9h1

You can see the vector file is in vectors/data/:

ls vector/data/
QmPTKSr1uwExGTVByDUoVm4CfrjU34oNV6gfvWvufaN9h1_block_models_from_epoch_471000_to_471010.json

Finally, execute the vector unit tests:

$ go test -v ./vector/...
=== RUN  TestExecuteVectors
=== RUN  TestExecuteVectors/QmPTKSr1uwExGTVByDUoVm4CfrjU34oNV6gfvWvufaN9h1_block_models_from_epoch_471000_to_471010.json
--- PASS: TestExecuteVectors (65.08s)
   --- PASS: TestExecuteVectors/QmPTKSr1uwExGTVByDUoVm4CfrjU34oNV6gfvWvufaN9h1_block_models_from_epoch_471000_to_471010.json (0.02s)

Push your changes to VECTOR_MANIFEST up execution in CI.

Running vector benchmarks

The vectors can be executed as part of a Go benchmark suite. The benchmarks output numbers of blockstore gets per operation as a custom metric.

Due to the length of time needed to execute, it is recommended to supply a custom benchtime for running each benchmark and an overall timeout for the test run that is greater than the expected benchtime for all benchmarks.

To run benchmarks for all the vectors

go test -v -bench=BenchmarkExecuteVectors/.* -benchtime 1m -timeout 30m  ./vector/...

To run benchmarks for a single vector:

go test -v -bench=BenchmarkExecuteVectors/QmYhszKDYDFvXyFQF8KefF4M8gcNaoTC8FMYFcgKmfq6pi_miner.* -benchtime 1m -timeout 30m  ./vector/...

Use go test -run=^$ to disable running the vector tests.

Documentation

Index

• type Base64EncodedBytes
  • func (b Base64EncodedBytes) MarshalJSON() ([]byte, error)
  • func (b Base64EncodedBytes) String() string
  • func (b *Base64EncodedBytes) UnmarshalJSON(v []byte) error
• type Builder
  • func NewBuilder(cctx *cli.Context) (*Builder, error)
  • func (b *Builder) Build(ctx context.Context) (*BuilderSchema, error)
• type BuilderExpected
• type BuilderSchema
  • func (bs *BuilderSchema) Persist(path string) error
• type Metadata
• type Options
• type Parameters
• type Runner
  • func NewRunner(ctx context.Context, vectorPath string, cacheHint int) (*Runner, error)
  • func (r *Runner) BlockstoreGetCount() int64
  • func (r *Runner) Reset()
  • func (r *Runner) Run(ctx context.Context) error
  • func (r *Runner) Validate(ctx context.Context) error
• type RunnerExpected
• type RunnerSchema

Types

type Base64EncodedBytes

type Base64EncodedBytes []byte

Base64EncodedBytes is a base64-encoded binary value.

func (Base64EncodedBytes) MarshalJSON

func (b Base64EncodedBytes) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshal for Base64EncodedBytes

func (Base64EncodedBytes) String

func (b Base64EncodedBytes) String() string

func (*Base64EncodedBytes) UnmarshalJSON

func (b *Base64EncodedBytes) UnmarshalJSON(v []byte) error

UnmarshalJSON implements json.Unmarshal for Base64EncodedBytes

type Builder

type Builder struct {
	From          int64
	To            int64
	Tasks         []string
	AddressFilter string
	Description   string
	// contains filtered or unexported fields
}

func NewBuilder

func NewBuilder(cctx *cli.Context) (*Builder, error)

func (*Builder) Build

func (b *Builder) Build(ctx context.Context) (*BuilderSchema, error)

type BuilderExpected

type BuilderExpected struct {
	Models map[string][]interface{}
}

type BuilderSchema

type BuilderSchema struct {
	Meta   Metadata           `json:"metadata"`
	Params Parameters         `json:"parameters"`
	CAR    Base64EncodedBytes `json:"car"`
	Exp    BuilderExpected    `json:"expected"`
}

func (*BuilderSchema) Persist

func (bs *BuilderSchema) Persist(path string) error

type Metadata

type Metadata struct {
	Version     string `json:"version"`
	Description string `json:"description"`
	Network     string `json:"network"`
	Date        int64  `json:"time"`
}

type Options

type Options map[string]interface{}

type Parameters

type Parameters struct {
	From          int64    `json:"from"`
	To            int64    `json:"to"`
	Tasks         []string `json:"tasks"`
	AddressFilter string   `json:"address-filter"`
}

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

func NewRunner

func NewRunner(ctx context.Context, vectorPath string, cacheHint int) (*Runner, error)

func (*Runner) BlockstoreGetCount

func (r *Runner) BlockstoreGetCount() int64

func (*Runner) Reset

func (r *Runner) Reset()

func (*Runner) Run

func (r *Runner) Run(ctx context.Context) error

func (*Runner) Validate

func (r *Runner) Validate(ctx context.Context) error

type RunnerExpected

type RunnerExpected struct {
	Models map[string]json.RawMessage
}

type RunnerSchema

type RunnerSchema struct {
	Meta   Metadata           `json:"metadata"`
	Params Parameters         `json:"parameters"`
	CAR    Base64EncodedBytes `json:"car"`
	Exp    RunnerExpected     `json:"expected"`
}