indexer

package
v1.1.28 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 11, 2021 License: GPL-3.0 Imports: 39 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrBackOff = errors.New("back off something is not working well")

ErrBackOff signals that an error was received from the server

View Source 
var ErrCouldNotCreatePolicy = errors.New("could not create policy")

ErrCouldNotCreatePolicy -

View Source 
var ErrEmptyEnabledIndexes = errors.New("empty enabled indexes slice")

ErrEmptyEnabledIndexes signals that an empty slice of enables indexes has been provided

View Source 
var ErrNegativeCacheSize = errors.New("negative cache size")

ErrNegativeCacheSize signals that a invalid cache size has been provided

View Source 
var ErrNilAccountsDB = errors.New("nil accounts db")

ErrNilAccountsDB signals that a nil accounts database has been provided

View Source 
var ErrNilDataDispatcher = errors.New("nil data dispatcher")

ErrNilDataDispatcher signals that an operation has been attempted to or with a nil data dispatcher implementation

View Source 
var ErrNilDatabaseClient = errors.New("nil database client")

ErrNilDatabaseClient signals that an operation has been attempted to or with a nil database client implementation

View Source 
var ErrNilElasticProcessor = errors.New("nil elastic processor")

ErrNilElasticProcessor signals that an operation has been attempted to or with a nil elastic processor implementation

View Source 
var ErrNilOptions = errors.New("nil options")

ErrNilOptions signals that structure that contains indexer options is nil

View Source 
var ErrNilPubkeyConverter = errors.New("nil pubkey converter")

ErrNilPubkeyConverter signals that an operation has been attempted to or with a nil public key converter implementation

View Source 
var ErrNilShardCoordinator = errors.New("nil shard coordinator")

ErrNilShardCoordinator signals that a nil shard coordinator was provided

View Source 
var ErrNoElasticUrlProvided = errors.New("no elastic url provided")

ErrNoElasticUrlProvided -

View Source 
var ErrReadPolicyFile = errors.New("error while reading policy file")

ErrReadPolicyFile signals that a read error occurred while reading policy file

View Source 
var ErrReadTemplatesFile = errors.New("error while reading template file")

ErrReadTemplatesFile signals that a read error occurred while reading template file

View Source 
var ErrWriteToBuffer = errors.New("error while writing to buffer")

ErrWriteToBuffer signals that a write error occurred

Functions

func GetElasticTemplatesAndPolicies added in v1.1.6 

func GetElasticTemplatesAndPolicies(path string, useKibana bool) (map[string]*bytes.Buffer, map[string]*bytes.Buffer, error)

GetElasticTemplatesAndPolicies will return elastic templates and policies

func NewDataDispatcher added in v1.1.6 

func NewDataDispatcher(cacheSize int) (*dataDispatcher, error)

NewDataDispatcher creates a new dataDispatcher instance, capable of saving sequentially data in elasticsearch database

func NewElasticClient added in v1.1.6 

func NewElasticClient(cfg elasticsearch.Config) (*elasticClient, error)

NewElasticClient will create a new instance of elasticClient

Types

type AccountBalanceHistory added in v1.1.6 

type AccountBalanceHistory struct {
	Address   string `json:"address"`
	Timestamp int64  `json:"timestamp"`
	Balance   string `json:"balance"`
}

AccountBalanceHistory represents an entry in the user accounts balances history

type AccountInfo added in v1.1.6 

type AccountInfo struct {
	Nonce      uint64  `json:"nonce"`
	Balance    string  `json:"balance"`
	BalanceNum float64 `json:"balanceNum"`
}

AccountInfo holds (serializable) data about an account

type ArgDataIndexer added in v1.1.6 

type ArgDataIndexer struct {
	ShardCoordinator   sharding.Coordinator
	Marshalizer        marshal.Marshalizer
	EpochStartNotifier sharding.EpochStartEventNotifier
	NodesCoordinator   sharding.NodesCoordinator
	Options            *Options
	DataDispatcher     DispatcherHandler
	ElasticProcessor   ElasticProcessor
}

ArgDataIndexer is struct that is used to store all components that are needed to create a indexer

type ArgElasticProcessor added in v1.1.6 

type ArgElasticProcessor struct {
	IndexTemplates           map[string]*bytes.Buffer
	IndexPolicies            map[string]*bytes.Buffer
	Marshalizer              marshal.Marshalizer
	Hasher                   hashing.Hasher
	AddressPubkeyConverter   core.PubkeyConverter
	ValidatorPubkeyConverter core.PubkeyConverter
	Options                  *Options
	DBClient                 DatabaseClientHandler
	EnabledIndexes           map[string]struct{}
	AccountsDB               state.AccountsAdapter
	Denomination             int
	TransactionFeeCalculator process.TransactionFeeCalculator
	IsInImportDBMode         bool
	ShardCoordinator         sharding.Coordinator
}

ArgElasticProcessor is struct that is used to store all components that are needed to an elastic indexer

type Block 

type Block struct {
	Nonce                 uint64        `json:"nonce"`
	Round                 uint64        `json:"round"`
	Epoch                 uint32        `json:"epoch"`
	Hash                  string        `json:"-"`
	MiniBlocksHashes      []string      `json:"miniBlocksHashes"`
	NotarizedBlocksHashes []string      `json:"notarizedBlocksHashes"`
	Proposer              uint64        `json:"proposer"`
	Validators            []uint64      `json:"validators"`
	PubKeyBitmap          string        `json:"pubKeyBitmap"`
	Size                  int64         `json:"size"`
	SizeTxs               int64         `json:"sizeTxs"`
	Timestamp             time.Duration `json:"timestamp"`
	StateRootHash         string        `json:"stateRootHash"`
	PrevHash              string        `json:"prevHash"`
	ShardID               uint32        `json:"shardId"`
	TxCount               uint32        `json:"txCount"`
	SearchOrder           uint64        `json:"searchOrder"`
}

Block is a structure containing all the fields that need 

to be saved for a block. It has all the default fields
plus some extra information for ease of search and filter

type DataIndexerFactory added in v1.1.6 

type DataIndexerFactory interface {
	Create() (Indexer, error)
	IsInterfaceNil() bool
}

DataIndexerFactory can create new instances of Indexer

type DatabaseClientHandler added in v1.1.6 

type DatabaseClientHandler interface {
	DoRequest(req *esapi.IndexRequest) error
	DoBulkRequest(buff *bytes.Buffer, index string) error
	DoBulkRemove(index string, hashes []string) error
	DoMultiGet(query objectsMap, index string) (objectsMap, error)

	CheckAndCreateIndex(index string) error
	CheckAndCreateAlias(alias string, index string) error
	CheckAndCreateTemplate(templateName string, template *bytes.Buffer) error
	CheckAndCreatePolicy(policyName string, policy *bytes.Buffer) error

	IsInterfaceNil() bool
}

DatabaseClientHandler is an interface that do requests to elasticsearch server

type DispatcherHandler added in v1.1.6 

type DispatcherHandler interface {
	StartIndexData()
	Close() error
	Add(item workItems.WorkItemHandler)
	IsInterfaceNil() bool
}

DispatcherHandler defines the interface for the dispatcher that will manage when items are saved in elasticsearch database

type ElasticProcessor added in v1.1.6 

type ElasticProcessor interface {
	SaveShardStatistics(tpsBenchmark statistics.TPSBenchmark) error
	SaveHeader(header data.HeaderHandler, signersIndexes []uint64, body *block.Body, notarizedHeadersHashes []string, txsSize int) error
	RemoveHeader(header data.HeaderHandler) error
	RemoveMiniblocks(header data.HeaderHandler, body *block.Body) error
	SaveMiniblocks(header data.HeaderHandler, body *block.Body) (map[string]bool, error)
	SaveTransactions(body *block.Body, header data.HeaderHandler, txPool map[string]data.TransactionHandler, selfShardID uint32, mbsInDb map[string]bool) error
	SaveValidatorsRating(index string, validatorsRatingInfo []workItems.ValidatorRatingInfo) error
	SaveRoundsInfo(infos []workItems.RoundInfo) error
	SaveShardValidatorsPubKeys(shardID, epoch uint32, shardValidatorsPubKeys [][]byte) error
	SetTxLogsProcessor(txLogsProc process.TransactionLogProcessorDatabase)
	SaveAccounts(accounts []state.UserAccountHandler) error
	IsInterfaceNil() bool
}

ElasticProcessor defines the interface for the elastic search indexer

func NewElasticProcessor added in v1.1.6 

func NewElasticProcessor(arguments ArgElasticProcessor) (ElasticProcessor, error)

NewElasticProcessor creates an elasticsearch es and handles saving

type Event added in v1.0.114 

type Event struct {
	Address    string   `json:"address"`
	Identifier string   `json:"identifier"`
	Topics     []string `json:"topics"`
	Data       string   `json:"data"`
}

Event holds all the data needed for an event structure

type FeesProcessorHandler added in v1.1.18 

type FeesProcessorHandler interface {
	ComputeGasUsedAndFeeBasedOnRefundValue(tx process.TransactionWithFeeHandler, refundValueStr string) (uint64, *big.Int)
	ComputeTxFeeBasedOnGasUsed(tx process.TransactionWithFeeHandler, gasUsed uint64) *big.Int
	ComputeMoveBalanceGasUsed(tx process.TransactionWithFeeHandler) uint64
}

FeesProcessorHandler defines the interface for the transaction fees processor

type Indexer 

type Indexer interface {
	SetTxLogsProcessor(txLogsProc process.TransactionLogProcessorDatabase)
	//SaveBlock TODO add a structure instead of these params
	SaveBlock(body data.BodyHandler, header data.HeaderHandler, txPool map[string]data.TransactionHandler,
		signersIndexes []uint64, notarizedHeadersHashes []string, headerHash []byte)
	RevertIndexedBlock(header data.HeaderHandler, body data.BodyHandler)
	SaveRoundsInfo(roundsInfos []workItems.RoundInfo)
	UpdateTPS(tpsBenchmark statistics.TPSBenchmark)
	SaveValidatorsPubKeys(validatorsPubKeys map[uint32][][]byte, epoch uint32)
	SaveValidatorsRating(indexID string, infoRating []workItems.ValidatorRatingInfo)
	SaveAccounts(acc []state.UserAccountHandler)
	Close() error
	IsInterfaceNil() bool
	IsNilIndexer() bool
}

Indexer is an interface for saving node specific data to other storage. This could be an elastic search index, a MySql database or any other external services.

func NewDataIndexer added in v1.1.6 

func NewDataIndexer(arguments ArgDataIndexer) (Indexer, error)

NewDataIndexer will create a new data indexer

type Miniblock added in v0.0.5 

type Miniblock struct {
	Hash              string `json:"-"`
	SenderShardID     uint32 `json:"senderShard"`
	ReceiverShardID   uint32 `json:"receiverShard"`
	SenderBlockHash   string `json:"senderBlockHash"`
	ReceiverBlockHash string `json:"receiverBlockHash"`
	Type              string `json:"type"`
}

Miniblock is a structure containing miniblock information

type NilIndexer 

type NilIndexer struct {
}

NilIndexer will be used when an Indexer is required, but another one isn't necessary or available

func NewNilIndexer 

func NewNilIndexer() *NilIndexer

NewNilIndexer will return a Nil indexer

func (*NilIndexer) Close added in v1.1.6 

func (ni *NilIndexer) Close() error

Close will do nothing

func (*NilIndexer) IsInterfaceNil 

func (ni *NilIndexer) IsInterfaceNil() bool

IsInterfaceNil returns true if there is no value under the interface

func (*NilIndexer) IsNilIndexer 

func (ni *NilIndexer) IsNilIndexer() bool

IsNilIndexer will return a bool value that signals if the indexer's implementation is a NilIndexer

func (*NilIndexer) RevertIndexedBlock added in v1.1.6 

func (ni *NilIndexer) RevertIndexedBlock(_ data.HeaderHandler, _ data.BodyHandler)

RevertIndexedBlock will do nothing

func (*NilIndexer) SaveAccounts added in v1.1.6 

func (ni *NilIndexer) SaveAccounts(_ []state.UserAccountHandler)

SaveAccounts won't do anything as this is a nil implementation

func (*NilIndexer) SaveBlock 

func (ni *NilIndexer) SaveBlock(_ data.BodyHandler, _ data.HeaderHandler, _ map[string]data.TransactionHandler, _ []uint64, _ []string, _ []byte)

SaveBlock will do nothing

func (*NilIndexer) SaveRoundsInfo added in v1.1.6 

func (ni *NilIndexer) SaveRoundsInfo(_ []workItems.RoundInfo)

SaveRoundsInfo will do nothing

func (*NilIndexer) SaveValidatorsPubKeys 

func (ni *NilIndexer) SaveValidatorsPubKeys(_ map[uint32][][]byte, _ uint32)

SaveValidatorsPubKeys will do nothing

func (*NilIndexer) SaveValidatorsRating added in v0.0.5 

func (ni *NilIndexer) SaveValidatorsRating(_ string, _ []workItems.ValidatorRatingInfo)

SaveValidatorsRating --

func (*NilIndexer) SetTxLogsProcessor added in v1.0.114 

func (ni *NilIndexer) SetTxLogsProcessor(_ process.TransactionLogProcessorDatabase)

SetTxLogsProcessor will do nothing

func (*NilIndexer) UpdateTPS 

func (ni *NilIndexer) UpdateTPS(_ statistics.TPSBenchmark)

UpdateTPS will do nothing

type Options 

type Options struct {
	IndexerCacheSize int
	UseKibana        bool
}

Options structure holds the indexer's configuration options

type ScResult added in v1.0.114 

type ScResult struct {
	Hash           string `json:"hash"`
	Nonce          uint64 `json:"nonce"`
	GasLimit       uint64 `json:"gasLimit"`
	GasPrice       uint64 `json:"gasPrice"`
	Value          string `json:"value"`
	Sender         string `json:"sender"`
	Receiver       string `json:"receiver"`
	RelayerAddr    string `json:"relayerAddr,omitempty"`
	RelayedValue   string `json:"relayedValue,omitempty"`
	Code           string `json:"code,omitempty"`
	Data           []byte `json:"data,omitempty"`
	PreTxHash      string `json:"prevTxHash"`
	OriginalTxHash string `json:"originalTxHash"`
	CallType       string `json:"callType"`
	CodeMetadata   []byte `json:"codeMetaData,omitempty"`
	ReturnMessage  string `json:"returnMessage,omitempty"`
}

ScResult is a structure containing all the fields that need to be saved for a smart contract result

type TPS 

type TPS struct {
	LiveTPS               float64  `json:"liveTPS"`
	PeakTPS               float64  `json:"peakTPS"`
	BlockNumber           uint64   `json:"blockNumber"`
	RoundNumber           uint64   `json:"roundNumber"`
	RoundTime             uint64   `json:"roundTime"`
	AverageBlockTxCount   *big.Int `json:"averageBlockTxCount"`
	TotalProcessedTxCount *big.Int `json:"totalProcessedTxCount"`
	AverageTPS            *big.Int `json:"averageTPS"`
	CurrentBlockNonce     uint64   `json:"currentBlockNonce"`
	NrOfShards            uint32   `json:"nrOfShards"`
	NrOfNodes             uint32   `json:"nrOfNodes"`
	LastBlockTxCount      uint32   `json:"lastBlockTxCount"`
	ShardID               uint32   `json:"shardID"`
}

TPS is a structure containing all the fields that need to 

be saved for a shard statistic in the database

type Transaction 

type Transaction struct {
	Hash                 string        `json:"-"`
	MBHash               string        `json:"miniBlockHash"`
	BlockHash            string        `json:"-"`
	Nonce                uint64        `json:"nonce"`
	Round                uint64        `json:"round"`
	Value                string        `json:"value"`
	Receiver             string        `json:"receiver"`
	Sender               string        `json:"sender"`
	ReceiverShard        uint32        `json:"receiverShard"`
	SenderShard          uint32        `json:"senderShard"`
	GasPrice             uint64        `json:"gasPrice"`
	GasLimit             uint64        `json:"gasLimit"`
	GasUsed              uint64        `json:"gasUsed"`
	Fee                  string        `json:"fee"`
	Data                 []byte        `json:"data"`
	Signature            string        `json:"signature"`
	Timestamp            time.Duration `json:"timestamp"`
	Status               string        `json:"status"`
	SearchOrder          uint32        `json:"searchOrder"`
	SmartContractResults []ScResult    `json:"scResults,omitempty"`
	SenderUserName       []byte        `json:"senderUsername,omitempty"`
	ReceiverUserName     []byte        `json:"receiverUsername,omitempty"`
	Log                  TxLog         `json:"-"`
	// contains filtered or unexported fields
}

Transaction is a structure containing all the fields that need 

to be saved for a transaction. It has all the default fields
plus some extra information for ease of search and filter

func (*Transaction) GetData added in v1.1.18 

func (t *Transaction) GetData() []byte

GetData will return transaction data field

func (*Transaction) GetGasLimit added in v1.1.18 

func (t *Transaction) GetGasLimit() uint64

GetGasLimit will return transaction gas limit

func (*Transaction) GetGasPrice added in v1.1.18 

func (t *Transaction) GetGasPrice() uint64

GetGasPrice will return transaction gas price

func (*Transaction) GetRcvAddr added in v1.1.18 

func (t *Transaction) GetRcvAddr() []byte

GetRcvAddr will return transaction receiver address

func (*Transaction) GetValue added in v1.1.18 

func (t *Transaction) GetValue() *big.Int

GetValue wil return transaction value

type TxLog added in v1.0.114 

type TxLog struct {
	Address string  `json:"scAddress"`
	Events  []Event `json:"events"`
}

TxLog holds all the data needed for a log structure

type ValidatorsPublicKeys 

type ValidatorsPublicKeys struct {
	PublicKeys []string `json:"publicKeys"`
}

ValidatorsPublicKeys is a structure containing fields for validators public keys

type ValidatorsRatingInfo added in v0.0.5 

type ValidatorsRatingInfo struct {
	ValidatorsInfos []workItems.ValidatorRatingInfo `json:"validatorsRating"`
}

ValidatorsRatingInfo is a structure containing validators information

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.