VulcanizeDB transformers for watching ETH and token value transfers
This repo contains transformers for indexing of ETH and token balances for account addresses.
This transformer works by filtering through all eth logs that have one of these topic0. These events are unpacked and converted to generic "Value Transfer" records. Token balances for user accounts are then constructed as views on these records. It then polls an archival Eth node to retrieve balances for these accounts and generate eth balance records.
- Setup VulcanizeDB
- Switch to
- Setup config for
- Setup Postgraphile to expose the
These transformers are run as plugin to VulcanizeDB's
To begin, setup VulcanizeDB as described here.
Once vulcanizeDB is setup and built, run vulcanizeDB in
to begin syncing headers into Postgres. It is vital that this sync process begins at a block before the
account.start field below.
lightSync has begun, we can run the
composeAndExecute command to compose and execute our account transformer. To
do so, we use a normal
compose config with two additional parameter maps:
[token] addresses = [ "0x0000000000085d4780B73119b644AE5ecd22b376", ] [token.equivalents] 0x0000000000085d4780B73119b644AE5ecd22b376 = [ "0x8dd5fbCe2F6a956C3022bA3663759011Dd51e73E" ] [account] start = 0 addresses = [ "0x48E78948C80e9f8F53190DbDF2990f9a69491ef4", "0x009C1E8674038605C5AE33C74f13bC528E1222B5" ]
contract.addresses are a list of the token addresses we want to track balances for, these addresses are used create token balance views.
This can be updated at runtime by adding new contract addresses to the
accounts.contract_addresses table in Postgres:
CREATE TABLE accounts.contract_addresses ( contract BYTEA PRIMARY KEY );
contract.equivalents is used to manually map contract addresses which represent the same token and need to be tracked
as such. For example, TrueUSD as shown above has a proxy contract
contract that was recently upgraded to from a direct implementation at
These two addresses do not emit each other's events and so to track the balance of TrueUSD we are configuring our
transformer to watch events emitted from both these addresses as though they all belong to
account.start is used to specify when to begin watching events and producing token and eth balance records for the user accounts,
this needs to be set to a block lower than the deployment block of any tokens we want to track. Additionally, this block number must fall within
the contiguous set of unchecked_headers (this is important if we need to restart a sync, we will need to restart from the lowest unchecked header)
account.addresses is used to specify which user account addresses we want to track and index ETH balance and token balance
records for. This can be updated at runtime by adding new addresses to the
accounts.addresses table in Postgres:
CREATE TABLE accounts.addresses ( address BYTEA PRIMARY KEY );
Currently, this config's
ipcPath needs to point to an archival node endpoint in order to track ETH balances, this will be deprecated
by the use of state diff data in the near future.
To expose the transformed data over Postgraphile, we need to modify our Postgraphile config.ts to include the "accounts" schema
["public", "accounts"]). After this, we should be able to expose graphQL endpoints as usual.
The transformer processes value transfer events from all contract addresses into uniform records of the form:
CREATE TABLE accounts.token_value_transfers ( id SERIAL PRIMARY KEY, header_id INTEGER NOT NULL REFERENCES headers (id) ON DELETE CASCADE, block_number BIGINT NOT NULL, name VARCHAR NOT NULL CHECK (name <> ''), dst VARCHAR(42), src VARCHAR(42), amount NUMERIC, contract VARCHAR(42) NOT NULL, log_idx INTEGER NOT NULL, tx_idx INTEGER NOT NULL, raw_log JSONB, UNIQUE (header_id, tx_idx, log_idx) );
A view on a join of these records with the
accounts.addresses table, the
accounts.contract_addresses table, and
public.headers table is used to construct our users' token balance records:
CREATE OR REPLACE VIEW accounts.address_token_balances AS SELECT accounts.addresses.address AS address_hash, accounts.contract_addresses.contract AS token_contract_address_hash, public.headers.block_number, ((SELECT COALESCE(SUM(amount),0) FROM accounts.token_value_transfers WHERE accounts.token_value_transfers.block_number <= public.headers.block_number AND accounts.token_value_transfers.dst = accounts.addresses.address AND accounts.token_value_transfers.contract = accounts.contract_addresses.contract) - (SELECT COALESCE(SUM(amount),0) FROM accounts.token_value_transfers WHERE accounts.token_value_transfers.block_number <= public.headers.block_number AND accounts.token_value_transfers.src = accounts.addresses.address AND accounts.token_value_transfers.contract = accounts.contract_addresses.contract)) AS "value" FROM accounts.token_value_transfers, accounts.addresses, public.headers, accounts.contract_addresses GROUP BY accounts.addresses.address, accounts.contract_addresses.contract, public.headers.block_number;
Which produces a view equivalent to the below table:
CREATE TABLE accounts.address_token_balances ( address_hash BYTEA NOT NULL, block_number BIGINT NOT NULL, token_contract_address_hash BYTEA NOT NULL, value NUMERIC, );
Additionally, for each user account it is configured with, it fetches their ETH balances and persists them as coin balance records of the form:
CREATE TABLE accounts.address_coin_balances ( id SERIAL PRIMARY KEY, header_id INTEGER NOT NULL REFERENCES headers (id) ON DELETE CASCADE, address_hash BYTEA NOT NULL, block_number BIGINT NOT NULL, value NUMERIC(100,0), value_fetched_at TIMESTAMP WITHOUT TIME ZONE, inserted_at TIMESTAMP WITHOUT TIME ZONE NOT NULL, updated_at TIMESTAMP WITHOUT TIME ZONE, UNIQUE (address_hash, block_number) );
It also syncs the users (and only the users) transactions and receipts into the core vDB
CREATE TABLE light_sync_transactions ( id SERIAL PRIMARY KEY, header_id INTEGER NOT NULL REFERENCES headers(id) ON DELETE CASCADE, hash VARCHAR(66), gaslimit NUMERIC, gasprice NUMERIC, input_data BYTEA, nonce NUMERIC, raw BYTEA, tx_from VARCHAR(44), tx_index INTEGER, tx_to VARCHAR(44), "value" NUMERIC, UNIQUE (header_id, hash) );
CREATE TABLE light_sync_receipts( id SERIAL PRIMARY KEY, transaction_id INTEGER NOT NULL REFERENCES light_sync_transactions(id) ON DELETE CASCADE, header_id INTEGER NOT NULL REFERENCES headers(id) ON DELETE CASCADE, contract_address VARCHAR(42), cumulative_gas_used NUMERIC, gas_used NUMERIC, state_root VARCHAR(66), status INTEGER, tx_hash VARCHAR(66), rlp BYTEA, UNIQUE(header_id, transaction_id) );
If you notice a value transfer type event is missing from the ones we are already tracking, please feel free to submit a PR to introduce the event or submit an issue to note it for inclusion.
You should have received a copy of the GNU Affero General Public License along with this program.
|You should have received a copy of the GNU Affero General Public License along with this program.|