The hcexplorer repository is a collection of golang packages and apps for Hcd data collection, storage, and presentation.
../hcexplorer The hcexplorer daemon. ├── blockdata Package blockdata. ├── cmd │ ├── rebuilddb rebuilddb utility, for SQLite backend. │ ├── rebuilddb2 rebuilddb2 utility, for PostgreSQL backend. │ └── scanblocks scanblocks utility. ├── dcrdataapi Package dcrdataapi for golang API clients. ├── db │ ├── dbtypes Package dbtypes with common data types. │ ├── dcrpg Package dcrpg providing PostgreSQL backend. │ └── dcrsqlite Package dcrsqlite providing SQLite backend. ├── public Public resources for block explorer (css, js, etc.). ├── explorer Package explorer, powering the block explorer. ├── mempool Package mempool. ├── rpcutils Package rpcutils. ├── semver Package semver. ├── stakedb Package stakedb, for tracking tickets. ├── txhelpers Package txhelpers. └── views HTML templates for block explorer.
The root of the repository is the
main package for the hcexplorer app, which has
several components including:
- Block explorer (web interface).
- Blockchain monitoring and data collection.
- Mempool monitoring and reporting.
- Data storage in durable database (sqlite presently).
- RESTful JSON API over HTTP(S).
After hcexplorer syncs with the blockchain server via RPC, by default it will begin
listening for HTTP connections on
http://127.0.0.1:7777/. This means it starts
a web server listening on IPv4 localhost, port 7777. Both the interface and port
are configurable. The block explorer and the JSON API are both provided by the
server on this port. See JSON REST API for details.
Note that while hcexplorer can be started with HTTPS support, it is recommended to employ a reverse proxy such as nginx. See sample-nginx.conf for an example nginx configuration.
A new database backend using PostgreSQL was introduced in v0.9.0 that provides
expanded functionality. However, initial population of the database takes
additional time and tens of gigabytes of disk storage space. To disable the
PostgreSQL backend (and the expanded functionality), hcexplorer may be started with
-l for short) command line flag.
JSON REST API
The API serves JSON data over HTTP(S). All
API endpoints are currently prefixed with
http://localhost:7777/api/stake), but this may be configurable in the future.
|Verbose block result||
|Block X (block index)|
|Verbose block result||
|Block H (block hash)|
|Verbose block result||
|Block range (X < Y)|
|Summary array for blocks on
|Summary array with block index step
|Size (bytes) array||
|Size array with step
|Transaction T (transaction id)|
|Details for input at index
|Details for output at index
|Summary of last 10 transactions||
|Verbose transaction result for last
|Summary of last
|Verbose transaction result for last
|Stake Difficulty (Ticket Price)|
|Current sdiff and estimates||
|Sdiff for block
|Sdiff for block range
|Current sdiff separately||
|Current pool info (size, total value, and average price)||
|Pool info for block
|Pool info for block range
*For the pool info block range endpoint that accepts the
arrays url query,
a value of
true will put all pool values and pool sizes into separate arrays,
rather than having a single array of pool info JSON objects. This may make
parsing more efficient for the client.
|Ticket fee rate summary||
|Ticket fee rate list (all)||
|Ticket fee rate list (N highest)||
|Detailed ticket list (fee, hash, size, age, etc.)||
|Detailed ticket list (N highest fee rates)||
|Endpoint list (always indented)||
All JSON endpoints accept the URL query
indent=[true|false]. For example,
/stake/diff?indent=true. By default, indentation is off. The characters to use
for indentation may be specified with the
indentjson string configuration
Important Note About Mempool
Although there is mempool data collection and serving, it is very important to keep in mind that the mempool in your node (hcd) is not likely to be the same as other nodes' mempool. Also, your mempool is cleared out when you shutdown hcd. So, if you have recently (e.g. after the start of the current ticket price window) started hcd, your mempool will be missing transactions that other nodes have.
Command Line Utilities
rebuilddb is a CLI app that performs a full blockchain scan that fills past block data into a SQLite database. This functionality is included in the startup of the hcexplorer daemon, but may be called alone with rebuilddb.
rebuilddb2 is a CLI app used for maintenance of hcexplorer's
(a.k.a. DB v2) that uses PostgreSQL to store a nearly complete record of the
Hcd blockchain data. See the README.md for
rebuilddb2 for important usage information.
scanblocks is a CLI app to scan the blockchain and save data into a JSON file. More details are in its own README. The repository also includes a shell script, jsonarray2csv.sh, to convert the result into a comma-separated value (CSV) file.
package dcrdataapi defines the data types, with json tags, used by the JSON
API. This facilitates authoring of robust golang clients of the API.
package dbtypes defines the data types used by the DB backends to model the
block, transaction, and related blockchain data structures. Functions for
converting from standard Hcd data types (e.g.
wire.MsgBlock) are also
package rpcutils includes helper functions for interacting with a
package stakedb defines the
ChainMonitor types for
efficiently tracking live tickets, with the primary purpose of computing ticket
pool value quickly. It uses the
database.DB type from
github.com/coolsnady/hcd/database with an ffldb storage backend from
github.com/coolsnady/hcd/database/ffldb. It also makes use of the
handles connecting new blocks and chain reorganization in response to notifications
package txhelpers includes helper functions for working with the common types
chainhash.Hash, and others.
dcrsqlite are currently designed only for internal
use internal use by other hcexplorer packages, but they may be of general value in
chainMonitortype and its
BlockConnectedHandler()method that handles block-connected notifications and triggers data collection and storage.
BlockDatatype and methods for converting to API types.
blockDataCollectortype and its
CollectHash()methods that are called by the chain monitor when a new block is detected.
BlockDataSaverinterface required by
chainMonitorfor storage of collected data.
ChainDBtype, which is the primary exported type from
dcrpg, providing an interface for a PostgreSQL database.
- A large set of lower-level functions to perform a range of queries given a
*sql.DBinstance and various parameters.
- The internal package contains the raw SQL statements.
sql.DBwrapper type (
DB) with the necessary SQLite queries for storage and retrieval of block and stake data.
wiredDBtype, intended to satisfy the
APIDataSourceinterface used by the hcexplorer app's API. The block header is not stored in the DB, so a RPC client is used by
wiredDBto get it on demand.
wiredDBalso includes methods to resync the database file.
package mempool defines a
mempoolMonitor type that can monitor a node's
mempool using the
OnTxAccepted notification handler to send newly received
transaction hashes via a designated channel. Ticket purchases (SSTx) are
triggers for mempool data collection, which is handled by the
mempoolDataCollector class, and data storage, which is handled by any number
of objects implementing the
See the GitHub issue tracker and the project milestones.
- Go 1.8.3 or newer.
hcd(>=1.1.0) synchronized to the current best block on the network.
Build from Source
The following instructions assume a Unix-like shell (e.g. bash).
Verify Go installation:
go env GOROOT GOPATH
$GOPATH/binis on your
dep, the dependency management tool
go get -u -v github.com/golang/dep/cmd/dep
Clone the hcexplorer repository
git clone https://github.com/coolsnady/hcexplorer $GOPATH/src/github.com/coolsnady/hcexplorer
dep ensure, and build executable
cd $GOPATH/src/github.com/coolsnady/hcexplorer dep ensure # build hcexplorer executable in workspace: go build # or to install hcexplorer and other tools into $GOPATH/bin: go install . ./cmd/...
The sqlite driver uses cgo, which requires gcc to compile the C sources. On Windows this is easily handled with MSYS2 (download and install MinGW-w64 gcc packages).
If you receive other build errors, it may be due to "vendor" directories left by
dep builds of dependencies such as hcwallet. You may safely delete vendor
folders and run
dep ensure again.
First, update the repository (assuming you have
master checked out):
cd $GOPATH/src/github.com/coolsnady/hcexplorer git pull origin master dep ensure go build
Look carefully for errors with
git pull, and reset locally modified files
Create configuration file
Begin with the sample configuration file:
cp sample-hcexplorer.conf hcexplorer.conf
Then edit hcexplorer.conf with your hcd RPC settings.
Indexing the Blockchain
If hcexplorer has not previously been run with the PostgreSQL database backend, it is necessary to perform a bulk import of blockchain data and generate table indexes.
- Create the hcexplorer user and database in PostgreSQL (tables will be created automatically).
- Set your PostgreSQL credentials and host in both
rebuilddb2 -uto bulk import and index.
- In case of errors, or schema changes, the tables may be dropped with
Finally, launch the hcexplorer daemon and allow the databases to sync new blocks. The SQLite database sync takes about an hour the first time. On subsequent launches, only new blocks are scanned.
Yes, please! See the CONTRIBUTING.md file for details, but here's the gist of it:
- Fork the repo.
- Create a branch for your work (
git branch -b cool-stuff).
- Code something great.
- Commit and push to your repo.
- Create a pull request.
Note that all hcexplorer.org community and team members are expected to adhere to the code of conduct, described in the CODE_OF_CONDUCT file.
This project is licensed under the ISC License. See the LICENSE file for details.
There is no documentation for this package.