command module
v0.1.7 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Sep 30, 2022 License: BSD-3-Clause Imports: 21 Imported by: 0


license Forum GoDoc


This is a Go application (originally written by Dvir Volk) which supports reading, indexing and searching using two search engines:

with the following datasets:

  • Wikipedia Abstract Data Dumps: from English-language Wikipedia:Database page abstracts. This use case generates 3 TEXT fields per document.
  • pmc: Full text benchmark with academic papers from PMC.

Getting Started

Download Standalone binaries ( no Golang needed )

If you don't have go on your machine and just want to use the produced binaries you can download the following prebuilt bins:

OS Arch Link
Windows amd64 (64-bit X86) document-benchmark-windows-amd64.exe
Linux amd64 (64-bit X86) document-benchmark-linux-amd64
Linux arm64 (64-bit ARM) document-benchmark-linux-arm64
Darwin amd64 (64-bit X86) document-benchmark-darwin-amd64
Darwin arm64 (64-bit ARM) document-benchmark-darwin-arm64

Here's how bash script to download and try it:

wget -c$(uname -mrs | awk '{ print tolower($1) }')-$(dpkg --print-architecture).tar.gz -O - | tar -xz

# give it a try
./document-benchmark --help
Installation in a Golang env

The easiest way to get and install the benchmark utility with a Go Env is to use go get and then go install:

# Fetch this repo
go get
cd $GOPATH/src/

Try it out

To try it out locally we can use docker in the following manner to spin up both a Redis and Elastic environments:

sudo sysctl -w vm.max_map_count=262144
docker run -d -p 9200:9200 -p 9300:9300 -e "ELASTIC_PASSWORD=password"
docker run -d -p 6379:6379 redis/redis-stack:edge
  • Retrieve the wikipedia dataset, and populate with 1000000 documents:
  • Populate into RediSearch:
./bin/document-benchmark -hosts "" -engine redis -file enwiki-latest-abstract.xml -maxdocs 100000
  • Populate into ElasticSearch:
./bin/document-benchmark -hosts "" -engine elastic -password "password" -file enwiki-latest-abstract.xml -maxdocs 100000
  • Run the RediSearch benchmark:
./bin/document-benchmark -hosts "" -engine redis -benchmark search -file enwiki-latest-abstract.xml
  • Run the ElasticSearch benchmark:
./bin/document-benchmark -hosts "" -engine elastic -password "password" -file enwiki-latest-abstract.xml -benchmark search 


The Go Gopher

There is no documentation for this package.


Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL